audio zoom

The main technology of audio zoom is beamforming or spatial filtering. It can change the direction of the audio recording (that is, it senses the direction of the sound source) and adjusts it as needed. In this case, the optimal direction is a supercardioid pattern (pictured below), which enhances sound coming from the front (that is, the direction the camera is facing directly), while attenuating sound coming from other directions (background noise). ).

The basis of this technology is that it is necessary to set up an omnidirectional microphone as much as possible: the more microphones and the farther away, the more sound can be recorded. When a phone is equipped with two microphones, they are usually placed at the top and bottom to maximize the distance between each other; and the signals picked up by the microphones will be in the best combination to form a supercardioid directivity.

The image on the left is a typical audio recording; the audio zoom on the image on the right has a supercardioid directivity, which is more sensitive to the target source and reduces background noise.

The result of this high directivity is obtained using a non-directional receiver by setting different gains for each group of individual microphones at various locations on the phone, then summing the phases of the spikes to enhance the desired sound and destroy the side wave to reduce off-axis interference.

At least, in theory. In fact, beamforming in smartphones has its own problems. On the one hand, cell phones cannot use the condenser microphone technology found in large recording studios, but must use electret transducers—miniature MEMS (micro-electro-mechanical systems) microphones that require very little power to function. Furthermore, in order to optimize intelligibility and control the characteristic spectral and temporal artifacts that occur with spatial filtering (such as distortion, bass loss, and overall sound with severe phase interference/nasality), smartphone manufacturers must not only carefully consider Microphone placement, too, must rely on its own unique combination of sound features, such as equalizers, voice detection, and noise gates (which themselves can cause audible artifacts).

So logically, each manufacturer has its own unique beamforming method combined with proprietary technology. That said, each of the different beamforming techniques has its strengths, from speech de-reverberation to noise reduction. However, beamforming algorithms can easily amplify wind noise in the recorded audio, and not everyone can or want to use an additional windshield to protect the MEMS. And why don’t the microphones in smartphones do more processing? Because that compromises the frequency response and sensitivity of the microphone, manufacturers tend to rely on software to reduce noise and wind noise.

In addition, it is impossible to simulate the real wind noise in a natural acoustic environment under laboratory conditions, and so far there is still no good technical solution to deal with it. As a result, manufacturers must develop unique digital wind protection technologies (which can be applied regardless of the product’s industrial design limitations) based on the evaluation of the recorded audio. Nokia’s OZO Audio Zoom records sound aided by its windproof technology.

Like noise cancellation and many other popular techniques, beamforming was originally developed for military purposes. Phased transmitter arrays were used as radar antennas during World War II, and today they are used for everything from medical imaging to musical celebrations. As for phased microphone arrays, they were invented in the 70s by John Billingsley (no, not the actor who played Dr. Volash in Star Trek: Enterprise) and Roger Kinns. Although the performance of this technology in smartphones has not improved significantly over the past decade, some handsets are oversized, some have multiple sets of microphones, and some even have more powerful chipsets. The mobile phone itself has a higher level, making the audio zoom technology more effective in various audio applications.

In N. van Wijngaarden and E. H. Wouters’ paper “Enhancing Sound by Beamforming Using Smartphones” states: “It comes to mind that surveillance countries (or companies) may use specific beamforming techniques to spy on all inhabitants .But to the extent of mass surveillance, how much impact can a smartphone’s beamforming system have? […] In theory, if the technology becomes more mature, it could become a weapon in the surveillance state’s arsenal, but That’s still a long way off. The specific beamforming technology on smartphones is still relatively uncharted territory, and the lack of mute technology and the inconspicuous synchronization options reduce the possibility of covert listening.