Netflix's ingenious Android update isn't about 'studio quality' audio
The technology media coverage of the Netflix Android update makes it sound like the streaming service only delivers "studio-quality audio" on smartphones. In reality, the update promises something way more significant for mobile devices. Netflix's grossly misinterpreted blog post details how it uses a new audio compression format introduced with Android 9.0 Pie to make movies and TV shows sound better on smartphone speakers.
Crux of the update involves optimizing movies for smartphone speakers
Traditional movie audio is mixed for sophisticated cinema sound setups capable of reproducing a wide dynamic range, from a whisper to an explosion. The narrow dynamic range of TV and smartphone speakers makes them incapable of faithfully reproducing such sound. Controlling the dynamic range is therefore important for a satisfactory aural experience on mobile devices. This forms the crux of Netflix's latest Android update.
Figuring out how to equalize volume levels across content
Although existing audio formats also provide dynamic range control, they aren't particularly good at equalizing the wide variation in dynamic range between action and drama flicks, or ghazals and heavy metal music for that matter. This inherent failure to intelligently equalize audio forces users to continually reach for the remote when dialogues are too faint and explosions too overbearing at the same volume level.
Dialogues are key to perceived loudness
During the course of their research, Netflix engineers discovered that dialogue level is the most important benchmark driving users' perception of loudness in cinematic and television content. They also realized that simply compressing the audio spectrum tends to cause significant variance in dialogue levels between action films and TV shows, because the former are louder in general and the latter being relatively quieter.
Leveraging MPEG-D DRC metadata for intelligent dynamic range control
Netflix has figured out a way to utilize the sophisticated dynamic range compression capabilities of the Extended HE-AAC (xHE-AAC) audio format introduced with Android 9.0 Pie. The new MPEG-D DRC metadata is the special sauce that separates the Dynamic Range Control (DRC) capabilities of xHE-AAC vis-à-vis existing audio formats. It provides Netflix granular and intelligent control over audio levels.
Netflix explains the significance of DRC metadata in audio-level equalization
"If you switched from the action show to the live concert, you would likely be diving for the volume control to turn it down! When the drama comes on, you might not be able to understand the dialogue until you turn the volume back up."
Intelligently equalizing sound levels so you don't have to
Netflix has essentially figured out an intelligent way to exploit the MPEG-D DRC metadata capability of the xHE-AAC codec to dynamically compress audio for smartphone loudspeakers while equalizing the dialogue levels across all kinds of content. In practice, Netflix claims this saves users the annoyance of reaching for the remote control while browsing through the vastly disparate content genres available through its streaming repertoire.
Netflix claims positive results from focus groups testing
The official blog post reveals that test subjects who participated in A/B (comparative) testing switched away from built-in smartphone speakers 7% less often when listening to xHE-AAC, with reduced volume change interaction. The figure improved to 16% for high dynamic range content, showing that the new DRC algorithm was doing a good job of managing volume variations between different content.
High-fidelity sound on the move, even with poor internet connectivity
The improved audio format in the new update incorporates seamless bitrate/quality switching for an improved Netflix experience for those on the move. This allows it to deliver high fidelity sound with fast and stable internet connectivity, while also coping well with poor internet speeds and network congestion. All of this without the need to re-buffer each time the audio bitrate/quality is changed.
Netflix hints at carrying the feature over to Android TV
The DRC techniques perfected in this Android update are expected to be applied to other platforms supporting the audio codec. The blog post has tacitly revealed that the feature could come to Android TV and desktop apps in the future. Not everyone owns an expensive home theatre setup, so this feature would be a welcome addition to Android TV and desktop apps as well.
