Dolby Digital plus for web projects: Meet re.flow

October 17, 2015 by Jason Marsh

Editor's Note: This article serves as the main index of a full examination of the building of the audio/visual experience named "re.flow". This post, as well as the following posts, are intended to give developers a view into the thought process behind building this interactive experience, the methods employed to generate multichannel sound and the steps to delivering that experience.

Jason Marsh, Founder/CEO and VR Information Architect for Flow will walk developers through the process step-by-step, and highlight the use of all tools, frameworks and services involved.

Dolby Developer is proud to be involved in this project, and is excited to have Jason share his knowledge to help other web developers start building better audio into web experiences. It is hoped that this experiment helps make the case for putting audio first in development experiences. 

THE DEVELOPER'S CASE

This is the first in a series of posts describing the creation of “re.flow” by Jason Marsh at the request of Dolby Laboratories to show off the possibilities of spatial audio with nothing other than a modern browser.

Dolby Digital Plus in the browser is significant not only because it affords playback of multi-channel content when connected to a home theater or headphones on systems that provide surround virtualization, but because it provides a premium consistent audio experience to the browser by incorporating a rich set of metadata about the audio that is part of the Dolby Digital Plus codec. This way, Dolby encoded content sounds great regardless of the system it’s being played back on.

The guidelines for the project were quite simple: create a WebGL project that features Dolby Digital Plus 5.1 surround audio. WebGL is used to create highly dynamic, smoothly animating graphics like you would expect from gaming consoles.

It took me under five minutes to realize that this was the perfect project for me: I have a Masters in Music composition, I’ve been programming forever, years ago I produced binaural projects for Bose, and I’ve been doing WebGL programming full-time for the past year working on Virtual Reality (WebVR). A project made in heaven, right?

THE PITCH

The original pitch I made was a silly sketch of planes floating in space, each synchronized to an audio track, and each with a custom visualizer for that track. Luckily, to their credit and not mine, for some reason the Dolby folks trusted me.Before reading further, you have to try the end result. The actual experience is hard to describe, and since it is a web project, why bother? Use the Microsoft Edge browser on Windows 10, if you have it, or the other modern browsers including Chrome if you don’t. The audio won’t be as good without the Dolby processing in Microsoft Edge, but it will all work.

Take a look now if you haven’t already done so: http://dolby.flow.gl

Did you like it? I can hope so!

THE CRITICAL STEPS

The main aspects of this project, and tools used were:

AUDIO PRODUCTION:

  • Producing the music, 16 clips in all, in Ableton Live and hiring the vocalists.
  • Obtaining 5.1 surround studio hardware.
  • Taking stereo mixes exported from Ableton Live into Adobe Audition CC 2015 to create the 5.1 panned ec3 files.
  • Wrapping the ec3 files into mp4 files that the browser will be able to use.

AUDIO PROGRAMMING

  • Building the music sequencer using the Web Audio Context (all in JavaScript) to simultaneous play 8 tracks, synchronized to the millisecond
  • Building the sequencer user interface
  • Saving the sequencer to a database in the cloud

WEBGL DESIGN AND PROGRAMMING

  • Creation of the 3D scene using ThreeJS and GLAM
  • Implementation of animated textures in GLSL shaders
  • Animation engine to sync up the spatial audio mixes to the graphics
  • Pickers and interface elements within the 3D scene

Finally, though not within the scope of the original project, I went ahead and created a WebVR version so that I could experience it all in Virtual Reality. While this is not ready for prime time and only works on pre-beta browsers, it is a great experience and ultimately a powerful application of all these technologies.

DIGGING INTO THE DETAILS

This post is the first of four posts describing various aspects of the project. So into the details we go: