Audio Programming Guide

Table of Contents

  1. Introduction
    1.1 Device names
    1.2 Programming concept
    1.3 Demo application
    1.4 Feature limitations
  2. Application notes
    2.1 Disable audio processing for HDMI input
    2.2 Hot audio input switching
    2.3 Closing the stream
    2.4 Stereo microphone
    2.5 HDMI input audio
    2.6 Audio glitches

1. Introduction

This article outlines the concept of accessing standard audio devices in the Dolby Voice Room system. Starting with the version 2.0 release, the Dolby Voice Hub app can access standard audio devices in the system. These devices include:

  • HDMI input audio device
  • HDMI output audio devices (for both HDMI1 and HDMI2)
  • Stereo microphones in the Dolby Voice Camera
  • Microphones in third-party cameras
  • Mono capture and rendering device on the Dolby Conference Phone (new in the Dolby Voice Room 3.0 release)
  • Encoded capture and rendering device on the Dolby Conference Phone (new in the Dolby Voice Room 3.0 release)

The app may access these audio devices for various use cases. For example, the app can route HDMI input audio to the TV when the user connects a computer to the HDMI input port of the Dolby Voice Hub. In the Dolby Voice Room 3.0 release, the app can route HDMI input audio to the Dolby Conference Phone when the system is on a call to avoid air-coupling where the content audio rendered on the TV can be captured by the Dolby Conference Phone microphone and result in degraded audio experience.

1.1. Device names

Device name Type Description Available in Dolby Voice Room release
HDMI-Input Audio Audio capture HDMI input audio capture device Labeled as "AverMedia C353-Input Audio" in 2.x, label changed in 3.0
Microphones - Dolby Voice Camera Audio capture Stereo microphones from the Dolby Voice Camera 2.x
HDMI-Output Audio Monitor 1 Audio output HDMI 1 output for audio 2.x
HDMI-Output Audio Monitor 2 Audio output HDMI 2 output for audio 2.x
Third-party names Audio capture Third-party camera microphones, for example, the Logitech Brio has a stereo microphone 2.x
Third-party names Audio output Third-party audio rendering devices, for example, a USB speaker 2.x
Dolby Voice Conferencing Input Audio capture Encoded capture device from the Dolby Conference Phone 3.0
Dolby Audio Input Audio capture Raw, mono audio capture from the Dolby Conference Phone 3.0
Dolby Audio Output Audio output Raw, mono audio output on the Dolby Conference Phone 3.0

In the Dolby Voice Room 3.0 release, the HDMI input device will appear or disappear to the application based on the source device state.

To list all media devices in the Dolby Voice Room, go to the devtools console and enter "navigator.mediaDevices.enumerateDevices()". Open the returned promise and examine the device names.

When HDMI input source device is available

image

When HDMI input source device is not available

image

1.2. Programming concept and use cases

The following diagram shows the most commonly used audio devices in the Dolby Voice Room system and their common use cases.

image

The audio devices can be accessed through standard WebRTC getUserMedia APIs. The app can use Web Audio APIs to process audio to enrich the experience. For an overview of Web Audio API compatibility with the Dolby Voice Room, refer to this page.

Use case index Use case
1 Using "Dolby Voice Conferencing Input" device from a WebRTC application automatically offloads audio processing to the conference phone. The system will bypass the built-in WebRTC Voice Engine and rely on the conference phone to process encoded audio. This is useful for supporting best-in-class audio communication experience with Dolby Voice technology.
2 A WebRTC application could use "Dolby Audio Input" device but it is not recommended because the "Dolby Audio Input" device is optimized for voice recognition applications. However, if an application supports multiple audio streams, it could render audio on "Dolby Audio Output" while the system is on a call. A typical application is where content audio is rendered on the Dolby Conference Phone speaker while the main conference audio flows to the Dolby Conference Phone at the same time.
3 Using "Dolby Audio Output" device, the application can stream audio to the Dolby Conference Phone regardless of the system call state. A typical application is the HDMI input audio routes to the Dolby Conference Phone during the call.
4 A WebRTC application could capture audio from HDMI input device, encode it with a preferred audio codec, then send as a separate audio stream in the peer connection, or as a separate peer connection altogether to the service.
5 An app can route and stream audio for HDMI input and output devices. For example, when out of a call, the app can choose to route HDMI input audio to the TV speaker.

1.3. Demo application

The Dolby Voice Room demo application shows how an application can direct HDMI input audio to TV audio programmatically. To configure the Dolby Voice Room system to load the demo application, refer to this page.

1.4 Feature limitations

Starting with the Dolby Voice Room 2.0 release, the system does not support multiple peerconnections with audio streams. This limitation has been removed in the 3.0 release.

2. Application notes

2.1 Disable audio processing for HDMI input

Audio processing must be turned off for HDMI-Input Audio to ensure a good audio experience. For other devices, audio processing should be turned on (default); this can be done by setting additional media constraints during the getUserMedia call.

Get user stream for HDMI input

let audioId = 'default';  // e.g. use default audio input
Promise.resolve()
   .then(() =>
   {
      return navigator.mediaDevices.enumerateDevices();
   })
   .then((devices) =>
   {
      console.log('Devices', devices);

      let audioConstraints;

      audioConstraints = {deviceId: {exact : audioId } };
      if (audioId)
            {
         // For HDMI Input: without applying any audio processing for echo cancellation,
         // hpf, noise suppression, etc.
         const wcarray = Object.values(devices);
         const hdmiinputaudio = wcarray.filter((audio) =>
            audio.label.indexOf('HDMI-Input') !== -1 && audio.kind == 'audioinput'
         );

         if ((hdmiinputaudio && hdmiinputaudio.length
            && hdmiinputaudio.find((audio) => audioId == audio.deviceId))
         )
         {
            console.log('Remove audio processing');
            // Constraints for HDMI Input
            audioConstraints.echoCancellation = false;
            audioConstraints.noiseSuppression = false;
            audioConstraints.autoGainControl = false;
         }
         let constraints = {
           audio: audioConstraints,
           video: false
         };
         console.log('Constraints', constraints);
         return navigator.mediaDevices.getUserMedia(constraints);
      }
      else
      {
        return Promise.reject();
      }
      })
      .then((stream) =>
      {
        console.log('Stream:', stream)
        return stream;
      })

2.2 Hot audio input switching

In the event that the app needs to support switching audio input devices while the active input is being routed to the local playback device, the following application notes apply:

  • An issue can occur when trying to switch the audio input stream while performing local playback, where one stream is audio processed and the other is not
  • Switching the audio input stream without closing and re-opening the playback device can result in no sound being heard
  • To enable switching between audio input streams (processed & non-processed audio), the app can implement the following logic:
    • If the Audio Input is switched, then set the Audio Output Device_id to an empty string
    • Then set the Audio Output Device_id to the id of the selected device
    • This effectively closes and re-opens the playback device

2.3 Closing the stream

Stopping all audio tracks (in a stream that has only audio tracks) is sufficient for the Web App Engine to close the stream (e.g., before reloading the page). The next getUserMedia request from the same device will result in a "device busy" error, so the app must force close the audio stream. This can be done by setting the HTMLMediaElement src property to an empty string as shown in the following sample code.

/**
 * Stopping tracks and closing stream
 * @param element HTMLMediaElement audio/video element stream is attached to
 */
let closeMediaStream = (element) =>
{
    try {
        let stream = element.srcObject;

        if (!stream)
            return;

        console.log('Stopping stracks');
        let tracks = stream.getTracks();

        for (let i=0, len=tracks.length; i < len; i++)
        {
            tracks[i].stop();
        }

        console.log('Closing stream');
        element.src = '';
    }
    catch(exc)
    {
        console.error('Could not end tracks', exc.message);
    }

}

2.4 Stereo microphone

If the app tries to access stereo tracks from a stereo microphone it needs to turn off echoCancellation in the mediaConstraints. If echoCancellation is turned on, then both channels will have downmixed mono audio.

2.5 HDMI input audio

When the state of the HDMI input port changes, the system will not trigger a devicechange event. In the Dolby Voice Room 2.x release, the app is expected to rely on a configuration change event for Device.Status.HDMIInputDetected and enumerate devices to obtain access to the HDMI input audio device. Starting with the Dolby Voice Room 3.0 release, the app no longer needs to rely on the configuration change event since the device (HDMI-Input Audio and HDMI-Input Video) will be dynamically added or removed based on the source device state.

2.6 Audio glitches

A call to navigator.mediaDevices.enumerateDevices() can cause interruptions during audio playback; therefore, the app shall avoid calling this API too frequently.