App Automation

The purpose of the app automation feature is to provide an external stimulus to the Dolby Conference Phone to permit test automation. The phone can be controlled via a set of URI commands that simulate user input to the phone's touch screen. The Dolby Conference Phone can also be configured to record a user's touch input; these gestures are saved to a file on the phone. This feature can be enabled via a configuration parameter on the device. The file produced by the Dolby Conference Phone can then be used as-is or modified to provide a source of playback commands for a script.

Basic Steps:

  • Get a power user key from Dolby
  • Enable app recording on the Dolby Conference Phone
  • Initiate an app recording session
  • Have someone push buttons on the Dolby Conference Phone touch screen
  • Stop the recording
  • Download the app recording file
  • Modify the recording file
  • Run a script to play back the file

Note that this works on Dolby Conference Phone software version 3.3 and later.

Enable App Automation and Make a Recording

For security and privacy reasons, UI recording is not enabled unless the app automation feature is unlocked with a power user key that is provided by Dolby. To enable this feature, email the Dolby Conference Phone's serial number to DolbyVoiceSupport@dolby.com. The Dolby Conference Phone serial number can be found on a label on the bottom of the phone or via the phone's interface: Settings > ... > About > General

Enable and Record a User's Session

Enabling and controlling app recording on the Dolby Conference Phone makes use of the phone's shell. Use PuTTY or a similar SSH or telnet program to log into the phone. The user name is root; the password is 1739.

  • Enter the system poweruser key command with the acquired key to enable the poweruser commands.
  • Enter the system poweruser appautomation enable command to enable app recording.
  • Enter the configure Device.AppRecording true to start the application recording.
  • Have someone push buttons on the Dolby Conference Phone touch screen to perform a task such as start a call.
  • Enter the configure Device.AppRecording false command to turn off recording. This creates a recording file named app_recording.dpf which is bundled with the syslog files on the Dolby Conference Phone.

The following screen capture shows an example of these commands entered in a Dolby shell:

login as: root
Enter password: ****

+- - - - - - - - - - - - - - - - - - -+
+       Dolby Conference Phone        +
+- - - - - - - - - - - - - - - - - - -+

Dolby>system poweruser key
Your device serial ID is 40084
Power user key is not set yet.

Dolby>system poweruser key A949:10CE:352B:0BC3
Your device serial ID is 40084
Power user key is set to: A949:10CE:352B:0BC3
Validating ... Key is valid.
Current time is: Wed Jun 14 15:07:44 2017
Key expires: Mon Jan 18 19:14:07 2038

Dolby>system poweruser appautomation enable
App Automation is enabled.

Dolby>configure Device.AppRecording true

Dolby>configure Device.AppRecording false
Dolby>

Download the Recording File

Download the recording file from the Dolby Conference Phone using the "download syslog" process. Get the Dolby Conference Phone's IP address via Settings > ... > About > General. Then, using a web browser, enter the IP address into the URL field (ignore the certificate warnings). The login username is admin and the password is 1739. Click on the Tools tab and expand the Utilities field. Click on the link in the first line, "You can download syslog files using this link."

The syslog file generated by the example above is named DVCP-D0040084-20170614152316.tar.gz Move the file to a different directory, if desired, and then untar it via a tar command or an app such as 7-Zip.

An example subdirectory containing the syslog files with the app_recording.dpf file:

C:\__some_path__\DVCP-D0040084-20170614160531>dir
 Volume in drive C is Default
 Volume Serial Number is 14FD-2800

 Directory of C:\__some_path__\DVCP-D0040084-20170614160531\DVCP-D0040084-20170614160531

06/14/2017  04:07 PM    <DIR>          .
06/14/2017  04:07 PM    <DIR>          ..
06/14/2017  04:05 PM             3,482 .client.json
06/14/2017  04:05 PM               668 .prov.json
06/14/2017  04:05 PM            25,919 .status.json
06/14/2017  04:05 PM             1,541 app_recording.dpf
06/14/2017  04:07 PM    <DIR>          extra
06/14/2017  04:07 PM    <DIR>          log
               4 File(s)         31,610 bytes
               4 Dir(s)   6,135,726,080 bytes free

The contents of the app_recording.dpf file are shown below.

The output contains two types of lines:

  • URI commands that define the touchscreen gestures
  • the names of the pages that the Dolby Conference Phone is displaying

The numbers in the left column are the time in milliseconds after the recording was started.

The UI commands are described in the next section.

1738 ui/mouse?event=press&localX=280&localY=62&button=1&buttons=1
1874 ui/mouse?event=move&localX=280&localY=61&button=0&buttons=1
1881 ui/mouse?event=release&localX=277&localY=61&button=1&buttons=0
CredentialInputPageOverlay
3348 ui/mouse?event=tap&localX=154&localY=143&button=1&buttons=0
4027 ui/mouse?event=press&localX=251&localY=247&button=1&buttons=1
4200 ui/mouse?event=move&localX=247&localY=247&button=0&buttons=1
4204 ui/mouse?event=release&localX=247&localY=247&button=1&buttons=0
4508 ui/mouse?event=tap&localX=240&localY=193&button=1&buttons=0
5142 ui/mouse?event=tap&localX=248&localY=86&button=1&buttons=0
5668 ui/mouse?event=tap&localX=241&localY=271&button=1&buttons=0
6211 ui/mouse?event=tap&localX=327&localY=90&button=1&buttons=0
6696 ui/mouse?event=tap&localX=148&localY=90&button=1&buttons=0
7245 ui/mouse?event=tap&localX=333&localY=143&button=1&buttons=0
7845 ui/mouse?event=press&localX=162&localY=195&button=1&buttons=1
7937 ui/mouse?event=move&localX=160&localY=195&button=0&buttons=1
7976 ui/mouse?event=release&localX=156&localY=195&button=1&buttons=0
8353 ui/mouse?event=tap&localX=147&localY=212&button=1&buttons=0
8993 ui/mouse?event=press&localX=437&localY=208&button=1&buttons=1
9084 ui/mouse?event=move&localX=437&localY=207&button=0&buttons=1
9088 ui/mouse?event=release&localX=437&localY=207&button=1&buttons=0
10226 ui/mouse?event=tap&localX=296&localY=214&button=1&buttons=0
ConnectScreen
ConferenceScreen
26477 ui/mouse?event=tap&localX=423&localY=138&button=1&buttons=0
TransitionOverlay
HomeScreen

Explanation of URI Commands

There are 2 major commands. One controls the mouse events and the second command enables the notification of UI page changes.

Mouse Events

The Dolby Conference Phone records finger gestures as mouse events. There are four types of mouse events: press, release, move, and tap. A mouse "click" is a pair of events (i.e., press & release). The move event is only interesting during a "drag" operation (i.e., press, move, move,....., move, release). Since a "pointer" is not displayed on the embedded UI, a regular move event has no significance. However, if the user's finger shifts position a little while touching the screen, a button press may be recorded as a set of press, move, release commands. If the finger does not drift, it will be recorded as a single tap command.

URI://ui/mouse?event=(press|release|move|tap)&localX=X_pos&localY=Y_pos&button=0|1&buttons=0|1

event: press|release|move|tap

localX & localY: mouse position in pixels. 0,0 is the upper left corner of the embedded UI. 479,271 is the lower right corner

button: The button causing the press or release event. 0 - no button, 1 - left button (default button when touching the embedded UI). All other button values are not applicable to the Dolby Conference Phone. For more information about button, refer to the QT documentation.

buttons: The state of the buttons during the event. This is significant for a drag operation in which all move events should have a value of 1 (i.e., button pressed down). For more information about buttons, refer to the QT documentation.

Examples:

Mouse press

URI://ui/mouse?event=press&localX=316&localY=186&button=1&buttons=1
URI://ui/mouse?event=move&localX=318&localY=186&button=0&buttons=0
URI://ui/mouse?event=release&localX=316&localY=186&button=1&buttons=0

Mouse press using a single tap command

URI://ui/mouse?event=tap&localX=234&localY=67&button=1&buttons=0

UI Page Changes

Enable Notification

During the recording process, all Dolby Conference Phone notifications are monitored, but only the page changes are written to the recording file. To track the behavior of the app, the change page notifications must be monitored during playback. The following command turns on the Dolby Conference Phone's notification process. During playback, the notifications can be parsed to match on a page change event (see next section).

URI://ui/display?feedback=true|false

Notification API (Poll for Page Changes)

Once the notification feature has been turned on, the following command is used to poll for all Dolby Conference Phone notifications.

https://{IP address}/notification

The playback script or tool can parse the Dolby Conference Phone notifications to identify page change events. Since the expected page name is embedded in the playback file, the contents of the page change event notification can be compared to the expected outcome. The following line is an example of a raw text notification that contains page change information that can be parsed.

{"resource":"ui","display": {"current":"CredentialInputPageOverlay" } }

Playback Example

The app_recording.dpf file can be used to simulate user input to the Dolby Conference Phone's touch screen for test automation purposes.

Several approaches, i.e., scripts or tools, can be used to send the URI commands to the Dolby Conference Phone. A simple method is to manually enter URI commands in a web browser to execute them, e.g.,

https://10.112.102.119/ui/mouse?event=tap&localX=240&localY=193&button=1&buttons=0

The example below makes use of a simple Python script that is provided as-is. The Python file is included with the SDK release.

The following command must be inserted and be the first line in the recording file.

0000 ui/display?feedback=true

In this example, the Dolby Conference Phone must be on its home page. If the Dolby Conference Phone is on another page, the playback button press commands will not match and may cause unexpected behavior. To start a test on the home page, the ‘showHomePage’ URI command can be used:

https://ip_address/ui/display?showhomepage=true

where "ip_address" should be replaced with the phone's IP address.

Play back the recording file

Before commands can be sent to the Dolby Conference Phone, the script or tool must log in via this command

https://ip_address/auth/login?username=admin&&password=1739&&source=web_ui

where "ip_address" should be replaced with the phone's IP address.

The script was run from a Windows Command Prompt:

> python playback.py IP_address app_recording.dpf

The following shows the program's output; the Dolby Conference Phone touch screen shows the buttons being pushed and new pages being presented.

C:\App_Automation_testing>python playback.py 10.112.102.119 app_recording.dpf
Start...
Expecting:  CredentialInputPageOverlay
Actual:  CredentialInputPageOverlay

Expecting:  ConnectScreen
Actual:  ConnectScreen

Expecting:  ConferenceScreen
Actual:  ConferenceScreen

Expecting:  TransitionOverlay
Actual:  TransitionOverlay

Expecting:  HomeScreen
Actual:  HomeScreen

...Done

C:\App_Automation_testing>

A python playback script is provided as an example on how to interface to the Dolby Conference Phone. Depending on your environment, the script may need modifications and/or installation of other packages.

Python Example