WristMan: An Unobtrusive Wrist-watch Interface for ...



WristMan: An Unobtrusive Wrist-watch Interface for Controlling Music Playback

Scott Batura, Daniel Nealey, Gaetano Borriello

Department of Computer Science and Engineering

University of Washington, Seattle, WA

{batura, dnealey, gaetano} @ cs.washington.edu

ABSTRACT

Current portable media players depend on a single unit to provide both the audio storage and the user interface for the device. These types of designs require the device and user to be within close proximity to one another, which is often inconvenient and unnecessary for certain users. Many commercial products have attempted to solve this problem by providing basic remote controls, which are constrained by their limited functionality and lack of customization. In this paper, we present the WristMan system, which solves these problems by separating the user interface from the audio storage and playback device. The WristMan interface, which runs on a DisplayMote, allows the user to adjust playback using common media playback controls from a remote setting. This wireless configuration allows the user to move away from the storage device (Intel Personal Server) and control audio output from anywhere in the local vicinity. Our paper describes the WristMan system architecture and use scenarios in addition to providing an analysis of the effectiveness of our implementation. Finally, we provide a discussion about future projects and ideas that our work could be used for.

1. INTRODUCTION

One problem facing portable media players currently on the market is their reliance on a single unit acting as both the playback device and the user interface. Interacting with a single device in this manner may lead to many inconvenient situations, such as having to hold and manipulate a bulky device or tangling wires that provide sound output and power. In addition, the user is forced to have the playback device near them or even on their person, where as it might be optimal to have the device connected to audio and video equipment.

Our project, named WristMan, attempts to solve this issue by separating the playback device from the unit’s interface. The system consists of a system-on-a-chip device called a DisplayMote [1], and an Intel Personal Server [2][3]. These two components communicate over a packet radio to seamlessly integrate the user-controlled interface and the media storage/playback device of our system. The DisplayMote functions as the interface, which is presented on an unobtrusive wristwatch style unit. The Personal Server holds the user’s files while also providing the functionality for handling requests from the DisplayMote in addition to decoding and playing audio. This system allows the user to manipulate the interface (DisplayMote) freely without requiring the storage/playback device (Personal Server) to be on their person.

[pic]

Figure 1. The DisplayMote combines a microprocessor, a 64x128 LCD, four buttons and a packet radio into a convenient wristwatch form-factor.

As a typical use scenario, imagine that an individual would like to listen to their music collection while they work. The user would first power on the Personal Server, which would automatically launch the server message handler program, and power the DisplayMote, which would establish a connection to the server. Once the connection is made, the user is presented with the Welcome Screen on the DisplayMote that allows them to select between browsing their song library and configuration options. If they choose to browse their song library, they will be presented with a scrolling list that shows them the root of their media library. From this menu, they can select to browse other directories or listen to songs and playlists. After a music file is selected, the user has the option of manipulating playback with the following controls: Play, Pause, Stop (which takes them back to the last folder they were in), and Track Forward and Back (if in playlist mode). All of theses selections require no interaction with the actual media device (after powering it on) as the DisplayMote interface provides support for all of these functions.

In the following section of our paper, we will compare WristMan to similar projects and analyze the differences between them. After that, in the third section, we will describe in detail how our system operates, along with descriptions of our architecture and its implementation. Section four will present our evaluations of the system functionality and its interface along with an analysis of our communication protocols and their reliability. The final sections will provide our conclusions about the WristMan system in addition to exploring future projects based on our work.

2. RELATED WORK

Upon analyzing MP3 devices presently on the market, we have found that few make an attempt at interface separation. The Apple iPod, which is probably the most widely recognized media player, does little to separate the interface from the playback device [6][7]. The iPod is primarily marketed as a media player to be worn on the user’s person, so its interface and controls are strictly on the device. Apple does offer a wired remote upgrade for the iPod but it forces the user to handle the media device directly to control music playback. Other devices, such as the Creative Nomad Jukebox [6], offer wired remotes that allow users to control playback without directly handling the playback device. These solutions don’t achieve interface separation either because the user’s movement and proximity to the device are restricted. There are also many devices that offer infrared remotes, none of which display track titles or give other playback information [10]. WristMan dose not suffer any of these limitations, as the user can be up to approximately 20 feet away from the playback device while sill maintaining control over directory browsing and audio playback via the DisplayMote interface.

In the domain of cell phone-computer interaction, there are two projects that are relatively similar to WristMan. The first project we researched is called Bemused, which is a two-part software program to allow a user to control their media using a cell phone [4]. The first component is a PC software program that sends information about the user’s media library to the cell phone and also controls song playback. The second component is the cell phone software that provides the user interface for media playback. This project accomplishes goals similar to those of WristMan. However, the Bemused server software cannot run on standard embedded platforms because it is reliant on floating-point based operations [5]. This requirement limits the Bemused project to operating on general PCs instead of smaller, portable platforms such as the Personal Server.

[pic]

Figure 2. The Bemused interface provides normal playback controls on a cell phone platform [4]

MP3Radio, a similar project, is currently being developed and hopes to achieve the same capabilities as the Bemused application, but with different media decoding methods that allow it to run on embedded platforms [5]. This project is targeting the Personal Server as its media playback device and Nokia cell phones as the interface platform. Cell phones may seem like an ideal platform for these projects, as they have become commonplace throughout the world. However, we feel that their use for media playback control is limited by the scope of their physical interface, which was primarily designed to act as a portable telephone.

3. APPROACH AND IMPLEMENTATION

We have completely separated our software components to reflect the ideology of our project: separation of interface and playback device. Our interface platform, a DisplayMote, runs TinyOS, which is a language for embedded systems development that provides support for tasks, events and commands [8]. The device that we used for file storage, media playback and remote message handling was the Intel Personal Server, which is a small, battery-powered computer that runs Linux [2]. The Personal Server also has the capability of having a Mote directly connected to it that allows us to communicate with our DisplayMote over the packet radio. This section will discuss the implementation of the two main components in our system: the wireless interface software and the server and playback software.

[pic]

Figure 3. The separation of interface (DisplayMote) and media storage/audio playback (Intel Personal server and associated hardware).

A. Wireless Interface Software

We decided to use the DisplayMote/TinyOS platform for our user interface software because it combines intuitive button based controls with a simple LCD display. The following section will demonstrate the simplicity of this interface by walking through a typical user driven scenario. After this demonstration, we will provide specific details on the software implementation of the WristMan system.

i. Demonstration

For the purpose of this demonstration, we will assume that a user wants to start the device from its powered-off state. The first screen that the user sees when powering up the device is the “Attempting Connection” screen, shown in Figure 4. This message indicates that the mote is repeatedly sending out packets to the Personal Server with the intent of establishing a remote connection.

[pic]

Figure 4. The DisplayMote software initially tries to establish a remote connection with the Personal Server, show by the “Attempting Connection” screen.

When the Personal Server detects an “establish connection” message from the DisplayMote, an acknowledgement is sent confirming the signal. When the DisplayMote receives the acknowledgement the “Welcome to WristMan” screen is displayed, indicating that the communication startup handshake was successful. This initial screen, shown in Figure 5, presents the user with two different selections: “Browse Library”, which allows the user to remotely play and control audio files on the Personal Server, and “Options”, which contains various settings and customizations that the user can change. The user can use the menu browsing buttons on the left side of the mote to move up or down the current menu.

[pic]

Figure 5. The “Welcome to WristMan” screen allows the user to select between browsing their media library and setting system options.

Selecting the “Browse Library” option and pressing the top-right “Select” button, the user will be presented with the base of their media directory, as seen in Figure 6.

[pic]

Figure 6. The user can enter nested directories and choose songs and playlists to play after selecting “Browse Library”. Directories are indicated by “/” following the file name while Playlists are indicated by “>”.

From here, the user can use the buttons on the left side of the mote to scroll though this list, or they can select a song to play, a playlist to play, or a directory to browse by pressing the top-right button. If they choose to browse a directory, they will be presented with a screen similar to this that shows the contents of that directory. If they choose to play a song, they will be presented with the now playing screen, seen in Figure 7.

[pic]

Figure 7. The “Now Playing” screen shows the presently playing song or playlist and allows the user to use common CD player-like controls.

During the “Now Playing” screen, the top-right button acts as a pause/play button common to most MP3 and CD players. If they press the lower-right button, the song stops playing and the previous directory browsing screen will be shown again. If the user had selected a playlist to play, the top-left button will act as a track forward button, and the lower-left button will act as a track back button. Much like a CD player, one press of the track back button will restart the current song from the beginning. Another press of the track back button will play the previous song in the playlist.

ii. Design and Implementation

During the design of the interface, we were careful to take into consideration existing designs from popular music players. Many of our playback controls, such as track forward and back, were designed to mimic the interface common to devices like the iPod [7] and programs like WinAmp [9]. By doing this, we felt that our system would be easy for most people to use because they are already familiar with the design. While designing the browse capability, we chose to add a character to the end of each file name to designate what that file is, which can be seen in Figure 6. These designations are “>” for a playlist, “/” for a directory, and if there is no designation, it is a normal MP3 file. This makes it easy to see what each file in the menu does while using very little screen space.

The interface software for the DisplayMote was written in the nesC language that is intrinsic to TinyOS. Our program follows the standard TinyOS convention and is divided into three modules: RadioM, which provides all of the messages necessary for the interface to communicate with the server; StateM, which manages which state the interfaces software is in and how it should interpret messages from the user and radio; and MenuM, our menuing system that provides a dynamic menu system for browsing the user’s library. The interfaces Radio, State and Menu allow these modules to communicate with each other and these modules are wired together through WatchConfiguration.

[pic]

Figure 8. TThe modules, interfaces and configuration of the TinyOS interface software written for the DisplayMoteT

The principle implementation challenge we faced was figuring out how to make the modules interact properly in the state machine paradigm of TinyOS. Because TinyOS is an embedded, synchronous language, it forbids a main execution loop and delay timers in favor of its own Event, Task and Command system

We decided that most of the implementation pertaining to the user interface should be driven by synchronous tasks that are queued by the operating system. This ensured that system features such as the radio took higher precedence than drawing to the screen, which didn’t require “real-time” performance. All of our radio communications are handled from either task or event contexts depending on the relative importance of the radio message that was being sent.

The radio module, RadioM, sends and receives all messages that are needed to operate the interface. For each possible message that the system needs to send, there is a command specified for these messages in the Radio interface, which allows the other modules to call the commands when needed. This module also receives messages from the server software. These messages are not handled in the Radio module itself (except for error handling), but passed on to the State module that is the primary controller of the system.

The State module, StateM, is responsible for controlling the primary operation of the system. This module maintains which state the interface is in, interprets messages from the server, processes input from the user and sends information to the Menu module for display. These operations are centralized to avoid data concurrency issues between the State and Menu modules. Most of the data necessary to operate this system is stored in this module, with the exception of the radio messages and the file listings, which are stored in the Radio and Menu modules, respectively.

The Menu module, MenuM, is responsible for rendering the actual user interface on the DisplayMote. Since it is “wired” to Waylon Brunette’s DisplayMote modules [1], it handles all calls that render information on the screen, capture button presses (passing them to the State module) and stores all the file listings. There are also methods in this that help the other modules display information, such as the “Now Playing” and “Attempting Connection” messages.

A primary design goal of the Menu module was to make it portable beyond this project. With a few subtle modifications, we will be able to take our menu system and allow other TinyOS developers use it to drive the DisplayMote’s text capabilities. The menu system is fairly flexible in its behavior, and with the implementation or modification of a few methods, it should be adaptable to other systems. This menu system is also dynamic, which means that only the first few items (equal to the number of displayable rows) in the menu need to be added, with the rest added in when the user scrolls up and down. This allows for quicker startup times, especially when the items to be displayed are transmitted through the packet radio.

B. Server and Playback Software

The WristMan server software is composed of two distinct parts: the music database and the remote communication handler, both of which are written in the C language. The music database is a memory resident mirror of the user’s music file structure while the remote communication handler processes all radio messages sent from the DisplayMote. Together, the music database and the remote communication handler allow the server software to recognize incoming messages and take appropriate actions based on the type of message. This design allows the user to remotely control the playback of music and audio files that are resident on the Personal Server.

[pic]

Figure 9. The Server and Playback system features two main components: the Remote Communications Handler that interprets messages from the DisplayMote and the Music Database that stores the file system, which is referenced by the remote communication handler.

i. Music Database

The first task of the server software is to recreate the file structure of the user’s music directory, which will server as the music database of our system. It is necessary to load this file structure in memory so that quick lookups can be performed to locate files and their associated information. This memory resident database is created by a recursive function that traverses the tree-structure of the media directory and adds files to our database as they are encountered. The algorithm omits directories that do not contain music files or valid playlist files. In addition to recreating the tree-structure of the media directory, a flat array with all of the directory entries is also created. Unique keys are assigned to each of the directories in this array so that they can be indexed directly, regardless of their depth in the tree. Each music or playlist file is then assigned the key of its parent directory along with file offset into the directory as shown in Figure 10. Finally, each file is also marked to be a directory, playlist, or music file. Using these three attributes (directory key, file offset, and file type) any file located in the user’s media directory can be quickly referenced via our database.

[pic]

Figure 10. Directories within the media file system are assigned unique keys that correspond to the index of the directory key table. Individual files are referenced by a local offset within the currently indexed directory.

After the music database has been constructed, the server software opens a port for remote communications and then waits for the DisplayMote to send an establish connection signal. When the server receives an establish connection signal, it then sends an acknowledgement telling the DisplayMote that a proper connection has been made. After the two devices have established a handshake with one another, the server software enters its primary processing loop, otherwise known as the remote communication handler.

A brief pseudo-code example for the basic flow of the server software is given as follows:

// Create the memory resident database

file_sys = build_file_system();

// Open a port for remote communications

open_port();

// Handshake with DisplayMote

establish_connection();

// Remote Communication Handler

// Determine what actions need to be performed // based on incoming messages

for(;;)

{

packet = read_packet();

handle_packet(file_sys, packet);

}

ii. Remote Communication Handler

The remote communication handler functions as a message fetch and execute engine. Messages originating from the DisplayMote are read through the communications port and then executed based on the type of the message. The first request that the DisplayMote makes is always a directory request for the root directory of the Personal Server’s media folder.

When the request directory message is received, the remote communication handler will package up to the first eight entries of the database root and send them back to the DisplayMote. The reason that up to eight entries are sent is because the DisplayMote is only able to display eight lines of text. Additional entries are only sent when the user browses up or down. Before these entries are sent, the server will send a message informing the DisplayMote of the total number of files that belong to the directory that was requested. After the initial directory size message is sent, the server then transmits the file entry information to the Mote. Each file entry message contains the lookup attributes (directory key, file offset, file type) and the name of the file. This information is used by the DisplayMote to print the name and type of the file to the screen. The lookup attributes can then be used to request new directories, and play music or playlist files. If the user selects another directory to view, the directory request process repeats and new file entries are sent to the DisplayMote.

If a user selects a song or playlist to play, the Display Mote sends a play file message that includes the lookup attributes to the remote communication handler. The lookup attributes are then used by the database to locate the file path that corresponds to the song or playlist that was selected on the DisplayMote. First, the directory key indexes the array of directory entries, then the file offset is used to locate the specific file within the directory. Finally, the file type is used to determine the method of playback (single audio file, or playlist). The remote communication handler then executes Madplay to control audio playback of that file.

iii. Madplay

We selected Madplay for audio playback primarily because it does not use a floating-point based Mp3 decoding algorithm [11]. This is an important feature because the ARM processor on the Intel Personal Server does not have a floating-point unit and cannot efficiently emulate those operations. Therefore, our only choice was to select an Mp3 decoding algorithm that was implemented with integer or fixed-point based operations. While Madplay suits this specific requirement, it was not designed to allow external processes to control audio playback. Instead, Madplay was designed to allow the user to control playback via the terminal and keyboard. Therefore we modified Madplay so that external commands could be sent via UNIX named pipes. This modification allowed the remote communication handler to send playback commands directly to Madplay without interacting with a terminal or keyboard.

4. EVALUATION

The WristMan implementation should be evaluated based on the following two characteristics: functionality of the system, and the simplicity of the user interface. Successful system functionality is relatively easy to gauge because we can analyze the features based on whether they work properly and within a reasonable time frame. However, analyzing the user interface is more difficult because individuals will have differing opinions about the aesthetics of the menu system and how the buttons should be utilized.

A. Functionality

We have implemented most of the standard features of a music playback device including play/pause, stop, track forward, and track back. Unfortunately, additional features such as fast forward and rewind would be very difficult to add primarily because of the four-button limit imposed by the DisplayMote. However, future versions of the DisplayMote will incorporate additional buttons, making the additional features possible.

i. Latency

The real metric for evaluating the functionality of the feature set is the latency involved in sending messages to and from the WristMan server software. Our tests show that the round trip latency for sending, processing and re-transmitting a single message is approximately 40-50 milliseconds. These delays are small enough so that the majority of communications between the DisplayMote and Personal Server are transparent to the user.

ii. Reliability

Another issue with the communications is the reliability of the DisplayMote’s packet radio. Sending single packets per message is not an ideal communication strategy because messages will occasionally be dropped. Our method for dealing with this behavior is to transmit multiple identical radio packets per message. Testing has shown that 3-4 packets per message will guarantee with an extremely high probability that no message will be lost. However, this method of “flooding” packets is not ideal as it adds additional latency to communications between the DisplayMote and the Personal Server. The user can perceive this latency during a directory request, when up to 8 messages are sent to the DisplayMote. There is roughly a 1.5-2 second delay for directory requests using this protocol.

An alternative to using the packet flooding method would be to use a protocol where each message is acknowledged by a return message. A timeout mechanize is necessary for this protocol so that if a return acknowledgement is not received within a specific amount of time, the message is rebroadcast. However, the acknowledgement method is still not ideal because of the frequency of dropped packets with the DisplayMote’s packet radio. Each timeout will affect the latency and produce results that are usually worse than the simple flooding strategy. Ultimately, to solve this problem a more robust technology such as Bluetooth will need to be used to ensure reliable transmissions.

B. User Interface

In general, user interfaces are a very subjective and hotly debated topic. However, for music playback devices, users have grown accustomed to a standard set of controls and features. We have worked these expectations into the design of WristMan so that users can easily transition from using standard audio devices to our product. For instance, music playlists work in WristMan just like in a car stereo or portable CD player. Ultimately, our goal is for users to be able to pick up and use the device without any previous knowledge of how it works. To this end we have envisioned adding descriptive labels next to the buttons so that users can clearly understand their functionality.

i. Limitations

In addition to providing standard audio controls with our interface, we have strived to incorporate known designs for menu browsing and file selection from popular Mp3 players such as the iPod. Unfortunately we cannot incorporate the full functionality of these devices because of the limitations of the DisplayMote such as the screen size and number of buttons. Currently, we can only show up to eight lines of text at a time on the screen, which also means that we can only display eight music files at a time. Additionally, we only have four buttons to work with so controls such as fast forward and rewind cannot be implemented in a logical manner. However, in a sense these limitations work to our advantage because they reduce the complexity of the device as seen by the user.

5. FUTURE WORK

While we feel overall that our project was successful in meeting our goal of separating the interface from the playback device, there are some aspects that need to be changed. These changes are highlighted in this section, with further explanation of why WristMan would benefit from these changes.

One of the largest hindrances to the practicality of our system is the packet radio. As talked about in our evaluation, it suffered from reliability issues that were hard to work around. While we designed a protocol that was relatively robust, it was not efficient for communicating more than a few packets at a time. Also, due to the heavy reliance on Mote type devices (on both the interface DisplayMote and the Personal Server’s Mote), our system is not very platform-independent. Since the DisplayMote is a device that is still being developed, we believe that it would be beneficial to add a Bluetooth radio to the DisplayMote so that a more robust and portable protocol could be implemented. The Bluetooth protocol would then allow us to port the server and playback software to any device that was capable of working with Bluetooth, such as PCs, laptops, cell phones and hand-held computers.

One input mechanism that we experimented with but never incorporated into our final design was the DisplayMote accelerometer. The primary reason we decided not to use the accelerometers was that they were difficult to work with and the software API was premature. Given a longer development cycle, it may have been possible to utilize the accelerometers in our system. This increased functionality would allow us to replace various button operations with accelerometer-based controls. Accelerometers would allow us to support a more rigid input paradigm of using wrist-based movements for menu navigation and using the buttons only for playback control. The drawback of using accelerometers, however, is that most users are already familiar with using button-based devices and may not be accepting of new, untraditional interfaces.

When we were designing the file architecture, we wanted to make sure that we did not impose any limitations on the types of files we could use this system with. As it presently stands, our system specifically looks for MP3 and playlist files only; however, with some minor modifications, we could easily support other types of audio files. If we used a server and playback platform that supported a display, it would even be possible to extend our system to support multimedia applications like video or presentation software. The interface software does not need to be heavily modified, as many of these applications are designed to operate using common controls such as play, pause, forward and back.

The last change that would probably be necessary is rewriting our playback software. While we were able to achieve a high degree of control over Madplay, we still had to make a number of compromises over the behavior of program. For instance, currently it is very difficult to detect when a song in a playlist has finished playing, which means that we cannot easily display the current song of a playlist on the Display Mote. Because the actual MP3 decoding is distributed as a library, we could write a player that operates this library to achieve tighter integrating into our server software. This modification would reduce the number of external program calls and increase the flexibility of the playback software. However, due to time constraints, we were unable to integrate these additions into our system.

6. CONCLUSIONS

Our original goal for the WristMan project was to create a media player system that separated the interface from the actual playback device. As our project requires no interaction with the Personal Server, we believe that we were successful in our goal.

Other media players ignore this separation or provide inadequate solutions. Most commercial devices, such as the Apple iPod or the Creative Nomad line, offer either no interface separation or partial separation such as a wired or infrared remote. The Bemused and MP3Radio cell phone projects provide interface separation similar to WristMan; however, they are more cumbersome and obtrusive than WristMan’s wristwatch form-factor. We feel that WristMan provides interface separation, is non-obtrusive and is a compact solution.

The WristMan system is implemented as two separate modules: the interface and the server software. The interface software, written in nesC for TinyOS, is comprised of three modules: the radio module, the state module and the menu module. These three modules are wired together to provide the user interface on the DisplayMote. The server and player software is comprised of three components: the Madplay music player, the Music Database file system and the Remote Communications Handler program. The Remote Communications Handler processes messages to and from the DisplayMote and also controls the player software. The Music Database builds the file system from the user’s Media library and allows the directory contents to be sent to the DisplayMote.

In our evaluation, we found that we successfully created a practical music playback system. We measured the latency of our communication protocol and found that the latency was small enough to consider the system response “real-time”. The only operation that had a significant delay was sending directory listings to the DisplayMote, which took about two seconds. This delay is common to other programs like Bemused that need to list the songs in the user’s library and should not be considered a hindrance.

Our future work on this project would most likely be on improving the communication protocol. As it presently stands, there are very rare communication errors, but with a Bluetooth radio, these errors would not happen at all. Since the DisplayMote hardware is not finalized, it may be possible to add Bluetooth to the DisplayMote, which would make the WristMan server and playback system portable to any platform that supports Bluetooth.

7. ACKNOWLEDGEMENTS

We would like to thank Waylon Brunette from the University of Washington for helping us with the DisplayMotes and our project in general.

Trevor Pering from Intel Research has been instrumental in helping us receive prototype devices such as the Personal Server, the Slappy Audio board and the DisplayMotes. He also helped us with several of the implementation difficulties we faced.

8. REFERENCES

[1] Waylon Brunette, Alan Rea and Gaetano Borriello. Extended Sensor Mote Interfaces for Ubiquitous Computing. International Conference of Ubiquitous Computing 2003.

[2] Intel Personal Server. HT

[3] R. Want, et al. The Personal Server: Changing the Way We Think about Ubiquitous Computing. Proc. International Conference of Ubiquitous Computing 2002.

[4] Bemused Project.

[5] Conversations with Richard Bannon, MP3Radio project.

[6] HotMP3Gear. Comparison of Players. HT

[7] Apple iPod.

[8] TinyOS.

[9] WinAmp.

[10] . Hard drive-based MP3 Players.

[11] UnderBit. Madplay Program.

9. APPENDIX

To view the complete project web for WristMan, including all documentation and source code, visit

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download