R



R.A.T.S.

Real-Time Action Tracking System

Brett Newlin and Ryan Frazier

Computer Science and Engineering

University of Washington, Seattle, WA

{newlinbr, okard}@cs.washington.edu

ABSTRACT

Judges in extreme sport events currently rely on a quick eye and fast interpretation to determine the point score of participants. Unfortunately, judges do make mistakes. Without raw data to support a performance, judges may be prone to inaccuracies, or worse, they may let their personal biases figure into the equation. This paper describes the design and application of RATS, a real-time action tracking system that aims to provide this lacking data. Our system incorporates a 3-axis accelerometer, gyroscope, Atmega16L microcontroller, and a Mica2Dot mote mounted on a small PCB board. The board is designed to be placed right on extreme sport equipment (skateboard, snowboard, etc.) and should be small and light enough so to not inhibit a participant’s performance. It gathers acceleration and rotational data and transmits the data over radio in real-time to a second mote at a nearby base station. We’ve designed and built this unique ubiquitous computing system to blend data collection into the hostile environment of extreme sports. Resulting data can be manipulated to be displayed both numerically and graphically. With this data to analyze, extreme sport scores can be standardized and personal bias eliminated. In addition, athletes can improve their skills from the detailed feedback they receive.

1. INTRODUCTION

Bucky Lasek, a well-adorned X-Games skateboarder, approaches the edge of the half pipe. He inhales deeply to steady himself. He knows that in order to have a chance at beating Tony Hawk, who is currently in the lead of the summer X-Games Men’s half pipe skateboarding finals; he will have to have a breath-taking, near flawless run. As a judge for this competition, you feel your palms begin to sweat as the realization hits you that you are the deciding factor as to which athlete will be crowned the next X-Games champion. Near perfect analysis is needed for every trick performed so that you can fairly score the run. Thoughts of doubt creep into your mind: “What if I misinterpret a trick?” “What if I miss something?” “Was that a full 720?” “What if I blink?” Luckily you’ve got RATS. The real-time action tracking system that enables you to see acceleration and rotational data of a trick as is occurs.

|[pic] |

|Figure 1: Skateboarding is one example of an extreme sport |

Extreme sports are some of the newest sports that depend on the rating of judges to determine scores of a performance. While much has been done to provide fair criteria for point scoring, such as judging on style, form, and consistency, there still lacks hard, statistical data of the physical dynamics of a performance run. The human eye cannot see exactly how much air a skateboarder gets on his Max Vert event, or the precise number of degrees a snowboarder spins while he’s in mid-air above the half pipe. This lack of data makes it difficult to establish a solid baseline for standard scoring. In some cases, popularity or personal bias may also be inadvertent elements added to the judging process. RATS, however, sets out rectify these shortcomings. With the use of an accelerometer and gyroscope, 3-axis acceleration and rotational data along the plane of the board are collected during an athlete’s run. The data is then sent to a base station in real-time and is made available in visual and numerical form. By gathering and presenting this data, RATS attempts to aid and streamline the judging process of extreme sporting events, as well as provide valuable feedback for the athletes themselves.

The trick with implementing a system such as RATS is being able to gather this data without affecting the athlete in any way. Characteristics of an event should not, in any way, be compromised for the sake of data collection. The system should be small, lightweight, and literally unnoticeable to the participant while still being accurate and strong enough to withstand repeated impacts and grinds. Because there exist both summer and winter X-Games competitions and they are typically held outdoors, it must also be able to withstand different forms of weather. Data transmission must also be wireless. X-Games competitors would tend to dislike the fact that they nearly performed a mind-bending trick, just to be tripped up at the last second by their extension cord. A desired goal would also be to have these wireless data transmissions be performed in real-time to allow judges to see details about tricks as they occur. This would cut down the time needed to review statistics about a performance, which would keep the fans more engaged and provide the participants with quick results. Power consumption must also be kept to a minimum to ensure power doesn’t run out in the middle of an event. It would also be a benefit to avoid mounting a large, cumbersome power source to the extreme sport equipment.

The rest of this paper is as follows: Section 2 describes other work related to the subject of action tracking. Section 3 includes the specifics of the approach taken to create our action tracking system. Section 4 outlines the implementation issues we encountered. Evaluation information is found next in section 5. Section 6 reveals our conclusions about our design as well as possible future work to expand it. Sections 7, 8, and 9 contain acknowledgements, references, and appendices, respectively.

2. RELATED WORK

Sensor-based tracking in modern sports seems all but unheard of. You rarely hear of sensors being put on athletes or their equipment during competition. Despite this, there has been some recognized work done pertaining to this field.

1. Killer App

X-games sports aren’t the only extreme human activities that show a necessity for technological help in the judging process. Adjudicating the quality of attack moves in martial arts has also proven to be controversial at times. Ed Chi, of the Palo Alto Research Center and the Stanford University taekwondo team, has developed a ubiquitous computing system for taekwondo called “Killer App” that measures the power and accuracy at which an attack is made to the body. [3]

|[pic] |

|Figure 2: Chi describes taekwondo as an extreme human activity |

Using piezoelectric force sensors, signals are transmitted wirelessly whenever a considerable impact is made to a competitor’s body. These sensors are located at strategic points inside a vest that each competitor wears. When a sensor is triggered, data is sent in real-time to a judges table nearby. In addition, the rules of the sport indicate that at least two judges must verify a hit. Each judge is equipped with a specially made handset they can use to confirm a hit has been made. When at least two confirmations are made and the sensors detect a hit, points are added to the attacking competitor’s score. [3]

Chi had to deal with many of the same issues that we encountered during the designing of RATS. He needed to make his system robust and accurate in a hostile environment without inhibiting human performance. To enable the system to work in an environment with hard impacts, Chi incorporated piezoelectric sensors rather than accelerometers. Piezoelectric sensors create a voltage when squeezed and can be extremely rigid. Accelerometers would have a much lesser chance of surviving multiple hard impacts. He also made the system wireless to allow for freedom of movement of the competitors. Unlike fencing, where competitors are tethered and forced into linear movements, Killer App is wireless, thus allowing a full range of motion that is critical to the sport of taekwondo. Another similarity to RATS is the need for real-time data transmission. By providing instantaneous feedback of a match, judges can quickly verify and support what they perceive during a competition.

Support of an extreme activity with virtually undetectable attached devices is a non-trivial problem. Chi successfully blended this support into the fabric of the sport of taekwondo.

2. RATS Beta

The version of RATS described in this paper is an extension of a previous idea by Matt Nevitt,

Kabir Shahani, Brandon Tengan, and Geoffrey Velasco from the Information School at the University of Washington. They should be given credit for coining the name RATS. For our purposes, we’ll name their version RATS Beta.

Both RATS and RATS Beta are based on the same general premise: data acquisition of extreme sports, such as snowboarding, skateboarding, etc. RATS Beta provides the main basic ideas from which RATS attempts to implement an optimized configuration suitable for extreme sports competitions. The main goals were to track height and rotation of extreme athletes during competition in order to supply highly-accurate quantifiable data and reduce judging disputes. The RATS Beta team researched the criteria for judging extreme sports and found enough need to pursue their project idea. They read literature reviews and conducted interviews on techniques currently used for measurement to get a better understanding of what kind of technology was needed to support extreme sports. [6]

The technologies the Beta team implemented included a break-beam system to determine airtime and a 2-axis accelerometer connected to an iPaq to collect and store acceleration data. The break-beam system was set up at the lip of a skateboarding half pipe. When a skateboarder would accelerate up past the top of the half pipe, the beam would be broken. Upon the rider’s inevitable return due to gravity, the beam would be broken again. The time in between beam breaks can be used to calculate just how much height a rider achieved during airtime. The dual-axis accelerometer would constantly record acceleration information of the rider over the course of the run, and store this information on the connected iPaq. The accelerometer board and iPaq were connected to the skateboard via a large steel cage on the underside of the skateboard. While this effectively protected the parts inside, it did much in the way of inhibiting human performance by changing the weight and dynamics of the skateboard. A simple prototype of the front end display was created as well, which showed the airtime of each jump a skateboarder/snowboarder made on the screen as each one occurred. [6]

|[pic] |

|Figure 3: RATS Beta skateboard mounting design |

The following sections discuss the steps taken to design RATS and improve the solutions to goals that were lacking in RATS Beta.

3. APPROACH

Below is a list of all components that we used in our sensor board to achieve the goals described above:

• Atmel Atmega16L microcontroller with 7.3728 MHz crystal

• Mica2Dot Berkeley Mote

• STMicroelectronics 3-axis, digital accelerometer

• Analog Devices 300 degree per second gyroscope

• TI OPA350 op-amp

• Linear Technology 3.3V and 5V voltage regulator

• Resistors, Capacitors and Inductors

The software needed for our project is listed below:

• Protel

• TinyOS

• AVR Studio and JTAG programmer

1. Data Collection

The two sensors that will be generating data values are the STMicroelectronics 3-axis accelerometer and the Analog Devices 300 degree per second gyroscope. The accelerometer generates values with respect to its X, Y, and Z axes at a rate of 280Hz. The data value of each respective axis is 12 bits long. Therefore, the complete data load of the accelerometer is 12 bits per axis * 3 axes = 36 bits. Using these values you can determine the speed and direction of an object over time. It can also be used in correlation with takeoff and landing events to determine air time.

The second sensor, the gyroscope, produces rotational data along the plane of the board. Its values are used to detect the number of degrees of rotation that an object feels (with a ceiling of 300 degrees per second). While competitors may be able to spin faster than 300 degrees in a single second, this is the fastest gyroscope we found. It is an analog device that constantly generates 10 bit values which can be read at any rate. This makes the entire data load of the board per reading 46 bits long (36 from accelerometer and 10 from the gyro).

2. Data Handling

Neither the accelerometer nor the gyroscope speaks directly to the Mica2Dot Mote. The reasons for this are: 1) the accelerometer and the mote share no data protocol in which to communicate and 2) the gyroscope is a 5V analog device while the mote is a 3V digital device. To facilitate this communication we employed the Atmel Atmega 16L microcontroller. It gathers data from the accelerometer via the SPI interface they both support. The Atmega also supports ADC, which is the route through which it obtains the values from the gyroscope. Since the gyroscope operates at 5V and the microcontroller operates at 3V, a voltage divider was created. It effectively scales down the gyro output from 0.25V - 4.75V to 0V - 3.3V. After the voltage divider we have placed an OPA350 op-amp to act as a buffer between the Atmega16L and the gyroscope. A buffer is required here because the load capacitance of the gyroscope is only 1000pF.

Once we’ve gathered the complete 46 bit value, corresponding to an entire set of data from both the accelerometer and the gyroscope, the Atmega sends the data through the USART interface to the waiting mote. The Atmega microcontroller is programmed on-chip by using a JTAG programmer and WinAVR software. This allows for easy Atmega reprogramming should the need arise.

3.3 Data Transmission

We needed some way to wirelessly transmit the generated data from the board to a base station for analysis. We opted to use Mica2Dot motes to accomplish this since we had had experience using them before, and they proved to be reliable. The transmitting mote receives the data bits from the Atmega via their common USART interface and stores them in a packet. These packets are then sent over the RF frequency of 433MHz at 19Kbps and received by another mote. This mote is connected to its programming board, which, in turn, is connected to a base station computer via a serial interface. This mote stores the values it receives within each packet to a file, where they can be graphed and analyzed. Each mote is initially programmed with the appropriate TinyOS code, the platform on which the motes run, through the mote programming board serially attached to the base station computer. [9]

In addition to transmitting sensor data, we have also added some connections between the sending mote’s general I/O lines to some general I/O lines on the Atmega for remote programming and expandability. Using these I/O lines you can program the Atmega to reset the accelerometer and run a self test on the gyroscope from a remote computer.

3.4 PCB Design

Since a goal of our project was to attach the sensors and mote to sports equipment to collect data, we needed something more slightly mobile than a large protoboard attached to a heavy power supply. In order to do this we had to custom manufacture our own small, low-power printed circuit board. We accomplished this with the help of Bruce Hemingway who is familiar with the process as well as the Protel software used for schematic design and layout. We sent out our design to be manufactured and ended up getting 25 PCBs custom made for our project.

|[pic] |

|Figure 5: The final design of the RATS PCB |

4. IMPLEMENTATION

Along our path of design we came across several forks on our road to a completed product. At these intersections we had to weigh some pros and cons and determine what course of action we wanted to take. Described below are some of our solutions.

1. Power Struggle

Some components on our board operate at 5V while others operate at 3V. Deciding how to power a board with two different power and ground planes forced us to weigh some different options. The first option was simply to find parts that all operated at a common voltage. There are many very small lithium batteries available that can supply 3V to a load. The motes ran strictly on 3V. The accelerometer and the Atmega could also run at 3V. However, we failed to find a worthy gyroscope that could run at anything other than 5V. A second option was to have a different power source for each voltage. This would require two batteries, however, and would add an undesirable bulk to our design. We settled on a third option: using a 9V battery and voltage regulators. The Linear Technology 3.3V and 5V voltage regulators are used to convert the 9V power source into a 3.3V and 5V voltage source which will supply our the rest of our components. While smaller, lithium batteries may exist, the convenience of a simple, easily-obtainable 9V battery outweighed the issue of bulk in our case.

2. Wasting the Clock

One easily-fixable hiccup we experienced was getting our Atmega 16L microcontroller to operate using the 7.3728MHz crystal we provided for it. No matter what we seemed to try, it would only run at an undesirable 1 MHz. If you attempt to reconstruct this project, and in an effort to save you some time, we advise that you realize the Atmega16L microcontroller has an internal 1MHz clock, and that it is factory defaulted to run on this clock. Simply deactivate this setting.

4.3 Who Wants the Gyro ADC?

Since the gyroscope is an analog device, it obviously needs to be converted to digital signals in order to talk to a digital device. The question we ran into was whether to do the ADC directly on the mote or to go through the Atmega. If the ADC was done on the mote, the gyroscope data could be gathered independent of the Atmega. This would require that the mote be able to handle the analog-to-digital conversion while still transmitting data in real time. This seemed to be pushing the limits of the capabilities of the mote. Plus, there seemed to be no real gain of having the gyro independent of the Atmega since the accelerometer still needed it. We therefore decided to allow the Atmega to perform the ADC. This way we can keep the data from both sensors together and the mote can be dedicated to data transmission only.

4.4 It’s Got to Be Real

Real-time data transmission is a must for RATS. Being able to see data about a performance as it happens was one of the main goals of the project. We had to make some small sacrifices in order to ensure this, however. Taking the full resolution of samples from the accelerometer and gyro can overflow the bandwidth of the mote. Therefore, if data is received while the mote is busy, it will be disregarded. This, in effect, reduces the number of samples per second that will be transmitted, resulting in a slightly lesser accuracy. Luckily, this doesn’t dramatically affect the overall resulting data since there are still more than enough points to record the relatively slower action of human movement.

5. Mote to Mote

As stated before, motes have their own specific platform that they run on called TinyOS. The communication stack in TinyOS provides a very verbose header of 18 bits on each packet of data it creates. Many of the header bits are unnecessary for our purposes and effectively create a bottleneck for our data transmission speed. Unfortunately, we do not have enough time left in class to manipulate the TinyOS communication stack to suit our needs. We invite any who attempt to reconstruct this project to experiment in this area to eliminate this bottleneck.

5. EVALUATION

Testing was performed on each individual component of RATS as it was completed. We first tested to see if data collection by the sensors was working properly. We did this by way of seven segment displays that would show a hexadecimal value corresponding to the forces the sensors experienced. Once we were confident that we were, in fact, sending useful data, we then determined if the Atmega could effectively communicate with the on-chip mote. This involved a simple test code that would turn the LED of the mote on when it received an expected value from the Atmega across the UART line. Transmission of data across a radio frequency from the sending to the receiving mote was tested next in a similar way. The on-chip mote transmitted a predefined value to the base station mote and its LED was either illuminated or not based on whether that number was recognized. This predefined value was sent only periodically so we had, in effect, the most glorified blink program we’ve ever been fortunate enough to experience. We were surprised to find just how much satisfaction a simple blinking light could bring us.

After writing all the needed code and successfully soldering a full board together, we crossed our fingers to see how our custom-designed PCBs would perform. There did turn out to be a flaw on the boards. There was a connection that shorted the power and ground planes. This was not a flaw in the design but an oversight in the laying out of the board. It was fixable, luckily, and our system evaluation could continue. At the time this paper was written, we had not yet attached our sensor board to a skateboard and tested it at a skate park. So, in order to simulate the effects our board may feel when connected to extreme sports equipment, we employed the elaborate testing method of “jiggling” the board as well as making spinning and swooping motions. At the expense of some strange looks from other people in the lab, we collected enough data to perform a detailed analysis. We calculated the transmission speed and determined that data is sent in 3.38 packets per second, with each packet containing 4 data readings from each sensor. This equated to 13.53 data readings per second, 94.64 bytes per second, or 757.12 bits per second. This is close to what we were expecting. In an ideal case we could receive close to 70 data readings per second, but the TinyOS communication stack creates quite a bit of overhead. Nevertheless, 13.53 data readings per second seem sufficient to record human movements.

|[pic] |

|Figure 6: This is the result of leveling out the accelerometer’s x-axis after having it hang down. The maximum value corresponds |

|to a +2g acceleration while the largest negative value corresponds to a -2g acceleration. As you can see, the accelerometer |

|experienced a force of +1g (gravity) while it hung down, was then leveled to a point where it experienced virtually 0g, and was |

|then released into the previous hanging position. |

We imported the data we collected into a Microsoft Excel spreadsheet, converted the hexadecimal values to decimal, and plotted the data over time. Figures 6 and 7 show an example of the data collected from the accelerometer and gyroscope, respectively.

|[pic] |

|Figure 7: This is data collected from the gyroscope on our sensor board. This graph is the result of spinning the PCB back and |

|forth in our hand. A value of 255 corresponds to a 300o/s rotation counterclockwise while a value of 0 indicates a 300o/s rotation|

|in the clockwise direction. |

While these evaluation techniques are sufficient for our purposes, there is much more testing that could be done with better software.

In the near future we do plan on attaching our PCB to a skateboard and renting out a skate park with the help of Matt Nevitt, to obtain real-life performance metrics. More evaluation data should be forthcoming.

6. CONCLUSIONS AND FUTURE WORK

Extreme sports are fast-paced and high-impact, creating a hostile environment. Therefore, attempting to seamlessly integrate ubiquitous computing systems into these sports is a non-trivial problem. We feel that we have built a component that will allow the merging of data collection into the extreme sports environment. We incorporated a 3-axis accelerometer, a 300o/s gyroscope, an ATmega16L microcontroller, and Mica2Dot motes onto a small printed circuit board, designed to fit onto extreme sport equipment. It is able to detect and wirelessly transmit data corresponding to the acceleration and rotational forces felt by the equipment during an extreme sport performance. This system is not restricted to extreme sports alone and may also be used to sense acceleration and rotation on any object you wish to attach it. While it is our belief that we have succeeded in our goal of creating a system to gain statistical data of forces felt on an object, the goal of providing a fully-functional judging system for extreme sports judges and athletes was beyond the scope of our project, the details of which are described in the remainder of this section.

One major element RATS did not attempt to implement is a front-end system for displaying the data in real-time in a useful way. Currently, data is displayed in real-time to the base station computer screen, but it appears as a series of 2 byte hexadecimal numbers, virtually useless to any human being. Some software engineering will be required to parse the stream of bytes and import them into a graphing program that will continuously update itself to show the changing forces experienced by the object. As it is, we are able to store a number of readings into a text file and import the values into Microsoft Excel spreadsheet. From this we can generate impressive scatter plots showing the changes in movement the sensors recorded. This is how we currently show the data we collected.

More future work to be done involves engineering mounting systems for the sensor board to the various forms of extreme sport equipment. Every different type of equipment will most likely need a different type of mount. Since our main focus was on skateboarding, we studied the construction of skateboards and theorized that embedding the thin PCB into the body of the skateboard would be the most efficient mount in this case. Unfortunately, we did not have the time or capacity to put our idea into effect. As a compromise, we placed the sensor board on the underside of the skateboard near the metal trucks. This seemed to be the area where the board would have the least chance to be damaged.

We also had some thoughts on what aspects we might do differently the next time we designed an action-tracking system. In regard to wireless data transmission, we pondered the idea of eliminating the use of motes completely. With Bluetooth being widely used in mobile electronics today, it, or any of the 802.11 Wi-Fi technologies, would seem apt candidates for the job of wirelessly transmitting our sensor data. These, however, are just some post-development notions we’ve had. We welcome those interested to further explore these other options.

7. ACKNOWLEDGEMENTS

We’d like to thank the following people, for without their help RATS would not have been possible:

Bruce Hemingway – Bruce, a University of Washington lecturer and manager of the hardware lab, took on the job of designing and laying out the physical aspects of the PCB boards that were manufactured. Without his knowledge of Protel, the software used for the design, we would have had a steep learning curve to overcome. Not to mention, had we tried to do it ourselves, we would have most likely ended up with 25 beautiful but inescapably flawed PCBs.

Jonathan Lester – A grad student in the EE department of the University of Washington, Jonathan provided us with accelerometers as well as useful information on how to operate them. He referred us to companies where we could find dependable parts and fast delivery. We often conferred with Jonathan when we’d encounter a question involving electrical characteristics of components.

Waylon Brunette – As the TA for the Digital Systems Design class in which this project was conceived, Waylon proved to be an invaluable confidant. He provided us with TinyOS code that we used to configure the UART interface as well as packetize data. He also made the JTAG programmer available for us to use. It was in Waylon that we most frequently confided when we ran into frustrating obstacles regarding design issues.

Matt Nevitt – The mastermind. Matt and his team dreamed up the idea of this type of action tracking system and began experimenting with it in RATS Beta. Matt described the purpose of such a system and gave us a defined set of final design goals to meet. He will also continue his work using RATS as his new sensor board.

8. REFERENCES

[1] Analog Devices. ADXRS300EB ±300°/s Gyro Data sheet.

[2] Atmel Corporation. ATmega16L Complete Datasheet.

[3] Chi, E.H., Song, J., Corbin, G. “Killer App of Wearable Computing: Wireless Force Sensing Body Protectors for Martial Arts.”

[4] Crossbow Technology. Mica2Dot Datasheet.

[5] Linear Technology. LTC1761 Micropower Voltage Regulator datasheet.

[6] Nevitt, M. N., Shahani, K. K., Tengan, B. S., Velasco, G. R. “Real-Time Action Tracking System (RATS).”

[7] ST Microelectronics. LIS3L02DS 3-axis Accelerometer Data Sheet.

[8] Texas Instruments. OPA350 Op-amp datasheet.

[9] University of Washington. CSE 466 – Software for Embedded Systems. TinyOS experiments.

9. APPENDICES

All the material you would need to reconstruct our project: source code, schematics, etc. can be found here:

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download