Survey of Sports Video Analysis: Research Issues and ...

[Pages:4]Survey of Sports Video Analysis: Research Issues and Applications

J. R. Wang and N. Parameswaran School of Compute Science and Engineering

The University of New South Wales

{jennyw,paramesh}@cse.unsw.edu.au

Abstract

Video processing has found many applications in sports such as slow motion replay, pattern analysis, statistics collection, video archiving, etc. This paper reviews current research in sports video analysis and discusses the research issues of the field and the potential applications..

Keywords: tracking, sports analysis, semantic analysis, video cataloguing, sports tactics, highlight extraction.

1 Introduction

The development of high-speed digital cameras and video processing has attracted people's attention in sports video analysis. Much work has been done on sports video. Examples include multi-camera recording and replaying [Rui et al. 2000], ball tracking [Pingali et al. 1998], video analysis and summarization [Ekin & Tekalp 2003], highlight extraction [Rui et al. 2000], etc. Applications have been found almost in all sports, eg, tennis [DSI 2003; Pingali et al. 1998; QUESTEC 2003], baseball [Rui et al. 2000; Zhou et al. 2000], soccer [Ekin & Tekalp 2003], American football [Li & Sezan 2002], etc.

This paper reviews the development in the sports video analysis. It discusses several major research issues such as sports tactics summarization, ball/player tracking, game highlight extraction, computer-assisted refereeing, content insertion, etc. It also presents the potential applications of the techniques. Sport is an ever-green field and attracts big spending each year. The requirement and expectation of users have grown significantly since digital media is popular. It is certainly an area worthing investigation. The aim of this paper is to provide an insight and lead the discussion of sports video analysis.

2 Sports Video Analysis

Much research has been done on analysing sports video. Some are directly on video stream analysis and some are on the vicinity areas of video streams. This section discusses some of them.

2.1 Tactics analysis

One of the major aims of sports video analysis is to provide assistance for training. There is a need to summarize the play tactics from video streams. Much research has been done on classifying a play sequence into an existing tactic pattern and recognising unknown patterns. The examples include baseball tactics [Han et al 2002], soccer [Assfalg et al. 2002], American football [Babaguchi et al. 2002], and tennis [Sudhir et al. 1998, Wang & Paramesh 2003]. The techniques include multimodal integration [Han et al 2002], intermodal collaboration [Babaguchi et al. 2002], semi-automatic annotation [Assfalg et al. 2002] and feature classification [Sudhir et al. 1998, Wang & Paramesh 2003]. The complexity of the games includes structured games such as tennis, cricket, and unstructured sports such as soccer, American football, baseball and basketball.

We present tennis tactics as an example of tactics analysis [Wang & Paramesh 2003]. Tennis points are made of a sequence of shots. When a player repeats a particular sequence of shots, the game begins to take on the form of a series of patterns of play. US Tennis Association has recommended 58 winning patterns in single matches for training [USTA 1996]. These patterns are based on several strategic principles that have been well tested over time. Most professional players follow these patterns and games can be classified into one of the patterns.

Fifty eight patterns are also classified into five classes:

? serve and return (18 patterns); ? groundstroke (8 patterns); ? midcourt (8 patterns); ? net play (8 patterns); ? defensive play (16 patterns).

B B Receiver

T

T

3

1

Copyright ? 2004, Australian Computer Society, Inc. This paper appeared at the Pan-Sydney Area Workshop on Visual Information Processing (VIP2003), Sydney. Conferences in Research and Practice in Information Technology, Vol. 36. M. Piccardi, T. Hintz, X. He, M. L. Huang, D. D. Feng, J. Jin, Eds. Reproduction for academic, not-for profit purposes permitted provided this text is included.

2

A A Server

Figure 1 Serve wide to open the court pattern Legend: A, B = player; = path of player;

T = target; 1 = path of ball.

Figure 1 shows one of the serve and return patterns on deuce court, where player A serves the ball to the far end of right court of opponent and force player B to move to the right, and then returns the ball to the far left to win the point. The list of patterns can been found in Appendix. The technical details of each pattern can be seen in USTA [1996].

The idea of tennis tactics analysis is to classify a play sequence into one of these tactics or identify any unknown tactics from the movement of players and the trajectories of the ball.

2.2 Tracking

Tracking is one of most frequently used techniques in sports analysis. It has been used in tracking balls, players, referees, etc. It plays an important role in other analysis processes. Tracking techniques include colour clustering [Xu et al. 2003], colour mean-shift [Duan et al. 2003], feature extraction [Jin 1994; Jin et al. 2000; D'Orazio 2002], trajectory-based optimization [Yu et al. 2003], and multi-camera estimation [Hawke-Eye 2003; Pingali et al. 1998; Pingali et al. 2000]. Tracking objects include tennis balls, cricket balls, soccer balls, players, etc.

Multi-camera estimation has attracted much attention. The principle of the method can be illustrated in Figure 2.

the view of at least three cameras. Secondly, the accuracy is still not enough. Thirdly, the calibration is a difficult task and the accuracy of calibration directly affects the accuracy of detection. There are also other problems such as the speed of computation, the robustness of communication, replacement in the event of sudden fault, etc.

2.3 Highlight extraction

Highlight extraction is originally from the broadcast requirement. The idea is to provide audience with quick highlights of the game without going through a lengthy video stream. Highlight events include the score event in soccer and basketball games, the aces event in tennis games, touch down in rugby and American football games, and the catch event in cricket and baseball games. The work in this area includes Rui et al [2000], Nepal et al. [2001], and Yow et al. [1995]. It is a difficult task. Many successful methods are relying on audio analysis of the video stream where audience cheering provides the most convincing cue.

2.4 Content insertion

Content insertion is one of the applications of sports video analysis. Most sports are closely linked with commercial advertisement. There is a need of customising the video for local audience, which includes replacing some commercial banners in the field, inserting banners, opening a sliding message window, etc. The important issue in content insertion is the content should be placed at the right place and the right time.

The technique issues are also closely related to feature extraction and tracking. Usual steps include dominant region detection, orientation detection, tracking, content insertion, colour balancing, synchronisation, etc.

A typical example can be found in Wan et al. [2003b].

Figure 2 Ball tracking using multi-camera

Six fixed cameras are placed around the playing field at key vantage points - two at each end at a 30 degree angular displacement, and two side cameras, as shown in Figure 2 [Hawk-Eye 2003]. These synchronised cameras track the ball's entire trajectory from the moment it leaves the bowler's hand until it stops. The six cameras are gen-locked into two sets of three cameras. The resulting images are processed into a 3D image by the Hawk-Eye system which then calculates - in a split second - where the ball pitched, the extent of its lateral movement in the air and off the wicket, its velocity and bounce, and - if applicable - exactly where it contacted the batsman's pad.

Although being successfully demonstrated on TV, it has some problems. Firstly, the coverage area depends on the view angle of cameras. The visible area has to be within

2.5 Computer-assisted referee

An automatic line-call system using a computer has been attempted [QUESTEC 2003, Hawk-Eye 2003]. Hawk-eye system has been used in Wimbledon 2003 to produce a computer-generated replay which can help the commentary team to analyse the play in eight main areas [BBC 2003]. There is also an attempt to automatic detect off-side situation in soccer [Lam et al. 2003].

In line-call decision, accuracy, speed and robustness are the most important issues. The problem is compound that in most case, there is no semi-automatic solution. The decision is made either by human or completely by a machine. Line-call people are specialized professionals. There is a conflict of interests to combine machines with human judges. In the situation of off-side detection, there are other issues to be resolved such as which players are active and which are in passive mode. It is important for the final decision.

2.6 Landmark detection

Landmarks in sports video analysis include goal mouth, oval, side lines, corners, etc. They are important features for decision making. They are also important for determining other features. For example, Yu et al. [2003] use the size of goal mouth to determine the size of soccer ball in their ball tracking process. Detecting landmarks usually starts with the most significant features. For example, in soccer, we may start with goal mouth which has two vertical poles and a bar linking two ends. It would be much easier to detect than corners. However, after successfully detecting goal mouth, corners and side lines can be detected with reduced complexity. Wan et al. [2003a, 2003b] present some results on detecting goal mouth in soccer video. Yu et al. [2003] illustrate their work on detecting mid-oval.

3 Conclusion

We review the development of sports video analysis. The techniques developed will have many applications in video archive and retrieval, broadcasting, training and entertainment. For example, Hawke-Eye has been used in 2003 Wimbledon. Lucent ball tracking system has been used in live broadcasts by more than 20 television networks broadcasting in more than 70 countries, in addition to webcasts from an official tennis website (). We can envisage even wide application once the indexing and retrieving problem being resolved.

4 Acknowledgement

This project is supported by Australia Research Council Linkage Grant (LP0347156).

5 References

Assfalg, J; Bertini, M; Colombo, C & Del Bimbo, A (2002). Semantic Annotation of Sports Videos, IEEE MultiMedia 9(2):52-60.

Babaguchi, N; Kawai, Y & Kitahashi, T (2002). Event Based Video Indexing by Intermodal Collaboration, IEEE Transactions on Multimedia 4(1):68-75 (also available: citeseer.nj.367967.html).

BBC (2003). releases/stories/2003/06_june/10/hawk_eye.shtml, 18/06/2003.

D'Orazio, T; Ancona, N; Cicirelli, G & Nitti, M (2002). A ball detection algorithm for real soccer image sequences, Proc. ICPR, Canada.

DSI (2003). advantages-stories1.html#dsi.

Duan, L Y; Xu, M; Chua, T S; Tian, Q & Xu, C S (2003). A mid-level representation framework for semantic sports video analysis, Proc. of ACM MM'03, Berkeley, pp.33-44.

Ekin, A & Tekalp, A. M. (2003). Automatic soccer video analysis and summariza-tion, IEEE Trans. On Image Processing 12(7):796-807 (also available . ece.rochester.edu/~ekin/papers/CPapers/spie2003.pdf).

Han, M; Hua, W; Xu, W & Gong, Y H (2002). An integrated baseball digest system using maximum entropy method, Proc. of ACMMM 2002, pp.347-350.

Hawk-Eye (2003). 2977068.stm, 18/06/2003.

Jin, J S (1994). Computational simulation of depth perception in the human visual system. Proc. The 16th Annual Conference of the Cognitive Science Society. Georgia, pp.451-456.

Jin, J S; Zhu, Z. & Xu, G Y (2000). A stable vision system for moving vehicles. IEEE Trans. on Intelligent Transportation Systems, 1(1):32-39.

Lam, M; Chan, M; Leung, J; Wong, R, Hang, C C & Jin, J S (2003). Computer-assisted off-side detection in soccer matches, Technical Report, School of Information Technologies, University of Sydney.

Li, B. & Sezan, M. I. (2002). Event detection and summarization in American football broadcast video, Proc. SPIE conf. on Storage and Retrieval for Media Databases, vol.4676, pp.202-213.

Nepal, S; Srinivasan, U & Reynolds, G (2001). Automatic detection of 'Goal' segments in basketball videos, Proc. ACM MM'01, Ottawa, pp.261-269.

Pingali, G. S.; Jean, Y. & Carlbom, I. (1998). Real time tracking for enhanced tennis broadcasts, Proc. IEEE Comp. Vision and Patt. Rec. (CVPR), pp. 260-265.

Pingali, G.; Opalach, A. & Jean, Y. (2000). Ball tracking and virtual replays for innovative tennis broadcasts, Proc. of ICPR'00, Barcelona, pp.4152-4146.

QUESTEC

(2003).



q2001/news/1999/030599.htm.

Quinlan, J. R. (1993). C4.5: Programs for Machine Learning, San Mateo, CA: Morgan Kaufmann.

Rui, Y.; Gupta, A. & Acero, A. (2000). Automatically extracting highlights for TV Baseball programs, Proc. of ACM Multimedia'00, pp.105-115.

Sudhir, G; Lee, J C M & Jain, A K (1998). Automatic classification of tennis video for high-level contentbased retrieval. Proc. of IEEE Int. Workshop on Content-based Access of Image and Video Database, pp.81-90.

USTA (1996). Tennis Tactics ? Winning Patterns of Play, Champaign: Human Kinetics.

Wan, K; Yan, X; Yu, X & Xu, C S (2003a). Real-time goal-mouth detection in MPEG soccer video, Proc. of ACM MM'03, Berkeley, pp.311-314.

Wan, K; Yan, X; Yu, X & Xu, C S (2003b). Robust goalmouth detection for virtual content insertion, Proc. of ACM MM'03, Berkeley, pp.468-469.

Wang, J R & Paramesh, N (2003). A scheme for archiving and browsing tennis video clips, Proc. of IEEE Pacific-Rim Conf. on Multimedia (PCM'03), Singapore.

Xu, R Y D; Allen, J & Jin, J S (2003). Robust real-time tracking of non-rigid objects, Conferences in Research and Practice in Information Technology, VIP'03, Sydney.

Yow, D; Yeo, B L; Yeung, M & Liu, B (1995). Analysis

and presentation of soccer highlights from digital

video, Proc. ACCV'95.

Yu, X; Xu, C S; Leong, H W; Tian, Q; Tang, Q & Wan,

K W (2003). Trajectory-based ball detection and

tracking with applications to semantic analysis of

broadcast soccer video, Proc. of ACM MM'03,

Berkeley, pp.11-20.

Zhou, W.; Vellaikal, A. & Kuo, C. C. J. (2000). Rule-

based video classification system for basketball video

indexing, ACM Multimedia Workshops, pp.213-216.

6 Appendix: Tennis Tactics Patterns

Fifty eight patterns were recommended by United States

Tennis Association:

? serve and return (18 patterns): o serve wide to open the court (deuce court) o serve wide to open the court (ad court) o serve to the "T" to reduce the angles (deuce court) o serve to the "T" to reduce the angles (ad court) o serve at the body to jam the receiver o return deep crosscourt (deuce court against a wide serve) o return deep crosscourt (ad court against a wide serve) o return deep down the line (deuce court against an extreme wide serve) o return deep down the line (ad court against an extreme wide serve) o return deep down the middle (deuce court against a serve to the "T") o return deep down the middle (ad court against a serve to the "T") o return deep down the line (deuce court against a serve to the "T") o return deep down the line (ad court against a serve to the "T") o hit a forcing shot down the line (against a short, weak serve) o hit a forcing shot crosscourt (against a short, weak serve) o chip or drive down the line and come to the net (against a short, weak serve) o return low at the server's feet (against a serve-andvolleyer) o return down the line (against a serve-and-volleyer)

? groundstroke (8 patterns): o attack a short ball down the line (rally crosscourt) o attach a short ball crosscourt (rally crosscourt) o hit a severe angle (rally crosscourt to get short, wide ball) o drive inside-out through the court (from a ball down the middle) o drive inside-out off the court (from a ball in left half of the court) o hit looping drives to opponent's backhand (when driven deep) o attach a shrt ball (exchange sliced backhands) o hig high and deep down the middle (against deep shots in the middle)

? midcourt (8 patterns):

o drive hard and flat down the line (if ball is above

the net) o drive crosscourt for a winner (if ball is above the

net) o slice down the line (if ball is below the net) o drop shot down the line (if ball is below the net) o approach down the middle (from a ball in the

middle) o use an inside-out forehand approach (from a ball in

the middle) o use a looping topspin approach (from a deep, high-

bouncing shot) o move in and hit an approach volley (from a looped

shot)

? net play (8 patterns): o volley to opponent's weakness (if ball is above the

net) o volley deep down the line (if ball is below the net) o volley deep crosscourt (after your approach, if ball

is above the net) o serve wide and volley to the open court o serve to the "T" and volley behind opponent or to a

weakness o angle your overhead away (if lob is short) o aim your overhead crosscourt (if lob is deep) o let ball bounce and aim crosscourt (on high lobs) ? defensive play: 16 patterns. o Pass down the line (against a deep crosscourt

approach) o Pass crosscourt (against a deep crosscourt

approach) o pass crosscourt (against a moderate down-the-line

approach) o use a two-shot pass (against a moderate down-the-

line approach) o use a two-shot pass (against an approach up the

middle) o use a two-shot pass inside out (against an approach

up the middle) o overpower your opponent (against a weak approach

up the middle) o use two shots to pass, first crosscourt, then down

the line (against a deep sliced approach to

backhand) o use two shots to pass, first down the line, then

down the opposite line (against a deep sliced

approach to backhand) o use two shots to pass (against a deep sliced

approach to forehand) o use two shots to pass, first down the line, then

down the opposite line (against a deep sliced

approach to forehand) o use a two-shot pass (against a short, sliced

approach) o drive at opponent's right hip (against a short, weak

volley) o drive and then lob (against an approach to your

backhand) o drive and then lob crosscourt (against an approach

to your backhand) o hit a high defensive lob (against a deep approach

shot)

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download