TY - GEN
T1 - Multicamera audio-visual analysis of dance figures
AU - Ofli, F.
AU - Demir, Y.
AU - Erzin, E.
AU - Yemez, Y.
AU - Tekalp, A. M.
PY - 2007
Y1 - 2007
N2 - We present an automated system for multicamera motion capture and audio-visual analysis of dance figures. The multiview video of a dancing actor is acquired using 8 synchronized cameras. The motion capture technique is based on 3D tracking of the markers attached to the person's body in the scene, using stereo color information without need for an explicit 3D model. The resulting set of 3D points is then used to extract the body motion features as 3D displacement vectors whereas MFC coefficients serve as the audio features. In the first stage of multimodal analysis, we perform Hidden Markov Model (HMM) based unsupervised temporal segmentation of the audio and body motion features, separately, to determine the recurrent elementary audio and body motion patterns. Then in the second stage, we investigate the correlation of body motion patterns with audio patterns, that can be used for estimation and synthesis of realistic audio-driven body animation.
AB - We present an automated system for multicamera motion capture and audio-visual analysis of dance figures. The multiview video of a dancing actor is acquired using 8 synchronized cameras. The motion capture technique is based on 3D tracking of the markers attached to the person's body in the scene, using stereo color information without need for an explicit 3D model. The resulting set of 3D points is then used to extract the body motion features as 3D displacement vectors whereas MFC coefficients serve as the audio features. In the first stage of multimodal analysis, we perform Hidden Markov Model (HMM) based unsupervised temporal segmentation of the audio and body motion features, separately, to determine the recurrent elementary audio and body motion patterns. Then in the second stage, we investigate the correlation of body motion patterns with audio patterns, that can be used for estimation and synthesis of realistic audio-driven body animation.
UR - http://www.scopus.com/inward/record.url?scp=46449111503&partnerID=8YFLogxK
U2 - 10.1109/icme.2007.4284997
DO - 10.1109/icme.2007.4284997
M3 - Conference contribution
AN - SCOPUS:46449111503
SN - 1424410177
SN - 9781424410170
T3 - Proceedings of the 2007 IEEE International Conference on Multimedia and Expo, ICME 2007
SP - 1703
EP - 1706
BT - Proceedings of the 2007 IEEE International Conference on Multimedia and Expo, ICME 2007
PB - IEEE Computer Society
T2 - IEEE International Conference onMultimedia and Expo, ICME 2007
Y2 - 2 July 2007 through 5 July 2007
ER -