An Integrated Face Tracking and Facial Expression Recognition System

Angappan Geetha, Venkatachalam Ramalingam, Sengottaiyan Palanivel

.

DOI: 10.4236/jilsa.2011.34023                

Abstract

This article proposes a feature extraction method for an integrated face tracking and facial expression recognition in real time video. The method proposed by Viola and Jones [1] is used to detect the face region in the first frame of the video. A rectangular bounding box is fitted over for the face region and the detected face is tracked in the successive frames using the cascaded Support vector machine (SVM) and cascaded Radial basis function neural network (RBFNN). The haar-like features are extracted from the detected face region and they are used to create a cascaded SVM and RBFNN classifiers. Each stage of the SVM classifier and RBFNN classifier rejects the non-face regions and pass the face regions to the next stage in the cascade thereby efficiently tracking the face. The performance of tracking is evaluated using one hour video data. The performance of the cascaded SVM is compared with the cascaded RBFNN. The experiment results show that the proposed cascaded SVM classifier method gives better performance over the RBFNN and also the methods described in the literature using single SVM classifier [2]. While the face is being tracked, features are extracted from the mouth region for expression recognition. The features are modelled using a multi-class SVM. The SVM finds an optimal hyperplane to distinguish different facial expressions with an accuracy of 96.0%.

Keywords

Face Detection, Face Tracking, Feature Extraction, Facial Expression Recognition, Cascaded Support Vector Machine, Cascaded Radial Basis Function Neural Network

Conflicts of Interest

The authors declare no conflicts of interest.

References

[1]	P. Viola and M. Jones, “Rapid Object Detection Using a Boosted Cascade of Simple Features,” Proceedings of IEEE Computer So Ciety Conference on Computer Vision and Pat Tern Recognition, Kauai, 8-14 December 2001, Vol. 1, p. 511.
[2]	L. Carminati, J. Benois-Pineau and C. Jennewein, “Knowledge-Based Super Vised Learning Methods in a Classical Problem of Video Object Tracking,” Proceedings of IEEE International Conference on Image Processing, Atlanta, 8-11 October 2006, pp. 2385-2389.
[3]	H. A. Rowley, S. Baluj and T. Kanade, “Neural Network-Based Face Detection,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 20, No.1, 1998, pp. 23-38. doi:10.1109/34.655647
[4]	R. Feraud, O. J. Bernier, J. Vialle and M. Collobert, “A Fast and Accurate Face Detector Based on Neural Networks,” IEEE Transactions on Pat Tern Analysis and Machine Intelligence, Vol. 23, No. 1, 2001, pp. 42-53. doi:10.1109/34.899945
[5]	C. Papageorgiou, M. Oren and T. Poggio, “A General Famework for Object Detection,” Proceedings of International Conference on Computer Vision, Bombay, 4-7 January 1998, pp. 555-562.
[6]	C. Liu, “A Bayesian Discriminating Features Method for Face Detection,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 25, No. 6, 2003, pp. 725-740. doi:10.1109/TPAMI.2003.1201822
[7]	D. Freedman, “Active Contours for Tracking Distributions,” IEEE Transactions on Image Processing, Vol. 13, No. 4, 2004, pp. 518-526. doi:10.1109/TIP.2003.821445
[8]	H. T. Nguyen and A. W. M. Smeulders, “Fast Occluded Object Tracking by a Robust Appearance Filter,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 26, No. 8, 2004, pp. 1099-1104.
[9]	H.-T. Chen, T.-L. Liu and C.-S. Fuh, “Probabilistic Tracking with Adaptive Feature Selection,” Proceedings of International Conference on Pattern Recognition, Wash- ington DC, Vol. 2, 2004, pp. 736-739. doi:10.1109/TPAMI.2004.45
[10]	A. U. Batur and M. H. Hayes, “Adaptive Active Appearance Models,” IEEE Transactions on Image Processing, Vol. 14, No. 11, 2005, pp. 1707-1721. doi:10.1109/TIP.2005.854473
[11]	P. Corcoran, M. C. Ionita and I. Bacivarov, “Next Generation Face Tracking Technology Using AAM Techniques,” Proceedings of International Symposium on Signals, Systems and Circuits, Vol. 1, 2007, pp. 1-4. doi:10.1109/ISSCS.2007.4292639
[12]	K.-S. Cho, Y.-G. Kim and Y.-B. Lee, “Real-Time Expression Recognition System Using Active Appearance Model and EFM,” Proceedings of International Conference on Computational Intelligence and Security, Guang- zhou, Vol. 1, November 2006, pp. 747-750.
[13]	P. Yang, Q.-S. Liu and D. N. Metaxas, “Boosting Coded Dynamic Features for Facial Action Units and Facial Expression Recognition,” Proceedings of IEEE Conference on Computer Vision and Image Understanding, Minneapolis, 18-23 June 2007.
[14]	S Petar, S. Aleksic and A. K. Katsaggelos, “Automatic Facial Expression Recognition Using facial Animation Parameters and Multi-Stream HMMs,” IEEE Transactions on Information Forensics and Security, Vol. 1, 2006, pp. 3-11. doi:10.1109/TIFS.2005.863510
[15]	L. Ma and K. Khorasani, “Facial Expression Recognition Using Constructive Feed forward Neural Networks,” IEEE Transactions on Systems, Man and Cybernetics, Vol. 34, No. 3, 2004, pp. 1588-1595. doi:10.1109/TSMCB.2004.825930
[16]	N. Friedman, D. Geiger and M. Goldszmidt, “Bayesian Network Classifiers,” Machine Learning, Vol. 29, No. 2, 1997, pp. 131-163. doi:10.1023/A:1007465528199
[17]	N. Sebe, I. Cohen, A. Garg, M. Lew and T. Huang, “Emotion Recognition Using a Cauchy Na?ve Bayes Classifier,” Proceedings of International Conference on Pattern Recognition, Quebec City, Vol. 1, August 2002, pp. 17-20.
[18]	N. Esau, E. Wetzel, L. Kleinjohann and B. Kleinjohann, “Real-Time Facial Expression Recognition Using a Fuzzy Emotion Model,” IEEE Proceedings of Fuzzy Systems Conference, London, 2007.
[19]	A. Geetha, V. Ramalingam, S. Palanivel and B. Palaniappan, “Facial Expression Recognition—A Real Time Approach,” International Journal of Expert Systems with Applications, Vol. 36, No. 1, 2009, pp. 303-308. doi:10.1016/j.eswa.2007.09.002
[20]	I. Kotsia, N. Nikolaidis and I. Pitas, “Facial Expression Recognition in Videos Using a Novel Multi-Class Support Vector Machines Variant,” Proceedings of International Conference on Acoustics, Speech and Signal Processing, Honolulu, 15-20 April 2007, pp. 585-588.
[21]	S. Avidan, “Support Vector Tracking,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 26, No. 8, 2004, pp. 1064-1072.
[22]	J. C. B. Christopher, “A Tutorial on Support Vector Machines for Pattern Recognition,” Data Mining and Knowledge Discovery, Vol. 2, No. 2, 1998, pp. 121-167.
[23]	F. Yang and M. Paindavoine, “Implementation of an RBF Neural Network on Embedded Systems: Real-Time Face Tracking and Identity Verification,” IEEE Transactions on Neural Networks, Vol. 14, No. 5, 2003, pp. 1162-1175. doi:10.1109/TNN.2003.816035
[24]	S. Haykins, “Neural Networks: A Comprehensive Foundation,” Pearson Publication 2001, Asia.
[25]	M. T. Musavi, W. Ahmed, K. H. Chan, K. B. Faris and D. M. Hummels, “On the Training of Radial Basis Function Classifiers,” IEEE Transactions on Neural Networks, Vol. 5, No. 4, 1992, pp. 595-603.
[26]	K. Anderson and P. W. McOwan, “Robust Real-Time Face Tracker for Cluttered Environments,” Computer Vision and Image Understanding, Vol. 95, No. 2, 2004, pp. 184-200. doi:10.1016/j.cviu.2004.01.001