Recognition of Arabic Sign Language (ArSL) Using Recurrent Neural Networks

Manar Maraqa, Farid Al-Zboun, Mufleh Dhyabat, Raed Abu Zitar

College of Information Technology, Ajloun National University, Ajloun, Jordan;.
College of Information Technology, AL Al-Bayt University, Mafraq, Jordan;.
Department of Management Information Systems, Al-Isra’ Private University, Amman, Jordan;.
School of Engineering and Technology, New York Institute of Technology, Amman, Jordan..

DOI: 10.4236/jilsa.2012.41004                

Abstract

The objective of this research is to introduce the use of different types of neural networks in human hand gesture recognition for static images as well as for dynamic gestures. This work focuses on the ability of neural networks to assist in Arabic Sign Language (ArSL) hand gesture recognition. We have presented the use of feedforward neural networks and recurrent neural networks along with its different architectures; partially and fully recurrent networks. Then we have tested our proposed system; the results of the experiment have showed that the suggested system with the fully recurrent architecture has had a performance with an accuracy rate 95% for static gesture recognition.

Keywords

Arabic Sign Language; Feedforward Neural Networks; Recurrent Neural Networks; Gesture Recognition

Conflicts of Interest

The authors declare no conflicts of interest.

References

[1]	J. Eisenstein, S. Ghandeharizadeh, L. Golubchik, C. Shahabi, D. Yan and R. Zimmermann, “Device Independence and Extensibility in Gesture Recognition,” Proceedings of IEEE Virtual Reality, Los Angeles, 22-26 March 2003, pp. 207-214. doi:10.1109/VR.2003.1191141
[2]	K. Assaleh and M. Al-Rousan, “A New Method for Arabic Sign Language Recognition,” EURASIP Journal on Applied Signal Processing, Hindawi Publishing Corporation, New York, 2005, pp. 2136-2145.
[3]	K. Abe, H. Saito and S. Ozawa, “Virtual 3-D Interface System via Hand Motion Recognition from Two Cameras,” IEEE Transactions on Systems, Man and Cybernetics, Vol. 32, No. 4, 2002, pp. 536-540. doi:10.1109/TSMCA.2002.804821
[4]	J.-S. Kim, W. Jang and Z. Bien, “A Dynamic Gesture Recognition System for the Korean Sign Language (KSL),” SMC-B, Vol. 26, No. 2, 1996, pp. 354-359.
[5]	G. Fang, W. Gao and D. Zhao, “Large Vocabulary Sign Language Recognition Based on Fuzzy Decision Trees,” SMC-A, Vol. 34, No. 3, 2004, pp. 305-314.
[6]	G. Fang, W. Gao and D. Zhao, “Large-Vocabulary Continuous Sign Language Recognition Based on TransitionMovement Models,” SMC-A, Vol. 37, No. 1, 2007, pp. 19.
[7]	T. Shanableh, K. Assaleh and M. Al-Rousan, “SpatioTemporal Feature-Extraction Techniques for Isolated Ges ture Recognition in Arabic Sign Language,” SMC-B, Vol. 37, No. 3, 2007, pp. 641-650.
[8]	I. Infantino, R. Rizzo and S. Gaglio, “A Framework for Sign Language Sentence Recognition by Commonsense Context,” SMC-C, Vol. 37, No. 5, 2007, pp. 1034-1039.
[9]	Y. Yuan, Y. Liu and K. Barner, “Tactile Gesture Recognition for People with Disabilities,” IEEE International Conference on Acoustics, Speech, and Signal Processing, Vol. 5, Kyoto, 18-23 March 2005, pp. 461-464.
[10]	R. Salomon and J. Weissmann, “Evolutionary Tuning of Neural Networks for Gesture Recognition, Evolutionary Computation,” Proceedings of the 2000 Congress on Evolutionary Computation, Vol. 2, La Jolla, 16-19 July 2000, pp. 1528-1534.
[11]	F. Florez, J. Garcia and A. Hernandez, “Hand Gesture Recognition Following the Dynamics of a Topology-Preserving Network,” Proceedings of IEEE 5th International Conference on Automatic Face and Gesture Recognition, Washington DC, 19-20 May 2002, pp. 303-308.
[12]	M. Mohandes and S. Buraiky, “Automation of the Arabic Sign Language Recognition using the PowerGlove,” AIML Journal, Vol. 7, No. 1, 2007, pp. 41-46.
[13]	K. Murakami and H. Taguchi, “Gesture Recognition Using Recurrent Neural Networks,” Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Gaithersburg, 15-17 March 1991, pp. 237-242.
[14]	O. Al-Jarrah, A. Shatnawi and A. H. Halawani, “Recognition of Gestures in Arabic Sign Language using Neural Networks,” Proceeding of Artificial Intelligence and Soft Computing, Palma De Mallorca, 28-30 August 2006.
[15]	W. K. Pratt, “Digital Image Processing, PIKS Scientific Inside,” 4th Edition, A John Wiley & Sons, Inc., Hoboken, 2007.
[16]	A. Ford and A. Roberts, “Color Space Conversions,” 1998. http://www.poynton.com/PDFs/coloureq.pdf
[17]	A. Balaguer, “Analytical Methods for the Study of Color in Digital Images,” 2006. http://www.tesisenxarxa.net/TESIS_UIB/AVAILABLE/TDX-10271064116//tapb1de1.pdf
[18]	S. Wesolkowski, “Color Image Edge Detection and Segmentation: A Comparison of the Vector Angle and the Euclidean Distance Color Similarity Measures,” 1999. http://etd.uwaterloo.ca/etd/swesolko1999.pdf
[19]	R. Gonzalez, R. Woods and S. Eddins, “Digital Image Processing Using MATLAB,” Pearson Prentice Hall, Upper Saddle River, 2004.
[20]	C. Y. Wen and C. M. Chou, “Color Image Models and Its Applications to Document Examination,” Forensic Science Journal, Vol. 3, No. 1, 2004, pp. 23-32.
[21]	J. Weissmann and R. Salomon, “Gesture Recognition for Virtual Reality Applications Using Data Gloves and Neural Networks,” Proceedings of IEEE International Joint Conference on Neural Networks, Washington DC, 10-16 July 1999.
[22]	S. Hykin, “Neural Networks: A Comprehensive Foundation,” 2nd Edition, Prentice Hall, Upper Saddle River, 1998.
[23]	L. C. Jain, “Recurrent Neural Networks,” CRC Press, Boca Raton, 2001.
[24]	J. L. Elman, “Finding Structure in Time,” Cognitive Science, Vol. 14, No. 2, 1990, pp. 179-211. doi:10.1207/s15516709cog1402_1
[25]	S. Ismail and A. bin Ahmad, “Recurrent Neural Network with Backpropagation through Time Algorithm for Arabic Recognition,” Proceeding 18th European Simulation Multiconference, Magdeburg, 13-16 June 2004.
[26]	N. Sinha, M. Gupta and D. Rao, “Dynamic Neural Networks: An Overview,” Proceedings of IEEE International Conference on Industrial Technology, Vol. 1, Goa, 19-22 January 2000, pp. 491-496.
[27]	S. Zappacosta, G. Baldassarre and S. Nolfi, “Elman Neural Networks and Time Integration for Object Recognition,” Technical Report, Istituto di Scienze e Tecnologie della Cognizione, Roma, 2006.
[28]	T. Chen and V. So, “A Comparative Study of Recurrent Neural Network Architectures on Learning Temporal Sequences,” IEEE International Conference on Digital Object Identifier, Vol. 4, 1996, pp. 1945-1950.
[29]	P. Vamplew, “Recognition of Sign Language Using Neural Networks,” PhD Thesis, University of Tasmania, Newnham, 1996.
[30]	P. Werbos, “Backpropagation through Time: What It Does and How To Do It,” Proceedings of the IEEE, Vol. 78, No. 10, 1990, pp. 1550-1560. doi:10.1109/5.58337
[31]	S. Ismail and A. bin Ahmad, “Recurrent Neural Network with Backpropagation through Time Algorithm for Arabic Recognition,” Proceedings 18th European Simulation Multiconference, Magdeburg, 13-16 June 2004.
[32]	M. Maraqa and R. Abu-Zaiter, “Recognition of Arabic Sign Language (ArSL) Using Recurrent Neural Networks,” Proceeding of the First International Conference on Applications of Digital Information and Web Technologies, 2008, pp. 478-481.