FPGA-Based Traffic Sign Recognition for Advanced Driver Assistance Systems

Sheldon Waite, Erdal Oruklu

Department of Electrical and Computer Engineering, Illinois Institute of Technology, Chicago, USA.

DOI: 10.4236/jtts.2013.31001                

Abstract

This paper presents the implementation of an embedded automotive system that detects and recognizes traffic signs within a video stream. In addition, it discusses the recent advances in driver assistance technologies and highlights the safety motivations for smart in-car embedded systems. An algorithm is presented that processes RGB image data, extracts relevant pixels, filters the image, labels prospective traffic signs and evaluates them against template traffic sign images. A reconfigurable hardware system is described which uses the Virtex-5 Xilinx FPGA and hardware/software co-design tools in order to create an embedded processor and the necessary hardware IP peripherals. The implementation is shown to have robust performance results, both in terms of timing and accuracy.

Keywords

Traffic Sign Recognition; Advanced Driver Assistance Systems; Field Programmable Gate Array (FPGA)

Conflicts of Interest

The authors declare no conflicts of interest.

References

[1]	J. Urry, “The ‘System’ of Automobility,” Theory, Culture & Society, Vol. 21, No. 4-5, 2004, pp. 25-39. doi:10.1177/0263276404046059
[2]	E. Eckermann, “World History of the Automobile,” Society of Automotive Engineers, Warrendale, 2001.doi:10.4271/R-272
[3]	“Transportation: Motor Vehicle Accidents and Fatalities,” The 2012 Statistical Abstract, US Census Bureau, Suitland, 2011.
[4]	“Population,” The 2012 Statistical Abstract, US Census Bureau, Suitland, 2011.
[5]	“EuroFOT Study Demonstrates How Driver Assistance Systems Can Increase Safety and Fuel Efficiency,” EuroFOT,2012.http://www.eurofot- ip.eu/en/news_and_events/eurofot_study_ demonstrates_how_driver_assistance_systems_can_increase_safety_and_fuel_efficiency_acr.htm
[6]	M. Meuter, C. Nunn, S. M. Gormer, S. Muller-Schneiders and A. A. Kummert, “A Decision Fusion and Reasoning Module for a Traffic Sign Recognition System,” IEEE Transactions on Intelligent Transportation Systems, Vol. 12, No. 4, 2011, pp. 1126-1134. doi:10.1109/TITS.2011.2157497
[7]	C. Lai, “An Efficient Real-Time Traffic Sign Recognition System for Intelligent Vehicles with Smart Phones,” Proceedings of 2010 International Conference on Technologies and Applications of Artificial Intelligence,Hsinchu, 18-20 November 2010, pp. 195-202.doi:10.1109/TAAI.2010.41
[8]	V. Andrey, “Automatic Detection and Recognition of Traffic Signs Using Geometric Structure Analysis," Proceedings of SICE-ICASE International Joint Conference, Busan, 18-21 October 2006, pp. 1451-1456. doi:10.1109/SICE.2006.315823
[9]	D. Soendoro and I. Supriana, “Traffic Sign Recognition with Color-Based Method Shape-Arc Estimation and SVM,”Proceedings of 2011 International Conference on Electrical Engineering and Informatics, Bandung, 17-19 July 2011, pp. 1-6. doi:10.1109/ICEEI.2011.6021584
[10]	Y. Liu, H. Yu, H. Yuan and H. Zhao, “Real-Time Speed Limit Sign Detection and Recognition from Image Sequences,” Proceedings of 2010 International Conference on Artificial Intelligence and Computational Intelligence, Shenyang, 23-24 October 2010, pp. 262-267.doi:10.1109/AICI.2010.62
[11]	R. Kastner, T. Michalke, T. Burbach, J. Fritsch and C. Goerick, “Attention-based Traffic Sign Recognition with an Array of Weak Classifiers,” Proceedings of 2010 IEEE Intelligent Vehicles Symposium, San Diego, 21-24 June 2010, pp. 333-339. doi:10.1109/IVS.2010.5548143
[12]	M. A. Souki, L. Boussaid and M. Abid, “An Embedded System for Real-Time Traffic Sign Recognizing,” 3rd International Design and Test Workshop, IDT 2008, Monastir, 20-22 December 2008, pp. 273-276.
[13]	M. Muller, A. Braun, J. Gerlach, W. Rosenstiel, D. Nienhuser, J. M. Zollner and O. Bringmann, “Design of an Automotive Traffic Sign Recognition System Targeting a Multi-Core SoC Implementation,” Design,Automation & Test in Europe Conference and Exhibition (DATE), Dresden, 8-12 March 2010, pp. 532-537.
[14]	H. Irmak, “Real Time Traffic Sign Recognition System on FPGAs,” M.S. Thesis, Middle East Technical University, Ankara, 2010.
[15]	Frost & Sullivan, “Development of Low-cost DAS Technologies to Help Reach European Union’s Target to Increase Road and Driver Safety,” 2011. http://www.frost.com/prod/servlet/press-release.pag?docid=251082001
[16]	Continental AG, “MFC 2 Multi-Function Camera,” Datasheet, 2009. http://www.conti-online.com/generator/www/de/en/continental/industrial_sensors/themes/mfc_2/mfc_2_en.html
[17]	Continental AG, “Traffic Sign Recognition,” 2012. http://www.conti-online.com/generator/www/de/en/continental/automotive/general/chassis/safety/hidden/verkehrszei chenerkennung_en.html
[18]	J. Markoff, “Smarter than You Think: Google Car Drives Itself,” The New York Times, October 2010. http://www.nytimes.com/2010/10/10/science/10google.html?_r=1
[19]	A. C. Clark and E. N. Wiebe, “Color Principles—Hue, Saturation and Value,” North Carolina State University, Raleigh, 2002. http://www.ncsu.edu/scivis/lessons/colormodels/color_models2.html
[20]	D. M. Rouse and S. S. Hemami, “Quantifying the Use of Structure in Cognitive Tasks,” Proceedings of SPIE, Vol. 6492, Human Vision and Electronic Imaging XII, 649210, San Jose, 12 February 2007.doi:10.1117/12.707539
[21]	J. P. Serra, “Image Analysis and Mathematical Morphology,” Academic Press, Inc., Orlando, 1983.
[22]	D. G. Bailey, “Design for Embedded Image Processing on FPGAs,” Wiley-IEEE Press, Singapore, 2011. doi:10.1002/9780470828519
[23]	S. Mignot, “A Hardware-Oriented Connected-Component Labeling Algorithm,” Technical Report, GEPI—Observatoire de Paris, Paris, 2006.
[24]	J. Chen, M. K. Leung and Y. Gao, “Noisy Logo Recognition Using Line Segment Hausdorff Distance,” Pattern Recognition, Vol. 36, No. 4, 2003, pp. 943-955. doi:10.1016/S0031-3203(02)00128-0
[25]	E. Baudrier, F. Nicolier, G. Millon and S. Ruan, “BinaryImage Comparison with Local-Dissimilarity Quantification,” Pattern Recognition, Vol. 41, No. 5, 2008, pp. 14611478. doi:10.1016/j.patcog.2007.07.011
[26]	C. Y. Fang, S. W. Chen and C. S. Fuh, “Road-Sign Detection and Tracking,” IEEE Transactions on Vehicular Technology, Vol. 52, No. 5, 2003, pp. 1329-1341. doi:10.1109/TVT.2003.810999
[27]	M. F. Hashim, P. Saad, M. R. M. Juhari and S. N. Yaakob, “A Face Recognition System Using Template Matching And Neural Network Classifier,” Proceedings of 1st International Workshop on Artificial Life and Robotics, Kangar, May 2005, pp. 1-6.
[28]	H. Bay, A. Ess, T. Tuytelaars and L. Van Gool, “SpeededUp Robust Features (SURF),” Computer Vision and Image Understanding, Vol. 110, No. 3, 2008, pp. 346-359. doi:10.1016/j.cviu.2007.09.014
[29]	“Embedded System Tools Reference Manual EDK (v 13.2),” Xilinx, 2011. http://www.xilinx.com/support/documentation/sw_manuals/xilinx13_2/est_rm.pdf