Simulation and Implementation of Real-Time Vision-Based Control System for 2-DoF Robotic Arm Using PID with Hardware-in-the-Loop


Microsoft Kinect sensor has shown the research community that it's more than just an interactive gaming device, due to its multi-functional abilities and high reliability. In this work, online HIL (Hardware-in-the-Loop) experimental data are used to apply human motion imitation to a 2-degree of freedom Lego Mind storm NXT robotic arm. A model simulation of the dc motor used in this experiment is also present in this paper. The acquired input data from the Kinect sensor are processed in a closed loop PID controller with feedback from motors encoders. The applied algorithms solve the overlapping input problem, conducting a simultaneous control of both shoulder and elbow joints, and solving the overlapping input problem as well. The work in this paper is presented as a prototype to assure the applicability of the algorithms, for further development.

Share and Cite:

Al-Shabi, M. (2015) Simulation and Implementation of Real-Time Vision-Based Control System for 2-DoF Robotic Arm Using PID with Hardware-in-the-Loop. Intelligent Control and Automation, 6, 147-157. doi: 10.4236/ica.2015.62015.

Conflicts of Interest

The authors declare no conflicts of interest.


[1] Microsoft (2009) Project Natal.
[2] Li, Y. (2012) Multi-Scenario Gesture Recognition Using Kinect. The 17th International Conference on Computer Games, Louisville, 30 July-August 1 2012, 126-130.
[3] Machida, E., Cao, M., Murao, T. and Hashimoto, H. (2012) Human Motion Tracking of Mobile Robot with Kinect 3D Sensor. 2012 Proceedings of SICE Annual Conference, Akita, 20-23 August 2012, 2207-2211.
[4] Wang, Y., Yang, C., Wu, X., Xu, S. and Li, H. (2012) Kinect Based Dynamic Hand Gesture Recognition Algorithm Research. The 4th International Conference on Intelligent Human-Machine Systems and Cybernetics, Nanchang, 26-27 August 2012, 274-279.
[5] Zhang, C., Xu, J., Xi, N., Jia, Y. and Li, W. (2012) Development of an Omni-Directional 3D Camera for Robot Navigation. The 2012 IEEE/ASME International Conference on Advanced Intelligent Mechatronics, Taiwan, 11-14 July 2012, 262-267.
[6] Procházka, A., Vysata, O., Valis, M. and Yadollahi, R. (2013) The MS Kinect Use for 3d Modelling and Gait Analysis in the Matlab Environment. Technical Computing 2013, Prague.
[7] Li, B. (2013) Using Kinect for Face Recognition under Varying Poses, Expressions, Illumination and Disguise. Applications of Computer Vision (WACV), Tampa, 15-17 January 2013, 186-192.
[8] Prochazka, A., Kubicek, M. and Pavelka, A. (2006) Multicamera Systems in the Moving Body Recognition. 48th International Symposium ELMAR-2006 Focused on Multimedia Signal Processing and Communications, Zadar, June 2006, 45-48.
[9] Staranowicz, A. and Mariottini, G.-L. (2013) A Comparative Study of Calibration Methods for Kinect-Style Cameras. University of Texas at Arlington, Arlington.
[10] Li, Y. (2012) Hand Gesture Recongnition Using Kinect. University of Louisville, Louisville.
[11] Harrison, T.R. (2011) Chapter 367: Approach to the Patient with Neurologic Disease. In: Longo, D.L., Fauci, A.S., Kasper, D.L., Hauser, S.L., Jameson, J.L. and Loscalzo, J., Eds., Harrison’s Principles of Internal Medicine, McGraw Hill Professional, New York.
[12] Ivanescu, L. (2014) KEV3.
[13] Lasenby, J. and Stevenson, A. (2001) Using Geometric Algebra for Optical Motion Capture. Birkhauser, Boston.
[14] Ringer, M. and Lasenby, J. (2002) Multiple Hypothesis Tracking for Automatic Optical Motion Capture. Lecture Notes in Computer Science, 2350, 524-536.
[16] Dorf, R.C. and Bishop, R.H. (2001) Modern Control Systems. 9th Edition, Prentice-Hall, Englewood Cliff.
[17] Watanabe, R. (2015) NXT Motor Parameters.
[18] Kroon, D.-J. (2011) Mathworks.

Copyright © 2024 by authors and Scientific Research Publishing Inc.

Creative Commons License

This work and the related PDF file are licensed under a Creative Commons Attribution 4.0 International License.