Robot Position Control Using Force Information for Cooperative Work in Remote Robot Systems with Force Feedback

Satoru Ishikawa¹, Yutaka Ishibashi¹, Pingguo Huang², Yuichiro Tateiwa¹

¹Nagoya Institute of Technology, Nagoya, Japan.
²Gifu Shotoku Gakuen University, Gifu, Japan.

DOI: 10.4236/ijcns.2021.141001                

Abstract

This paper proposes robot position control using force information for cooperative work between two remote robot systems with force feedback in each of which a user operates a remote robot by using a haptic interface device while observing work of the robot with a video camera. We also investigate the effect of the proposed control by experiment. As cooperative work, we deal with work in which two robots carry an object together. The robot position control using force information finely adjusts the position of the robot arm to reduce the force applied to the object. Thus, the purpose of the control is to avoid large force so that the object is not broken. In our experiment, we make a comparison among the following three cases in order to clarify how to carry out the control effectively. In the first case, the two robots are operated manually by a user with his/her both hands. In the second case, one robot is operated manually by a user, and the other robot is moved automatically under the proposed control. In the last case, the object is carried directly by a human instead of the robot which is operated by the user in the second case. As a result, experimental results demonstrate that the control can help each system operated manually by the user to carry the object smoothly.

Keywords

Remote Robot System, Haptic Interface Device, Force Feedback, Cooperative Work, Robot Position Control, Experiment

1. Introduction

A number of researchers focus on remote robot systems with force feedback [1] [2] [3]. We can conduct various types of cooperative work such as remote surgery, work in outer space, deep sea, and disaster areas among multiple remote robot systems with force feedback [4] - [11]. Since we can feel the shape, softness, surface smoothness, and weight of a remote object through the reaction force against the object by using a haptic interface device, we can expect to improve the efficiency and accuracy of cooperative work among the systems largely. However, when force information is transmitted over a network such as the Internet, which does not guarantee the quality of service (QoS) [12], the quality of experience (QoE) [13] may seriously be degraded and the system may become unstable owing to the network delay, delay jitter, and packet loss. To solve the problems, we need to exert QoS control and stabilization control together. If the two types of control are not carried out in the systems, we cannot perform the work efficiently, and strong force may be applied to the remote object [4] [5] [6] [7] [8]. If the object is fragile, it may seriously be damaged.

In [5], the authors investigate the influence of network delay on cooperative work of carrying a wooden stick as an object together by the robot arms of two remote robot systems with force feedback. In the cooperative work, a user operates the two haptic interface devices with his/her both hands. They demonstrate that the average time of work and the force applied to the stick increase as the network delay becomes larger by experiment. In [6], where the cooperative work in [5] is handled, the authors make a comparison of three types of stabilization control by experiment. One is the reaction force control upon hitting [7]. Another is the stabilization control by viscosity [8], and the other is the stabilization control with filters [4]. Experimental results illustrate that the stabilization control with filters is the most effective. However, the force applied to the stick is large in [5] and [6]. If the force is too strong, the stick may be broken. To solve the problem, we need to avoid strong force.

On the other hand, robots should outperform or behave like humans as the final goal of our research. To clarify the goal of this study quantitatively, it is necessary to make a comparison between humans and robots. However, such a comparison has not been made sufficiently so far [9].

In this paper, we propose robot position control using force information to suppress the force applied to the object for two remote robot systems with force feedback. The proposed control finely adjusts the position of the robot arm to reduce the force applied to the object. We demonstrate the effectiveness of the proposed control under the stabilization control with filters by experiment. In the experiment, we make a comparison between the case in which the proposed control is carried out in both systems, the case in which the control is exerted in only one system, and the case in which a human directly carries the object instead of the robot arm in the second case. By the experiment, we can make a comparison between the robot arm and the human and clarify how to carry out the proposed control effectively.

The rest of this paper is organized as follows. In Section 2, we first outline the remote robot systems with force feedback. Next, we propose the robot position control using force information in Section 3. Then, we describe the experiment method in Section 4 and present experimental results in Section 5. Finally, Section 6 concludes the paper.

2. Remote Robot Systems with Force Feedback

2.1. System Configuration

We show the configuration of the two remote robot systems with force feedback (called systems 1 and 2 here) in Figure 1. The two systems are almost the same basically excluding the number of PCs in each system. Each system consists of a master terminal and a slave terminal. The master terminal in system 1 is composed of PC for haptic interface device and PC for video; in system 2, one PC is used for both haptic interface device and video. A haptic interface device (3D Systems Touch [14] ) is connected to PC for haptic interface device/PC for haptic interface device and video. The degree of freedom (DoF) of the device is three (i.e., the x, y and z axes). The slave terminal in system 1 consists of PC for industrial robot and PC for video; in system 2, one PC is used for both industrial robot and video. PC for industrial robot/PC for industrial robot and video is directly connected to the industrial robot. Also, a web camera is connected to PC for video/PC for industrial robot and video. The industrial robot has the robot arm (RV-2F-D [15] ), robot controller (CR750-Q [15] ), and force interface unit (2F-TZ561 [16] ). A force sensor (1F-FS001-W200 [16] ) is attached to the tip of the robot arm. The DoF of the robot arm is six, and the force sensor can measure force of six axes, only the three (x, y and z) axes are used in this paper. A toggle clamp hand is further linked to the force sensor. The hand is used to clamp an

Figure 1. Configuration of two remote systems with force feedback.

object by a toggle.

2.2. Remote Operation

In each system, a user at the master terminal can remotely operate the robot arm by using the haptic interface device while observing work of the robot with the web camera. The initial position about the stylus of the haptic interface device is set to the origin point of the industrial robot.

The master terminal acquires the position information from the haptic interface device every millisecond, calculates the reaction force, and outputs it via the device. Then, the position information is transmitted to the slave terminal by UDP. The reaction force $F_t^{\left(\text{m}\right)}$ outputted at time t (>0) through the haptic interface device is calculated as follows [3]:

$F_t^{\left(\text{m}\right)}=K_{\text{scale}}{F}_{t-1}^{\left(\text{s}\right)}$ (1)

where $F_t^{\left(\text{s}\right)}$ is the force received from the slave terminal at time t, and $K_{\text{scale}}$ is set to 0.33 [11] by a preliminary experiment in this paper.

The slave terminal transmits the position information of the robot arm and the information about force sensed by the force sensor to the master terminal every millisecond. Then, the slave terminal employs the real-time control function [17] of the industrial robot to obtain the information about the position of the industrial robot and to send instructions to the robot, and the terminal uses the real-time monitor function [17] of the industrial robot to get the information about the force sensor from the robot controller every 3.5 milliseconds. The two types of information are transmitted as different packets between the robot controller and PC for industrial robot/PC for industrial robot and video by UDP. At time t (>0), the position vector about the tip of the industrial robot arm $S_t$ is calculated as follows [3]:

$S_t=M_{t-1}+V_{t-1}$ (2)

where $M_t$ is the position vector about the stylus of the haptic interface device that is received from the master terminal at time t. $V_t$ is the moving velocity of the industrial robot arm at time t, and $\left|V_t\right|\le V_{\max }$, where $V_{\max }$ is the maximum value of $V_t$. We set $V_{\max }=5$ mm/ms [3] in this paper.

2.3. Cooperation between Systems

As described earlier, we handle the cooperative work in which an object (a wooden stick in Figure 2) is carried between the two robot arms (called the robot-robot case here), and between a human and the robot arm (called the human-robot case). In the former case, one user operates the two robot arms with his/her both hands (see Figure 2(a) and Figure 2(b)). The object is held by the two toggle clamp hands as shown in Figure 2(b). Note that the toggle clamp hand in system 2 is attached in the opposite direction to that in system 1. In the latter case, a human uses a reacher to grasp the object, and a user operates the robot arm as shown in Figure 2(a) and Figure 2(c), where the object is held by

Figure 2. Appearance of master and slave terminals. (a) Master terminals; (b) Slave terminals (robot-robot case); (c) Slave terminals (human-robot case).

the reacher and the toggle clamp hand. Note that only one system is used in this case. To avoid unstable phenomena of the systems, we carry out the stabilization control with filters [4] and disable the movement of each robot arm in the left-right and up-down directions (i.e., the y and z axes, respectively) in both cases.

3. Robot Position Control Using Force Information

To reduce the force applied to an object which is held by each robot arm, the robot position control using force information finely adjusts the robot position dynamically according to the force. Under the control, we get a new position vector ${\hat{S}}_t$ by adding $P$ to $S_t$ of Equation (2) as follows:

${\hat{S}}_t=S_t+P$ (3)

where $P$ is a position adjustment vector which decreases the difference in the position vector between the two robot arms to reduce the force applied to the object.

In [10], when a wooden stick is held as the object by the two robot arms, we move the one robot arm, and measure the force applied to the stick. As a result, we obtain the relation between the movement distance and force in the front-back direction (the x axis) as follows:

$P_x=a_x{F}_x$ (4)

where P_x is movement distance of the robot arm, and F_x is force vector which is sensed by force sensor. Also, the coefficient a_x is a function of the length l [cm] of the wooden stick [11].

$a_x=4.82\times {10}^{-2}l-1.16$ (5)

By using Equations (4) and (5), we can calculate the difference in the position vector between the two robot arms from the force applied to the stick with length of l.

When we carry the stick with length of L [cm] together by using the two systems, we find in [11] that we need to use a different value from L as l in Equation (5); there exists the optimal value of l (denoted by l_opt) for each value of L. This is because the type of work is different from that in [10].

4. Experiment Method

In our experiment, we conducted cooperative work of carrying a wooden stick as an object together. In the work, a user (one of the authors) operated one or two haptic interface devices with his/her hands while watching video. When the user operated the two haptic interface devise (i.e., the robot-robot case), we handle the case in which the robot position control using force information is carried out in the systems (called the control case here), and the case in which the control is not carried out in both systems (called the no control case). In the control case, we further two cases: Both systems carry out the control in one case, and only one system performs the control in the other case (the other system does not exert the control). When the user operates one haptic interface device, the robot arm follows automatically the other robot arm which is operated manually by the user (i.e., the robot-robot case). Note that the user employs only the haptic interface device of system 1. Then, we handle the case in which the control is carried out in both systems, and the case in which the control is performed in only one system. When the robot arm is operated automatically under the control, we also conducted the work with a reacher instead of the robot arm which is operated by the user as shown in Figure 2(c) (i.e., the human-robot case).

To move the stick in almost the same way in the experiment, as shown in Figure 2, building blocks were piled up ahead and behind the initial position of the stick, and a paper block was placed on each uppermost building block. The arrangement of the stick and blocks is shown in Figure 3. As discribed in Subsection 2.2, the initial position is 0 (i.e., the origin point). The human and user

Figure 3. Plane view of arrangement of stick and blocks.

moved the stick toward the paper blocks to touch the paper blocks while keeping the robot arms parrallel to each other. They touched the paper block on the front side in Figure 2 by the stick, and then they did that on the other side. Also, to move the stick at almost the same speed, they touched the first paper block at about 5 seconds from the beginning of each work and the second block at about 15 seconds; we determined the speed by carrying out a preliminary experiment. The user moved the stick while checking the time. The ratio of the moving distance of the haptic interface device to that of the industrial robot is set to 2:1 [6], and the ratio of the force is 1:3 [11] (i.e., $K_{\text{scale}}=0.33$ as described in Subsection 2.2). The master and slave terminals of each system were connected via a network emulator (NIST Net [18] ) instead of the network in Figure 1. The network emulator just connected the master and slave terminals of each system; that is, the produced network delay was 0 ms and packet loss rate was 0%. We used the wooden stick with height, width, and length of 1 cm, 1 cm, and 30 cm [11], respectively. The weight of the stick is approximately 0.44 gf per length of 1 cm; it is enough light compared with the weight of the robot hand.

We carried out a preliminary experiment to obtain the optimal value of l (l_opt) which is used in Equation (5). As a result, we obtained the following results: When the two robot arms are operated manually, the force applied to the stick is the minimum at $l_{\text{opt}}=40$ and 35 in the case where the control is carried out in both systems and in the case where the control is carried out in only one system, respectively [19]. When one robot arm is operated automatically, the force is the minimum at $l_{\text{opt}}=40$ and 55 in the case where the control is carried out in both systems and in the case where the control is carried out in only one system, respectively [20]. Also, the force is the minimum at $l_{\text{opt}}=150$ in the human-robot case [20]. In the control case, we used the value of l_opt. The work was conducted 10 times in random order in each case. We measured the force applied to the stick and obtained the average force and maximum force during the work; then, we calculated the averages of the 10 times (called the average of average force and the average of maximum force, respectively, in this paper).

5. Experimental Results and Discussions

We show the average of average force and the average of maximum force in the front-back (the x axis) direction in six cases in Figure 4 and Figure 5. The 95% confidence intervals of averages are also plotted in the figures.

We can observe that Figure 4 and Figure 5 have similar tendencies to each other. In the figures, we see that the average of average force and average of maximum force in the no control case are the largest, and those in the case where the control is carried out at one system in the robot-robot case are the second largest; however, the difference between the two cases is not so large. Thus, we performed one-way ANOVA to examine whether difference is significant. As a result of one-way ANOVA, we confirmed that there are significant differences among the six cases. Thus, we can say that the robot position control

Figure 4. Average of average force.

Figure 5. Average of maximum force.

using force information is effective in our experiment.

We also find in the figures that when the two robot arms are operated manually, the averages in the case where the control is carried out in both systems are smaller than those where the control is carried out in one system. Thus, we can say that the case in which the control is carried out in both systems is better than that in which the control is carried out in only one system, when the two robot arms are operated manually.

In the figures, we find that when the one robot arm is operated automatically, the averages in the case where the control is carried out in one system are smaller than those where the control is carried out in both systems. Therefore, we can say that the case in which the control is carried out in one system is superior to that in which the control is carried out in both systems, when the one robot arm is operated automatically.

We further observe that the average of average force and average of maximum force are the smallest in the human-robot case. This is because humans can work more flexibly than the robots which are operated manually and observe the work of the robots directly without the network in the experiment. It is necessary to improve the flexibility in the robot-robot case to achieve almost the same (or smaller) averages as those in the human-robot case as the quantitative goal of this study. This is for further study.

To examine the results in Figure 4 and Figure 5 in more detail, we show the position ( ${\hat{S}}_t$ in Equation (3)) and force of one or two robot arms in the front-back direction in the six cases versus the elapsed time from the beginning of the experiment in Figures 6-11. In the figures, robot arms 1 and 2 means robot arms of systems 1 and 2, respectively. The results are typical examples in our experiment. Figures 6-8 show the results in the case where the two robot arms were operated manually, and Figures 9-11 do those in the case where one robot arm was operated automatically.

From Figure 6(a) through Figure 10(a), we find that the positions of robot arms 1 and 2 are almost the same. In the figures, the position increases from 0 mm to about 40 mm for around 5 seconds and then decreases to −40 mm for approximately 10 seconds (totally 15 seconds as described in Section 4). In Figure 11(a), the position of robot arm 2 somewhat fluctuates since the robot arm

Figure 6. Position and force versus elapsed time when two robot arms are operated manually (no control case). (a) Position; (b) Force.

Figure 7. Position and force versus elapsed time when two robot arms are operated manually (control is carried out in both systems). (a) Position; (b) Force.

Figure 8. Position and force versus elapsed time when two robot arms are operated manually (control is carried out in one system). (a) Position; (b) Force.

Figure 9. Position and force versus elapsed time when one robot arm is operated automatically (control is carried out in both systems). (a) Position; (b) Force.

Figure 10. Position and force versus elapsed time when one robot arm is operated automatically (control is carried out in one system). (a) Position; (b) Force.

Figure 11. Position and force versus elapsed time in human-robot case. (a) Position; (b) Force.

followed the human. In Figure 6(b), the absolute force is large during the cooperative work. This is because the difference in position between the two robot arms tends to be large as shown in Figure 6(a). In Figure 7(b), the absolute force is kept small. In Figure 8(b), the absolute force is larger than that in Figure 7(b). Also, the absolute force in Figure 9(b) is larger than that in Figure 7(b), and the absolute force in Figure 10(b) is smaller than that in Figure 8(b). The absolute force in Figure 10(b) is smaller than that in Figure 9(b), where large force is applied to the stick at the beginning of the work and at the change of the moving direction. In Figure 11(b), the absolute force in the human-robot case is kept much smaller than that in the robot-robot case. Thus, we need to suppress the force in the robot-robot case; this is for further study as described earlier. The relations of the absolute force among Figure 6(b) through Figure 11(b) are similar to those of the averages in Figure 4 and Figure 5. In the figures, we find that the force has small vibrations. The vibrations were hardly perceived by the user. This is because the effects of the stabilization control.

From the above considerations, we can say that the case in which the robot position control using force information is carried out in both systems is better than that in which the control is carried out in only one system, when the two robot arms are operated manually. Also, the case in which the control is carried out in only one system is superior to that in which the control is carried out in both systems, when the one robot arm is operated automatically. Therefore, we can conclude that the control can help each system operated manually to carry the object smoothly.

6. Conclusions

This paper proposed the robot position control using force information for cooperative work between two remote robot systems with force feedback. As cooperative work, we dealt with work in which two robots carry an object together. In our experiment, we conducted the cooperative work between the two robots which are operated manually by a user with his/her both hands. Also, we performed the work between one robot which is operated manually by a user and the other robot which is operated automatically under the robot position control using force information. As a result, we found that the case in which the control is carried out in both systems is the most effective when the two robots are operated manually. Also, the case in which the control is carried out in only one system is the most effective when the one robot is operated automatically. Thus, the control can help each system operated manually to carry the object smoothly.

In our experiment, the work was conducted by one user. We plan to perform the work with two users. We will also improve the flexibility and suppress the force applied to the stick in the robot-robot case as in the human-robot case. Furthermore, it is important to carry out the experiment with various moving speeds of robot arms.

Acknowledgements

This work was supported by JSPS KAKENHI Grant Number 18K11261.

Conflicts of Interest

The authors declare no conflicts of interest regarding the publication of this paper.

References

[1]	Ohnishi, K., Katsura, S. and Shimono, T. (2010) Motion Control for Real-World Haptics. IEEE Industrial Electronics Magazine, 4, 16-19. https://doi.org/10.1109/MIE.2010.936761
[2]	Miyoshi, T. and Terashima, K. (2006) A Stabilizing Method for Non-Passive Force-Position Teleoperating System. Short report, The 35th SICE Symposium on Control Theory, Japan, 127-130.
[3]	Suzuki, K., Maeda, Y., Ishibashi, Y. and Fukushima, N. (2015) Improvement of Operability in Remote Robot Control with Force Feedback. 2015 IEEE 4th Global Conference on Consumer Electronics, Osaka, Japan, 27-30 October 2015, 16-20. https://doi.org/10.1109/GCCE.2015.7398509
[4]	Huang, P., Miyoshi, T. and Ishibashi, Y. (2019) Enhancement of Stabilization Control in Remote Robot System with Force Feedback. International Journal of Communications, Network and System Sciences, 12, 99-111. https://doi.org/10.4236/ijcns.2019.127008
[5]	Taguchi, E., Ishibashi, Y., Huang, P. and Tateiwa, Y. (2018) Experiment on Collaborative Work between Remote Robot Systems with Haptics. IEICE General Conference, B-11-17. (In Japanese)
[6]	Taguchi, E., Ishibashi, Y., Huang, P. and Miyoshi, T. (2020) Comparison of Stabilization Control in Cooperation between Remote Robot Systems with Force Feedback. International Journal of Mechanical Engineering and Robotics Research, 9, 87-92. https://doi.org/10.18178/ijmerr.9.1.87-92
[7]	Arima, R., Huang, P., Ishibashi, Y. and Tateiwa, Y. (2018) Softness Assessment of Object in Remote Robot System with Haptics: Comparison between Reaction Force Control upon Hitting and Stabilization Control. IEICE Technical Report, CQ2017-98. (In Japanese)
[8]	Rikiishi, T., Ishibashi, Y., Huang, P., Miyoshi, T., Ohnishi, H., Tateiwa, Y., et al. (2017) Stabilization Control by Viscosity in Remote Robot System with Haptics. IEICE Society Conference, BS-7-21.
[9]	Qian, Q., Ishibashi, Y., Huang, P. and Tateiwa, Y. (2020) Cooperative Work among Humans and Robots in Remote Robot Systems with Force Feedback: Comparison between Human-Robot and Robot-Robot Cases. 2020 8th International Conference on Information and Education Technology, Okayama, Japan, March 2020, 306-310. https://doi.org/10.1145/3395245.3396418
[10]	Qian, Q., Osada, D., Ishibashi, Y., Huang, P. and Tateiwa, Y. (2018) Human Perception of Force in Cooperation between Remote Robot Systems with Force Feedback. 2018 4th IEEE International Conference on Computer and Communications, Chengdu, China, December, 2018.
[11]	Ishikawa, S., Ishibashi, Y., Huang, P. and Tateiwa, Y. (2019) Effect of Robot Position Control with Force Information for Cooperative Work between Remote Robot Systems. 2019 2nd World Symposium on Communication Engineering, Nagoya, Japan, 20-23 December 2019, 210-214. https://doi.org/10.1109/WSCE49000.2019.9041108
[12]	ITU-T Rec. I. 350 (1993) General Aspects of Quality of Service and Network Performance in Digital Networks.
[13]	ITU-T Rec. G. 100/P. 10 Amendment 1 (2007) New Appendix Idefinition of Quarity of Ecperience (QoE).
[14]	3D Systems Touch. https://www.3dsystems.com/haptics-devices/touch.
[15]	RV-2F Series. https://www.mitsubishielectric.co.jp/fa/products/faspec/point.do?kisyu=/robot&formNm=RV-2F_RV-2F%28B%29_1, (in Japanese)
[16]	CR750/CR751 Controller. https://www.mitsubishielectric.co.jp/fa/products/faspec/point.do?kisyu=/robot&formNm=CR75x-Q_CR750-Q_1&category=ex&id=spec, (in Japanese)
[17]	CR750/CR751 Series Controller, CR800 Series Controller Ethernet Function Instruction Manual. http://dl.mitsubishielectric.co.jp/dl/fa/members/document/manual/robot/bfp-a3379/bfp-a3379b.pdf, (in Japanese)
[18]	Carson, M. and Santay, D. (2003) NIST Net: A Linux-Based Network Emulation Tool. ACM SIGCOMM Computer Communication Review, 33, 111-126. https://doi.org/10.1145/956993.957007
[19]	Ishikawa, S., Ishibashi, Y., Huang, P. and Tateiwa, Y. (2020) Experiment on Robot Position Control Using Force Information in Cooperation between Remote Robot Systems with Force Feedback. IEICE Technical Report, CQ2020-18. (In Japanese)
[20]	Ishikawa, S., Ishibashi, Y., Huang, P. and Tateiwa, Y. (2020) Effects of Robot Position Control Using Force Information in Remote Robot Systems with Force Feedback: Comparison between Human-Robot and Robot-Robot Cases. 2020 2nd International Conference on Computer Communication and the Internet, Nagoya, Japan, 26-29 June 2020, 179-183. https://doi.org/10.1109/ICCCI49374.2020.9145962