Remote Operation SW for USV: Part I. Integrated Mission Planning System


The integrated Mission Planning System (MPS) of Unmanned Surface Vehicle (USV) refers to the process which can recognize, decide, plan situations and carry out missions, such as human beings, for all incidental or complex events occurring at sea. In the actual operating environment, it is necessary to develop a simulation software environment and analyze, verify it in advance so as to make an appropriate mission plan considering equipment, sensor, fuel, and other available resources. The existing USV mission planning process methodology has several limitations in the analysis of USV missions because the scenario to be tested is limited and autonomy of USV is not considered sufficiently. To overcome these problems, we constructed a process that considers various missions and is more autonomous, and an integrated environment in which to experiment. In this study, we designed a multi-agent based USV Integrated Mission Planning System and modeled each component. In addition, we constructed the USV remote operation S/W based on M&S that user can experiment with the modeled process and verified the usefulness of the developed system through simulations.

Share and Cite:

Heo, J. , Kim, J. and Kwon, Y. (2018) Remote Operation SW for USV: Part I. Integrated Mission Planning System. World Journal of Engineering and Technology, 6, 806-815. doi: 10.4236/wjet.2018.64053.

1. Introduction

Recently, according to the development of science and technology and changes in the concept of war performance, the value of strategic use of unmanned systems has been increasing because the idea of life emphasis has increased. As the battlefield expands to the ground, sea, air, and even cyber space, the importance of operations to secure the superiority of battlefield is increasing [1] [2] . Especially, there are a lot of cases in which unmanned systems are used in combat situations where there is a high risk or a human is difficult to carry out. One of the marine unmanned systems, USV is extended to various missions such as marine surveillance and reconnaissance, target search, and mine search, and as shown in Figure 1, the operator can monitor at the remote operation station (ROS). In order to give a specific command to USV regarding mission performance, a remote operation system equipped with a series of communication and sensor equipment is required [3] , and it is mainly built on the ground. In addition, there are active researches on algorithms that perform tasks such as radar, lidar, GPS, etc. to identify obstacles, and enemies [4] . In recent years, research has been conducted on how to effectively search mines using robot software platform, Robot Operating System (ROS) [5] . As such, USV performs various missions in the ocean and requires various algorithms and autonomous operation process to accomplish this.

Therefore, it is necessary to study the Integrated Mission Planning Process Design that can plan the mission considering the equipment operation situation according to the surrounding situation when performing the mission. Although the USV is expected to be able to operate autonomously in the marine battlefield, it is having trouble due to constraints such as communication problems and complicated environment compared to other unmanned systems [6] [7] . In addition, autonomy is not sufficiently considered in the existing autonomous unmanned system research, and research on the simulation experimental framework is insufficient. In this study, we modelled the integrated MPS and developed a simulation environment based on Modeling & Simulation (M&S) that can analyze and verify processes. This simulation-based integrated MPS process can be presented as an alternative to in-situ testing that involves high cost and risk in actual USV development [8] [9] . This paper is divided into Part I

Figure 1. USV operation system.

and Part II. In Part I, modeling and verification of the Integrated Mission Planning System is discussed. In Part II, we will discuss the experimental framework that can be simulated with the above-mentioned model. Therefore, the Section 2 of this article defines the integrated mission planning process structure and describes each internal model. In Section 3, we briefly review MPS which we designed, and suggest about future research directions.

2. Integrated Mission Planning Process Design

The integrated mission planning process of USV is an intelligent system that has high-level decision-making functions such as mission planning, path planning, and equipment operation planning to smoothly perform a given mission in an unknown marine environment [10] . Depending on the given mission and environment, various situations such as path re-planning, obstacle avoidance, unmanned ship detection, fuel shortage, and even equipment failure can be encountered. Since they have different characteristics, it is necessary to consider 1) maximization of consistency of each component, 2) independence of each module, and 3) ease of extension of additional modules. In this regard, we need an integrated process model and architecture that can efficiently plan the mission. Figure 2 shows a schematic diagram of the integrated mission planning process based on simulation.

As shown in Figure 2, this simulation model can be divided into USV model and Space model, and Experimental Frame (EF) model. The USV model is

Figure 2. Simulation based integrated mission planning process architecture.

classified into Multi-Agent Process model and USV Platform model, which are the core of integrated mission planning process. The remaining elements are included in the Space model. In detail, it can be classified into a physical model for visualization such as platform model and space model, and a logical model, Multi-Agent Process Model. Each model is introduced in detail in the next section.

2.1. Integrated Mission Planning Process Modeling

In this study, we introduced an artificial intelligence system implementation methodology to effectively model logical elements such as situation awareness, mission planning, resource allocation, and scheduling. The integrated mission planning process model is based on the concept of hierarchical encapsulation & abstraction principle (HEAP), which is a multi-agent modeling technique proposed by Zeigler and Chi [11] [12] .

In Figure 3, the unit agent consists of four parts: Knowledge Base (KB), Fact Base (FB), Macro Function (MF), and Inference Engine (IE). KB is a long-term memory that stores various know-how acquired information through knowledge mining. The FB is available as a kind of short-term memory temporarily stored in which the currently input information and the fact information confirmed through the KB. System Entity Structure (SES) Ontology, IF-THEN Rule and Discrete Event Simulation (DEVS) can be used to express information and knowledge used in KB or FB. Functions for supporting such detailed reasoning are handled by MF. Macro functions are responsible for a variety of functions such as abstraction, identification, classification, validation, goal setting, decision, planning, and resource allocation, which are the main roles of agents such as fuzzy or non-fuzzy operations, deep learning and A* algorithm. Finally, IE performs situation awareness, object identification, and state estimation through final inference on KB, FB, MF and I/O information. Thus, each module is modeled as an agent that includes four parts: KB, FB, MF, and IE. For example, Figure 4 shows an example of a situation awareness agent modeled in this study.

Figure 3. Unity agent structure.

Figure 4. Situation awareness based on agent.

When the MF receives the information about the situation (detection of Unmanned Vehicles), the FB determines the type of the information (enemy) and decides the next action (tracking the enemy) according to the rule in KB.

2.2. Integrated Mission Planning Process Structure

As mentioned above, the M&S based integrated mission planning process structure is classified into 1) Multi-Agent Process, 2) Platform, 3) Space, and 4) EF (Experimental Frame). This section introduces each model in detail.

2.2.1. Multi-Agent Process

The most important part of this study is the process that takes overall responsibility for the logical behavior of USV. This shows a multi-agent hierarchical structure combining the above-described unit agents of different functions. In Figure 5, this process introduced Norman’s seven-step model of human behavior processing in cognitive science [10] [11] . This model is fundamentally same with artificial intelligence or humanoid that approach three basic elements:

Figure 5. Multiple agent hierarchy structure.

awareness, decision making, and behavior. However, the recognition factor is expressed in more detail. First, the self-estimation agent is responsible for the identification of objects other than oneself. The identification agent is responsible for identifying and verifying all of its information and status. If the current status of the other party is confirmed, it is necessary to recognize the overall situation including the relationship between them. The agent for this is the situation awareness.

If the situation is recognized through this agent, a decision-making agent in charge of the decision is activated. In other words, it is decided whether the current situation is normal and whether to continue the mission according to the existing plan, or whether the current situation should be taken as an abnormal contingency and to take emergency measures and establish a new plan.

Then, the task re-planning agent establishes a plan for the detailed tasks that are roughly required according to the decision. And it determines the optimal order of action from the current situation to the target situation. The task plan thus determined is again classified through the path planning and resource allocation agents. The path-planning agent is responsible for the specific route planning of the USV. The resource allocation agent establishes a detailed schedule based on the work plan, such as the timing of input of resources such as sensors, weapons, communication equipment, and mission time.

2.2.2. Platform

The Multi-Agent Process represents only the logical part of the USV. However, the logical part can only be described in terms of the actual USV by interfacing with the physical part. The physical model of the USV for this is the platform. The platform model consists of four detailed models: C2, Sensor, Engagement, and Movement. C2 represents command and control of USV, and sensor represents sensor information such as radar, sonar, and LIDAR. Engagement and movement represent the weapon and movement path of USV.

2.2.3. Space & EF

The Space model is a model for representing and managing the entire mission environment including the USV, and shares the same spatial information between the Multi-Agent Process and the Platform. The EF model is a model for analyzing and verifying the process in a simulation environment. It consists of a Generator model that generates scenarios and a Transducer model for the final Mission of Effectiveness (MOE) analysis.

3. Simulation and Verification

Section 2 introduces the integrated mission planning process structure for USV operation. Based on this structure, we designed an agent-based model and interface environment using C# and Unity3D programs. Figure 6 shows agent pseudo code examples for each model modeled in this study.

In order to verify that modeling process, we created a mission scenario and confirmed through simulation that the mission information transfer between the Multi-Agent Process and the platform is well done in the interface. Figure 7 shows the process of Multi-Agent Process and Situation Awareness Agent in case of avoidance of obstacles, and shows scenarios in which the existing route is re-planned. Figure 8 shows the simulation execution screen of the interface developed in this study.

Based on this research, it is necessary to define the MOE for various situations such as engagement, search, avoidance, and failure as well as obstacle avoidance, and analyze the mission effect through iterative simulation. In order to do this, an interface based experimental framework should be constructed to enable

Figure 6. Agent-specific pseudo-code for each model modeled in this study.

Figure 7. Process of multi agent and situation awareness agent in avoidance of obstacles and scenarios for re-planning the current path.

iterative simulation, and it is necessary to derive the evaluation items on the effectiveness of USV missions. The Part II introduces the simulation-based interface and presents a research direction on MOE analysis.

4. Conclusion

In this paper, Part I, we defined and briefly simulated the Integrated Mission Process model, which enables usable resources and appropriate situation judgment in the complex marine battlefield environment. The core differentiations of this study can be classified into three. First, 1) modular system based Multi-Agent has ease of implementation, scalability, and reusability. Therefore, it can be easily applied to various studies in the future. The second is 2) simulation-based integrated environment. The key points for USV tactical operation can be confirmed through simulation and visualization beforehand. Finally, 3)

Figure 8. Simulation execution display of interface developed in this study.

Mission of Effectiveness (MOE) analysis module of USV is easy to be installed in this environment. Since the simulation results are collected and analyzed independently, the mission effect analysis can be easily performed. This is covered in detail in Part II. Therefore, in Part II, a simulation-based interface was constructed through the modeling process, and it was repeatedly simulated to study the mission effect of USV. In addition, we introduce the interface design mentioned above and the simulation experiment frame for analysis of mission effect. In the future, MOE will be defined through consultation with experts, and optimal operating conditions will be derived through the study of mission effectiveness and conformity analysis. In addition, it will improve the effectiveness of this model through iterative simulation.


This work was supported by the Agency for Defense Development (ADD) under the Contract No. UD160008DD.

Conflicts of Interest

The authors declare no conflicts of interest.


[1] Kim, J.H., Lyu, Y.J., Yu, C.W. and Lee, D.H. (2016) Interface Form Based Experimentation Framework for Mission Planning Analysis of Multi-Mission USV. Journal of the Korean Operations Research and Management Society, 4, 6306-6311.
[2] Lee, H.W., Roh. M., Ham, S.H., Zhao, L., Kim, N.W., Ha, S., Woo, J.H., Jung, W.H. and Yu, C.W. (2017), A Study on Integrated Simulation Method for Mine Detection Mission of USV. Korean Journal of Computational Design and Engineering, 9, 306-316.
[3] Kim, J.-H., Chi, S., Kwon, Y. and Kim, J. (2018) A Study of Effectiveness Analysis of USV Mission Planning. Vol. 20, 2279-2280
[4] Son, N.S., et al. (2014) Development of USV Autonomous Navigation System. Bulletin of the Society of Naval Architects of Korea, 51, 18-22.
[5] Gooding, T.R. (2001) A Framework for Evaluating Advanced Search Concepts for Multiple Autonomous Underwater Vehicle (AUV) Mine Countermeasures (MCM) Naval Postgraduate School, Monterey.
[6] Zeigler, B.P., et al. (2000) Theory of Modeling and Simulation. Academic Press, Cambridge.
[7] Liu, Z.X., et al. (2016) Unmanned Surface Vehicles: An Overview of Developments and Challenges. Annual Reviews in Control, 41, 71-93.
[8] Duan, L., Luo, B., Li, Q.Y. and Yu, G.H. (2016) Research on Intelligence, Surveillance and Reconnaissance Mission Planning Model and Method for Naval Fleet. IEEE Conference on Control and Decision (CCDC), Yinchuan, 28 May 2016, 2419-2424.
[9] Thomas, G., Schneider, M. and Otto, P. (2008) Multi System Mission Control for Teams of Unmanned Marine Vehicles-Software Structure for Online Replanning of Mission Plans. 7th International Conference on Computer Applications and Information Technology in the Maritime Industries (COMPIT), Liège, April 2008, 185-199.
[10] Okan, T. (2014) Adaptive Decision Making in Agent-Based Simulation. Simulation, 90, 815-832.
[11] Domenico, P., Venticinque, S. and Aversa, R. (2013) Agent-Based Design for UAV Mission Planning. 2013 Eighth International Conference on P2P, Parallel, Grid, Cloud and Internet Computing, Compiegne, 28-30 October 2013, 76-83.
[12] Hwang, H.-G., Kim, H.-W., Kim, B.-S., Woo, Y.-T., Shin, I.-S., Shin, J.-H., Lee, Y.-J. and Choi, B.-W. (2017) A Development of Integrated Control System for Platform Equipments of Unmanned Surface Vehicle (USV). Journal of the Korea Institute of Information and Communication Engineering, 21, 1611-1618.

Copyright © 2024 by authors and Scientific Research Publishing Inc.

Creative Commons License

This work and the related PDF file are licensed under a Creative Commons Attribution 4.0 International License.