Multi Agent Based Power Distribution System Restoration—A Literature Survey

Multi agent system (MAS) is one of the most dominant research wings which consist of several agents who interact with each other to achieve a common objective. MAS has been developed for a wide range of applications in power systems. Power system restoration is a main application of that. Researchers present several architectures for fault identification, isolation and restoration of the power system. This paper presents a complete literature review on available architectures for power distribution restoration and future trends in MAS based power system restoration.


Introduction
Power System has become very complicated with restructuring and excessive power demands. It has increased more than ever with newly integrated net metering systems, electric vehicles and smart grid devices. A single fault can result in massive cascading effects, affecting the power supply as well as the power quality [1]. Power quality has become one of the most important factors to be considered by electricity distribution utilities. Availability is the most important factor when it comes to power quality.
System Average Interruption Duration Index (SAIDI) is the best and the most commonly used reliability indicator. SAIDI measures the average outage duration for each customer served, and is calculated for certain location by multiplying the durations of the interruption by interrupted customers to find customer-minutes, then the total customer-minus will be found out by taking the summation of all interruptions. Then the total customer-minutes will be divided by the total number of customers. SAIDI has a direct relation with "Customer Minutes of Interruption" (CMI) which represents the average annual outage minutes per customer served. In order to develop SAIDI, faster restoration strategy is essential.
Faults occurring in the distribution system reduce the availability of electricity and the time to clear that fault is very critical. Therefore, distribution automation systems are introduced to automate the fault detection, isolation and restoration process [2]. Various approaches have so far been proposed to obtain the target configuration, such as expert systems, heuristics, soft computing and mathematical programming (MP) [3]. Nagta, T. et al.
give a brief description of each of those proposed systems and their advantages and disadvantages [3]. Heuristics and expert systems have problems in achieving the optimal solution whereas MP is able to obtain the optimum solution; however, more engineering judgmental to formulate restoration problem is needed and due to its difficulty, its execution time is extended. In the case of Soft Computing, it is easy to implement, but it is difficult to obtain the optimal solutions. And also, it takes lengthy computation time until the solution.
Each of above systems relies on SCADA systems. For large scale power systems, when all decisions are being taken by the control center, it will be very difficult to handle all these calculations and communications. They are built upon proprietary communication and signaling protocols. Future power systems aim is to allow for flexible integration of different power system equipment. However, the vendor specific control protocols used in commercial SCADA systems for power systems restricts the interconnectivity and interoperability and this results to increase deployment costs and prevent power system rollout.
Several definitions for an agent are given as per Wooldridge defines that agent is a software component which has been given the autonomy that provides an interoperable interface to an arbitrary system and/or behaves like a human agent, working for some clients in pursuit of its own agenda [4]. Multi agent systems (MAS) are made up of multiple interacting intelligent agents. A collection of social, collaborative agents constitute a multi agent system. All the agents will inherit a set of common goals, and react to change in the environment, make decisions to achieve those goals, or help other agents in achieving their goals [4]. Other than a software agent MAS is now being used as a physical agent.
Why multi agent systems are more suitable for restoration problem was a debatable question. It can be found from the literature clearly why it is more suitable in this scenario. The definition of agency proposed by Wooldridge gives a good understanding on MAS [5]. By extending the autonomy in the definition of as agent to flexible autonomy, Woodridge extends the concept of as agent into an intelligent agent [6]- [8]. When an agent has become an intelligent agent it will comprises of following characteristics.
Reactivity: Agents are designed in such a way that they can recognize changes in the environment not only that but they also can react and response to those in a timely manner. Most importantly they can take some actions depending upon those perceptions and functions which are designed to be accomplished.
Pro-activeness: Though changes happen in the environment agents will not deviate from its goals which means agent will dynamically change its behaviors in order to achieve its objectives. This goal oriented behavior is called pro-activeness. For example if an agent loses connection between an agent which should be communicated to achieve his objective, he will contact another agent who can provide necessary information. Wooldridge describes this pro-activeness as an agent's ability to "take the initiative".
Social ability: This emphasize that agents will act together to achieve common goals of the system. This behavior is not merely information sharing but it indicates that agents will be able to negotiate and interact in a cooperative manner. That capability is normally supported by an agent communication language (ACL), which allows agents to antithetical and cooperative rather than simply pass data.
Though a large number of surveys are being conducted [9] in the multi agent research wing lesser number of surveys actually speak about power system restoration. Importance of multi agent systems in system restoration and appropriateness of the MAS in system restoration are being analyzed by Acharya, D.S. et al. [10]. This paper categorized types of restorations such as connectivity restoration [11], power system restoration [12] and restoration of smart grid [13] and also presented several architectural features with some designs [12]- [14]. This work attempted to present a survey of the various areas, in control engineering, where multi agent systems have been successfully and efficiently implemented [10] [15].
A survey on multi-agent systems in control engineering was presented by Daneshfar, F. et al. [16]. As in most of surveys related to MAS this paper also analyzes methods and architectures that are being used in MAS for control engineering applications. This paper presented the essential terms and concepts that are associated with multi-agent systems, deliberate why it is more popular among control engineering applications and they also scrutinize recent publications and research works to find out and areas which MAS is much suitable. And they also explain reasons why MAS is being used in power distribution applications. Though these papers provide a good understanding of MAS none of those represents a comprehensive overview of available technologies and architectures.
This survey has been done following the method proposed by Kitchenham [17] [18] and the presentation template used by Aleti, A. et al. [19]. In order to achieve its research objectives classification of the current research topics on multi agent systems in power system applications was needed and current state of multi agent systems in power system applications research with respect to this classification was needed to be identified.
The literature review has been done using widely accepted literature search engines for literature searching mainly three key words are being used such as multi agent systems (MAS), distribution automation systems (DAS), power system restoration, electric vehicle charging, smart grids. Key words were refined and extended while the searching process. The concentration of this research is being given to multi agent based distribution restoration. Papers related to micro grids were excluded because one of the main ideas of the research was to identify more suitable architecture for MAS based restoration for a main grid. Due to the same reason as above, shipboard power restoration was also omitted. But it was necessary to identify applications of MAS in power system before comes in to the topic.
Therefore in the next section applications of MAS in the power system other than restoration is briefed. Papers related to power system restoration was then analyzed and categorized according to the architecture in the section 3 and their architectural features are thoroughly discussed. Optimization methods are then analyzed and briefed in Section 4. Section 5 presents the conclusion and the future directions of this research wing.

Application of MAS in Power System (Other than Restoration)
Due to inherent features multi agent systems are widely being applied in the power system automation. Other than the main features such as reactivity, proactive and social ability, Bellifemine, F. et al. emphasized some other features such as self-governing, mobile, honest, benevolent, Rational, learn. Agents can their own without direct human intercession and they have control over its activities and internal state, therefore agents are said to be self-governing. Agents can also travel among different nodes in a computer network which can be called as mobile. Since agents do not provide any false information intentionally they can be considered as honest. It can be benevolent; always trying to achieve what is asked of it. It can be rational, agents always act in order to achieve its designed objectives and never to thwart its goals being achieved. Most importantly they can learn by themselves in order to acclimatize itself to fit its environment and to the desires of its users [20] [21]. From the literature it can be found many applications and those are briefly described in the next section.

Distributed, Decentralized, Microgrids
A conventional distribution network transmits electricity from a central power sources to consumers, and the power dispatch is centrally controlled [22]- [25]. In recent times however, technological progressions, security apprehensions, regulatory policies and environmental considerations are changing the background of electricity generation and transmission by decreasing the grid's dependency on large centralized generation facilities [26]- [29]. Resulting advantages such as reduced voltage drop and reduced power losses the concept of distributed generators (DGs) and energy storing concepts have arisen as non-conventional power sources to the main grid or the distribution network at a smaller and distributed scale. As a result several problems can also arise due to uncertainty and unpredictability because most of those DGs are not utility owned and also characterized by high degree of vagueness such as solar and wind [30]- [33] and the distribution network may perform differently from the traditionally anticipated behaviors [34]- [37]. Animatedly and efficiently managing the power dispatch in order to maintain the frequency by balancing supply and demand by considering the variability of DGs and loads has become a substantial research topic [38]- [41].
Niannian, C. et al. proposes complete multi-level control architecture described for master-slave organized microgrids with PE interfaced DGs [42]. Zhao, L. et al. presents an approach of using power line communication (PLC) technology, a multi-agent system (MAS)-based, decentralized, under-frequency load shedding (UFLS) scheme is scrutinized in this study for smart distribution networks with the communication constraints that each agent can only communicate with its neighboring agents [43] and proposed a decentralized reactive power control scheme to optimally control the switched capacitor in the system in order to abate system losses and maintain satisfactory voltage profile [44]. Also literature can be found on multi agent based designs for reactive power management in smart grids with renewable power sources to enhance the dynamic voltage stability, which is mainly based on controlling distributed static synchronous compensators [45]. Katherine, R. R. et al. proposes that prevailing and forthcoming devices at the inhabited level have the capability to deliver reactive power. Good examples are inverters which connect distributed generation such as solar panels and pluggable hybrid electric vehicles to the grid [19].

Electric Vehicles
A number of EVs are expected to grow rapidly within the next few years. The coordination and controlling of charging of electric vehicles is one the emerging trends in MAS. Unrestrained charging of EVs can affect unfavorably the normal operation of the power system, exclusively at the distribution level [46]. Karfopoulos, E. L. et al. presents a distributed, multi-agent EV management system. The proposed system gratifies energy demand of large population of electric vehicles by considering their owner's and confirming the efficient operation of the network. The proposed distributed control concept is a single-objective; non-cooperative, dynamic game which congregates to å-Nash equilibrium under the circumstance of feebly coupled VC agents. Simulations on a convincing MV feeder endorse the incapability of dumb charging and simple dual-tariff structures to assimilate a large number of EVs without distressing unfavorably the power networks landscapes. The paper also suggests that EVs can be used to smoothen the demand curve by "valley filling" by providing excess demand at the peak and charging at off peak, leading to intensification of load factor and minimization of energy losses [47].
Several other literatures can also be found with different optimizations and control algorithms [48]. A multi-agent system that harmonizes EV charging in distribution network using a distributed control technique, with search techniques and neural networks was described. The concept of feeding back the grid from EVS is made easier with MAS and it is a part which has to be considered in the future with restoration process because it is possible to use these storage as a backup power source for highly prioritized areas such as hospitals.

Electricity Market
Agent-based simulation has been a widespread method in modeling and analyzing electricity markets too [49], [50]. It has been researched during the last decay and several agent based techniques are being presented and separate reviews on this topic can be found [51]. In their paper Zhou, Z. et al. investigates several general-purpose agent-based simulation tools. Then, it reviews four widespread agent-based simulation packages developed for electricity markets and several agent-based simulation models reported in the literature.

Building Management
Intelligent building is defined as "One which integrates several systems to effectively manage resources in a harmonized manner to maximize: technical performance, investment and operating cost savings, flexibility" [52], [53]. One of the techniques to implement these systems is multi agent systems with reconfigurable hardware accelerate responses because of parallel processing [54] [55].

Load Forecasting
Load forecasting is one of the most important functions when it comes to economic concern. MAS is being used to find the unit commitment. A multi agent system for spatial electric load forecasting is particularly suited for simulating the different social dynamics involved in distribution systems [56]. This methodology recovers the spatial predicting procedures which usually consider the service zone as a static entity to model or simulate the spatial electric load forecasting in a city.

Fault Detection
The first stage of the process is fault detection and in most of the cases it is being done by transforming the wavelet [57] [58]. The wavelet transform usage is explicated by its high efficiency for the singularities discovery, which is a power system fault characteristic [58].
Several literature on fault detection suggest detecting the fault by calculating the entropy alongside with transformation of current signals, categorize those faults and tracking down the fault. It is done by calculating the entropy of wavelet coefficients of 3-phase current signals where various information related to the fault is included in those transient components which ultimately led to identify faults or abnormities in the power dis-tribution network as well as whole power system. The main advantage is that it can be used to deal with the fault and analyzing the reason behind [57]. El-Zonkoly, A.M. et al. suggests multi agent based fault detection method. In this technique he suggests relay agents at the borders of the network sections to measure the current signal and to recognize the type of fault if it is three lines to ground (3LG), single line to ground (LG), double line to ground (DLG) or a line to line (LL) fault and this is determined by taking the sum of absolute entropies of wavelet coefficients of the Clarke components to determine the type of fault if it is 3LG, LG, DLG, or LL fault. Moreover, it also gives information such as phases comprised in fault and the bus or line at which fault has arisen by taking the sum of absolute entropies of wavelet coefficients of the three phase currents (Ia, Ib, and Ic). Finally, the values of the sum of absolute entropies of wavelet coefficients of Clarke components are used to locate the fault. The mother wavelet used was "Symlets" in addition to Shannon entropy [57].

MAS Based Power System Restoration
Feasibility study in application of AI techniques in distribution networks is being carried out by Bouhouras. S. A [59]. MAS has basic features such as reactivity, pro-activity, social ability and other features such as autonomous, mobility, truthfulness, rationality, ability of learning [20]. As per the above discussion the conventional power grid system is built in a centralized and radial topology where power is generated and delivered from one end to the other. Unidirectional power flow analysis which is used in conventional approaches will no longer be effective to control novel renewable energy sources implemented at the consumption sector efficiently; new strategies are called for to facilitate the bidirectional power flow incurred by power production of the distributed energy resource units. Intelligent distribution automation will be required by means of decentralized power management as well as information and communication technologies to actualize smart grid modernization [60]. Therefore, MAS based power system restoration method has become more popular.

IEC 61850
The IEC 61850 standard permits for a structured grid automation system where acute tasks like grid protections, renewable energy sources management, synchronized measurements, and monitoring applications share the communication network [61] [62]. Papers related to IEC 61850 and MAS clearly describe how MAS can be used in the distribution system automation in accordance with IEC 61850. The solution is based on the amalgamation of IEC 61850 object based modeling and interoperable communication with IEC 61499 function block executable specification [63]. Using the developed simulation environment, Vyatkin, V. et al. demonstrates the possibility of multi agent control to attain self-healing grid through collaborative fault location and power restoration.

Jade
Dealing with software agents is done with agent oriented software and programming and this involves using agents as main unit of encapsulation [64]. There are several frameworks such as JADE, FIPAOS and Zeus which is all open sources JAVA based frameworks. A number of comparisons can be found from literature [65]. Most of features of JADE don't stand out in comparison to FIPAOS and Zeus. Therefore JADE is more popular than other two. JADE (Java Agent Development Framework) is a software framework which make easy the development of multi-agent applications in obedience with the FIPA specifications [4]. JADE can then be considered a middle-ware that put into effect an efficient agent platform and supports the development of multi agent systems.

Architecture
As per the definition of MAS, it is all about the coordination of a large number of agents to achieve a global objective [66] where several architectures are being developed for software agents. But when it comes to the topic of power system restoration limitations such as communications and requirement of high performance devices will create some constraints.
Two main architectures can be found from literature such as Hierarchical Model and Centralized model depending upon the communication and coordination strategy. Figure 1 shows three different architectures. As summarized in Table 1, there are both advantages and disadvantages associated with each architecture.   Communication and coordination will be done through a central agent.  All agents will be in the same level.  Defined hierarchy will be there and agents will only communicate with neighbors.  All agents will be in different levels.
 Simple in both processing and communication  High reaction time  Difficult to find the optimum solution due to unavailability of information Hybrid Model (3D)  Features of both centralized and hierarchical are included. Since it has both dimensions it can also be called as a 3D model.  There will be several levels so that at least one level will be with more than two agents.
 Less reaction time  Simple in both processing and communication  Bit complexin both processing and communication.  Disadvantages from both are included.

Centralized
In this model in Figure 2, there is a central agent that will coordinate and communicate with each agent and such a strategy usually contains a single central coordinator to control the entire system for system management, maintenance, and restoration [67] [68]. The speed of the process will be much higher than the hierarchical method but the need for high performance central computer and data jams could be disadvantages of this method. At the beginning this centralized coordination strategy was more popular than decentralized coordination where a decentralized coordination strategy was suggested to overcome such limitations [69] [70].

Hierarchical
This is the most conventional and used architecture shown in Figure 3. In most of the papers this model is being used as the architecture where the information gathered by a lower level agent is fed in to the upper level agent [71] [72]. The main advantage of this method is that the complexity of communication is very low. The data jam becomes low because sending of messages to the same agent by several agents will be avoided. The main disadvantage is that because of the hierarchy time consumption among communications will be increased and the  overall performance will be slower.
The other main disadvantage: since they only communicate with neighboring agents they only have immediate neighbor's information and therefore it is difficult to find the optimum path because of incompleteness of information.

Hybrid
Hybrid MAS structure shown in Figure 4 can take the advantages of both centralized and decentralized approaches and reach the optimal restoration plan [73]- [76]. In hybrid method they can make better decisions and since they are not just relying on centralized data, even failures in accessing the centralized data do not affect the MAS much [77]- [80].

Optimization
The new trend of MAS is optimizing the solution and optimization of architecture with genetic algorithm, swarm optimization and etc… Though a number of architecture is being introduced optimum solution cannot be achieved at the first instance to find out the optimum restoration path considering voltage levels, line capacities, transformer capacities and etc… Therefore optimization algorithms are being introduced recently [81].

Genetic Algorithm (GA) & Particle Swarm Optimization
From the literature it can be found new directions towards GA to be used in MAS [3] [82] [83]. Generally, GA has solid capability of optimal search. It has been used in the service restoration even with some shortcomings in MAS genetic algorithm. It can be used to hunt the best Pareto solution for the multiple objects service restoration problem, considering users' priority [82]. MAS GA is an amalgamation of MAS and GA and it is easy to realize. An individual in GA is treated as an agent having the capability of part perception, rivalry and selflearning technique. The aim of overall optima comes real through interactions between agents and environment as well as in the middle of agents. The competitive and self-healing comportments of agent are reflected by characters of survival of the most appropriate, diversity and self-learning [82] [83].
PSO shares several likenesses with evolutionary calculation methods such as Genetic Algorithms (GA). The system is initialized with a population of random solutions and searches for optima by updating generations. However, unlike GA, PSO has no evolution operators such as border and mutation. In PSO, the prospective solutions, called particles, fly among the problem space by following the current optimum particles [84]. It is also a new wing of MAS to use PSO.

Immune System Algorithm
These algorithms mostly are trying to solve the restoration problem by decentralized processing to survive in case of single point of failures. In this case although agents have access to their immediate neighbor's information optimal solution cannot be reached because of incomplete information which is necessary to reach such decisions. The immune system and the human body can be considered as the best example for a highly intelligent multi agent system. The immune system is the defender if human system formed by millions of cells. Multiagent based systems are now being implemented based on the same principle as the immune system of a human body.
Tang, X. et al. suggests a multi agent system which is capable of fault detection isolation and to discover an optimal path to restore power. In the search for optimal path, the algorithm considers constraints such as voltage violations, line capacities, and performs load shedding if necessary.
Similar to the immune system in a human body the suggested system also has the continuous learning capability through the integration of the power system [85]. This is the most important characteristic of immune system in a human body which is capable of learning and memorizing over the time. By trying to mimic some of the behaviors of immune system MAS for power reconfiguration and restoration can be designed.

Conclusion and Future Direction
Due to increments in complexities in the power system, MAS has become a dominant research wing. In this paper, applications of MAS in power system are identified. Identified research wings from derived taxonomy are then briefly described where power system restoration is a main topic. From the above analysis, it can be concluded that MAS will be a solution for emerging complexities with bidirectional power flows in the power system. In the later part of the paper, a systematic Literature Review of Multi Agent Based Distribution Restoration which includes a number of different approaches. Based on the analysis, the methods used for fault identification are discussed. The main different MAS architectures are identified and their advantages and disadvantages are analyzed. From identified architectures and features, it can be concluded that innovative architectures are desirable to be developed. The development of hybrid architecture will be more expedient especially for young researchers. Most importantly from this study it has been identified that more research on the optimization of MAS based restoration are needed today. There is a vacuum in this sector for optimizations.
For future directions, more research is needed for optimization of MAS based restorations and architectures. Optimizations like ant colony optimizations, Bacterial Algorithms are necessary to be embedded with MAS to accomplish the optimal solution. In addition, controlled charging of EVs using MAS to control load profile will be a future research wing. Further interconnection of MAS based restoration and MAS based EV charging systems can also be a good research topic.