Multi-ACO Application in Routing and Scheduling Optimization of Maintenance Fleet (RSOMF) Based on Conditions for Offshore Wind Farms

Reducing the operation and maintenance (O & M) cost is one of the potential actions that could reduce the cost of energy produced by offshore wind farms. This article attempts to reduce O & M cost by improving the utilization of the maintenance resources, specifically the efficient scheduling and routing of the maintenance fleet. Scheduling and routing of maintenance fleet is a non-linear optimization problem with high complexity and a number of constraints. A heuristic algorithm, Ant Colony Optimization (ACO), was modified as Multi-ACO to be used to find the optimal scheduling and routing of maintenance fleet. The numerical studies showed that the proposed methodology was effective and robust enough to find the optimal solution even if the number of offshore wind turbine increases. The suggested approaches are helpful to avoid a time-consuming process of manually planning the scheduling and routing with a presumably suboptimal outcome.


Introduction
The demand for energy in general and renewable energy in particular is growing, and one important source is wind energy with 486.749GW installed capacity at the end of 2016 [1].It has overtaken hydro energy in 2015 as the third largest source of power generation in the EU with a 15.6% share of total power capacity which has been increased remarkably from just tiny 2.4% in 2000 [2].The European Wind Energy Association (EWEA) estimated that by 2020, 230 GW of wind capacity will be installed in Europe and 735 GW will be installed by 2050 [3] [4].If the trend remains, it is no doubt that the target of 20% wind energy share of energy production in Europe will be met in 2020 [5] [6].However, the target of these installation capacities cannot be reached without a large-scale offshore wind development in increasingly remote and hostile locations.Today, large wind turbines (2 -10 MW) for both onshore and offshore from some wind equipment OEMs, such as Siemens and Vestas, are designed, tested and manufactured as economically viable alternatives to traditional fossil-fueled power generation and other renewable resources.Nevertheless, in the environments of offshore, installation is more difficult and expensive, and access to the wind farms for maintenance is also limited, and thus the Operations and Maintenance (O & M) cost can be very high which has been estimated at up to 20% -30% of overall lifetime costs for energy [7] [8].It is really making sense to reduce the O & M cost by optimizing scheduling and scheduling of maintenance fleet for offshore wind farms.
Because of harsh environments of offshore wind farms and spread of offshore wind turbine installations, the O & M costs can be estimated to 5 to 10 times more expensive than that of onshore wind farms [9] [10].In addition, the conditions for each offshore wind turbine may be very different based on SCADA system or other condition monitoring system for offshore wind farms.Since the difference of individual offshore wind turbine conditions, the maintenance actions to these turbines should be also different, such as inspection, repairing, replacement, or no service demand.Considering the condition of turbines, the weather windows, the availability of vessels and all other relevant factors, the maintenance scheduling and plan becomes very difficult and time-consuming.It is very practical and profitable for wind farm owner and other related organizations to develop automatic techniques for offshore wind turbines maintenance scheduling with multi vessels.
The importance of effective utilization of the support resources is being recognized, but there are few relevant works and a comprehensive solution has been found.Van Bussel and Schöntag developed the simulation program CONTOFAX [11] for determining the availability of a specific offshore wind farm.The program uses a Monte Carlo approach to simulate random weather windows and random failures of offshore wind turbines (OWTs).One type of service vessel needs to be specified in the program with travel time to OWTs, as well as the number of personnel and shifts.The program can evaluate the influence of different service vessels on the availability of the offshore wind farm and the O & M cost [12].However, the detailed deployments of the service vessels and personnel, and assignment of OWTs and routing decisions, are not enabled.[13] applied operations research to the optimization problem of fleet size and mix for an offshore wind farm, supporting the decision making on when and how many service vessels to acquire or rent in order to meet a given maintenance schedule.Besnard, et al. [14] [15] presented opportu-Z.Y. Zhang nistic maintenance planning for offshore wind farms.Opportunity maintenance is a form of preventive maintenance based upon "convenient" replacement of equipment items or components by taking the advantage of the unplanned or planned shutdown of a system where have suitable maintenance resources already on location.However, none of them considered the routing optimization of vessel fleet.The fleet size and mix problem is a strategic decision which should be made early in the design phase of a project.In comparison, the current article aims to investigate an operational decision problem that is, scheduling and routing of a maintenance fleet for offshore wind farms.For the available service vessels determined by the strategic decision, optimal routes need to be found to transport the personnel and spare parts/equipment from shore to the offshore wind turbines that are in need of service.After the maintenance service, the personnel and equipment should be brought back to shore.

Gundegjerde and Halvorsen
Comparable studies for the RSOMF problem can be found in routing problems of supply vessels for offshore installations.In the offshore oil and gas industry, a set of installations regularly requires supplies from an onshore supply depot and returns used material/equipment.Aas et al. [16] explore such a routing problem and provide a mathematical formulation of this routing problem as a mixed integer programming model.Gribkovskaia et al. [17] continue the previous work and develop several construction heuristics, as well as a tabu search algorithm for the routing problem.The routing problem investigated in these 2 articles only considers 1 single supply vessel.For small size offshore wind farms [18], 1 vessel could be enough to provide enough maintenance service.However, for maintenance in big size offshore wind farms, usually more vessels are involved.This paper aims to investigate the maintenance optimization problem, i.e.RSOMF based on wind turbine conditions for offshore wind farms that can be used to avoid a time-consuming process of manually planning the scheduling and routing.The problem is NP-hard problem and thus exact methods are difficult to solve for more than 20 -50 OWTs.Heuristic algorithms generally can be used to solve NP-hard problem [19] [20].There are several algorithms, such as Dijkstra's Algorithm and Genetic Algorithm (GA), could be possible to solve the NP-hard problem.However, Dijkstra's Algorithm does blind search, and thus, wastes lot of time while processing, and in addition, it leads to acyclic graphs and most often cannot obtain the right shortest path (optimal solution) [21].GA is a stochastic and parallel search method, which can be used as an optimization technique for obtaining near-global optimum solutions of given problem [22].Generally, fitness evaluations by GA are often very expensive or highly time-consuming, especially for engineering optimization problems [23] and lack capability to explain why a particular solution was arrived [24].ACO is a swarm intelligence technology, which is very suitable to solve the routing problem, such as the travelling salesman problem (TSP) problem [25].ACO has already successfully applied in several applications, such as multi-UAV minimum time search with uncertain domains [26], irrigation scheduling [27], water Journal of Power and Energy Engineering alternating gas injection process [28], bridge inspection routing [29] and so on.All these problems were routing or scheduling related.RSOMF is a kind of routing problem with more complexity and more constraints.Therefore, Ant Colony Optimization (ACO) is modified as Multi-ACO to be applied to solve this problem.
The remaining sections of this article are organized as follows.Mathematical model of RSOMF is retrieved from a literature [30] [31] [32] [33] in Section 2. Section 3 proposes the algorithms of ACO and its modification Multi-ACO to solve the RSOMF problem, while some numerical studies are presented in Section 4. Finally, the conclusion is given in Section 5.

Mathematical Model of RSOMF
Essentially, the objective of the RSOMF problem is to achieve the cheapest maintenance operation in the defined time interval, which involves the costs on service vessels and production loss.All the vessels may have a fixed and a variable cost, similar to [13] for the vessel fleet size and mix optimization problem.The fixed cost stems from purchasing or renting the vessels.The variable cost depends on the operational expenditures, such as fuel.For routing and scheduling of the available vessels, the fixed cost is irrelevant for optimization.Therefore, only the variable costs are considered in the mathematical model.
There are n OWTs with the index of i.Therefore, the index is i in delivery process while the index is n + i when the vessels pick up the personnel and return to harbor.All vessels start from harbor (index 0) and finally return to harbor (index 2n + 1) as well every day.The definitions of relevant variables and description of mathematical model are given as following:

Sets
All nodes here represent the turbines, which need kinds of maintenance based on the results of condition monitoring.
Z + : the set of pick up nodes, , in which 0 is the harbor in delivery process while 2 1 n + is the harbor in pickup process.V: the set of service vessels.T: the set of days in the planning period;  represents the length of the period.

PE i
C : the penalty cost per day for the delaying maintenance task on turbine i, which is related the difference between preventive maintenance and corrective maintenance cost, and additional production loss due to delayed maintenance.

M i
T : the time needed for performing the maintenance task on turbine i; i L : the weight of spare parts and equipment for maintenance on turbine i. i P : the required personnel number for maintenance on turbine i.

MAX vd T
: the maximum working hours on day d for vessel v, which is used as the weather limitation for different vessels.

MAX v
L : the load capacity of vessel v.

MAX v P
: the personnel capacity of vessel v.

LATE i T
: the latest day to perform the maintenance task on turbine i without incurring a penalty.

Decision Variables
1, vessel travels from node to node on maintenance day 0, otherwise the number of delayed days for maintenance task on turbine i.
the preventive maintenance action is carried out before the turbine is down.
vid t : the time at which vessel v visits turbine i on maintenance day d.
vid k : the total load weight on vessel v just after it leaves node i on maintenance day d.vid q : the total personnel number on vessel v just after it leaves node i on maintenance day d.

Mathematical Model
Constraints: 1, , , , , , , , , , , ( ) ( ) 0, , , , , ( ) 0, , , , , ( , , , The objective function Equation ( 1) minimizes the sum of the sailing cost and the penalty cost associated with not performing maintenance within time window.Other fixed costs, such as personnel cost, are not presented in this function since they will happen anyway and the costs are fixed which would not affect the optimizing process and result.The means of the constraints are listed as follows [33]: 1) Equation ( 2) ensures that each OWT is visited only once for delivery and once for pick up.
2) Equations ( 3) and ( 5) ensure that each vessel leaves and returns the harbor only once every day.
4) Equation ( 7) means that if the vessel needs to present during the maintenance operation on 1 OWT, it will only leave the OWT when the operation is completed.
5) Equation ( 8) is precedence constraints which force the pickup is not done before completing the maintenance operation on the same OWT.
6) Equation ( 9) is soft constraints, which require that the maintenance task is performed within the preferred time.
7) Equation ( 10) keeps the travelling time compatibility of each vessel.
8) Equation ( 11) ensures the service vessels are not overloaded.9) Equation ( 12) expresses the compatibility requirements between routes and vessel loads.
10) Equation ( 13) ensures that no extra load added when the vessels pick up from OWTs.
11) Equations ( 14) and (15) describe the compatibility requirements between routes and personnel number on the vessels.
12) Equations ( 16) and ( 17) guarantee that neither of load or personnel number Z. Y. Zhang exceeding the vessel limitations.
13) Equation ( 18) imposes a maximal working time of the service vessels on each day.
14) Equation ( 19) means the time is counted from the vessels leaving the harbor.
15) Equation ( 20) set the delayed maintenance day to be non-negative.

Ant Colony Optimization (ACO)
Ant colonies can accomplish complex tasks that far exceed the individual capabilities of a single ant [34].ACO is a meta-heuristic technique which is inspired by the foraging behavior of the ant species [35].The ACO model was applied firstly to solve the Travelling Salesman Problem (TSP).The 2 main phases of the algorithm constitute the solution construction and the pheromone update.For TSP, m ants concurrently build a tour and select cities randomly at the beginning of the tour construction.At each construction step, ant k decides which city to visit next according to a random proportional rule.The probability with which ant k, currently at city i, chooses to go to city j is [34] [36]: , if where ij τ is the pheromone deposited on ( ) , which represents the visibility of city j towards city i which is inversely proportional to the distance ij d , α and β are 2 parameters which determine the relative in- fluence of the pheromone trail and the heuristic information, and k i N is the set of cities that ant k has not visited yet [34].
The pheromone trails are updated after tours constructing by evaporating at a constant rate and accumulating with new deposits: where 0 1 ρ < ≤ is the pheromone evaporation rate and is the amount of pheromone that ant k deposits on the arcs it has visited, defined as follows: ( ) where k C is the length of the tour T k built by ant k.By using this rule, the probability increases that forthcoming ants will use this arc.

Multi-ACO
ACO is a meta-heuristic technique which is a very good algorithm for solving optimization problem typically Travelling Salesman Problem (TSP) as mentioned above.In classical TSP problem, there are many cities and only 1 salesman.If there are 2 or more salesmen to travel all these cities and each city can and only can be traveled once, how to solve this Multi-TSP problem?The Z. Y. Zhang RSOMF problem may have 2 or more vessels which is very similar with Multi-TSP problem.This section describes the principle of Multi-ACO which is a modification from basic ACO as described in Section 3.1.
The idea of Multi-ACO is evolved from classical ACO.The procedure of classical ACO Algorithm solving TSP problem can be written as Figure 1.Multi-ACO has 2 or more groups with the same number of ants and each group has its own pheromone (group 1, group 2 … and pheromone1, pheromone 2 … accordingly).The procedure of the algorithm can be written as Figure 2 assuming that there are n salesmen, i.e. n groups of ants.The relevant functions used in ACO and Multi-ACO algorithms have been described in section 3.1.In each iteration of Multi-ACO, the ants with same index (k) in different groups visit nodes (cities) according to their own probabilities using roulette wheel selection principle.After all nodes are visited by ants with same index (k) for all groups, the pheromones are updated for all the groups, and then move to next index ( 1 k + ).After all nodes are visited by ants of all groups, the iteration number increases by 1 until maximum iterations or other termination criterion.Then the best routes of n groups with the same index are recorded as the best route.

Numerical Studies
In this section, we present the computational results obtained from solving the mathematical model presented in section 2.4 using the Multi-ACO algorithm presented in section 3.2.Figure 3 shows the illustration of the OWTs' locations in an offshore wind farm.Of course that the turbines can be in different wind farms.However, for simplifying, only turbines in 1 offshore wind farm are considered.Based on condition monitoring or performance monitoring of wind turbine, the turbine conditions may be very different which lead to different maintenance requirements: replacement, repair, or no service demand (inspection).Accordingly, the requirements of personnel, load of spare parts, maintenance time are different for different OWTs.
For offshore wind farms, the maintenance fleet can consist of different vessels with different speeds and costs.Each vessel has a limitation with respect to the vehicle capacity such as load capacity of spare parts, and number of personnel the vehicle can take.2 type of service vessels for offshore wind farms are extracted from [37] which are listed in Table 1.In the following 2 subsections, 2 cases will be presented based on the locations presented in Figure 3 and the parameters of service vessels presented in Table 1.Comparison studies are also presented in the final subsection to show the advantages of proposed algorithm comparing to commercial software.3 randomly as the first case.The parameters of these turbines are shown in Table 2. Time window in the table means that the maintenance action must be taken before the number of days passed or the penalty cost will be applied due to do the corrective maintenance instead of predictive maintenance.Penalty cost per hour may be varied but using fixed number for simplifying.Load and personnel requirements are weight of spare parts and the number of employees needed to carry out the maintenance action.Task duration means that the number of hours needed to fix the problem of the offshore wind turbines.Table 3 shows the maximum hours the service vessels can work mainly based on the weather forecasting and condition of vessels.The maximum working hours may be different or same for different service vessels.For distinguishing, we assume that the maximum working hours for each day are different.
The process of the program can be seen in Figure 2.There are 2 groups of ants and each of them represents a vessel.The routing of each group of ants represents the routing and scheduling of maintenance for 1 vessel.Based on the relevant research [38] [39] about the tuning of ACO and several trials, the number of ants in each group should be approximately the number of nodes the ants be visited which are the number of offshore turbines in this case.Therefore, the parameters of Multi-ACO are set as: number of ants of each group is 10, the maximum iteration is 200, important coefficient of pheromone α and β are set as 1 and 5 respectively, and the pheromone evaporation coefficient is 0.1.
The results of maintenance scheduling and routing with 8 offshore turbines are shown in Table 4.It is clear that only vessel of SWATH is used for maintenance service for the 8 turbines due to that it is cheaper and faster than Vessel Smit Bronco.In this table, D means delivery process while P means pick up (return) process.The final objective value is 2436.0 and its changing with the iteration is shown in Figure 4.
Figure 5 shows the illustration of scheduling and routing for 8 offshore wind turbines.At first glance, the routing seems not the optimal one for pick up processes in Figure 5(b) and Figure 5(d).However, if we consider that T61 in Figure 5(b) will be the last one to be completed the maintenance action and thus the personnel in this turbine should be the last ones to be picked up, and the same apply to T27 in Figure 5(d).

Computational Study with 36 Offshore Wind Turbines
In order to examine the Multi-ACO performance for a large number turbines' Z. Y. Zhang Journal of Power and Energy Engineering wind farm, a new offshore wind farm with 36 turbines that are selected from illustration in Figure 3 which are all turbines with kinds of service requirements.The information of 2 vessels is the same as shown in Table 1 and the maximum working hours for each day is the same as Table 3.The conditions and parameters of 36 turbines are not presented because of too much data.This would not influence the target of the case, which is to test the performance the Multi-ACO performance for large number of turbines.The parameters of Multi-ACO changes because of the increasing the number of wind turbine.The number of ants of each group is set as the same number of wind turbines, i.e. 36, and the maximum iteration is set as 300.The results are shown in Table 5 and Figure 6.The vessel SWATH and vessel Smit Bronco visit and repair, replacement 19 turbines and 17 turbines respectively.Vessel SWATH need 4 days to visit and maintain all these 19 turbines, and it needs 5.0581, 5.2515, 5.2674, and 6.6055 hours for each day which are less than that of the maximum working hours of this vessel in Table 3. Vessel Smit Bronco need 3 days to visit and maintain 17 turbines, and it needs 6.9796, 7.1976 and 7.1590 hours for each day which are also less than that of maximum working hours of this vessel in Table 3.The objective value of fitness function of Equation ( 1) is 11,391.89. Figure 7 and Figure 8 show the details illustration of the routing and scheduling of these 2 vessels.It is easy to see that vessel SWATH visit more turbines than vessel Smit Bronco even using longer time (1 more day).The reason is that the more turbines visited by vessel SWATH, the better (lower cost) of the routing and scheduling as long as the turbines can be visited (for repairing and replacement) within the time windows.It is common to get this type solution by Multi-ACO, which is one of the benefits comparing to scheduling and routing manually.

Comparison with Xpress ®
This section compares the proposed algorithm with existing commercial soft-  [42].Same cases are used as in [33] and the results between Xpress® and Multi-ACO are compared in Table 6.
For simplicity, "0" in Table 6 represents the node "Harbor", the first time the turbine number presents means the vessel delivery the personnel to this turbine, and the second time of the same number means the vessel come again to pick up the personnel to harbor.We can see from Table 6 that both methods can solve the problem effectively.However, almost all the results of Multi-ACO are better than existing commercial software, which shows the advantage of proposed algorithm.

Conclusions
This article proposes Multi-ACO algorithm to solve the scheduling and routing The numerical studies show that Multi-ACO is suitable to solve the RSOMF problem with multi vessels and multi wind farms with a number of constraints.
It can find relative optimal scheduling and routing of maintenance flee to reduce the O & M cost and the production loss and ensure the safety and reliability of offshore wind turbines at the same time.Even with big number of offshore wind turbines, Multi-ACO also shows its effectiveness and robust to find the optimal solution.
With the increasing of the number of offshore wind turbines, the time of Multi-ACO to find the optimal solution also increases.However, the time to find the solution is not very sensitive, because that comparing with the long-time maintenance scheduling and routing (days), some minutes or hours the time to be used is acceptable.
One issue of the heuristic algorithm is that no one can guarantee the solution is the best globally.What we can say is that the solution got by heuristic algorithm is relatively optimal.Multi-ACO has the same problem, which means that Z. Y. Zhang the scheduling and routing solution may be not the best but a relatively optimal solution.One way to improve the solution is that to run the program many times and choose the best one.
Even though this article has already achieved good solution, there are still some directions that can be extended.Even though the methodology can be used for many vessels, the case studies in this paper only consider two vessels.More vessels should be considered for future work.Weather constraints on the vessels are simplified as static feasible working hours.In further research, more accurate weather forecasts can be used.For example, some wind farms in north part of Norway are only accessible in summer time which is different from the case in this paper.The maintenance tasks modeled in this paper are limited to "light" tasks, excluding replacement of big sized components which requires special vessels, such as crane ships.These "heavy" tasks should be considered in the future work.

vijC
: the traveling cost of vessel v from node i to j. vij T : the time (hours) for vessel v traversing arc ( ) , i j .

Figure 1 .
Figure 1.The Implementation Steps of ACO.

Figure 2 .
Figure 2. The Implementation Steps of Multi-ACO.

Figure 3 .
Figure 3.The Illustration of the Location of Offshore Wind Turbines.
problems of maintenance fleet for offshore wind farms.The mathematical model of the RSOMF is retrieved from the existing literature.The aim of this article is to reduce the O & M cost for offshore wind farms by improving utilization of maintenance resources.The model involves an objective function and a number of constraints and thus is a combinational optimization problem with high complexity, which is very difficult to be solved by deterministic methods, and thus a heuristic algorithm, i.e.ACO, is modified as Multi-ACO to be used to solve this problem.

Table 1 .
Reference values for 2 type service vessels.

Table 3 .
Maximum working hours for each day.

Table 4 .
Results of scheduling and routing with 8 turbines.