Railway Transport Infrastructure Monitoring by UAVs and Satellites

Improving the rail transport security requires development and implementa-tion of neoteric monitoring and control facilities in conditions of increasing speed and intensity of the train movement and high level of terrorist threat. Use of Earth remote sensing (ERS), permitting to obtain information from large areas with a sufficiently high resolution, can provide significant assistance in solving the mentioned problems. This paper discusses the possibility of using various means of remote sensing such as satellites and unmanned aerial vehicles (UAV), also known as drones, for receiving information in different ranges of the electromagnetic spectrum. The paper states that joint using of both these means gives new possibilities in improving railroad security.


Introduction
The railway transportation safety demands constant attention from the staff and management of company that operates the facility. This applies to the locomotive drivers' condition monitoring, as well as to all services related to ensuring the smooth operation of the railway: manifestation that is called the human factor. These problems can be solved by a set of organizational and technical measures that improve discipline and working conditions of employees.
Unfortunately, there are still some technical factors, such as violation of the railway line integrity (rail rupture, destruction of railway arrow, etc.), which also require permanent monitoring. The another thing to mention is a problem of countering terrorist acts with the use of improvised explosive devices, which can lead to even more dire consequences. The use of remote sensing means such as piloted helicopters and drones could render substantial assistance in determining the scale of a railway catastrophe and ways of its elimination, including assistance to possible victims [1] [2] [3]. There is one more task connected to the topic of improving the safety of rail transportation. These are incidents related to natural phenomena including mudflows and avalanches, heavy snowfalls in the plains, and some others. Remote monitoring such phenomena could effectively help in its prediction and consequences management. To monitor and control the state of snow cover on the mountain slopes in avalanche prone areas, the UAV could be successfully used for reconstructing three-dimensional images of the terrain in the summer (without snow cover) and in the winter after heavy snowfalls. After comparison with such images. it will be possible to determine the height of the snow cover, and UAVs equipped with passive radar (radiometer) operated in centimeter or decimeter wavelengths will provide an opportunity to measure snowpack humidity [4]. These data could be added by measurements obtained from a ground-based radar network [5]. The combination of these factors (stockpile and humidity of snow) will allow creating a fairly accurate forecast of avalanches probability.
The railway is a transporter of dangerous consignments such as explosives, perilous for environment and poisonous chemical compounds, combustible and flammable materials. In case of an accident, it is necessary to promptly assess the scale of the disaster and make decisions on the prompt elimination of its consequences. The described examples widely show possible applications of UAVs and other remote sensing tools, including earth observation satellites, which can be used to ensure the safety of rail transportation.
The main goal of current paper is a comparison of the satellite and drone effectiveness in recording images of railway infrastructure in visual and microwave frequency range, and gives recommendation in their joint using. This matter was not analyzed before and main emphasis was mostly done on drones' applications in infrastructure inspection [6].

Advantages and Disadvantages of Remotely Piloted Aircrafts
Recent years have been characterized by an increasing use of UAVs with the expansion of tasks ranges, which are solved with their help. While in the past UAVs were used exceptionally for military purposes, now their civilian use is becoming extremely important. Military original interest to drones, which arose in the late 60s, was stipulated by a desire to avoid the risk for the crew lives during reconnaissance flights. The disadvantage of these drones was the need of returning to "home" territory after the flight mission for the treatment of the recorded photographic films and decryption of the material. That reduced the relevance of obtaining information.
Hereinafter, the most successful development of UAVs was implemented in Israel (tactical and operational UAVs) and the United States (tactical and strategic UAVs). Progress in the designing a new generation of reconnaissance UAVs is primarily associated with the emergence of digital photography, including cameras functioning in different wavelength ranges, navigation and communication through satellites. This significantly improved the performance of on-board equipment, especially the weight and size, and allowed improving the parameters of the UAV such as endurance, range and height of flight. It should also be noted that the cost of the UAV has sharply decreased, and access to satellite information has been improved in recent years.
Modern drones for civil purposes are characterized by low operation cost and no need for stationary airfields with paved runways. As a rule, they are launched from simplest ground airfields, mobile catapults or directly from the operator's hand. Remarkable progress has been achieved in the development of multi-rotor drone helicopters with electric power. UAVs of this type are characterized by simplicity of design and affordability. This is mainly due to the lack of such an essential for conventional helicopters a structural element as rotor hinge system, and the use asynchronous control of electric motors traction, that allows running the movement of the device in flight. Figure 1 shows a six-rotor UAV helicopter, which was used to obtain images of the Experimental Railway Ring (ERR) in Shcherbinka, Moscow region, in current project. Advantage of this type of UAV is also the fact that now their use in the exclusion zone of the railways at altitudes of less than 100 m does not require special permission for flights.
The disadvantage of multi-rotor electrical UAVs is a relatively short flight time usually less than an hour, and consequently restrictions on range and altitude of the flight. However, this is not essential when the operator controlling UAV flight is directly placed on the action place or near it. In this case, information about objects and events happening on earth surface can be obtained in real time and is compared with operator's own vision, what is essential, for example for making decisions in the elimination of railway accident consequences. An increase of the drone flight time can be achieved by replacing the batteries during intermediate landings, although the range limit remains, and it is still sometimes a significant factor.

Analysis of Satellite Resources for Obtaining Information
In the 1960s and 1970s the main users of photographic and radar satellite information were military, and this information itself was classified, but in recent decades a significant segment of consumers is the highly profitable civilian sector. In the first instance to the tasks to be solved by Earth observation satellites the following items should be included:  Research of natural resources;  Prediction, analysis and control of emergencies and their consequences;  Geological exploration;  Weather forecast;  Control of the environment and its sources of pollution;  Agriculture and prediction of harvest;  Forestry and forest protection from fire and pests;  Construction and other industries. The main advantage of the remote sensing optical satellites is the ability to Journal of Transportation Technologies obtain high-quality multispectral images, which resolution can reach 0.3 m or lesser. Nevertheless, visual spectrum satellites are not able to function in night or cloudy weather. While the first shortcoming can still be potentially overcome, for example, by using infrared optics, the cloudiness, which in mid-latitudes in winter can hide the earth for weeks or even months, makes impossible using even infrared satellites. All-weather satellites are radar ones, even though they have the worst resolution. At the same time, radar provides additional opportunities, for example, obtaining images of the recorded signal in different polarizations or finding underground objects [7].

The Databases of Satellite Information
Currently

Comparison of Information Received from Different Sources
To compare satellite and airborne images of the railway transport infrastructure, the Experimental Railway Ring in Shcherbinka was selected. The choice of this object was related to the following circumstances. Firstly, ERR is well studied and located close to Moscow. Secondly, there were no difficulties with getting the permission for UAVs flights over the ERR territory because of its experi-Journal of Transportation Technologies mental status. Figure 2 shows the Experimental Ring image taken from the Google Maps database. The center of the ERR ring is tagged with an arrow, its coordinates are 55.521373, 37.550190 (Northern latitude and Eastern longitude in degrees accordingly). Inside the Ring, a lot of summer country houses are located. The Experimental Railway Ring itself includes two double-tracks rings: outer and internal. The outer ring has the shape of a regular circle, and the inner ring contains a straight section.
A rail depot is visible in the lower right corner of this picture. A rectangle marks flight area of UAV which photos are used in this article. The another arrow in the rectangle marks the location of a water tower, which served as a landmark for UAV's flights.
The enlarged image of the water tower obtained from the satellite is shown in Figure 3. This is maximal scale that the Google Maps site allows. Roof of the tower has a conical shape and is covered with galvanized steel sheets. Joins between the sheets have the form of vertical ribs. The ribbed shape of the roof is visible even in the satellite image.  The same area near the water tower was shot from the board of the UAV equipped with a Canon EOS 5D Mark II camera with EF-S 17-55 f/2.8 IS USM lens. Camera was mounted on a gimbal controlled by an operator, Figure 1. The result of the photographing from flight altitude about 70 m is shown in Figure 4. The difference between these two images consists in the location of the rail wagons on the tracks near the water tower that caused by different time of the shooting. The satellite image was recorded in summer, and drone flights were performed in autumn. They also differ in size and direction of the local objects shadow, as the images were taken at different positions of the sun on the celestial sphere. This fact stresses the importance of sun-synchronous orbits, which were mentioned earlier and are commonly used for remote sensing satellites. Another advantage of using sun-synchronous orbits is the ability to determine whether an object is being built or not, and at what speed it is being built, judging by the length of the shadow at the images obtained from different orbit passes of the satellite. Usually this method is used for the interpretation of images recorded by military reconnaissance satellite, but it also can be used for cadastral surveying, when it is necessary to allocate from a large array of images new and illegally erected buildings. In that case, to determine the newly erected buildings and their height change it is enough to subtract the images.
If we compare the resolution of the two above photos, it is obvious that the UAVs image has a much better resolution than satellite one. This is clear since the altitude of the orbits of remote sensing satellites is about 600 km in comparison with the same parameter for drones that equals a few hundred meters or several kilometers. UAVs have another advantage over satellites because they are less dependent on weather conditions. In the presence of cloudiness, it is impossible to shoot in the visible spectrum from satellites. While UAVs can hover below the lower cloud boundary and take pictures of the terrain. As mentioned earlier, only radar imaging is weather-independent. Two low-resolution images of the Experimental Ring district obtained from the Google Maps website and the European Sentinel-1 radar satellite database (Sentinel-1, https://sentinel.esa.int/web/sentinel/missions/sentinel-1/observation-scenario/ar chive) are presented in Figure 5.
At the lower part of both images Ostafievo airport (left in the pictures) and ERR (right) are seen. Both presented images have approximately the same scale, but the visible-range image contains more detail than the radar one. It is worth mention that radar image clearly depicts runways of the airport. The scale of both images can be easily estimated, because a diameter of the outer circle of the railway testing area is 1912 m.
The main advantage of radar, as it was mentioned earlier, is the ability to record images during the whole day and in all-weather conditions. In some cases, radar images may be the only source of information. In this regard, it can be concluded that to solve various problems in the interests of railway safety, it is important to have access to different sources of remote sensing information.

Reconstruction of 3D Models of the Railway Infrastructure Objects from a Set of 2D Images
Modern image processing software allows 3D environment model reconstructing from a set of 2D images. This gives an ability to view objects from different angles and heights. Such technology can be used in designing complex objects of the railway infrastructure (bridges, tunnels, etc.), as well as in solving problems of their safety. In the recent time, the increasing interest in using 2D aerial images, collected by the UAV, in 3D model reconstruction is evident. It is very important to note that the UAV simplifies the process of taking images because operator can set the trajectory of the vehicle individually [8]. The multiple images are acquired by on board high resolution camera from multiangle and multiposition during the flight of the UAV. The essence of an image is a projection from a 3D scene onto a 2D plane, during which process the depth is lost. The 3D point corresponding to a specific image point is constrained to be on the line of sight. From a single image, it is impossible to determine which point on this line corresponds to the image point. If two images are available, then the position of a 3D point can be found as the intersection of the two projection rays. This process is referred to as triangulation. The key for this process is the relations between multiple views which convey the information that corresponding sets of points must contain some structure and that this structure is related to the poses and the calibration of the camera. The task of converting multiple 2D images into 3D model consists of a series of processing steps which brief overview is given here. For a more extensive treatment the reader is referred to textbooks such as [9] [10].
Camera calibration consists of internal (focal length, coordinates of the principal point, distortion coefficients) and external (position and orientation) parameters, without which at some level no arrangement of algorithms can work Internal parameters are independent of outer conditions and do not change from one snapshot to another snapshot. In most situations, the internal parameters are calculated by shooting calibration objects, the geometry of which is pre-known, but also it can be computed without their use by applying a large number of images with common points [9]. External calibration is performed by triangulating key points common in two or more images.
To synthesize a 3D image, the problem of finding a match between key points in different images has to be solved. This task is accomplished by searching for points with minimum different descriptors-vector parameters describing the point. Thereby the SIFT algorithm describes each key point by a vector of 128 parameters identified on the basis of local gradients in the vicinity of the point [11].
One of the most efficient algorithms for filtering false correspondences of certain key points is RANSAC, Random Sample Consensus, [12]. The essence of the mentioned method is iterative search for the best transformation matrix (fundamental matrix) between N randomly selected key points in the first image and the corresponding points in the second one. From the initials set of key points RANSAC algorithm iteratively removes such points, deviation of which as a result of the applying the found transformation from the corresponding points in another image, exceeds the specified threshold.
Depth determination serves as the most challenging part in the whole process, as it calculates the 3D component missing from any given image-depth. The correspondence problem, finding matches between two images so the position of the matched elements can then be triangulated in 3D space is the key issue here. Once the multiple depth maps are determined they have to be combined to create a final mesh by calculating depth and projecting out of the camera. Camera calibration will be used to identify where the many meshes created by depth maps can be combined together to develop a larger one, providing more than one view for observation.
Complete 3D mesh may be the final goal, but applying the color from the original photographs to the mesh is usually required. In this paper the "mosaic" approach [13] was used to create the final texture which implies two-step approach: it does blending of low frequency component for overlapping images to avoid seamline problem (weighted average, weight being dependent on a number of parameters including proximity of the pixel in question to the center of the image), while high frequency component, that is in charge of picture details, is taken from a single image-the one that presents good resolution for the area of interest while the camera view is almost along the normal to the reconstructed surface in that point.
As the initial data for the reconstructing a 3D model of the earth surface with the objects of the railway infrastructure located on it, 176 drone photos were used. A few examples of the original photos are shown in Figure 6.
The resulting 3D model of the surface under different camera angles and from different heights is shown in Figure 7.
A more complete picture of the obtained model is demonstrated in the video, which could be downloaded from webpage [14]. Some limitations of the data, which have been used for reconstruction of 3D image, should be noted:  A relatively small number of images;  Non-optimal flight path of the UAV;  Small overlap between images;  Absence of information about the orientation of the UAV and camera, as well as data on GPS/GLONASS coordinates. Nevertheless, the resulting model sufficiently accurate reflects the features of the area where the photography was made. The model was constructed using the mathematical software Pix4D Mapper (Pix4Dmapper, https://pix4d.com/product/pix4dmapper/).

Conclusion
The analysis of the possibilities of using remote sensing satellites and UAVs equipped with various sensors showed that they could be an effective monitoring tool for ensuring traffic safety on the railways. Their information also can be used for solving other tasks, for example in the design and construction of railways infrastructure. The paper shows that the effectiveness of the Earth remote sensing can be increased by combining information recorded by satellites and UAVs, and with the use of the modern image processing means, it becomes possible to build synthetic three-dimensional images of railway infrastructure.