Applying Photogrammetric Techniques to Determine Facade Decays: The Case Study of Zisa Palace, Italy

The aim of this paper is to present a non-destructive method for façade decay analysis and quantifying with many applications to architectural heritage (monuments, historical buildings, etc.). The proposed methodology is based on the combination of topographic surveys, digital photogrammetry and image processing techniques by means of four control points and one photography of the target façade, at least, that become metric rectified image at any plane defined by the user. It is possible to quantify any physical characteristic of the surface of a building as well as alterations on its surface that produce deviations from flatness. The Bundle adjustment was applied to a 2D reconstruction algorithm instead to a 3D reconstruction algorithm to control the flatness of a façade of a monument, the Zisa Palace in Palermo (Italy). Significant deviations of the points in the vicinity of the edge left of the façade with respect to the coordinates of the points of support were evidenced, indicating that the overall facade of the monument is not a flat surface. The results, with an estimated 10 accuracy, suggest that a structural failure in an area of the façade may be occurred after the restoration work or, alternatively, that such restoration was not performed successfully in that area.


Introduction
The improvement of surveying methods with photography of man-made structures (monuments, historical buildings, etc.) is an important contribution to architectural and cultural heritage recording, perceptual monitoring, information modeling, reverse engineering, 3-d modeling and documentation [1] to preservation and restoration of any valuable architectural or other cultural monument.
In this domain, to quantify decay is really a problem.In fact, in addition to describing the type of decay, it is essential that we are able to measure its extent, or the area it covers; its severity, or how advanced the decay is and the rate of decay over time, in order to decide whether remedial action is necessary.
Furthermore, we cannot establish whether our remedial actions are having any effect unless we can monitor the condition of the stone afterward.
No single technique is sufficient to measure stone deterioration, since decay takes many different forms.Some techniques, such as 3D laser scanning and fluorescence Lidar (light detection and ranging), look only at the surface, and they are well suited to decay that consists of a gradual loss of surface, leaving sound stone behind.Other techniques, such as ultrasonic measurements [2,3] thermography [4][5][6] or magnetic resonance imaging [7] are designed to probe below the surface, and these are useful where decay consists of a loss of cohesion within the stone, or the development of detached layers, blisters, or internal voids.
Nevertheless, before using more complex methods, photogrammetry plays an important role in quantifying decay [8].Photogrammetry is a technique for obtaining geometric information, e.g.position, size and shape of any object, that was imaged on photos before.
An improvement in traditional photographic documentation is a new method known as polynomial transform mapping that is, the use of multiple photographs from different angles to document more comprehensively the texture of stone surfaces.This gives the viewer the ability to control the angle of the light source in a given image using Java-based software [9].
To achieve a restitution of a 3D point one need the intersection between at least two rays (from photo to object point) in space or between one ray and the surface that includes this point.If more than two rays are available (the objects shows on three or more photos) a bundle solution is possible including all available measurements (on photos or even others) at the same time.
Most of the techniques for characterization are well established.Many of them are summarized by [10].In addition multi spectral imaging techniques have been used in the scientific analysis and documentation of objects, since images in different wavelengths provide information that human eye cannot see.
This paper shows the profitability of applying digital image analysis on material surfaces together with a data processing (the Bundle adjustment) in order to provide global and accurate information from visible photographs regarding the quality of a restoration.
The proposed method was applied to the Zisa Palace, that is one of the suburban residences built by the Norman kings of Sicily.
The building is a rectangular parallelepiped and the facades feature a series of large blind arcades.
The façade was partially destroyed by repairs in the fourteenth century, and the damage was increased by restoration work in the seventeenth century.The collapse occurred in line with the big staircase constructed in the seventeenth century, which jeopardized the resistance of the overall structural complex.
After the collapse of the north wing in 1971, substantial restoration work was undertaken, led by Giuseppe Caronia.The work of restoration started in March 1974 with the concern to conciliate the static requirements with those of a historical and aesthetic nature.The solution adopted was to cage the whole structural complex with a system of pre-stretched cables running vertically and horizontally inside the walls, and welded with the masonry by means of cement mortar injections [11].The innovation of the proposed method is the ease, rapidity and accuracy with which the flatness of the surface of a building can be checked, using a single frame with a semi-metric camera, some points of support and the Bundle adjustment data processing, highlighting any structural failure which produced deviations from flatness of the façade.

Digital Photogrammetry: Semi-Metric Camera
Unconventional photogrammetric techniques use photos which does not originate from a metric camera.
In fact with the use of semi-metric cameras the employment of réseau techniques in photographic cameras was established in the everyday work of architectural photogrammetry.The réseau, a grid of calibrated reference marks projected onto the film at exposure, allows the mathematical compensation of film deformations, which occur during the process of image acquisition, developing and processing.Semi-metric cameras with medium format offer a good compromise between a large image format and established camera technique.An overview on semi-metric cameras is given in [12].
The generic relationship between the image points and the object points can be represented by: We need at least 4 control points defined by their coordinates or distances in the object plane, for the determination of the parameters Lij.
Nevertheless, many parts of architectural objects can be considered as plane, such as a façade of a building.In this case, even if the photo is tilted with regard to the considered plane of the object, a unique perspective is enough to compute a rectified scaled image.
The Rolleimetric MSR software package, for example, provides scale representations of existing objects on the basis of rectified digital images.The base data is usually one or more photogrammetric images and/or amateur photographs of the object which are rectified at any planes defined by the user and are processed as a result of the rectification.
In the case of a perspective rectification, radial image displacements in the computed image will occur for points outside the reference.In the resulting orthophoto, the object model is represented by a digital terrain model [13].Hence, the shape of the surface is restricted to planes only and a minimum number of four control points in two dimensions have to be available.It can be shown that the relation of the object plane to the image plane is described by the projective equation of two planes [14]: where X and Y are the coordinates on the object's plane, x and y the measured coordinates on the image and a i , b i , c j (i = 1, 2, 3 and j = 1, 2) the 8 parameters describing this projective relation.The measurement of a minimum of 4 control points in the single photo leads to the evaluation of these 8 unknowns (a 1 , a 2 , As a result the 2D coordinates of arbitrary points on this surface can be calculated using those equations.This is also true for digital images of facades.Digital image processing techniques can apply these equations for every single pixel and thus produce an orthographic view of the object's plane, a so called orthophoto or orthoimage.
The digitization of photographic images offers to combine the advantages of film-based image acquisition such as large image format and geometric and radiometric quality, with the advantages of digital image processing.The raster files are transformed into object surfaces and digital image data is projected onto a three-dimensional object model.
Photogrammetric scanners are low cost and easy to handle and can be used for photogrammetric purposes.This holds especially for calibrated systems, where geometric accuracy in the order of 5 -10 μm is feasible [15].
Within the digital pre-processing stage some tasks were performed: image enhancement (histogram modification of individual images), multispectral image rectification by a software, image extraction and, finally, the construction of a digital mosaic from the set of overlapping rectified-images.

The Proposed Non-Destructive Analysis
Method: Façade Decays Quantifying The control points were 28 points of the main façade, that were detected by the topographic instrument Topcon ET2, on a basis at the distance of 25 m from the façade.
For the practical use of scanners in architectural photogrammetric applications the problem of necessary and adequate scan resolution has to be faced.On the one side the recognition of details has to be ensured and on the other side the storage medium is not unlimited.To scan a photographic film with a resolution equivalent to the film a scan of about 12 μm (2100 dpi) is required.
The pixel-imagine dimension and the pixel-real dimension were 0.070 mm and 15 mm (0.075 mm × 200) respectively.
A scanned image from a medium format film (6 × 6 cm 2 ) has about 5000 × 5000 pixel.To hold this data on disk required approximately 25 Mbytes for a black-andwhite scan and 75 Mbytes for a colored image.For a scanned color aerial image one would get a digital image of 1.2 Gbytes.
Assuming a graphic error of 0.3 mm, to which a value of 15 mm corresponds by an average scale 1:50 frames, a resolution of 360 DPI (in x, y) was used by a scanner Epson GT-8000.
The graphic output was obtained by a HP jet-ink.

Methodology
A relevant number of images of the monument was acquired by the semi-metric cameras Rolleiflex 3003 and Rolleiflex 6006.28 points of support at the different quotes labeled with P-I, P-II, P-III, P-IV, P-V in Figure 2 were chosen.These points were pointed out on Figure 2  The measurements were executed from three points located on a basis at 25 meters from the facade of the monument.
The image was rectified using a superabundant system of equations derived from the control points coordinates.The solution provided the least squares statistical parameters that allow an assessment of the reliability of the procedure for straightening.
The digital image was suitably scaled, rectified and  1.
In this work Bundle adjustment was applied to a 2D reconstruction algorithm, that is the software MSR.
resampled, so that it was used without further manipulation, using the MSR-CAD software that allows the manual factorization of the image.A reconstruction of this rectified image is represented in Figure 3.The parameters that were used are described in the previous paragraph.
The results of this application are reported on the last three columns of Table 1.
Significant deviations from the coordinates of the points of support were observed particularly in an area of the façade of the monument.These points were marked with the label * in the first column of Table 1.

Results
The values of the coordinates of the points of support were determined by the software MSR and MR2 and were compared with those obtained by the topographical methods.In particular, comparison between the coordinates provided from the software MSR and those obtained by the topographic survey was elaborated by bundle adjustment.

Accuracy
A statistical analysis was carried out using Student's t-test for comparison between two groups of points of support, the group of points having abscissa X > 30 m and the group of points having X < 30 m, with p-values less than 0.05 considered significant (X is the abscissa on the main façade of the monument).
Bundle adjustment is almost always used as the last step of every feature-based 3D reconstruction algorithm.It amounts to an optimization problem on the 3D structure and viewing parameters to obtain a reconstruction which is optimal under certain assumptions regarding the noise pertaining to the observed image features, minimizing the reprojection error between the image locations of observed and predicted image points, expressed as the sum of squares of a large number of nonlinear, real-valued functions [16].
The t-test applied to the group of control points with X > 30 m provided that the deviations of their coordinates are significantly different in comparison to the other group of control points with X < 30 m (p < 0.05), confirming that the main façade of the Zisa Palace cannot be considered a flat surface, and in particular, that an area of the façade lie on a plane different from that of the other area.

Conclusion
Non conventional photogrammetric method can be successful used to create metric photographs, output data obtained from them that are ready for measurements and fit to exact positions because output data are rectified and automatically classified images.The characterization of features by means of digital processing tools on any material surface offer a high degree of completeness and accuracy, because it is possible to obtain detailed positions of materials on facades, real measures of distances, to quantify failure or the quality of restoration.In this paper the profitability of applying digital image analysis together with a data processing (the Bundle adjustment) and a 2D reconstruction algorithm (the software MSR) showed global and accurate information about the facade of an ancient monument, the Zisa Palace in Palermo (Italy).In particular, significant deviations of the points of an area of the façade in the vicinity of its edge left with respect to the main door were detected.Hence, the main façade of the monument can be considered as consisting of two areas lying on two different planes.This result can be explained by a structural failure occurred after the restoration work done in the 70's, or alternatively that such restoration was not successful in that area of the façade.The innovation of the proposed method is therefore the ease, rapidity and accuracy with which the flatness of the surface of a building can be controlled, by means of a digital image analysis on material surfaces together with a data processing of a 2D reconstruction algorithm.
as well.Figure 2 represents one of the images acquired by the camera Rolleiflex 3003.The control points coordinates were detected by a topographic instrument Topcon ET2 (Faculty of Engineering of the University of Palermo, Italy).

Figure 2 .
Figure 2. A photogrammetric image of the façade of the Zisa Palace in Palermo (Italy) acquired by the semi-metric camera Rolleiflex 3003.The points of support are indicated at the different quotes P-i (i = I, ••• , V).Their coordinates are reported in Table1.

Figure 3 .
Figure 3.A reconstruction of a rectified image of the Zisa Palace obtained by the MSR-CAD software.The points of support are indicated at the different quotes labeled with P-i (i = I, ••• , V).Table 1.The coordinates of the points of support that were detected by the topographic instrument Topcon ET2 on the main façade of the Zisa Palace (Palermo, Italy) and the reprojection errors of these coordinates after the use of Bundle Adjustment.All values may be considered affected by a standard error ± 0.005.The points labeled with * are the points with X > 30 m, whose deviations are significantly different in comparison to the group of points with X < 30 m (p < 0.05).