Pixel Discontinuity Repairing for Push-Broom Orthorectified Images ()
1. Introduction
Orthorectification of imagery is essential for data to be used with other spatially referenced data sets. Pushbroom imaging systems collect data and analyze images by embedded real-time software as they scan tracks perpendicular to the flight line. The ground location of these pixels can jump dramatically from pixel to pixel because of the pitch, yaw, and roll of the aircraft as it conducts instrument scanning. Thus, pixel discontinuity (blank pixels) occurs quite often in orthoimages. In order to fill in these blank points several pixel repair methods have been proposed. However, before accurate measurements based on aerial photographs are performed, the distortion in photographs must be removed. All landscape metrics are sensitive to geometric distortion.
The pixel-by-pixel approach is one of the proposed approaches for rectifying an orthoimage from raw images [1]. Specifically, the associated DTM is used in generating the orthoimage and the correspondence between an image pixel and the conjugate ground object is characterized by the collinearity equation. The gray value of a ground object on the orthoimage is then resampled using the projected location value on the raw image. In principle, there are two ways to do this.
The first method called the bottom-up method [1,2]. Starting from the object space, each ground object is projected onto the image space. Conversely, the top-down method starts from the image space and projects each image pixel onto the object surface [1]. Because of the rapid scanning speed and linear scan lines of the push broom imager, the it is far more difficult to calculate to which scan line the ground objects correspond to in the bottom-up method. The top-down method is easier to operate when pixel discontinuities occur. In addition, in either approach, in order to repair pixel discontinuity, image resampling using interpolation is essential. The most commonly used resampling algorithms are the nearest neighbor (NN), and bilinear (BL) interpolations. The computational speed of the NN interpolation is rapid but jagged pixels occur. The BL interpolation, weighted by distance of neighboring pixels, generates smoother images [3].
This study proposes an algorithm for repairing blank pixels during the orthorectification process to eliminate pixel discontinuity. The algorithm employs bilinear (BL) interpolation and distance-weighted allocation to obtain the gray values of all orthoimage pixels [4-6]. This study evaluates the results of the proposed algorithm by using a near-infrared band of the preliminary hyperspectral images, which were captured by a hyperspectral imager of plants and the environment [7,8]. Simple comparisons of the results using NN and BL were also conducted to provide a reference for future improvements.
2. Blank Pixels Repairing for Orthorectified Images by Top-Down Scheme
Pixel discontinuity primarily results from variations in the pitch of an airplane and the low performance of embedded real-time software within the push-broom imaging system (Figure 1). The deviation of pixel location must be first analyzed to assess whether the proposed scheme is applicable. A stabilizer was installed between the airplane and the imagery system to filter out highfrequency noise during image acquisition. The angular rates of the pitch and heading were both within ±0.7% statistically according to altitude variations. The scheme was determined to be suitable because the deviation of the pixel location was less than 1.0 pixel.
After filtering out high-frequency aircraft vibrations using the embedded real-time analyzing software and stabilizer, the exterior orientation can be obtained from the IMU/INS and GPS system for geometric image correction [9]. After executing the top-down approach, the orthoimages contain dot-like blank pixels in the form of black lines that indicate dramatic variations in altitude (Figure 2).
2.1. Top-Down Approach
The top-down approach, also known as the ray-tracing approach, employs a collinear equation (Formulas 1 and 2) to cross-reference pixels from the image space with the object space (Figure 3).
(1)
(2)
where f is focal length of the camera and are rotation matrix coefficients calculated by the rotation orientations of each axis from image to object space. and represent the perspective center of the object and image coordinates. (X, Y, Z) is the image point (x, y) in the object coordinate [10].
2.2. Pixels Allocation Algorithm
For the top-down approach, this study applied the pixel interpolation method to improve the orthoimage quality through pixel allocation by using the Nearest Neighbor (NN) approach. This study also proposed another method for eliminating pixel discontinuity to improve the image quality. This approach is similar to the Bilinear Interpolation (BL) approach for image pixel allocation. The two methods are described below [6]. The flowchart is shown in Figure 4.
2.2.1. Nearest Neighbor (NN) Approach
To increase the computational efficiency of imagery orthorectification using the top-down approach, referenced gray values are typically allocated to the orthoimage directly using the NN method. A schematic diagram of pixel allocation using the NN method is shown in Figure 5. The distances between blank pixels and neighbor pixels should be compared to determine which is the nearest. To carry out this process, as Figure 5 shows, a search mask is applied to each blank pixel to determine its gray value according to the inverse square of the distance for each mask pixel. Furthermore, any blank pixels in the neighboring points must be removed and allocated a gray value.
2.2.2. Bilinear Interpolation (BL) Approach
To resolve blank pixels, this study also applies the BL method for imagery orthorectification to allocate gray values to the pixels, as shown in Figure 6. The position Pa (I, J) is a pixel point in the orthoimage, and I and J are positions in the orthoimage grid system. These results indicate that the four adjacent pixels are highly correlated. Thus, this study applies the BL and area-weighting method to allocate gray values to the adjacent pixels P (i, j), P (i + 1, j), P (i, j + 1), and P (i + 1, j + 1). For a given pixel of an orthoimage, consider the following example using P (i, j). Because P (i, j) is affected by other adjacent points, such as Pa, Pb, Pc, and Pd, repeated filling of the pixel value is required. As shown in Figure 4, this study assigned the gray value of Pa, Pb, Pc, Pd, and other points to P (i, j). Therefore, the gray value of point P (i, j) must be adjusted according to each gray valuefilled pixel and its corresponding weight.
The algorithm proposed in this study is detailed below. In the top-down approach, the pixels of the raw image are cross-referenced to the points Pa in the orthoimage grid system sequentially. For the BL method, the corresponding area of a pixel is used as the weighting factors for the gray value allocation around the neighboring image points of P (i, j), P (i + 1, j), P (i + 1, j + 1), and P (i, j + 1). The weighting factors of Pa are
, , and,
Figure 1. Sketch of the push-broom imaging system.
Figure 2. Black lines in orthoimages indicate dramatic altitude variations.
Figure 3. Sketch map of the top-down approach.
where dL = (I – i) and dS = (J – j) are the distance components between Pa(I, J) and P(i, j) in the line and sample directions.
After completing image orthorectification using the previous two steps, a weighted average calculation of the gray values based on the multiple gray values and weighting factors allocated to each image point must be performed. For example, point P (i, j) can be expressed using the following equation:
(3)
where GL represents the gray value of the pixel P (i, j),
Figure 4. Flowchart of proposed approach.
and W is the weighting factor of the pixel.
To complete the preceding steps, this study applied a dynamic range adjustment to the gray values of the pixels in the orthoimage to ensure their consistency with the range of gray values in the raw image. By applying the presented scheme, pixel discontinuity was eliminated,
Figure 5. Pixel allocation using the NN method.
Figure 6. Pixel allocation using the BL method.
and the quality of the orthoimages was substantially improved.
3. Experiments and Discussion
3.1. Experimental Data
We used hyperspectral images retrieved by the HOPE imager [11] to verify the NN and BL methods, and compared the results of these two methods. The HOPE imager is a push-broom imager developed in our colleagues’ previous study. Table 1 shows the specifications of the HOPE imager.
Figure 7 shows a raw image of the near-infrared band (817 nm) of the HOPE hyperspectral imager. The image was captured in the Shengang District of Taichung City, near a major highway interchange. The test field is shown as the red rectangle in Figure 7.