Vision-Based Vehicle Detection for a Forward Collision Warning System

A weather-adaptive forward collision warning (FCW) system was presented by applying local features for vehicle detection and global features for vehicle verification. In the system, horizontal and vertical edge maps are separately calculated. Then edge maps are threshold by an adaptive threshold value to adapt the brightness variation. Third, the edge points are linked to generate possible objects. Fourth, the objects are judged based on edge response, location, and symmetry to generate vehicle candidates. At last, a method based on the principal component analysis (PCA) is proposed to verify the vehicle candidates. The proposed FCW system has the following properties: 1) the edge extraction is adaptive to various lighting condition; 2) the local features are mutually processed to improve the reliability of vehicle detection; 3) the hierarchical schemes of vehicle detection enhance the adaptability to various weather conditions; 4) the PCA-based verification can strictly eliminate the candidate regions without vehicle appearance.


Introduction
To avoid the traffic accidents and to decrease injuries and fatalities, many advanced driving assistance systems (ADASs) have been developed paying more regard to situations on road for drivers, such as forward collision warning (FCW), lane departure warning (LDW), blind spot detection (BSD), etc.Based on our previous studies [1] [2] [3], this study focuses on developing a FCW system.World Journal of Engineering and Technology Extensive studies have been proposed for vision-based FCW systems [4] [5] [6] [7].Sun et al. [4] and Sivaraman et al. [5] generally divided the vision-based vehicle detection methods into three categories: knowledge-based, motionbased, and stereo-based methods.Knowledge-based methods integrate many features of vehicles to recognize preceding vehicles in images.Most of these methods are implemented by two steps: hypothesis generation (HG) and hypothesis verification (HV) [4].The HG step detects all possible vehicle candidates in an image; the vehicle candidates are then confirmed by the HV step to ensure correct detections.
Due to the rigidness and the specific appearance of vehicles, many features such as edges, shadow, symmetry, texture, color, and intensity can be observed in images.The features used in knowledge-based methods can be further divided into global features and local features according to whether the vehicle appearance is considered when using the features.A usable FCW system should be stable in various weather conditions and in various background scenes.Furthermore, the system should get high detecting rate with fewer false alarms by utilizing both local-and global features.
The weather adaptability of edge information is much better than other local features such as shadow and symmetry.The contour of a vehicle can represent by horizontal and vertical edges.However, the local edge features are not strict enough to describe vehicle appearance.The appearance based verification is better and need to support the vehicle detection based on edge information.
We here propose a FCW system combining the advantage of the global features and local features to improve the reliability of vehicle detection.The local feature is invariant to vehicle direction, lighting condition, and partial occlusion.
The global feature is used to search objects in the image base on whole vehicle configuration.In the detection stage, all vehicle candidates are located according to the local as many as possible to avoid missing targets; in the verification stage, each candidate is confirmed as real target by the global as strictly as possible to reduce the false alarms.The proposed FCW system detects vehicles relying on edges; thereby the proposed system can work well under various weather conditions.At first, the lane detector [1] [2] is performed to limit the searching region of preceding vehicles.
Then an adaptive threshold is used to obtain bi-level horizontal and vertical edges in the region.The vehicle candidate is generated depending on a valid horizontal edge and the geometry constrains vertical edges at two ends of the horizontal edge.In verification stage, the vehicle candidates are verified by vehicle appearance and principal component analysis (PCA).A principal component set is obtained by applying PCA to a set of canonical vehicle images.These components are used to dominate the appearance of vehicle-like regions; therefore, the regions of vehicle-like objects can be well reconstructed by the principal components.The closer the distance between an image and its reconstruction image is, the higher the probability of the object on the image being a vehicle is.( , ) symmetry 0.5 where h and w are the height and the width of the window, x i and y j are the xand y-locations in the window, eng(x i , y j ) is the energy of symmetry at (x i , y j ) defined by ( , ) otherwise Firstly, the areas for searching the vertical edges are set extending upward and downward from the horizontal edge endpoints, as the gray regions in Figure 3.
According to the candidate bottoms, the images of the vehicle candidates are extracted with ratio height/width being 1.In the detecting result, the candidate locations are represented by "U" shape bounding boxes.The bottom of a "U" shape bounding box is the candidate bottom.The left and right margins of the  bounding box are defined as the vertical line segments from (x l , y l ) to (x l , y l + w ) and from (x r , y (r) ) to (x r , y r + w ), respectively.w is the width of the candidate bottom, that is w = x r -x l .Each horizontal edge is individually determined if it belongs to a vehicle candidate.Around one vehicle target, there might be many bounding boxes which are set according to the horizontal edges at different y-positions.To delete the repetitions, the candidate being most closes to the host vehicle in a candidate group is reserved for the rest processes.That is, the bounding box with smallest y-position in image is reserved.

Vehicle Verification
In the verification stage, the vehicle candidates are determined applying PCA reconstruction.The reconstruction images are obtained according to the images of pre-trained eigenvehicles.Then, the image comparison methods are used to evaluate the likelihood of the vehicle candidates to e vehicle.

The Principle and Procedure of PCA-based Verification
Principal components analysis (PCA) is a statistical technique used to analyze and simplify a data set.Each principle component (eigenvector) is obtained with a corresponding data variance (eigenvalue).The larger the corresponding variance is, the more important the data information contained by the principle component is.For application of reducing data dimensionality, the principle components are sorted by the corresponding variance; and the original data are projected onto the principle components with the largest m variance.PCA is widely performed to image recognition and compression.

Eigenvehicle Generation
The images of the eigenvectors obtained by applying PCA to vehicle images are called "eigenvehicles".4.
Eigenvehicles are easily influenced by background content, foreground size, illumination variation, and imaged orientation.To solve most of the problems, the original vehicle images of the training set are manually identified and segmented fitting the contour.The color and illumination factors would be also eliminated by using edge map and applying normalization to the edge map.

Evaluation by The Reconstruction Error
For each vehicle candidate, the original image is firstly resized to 32 × 32 and transformed into a normalized edge map Γ.The edge map Γ subtracts the mean image Ψ to have the difference, and the difference is then projected onto the M' leading eigenvehicles.Each value representing a weight would be calculated by ( -), where i = 1, 2, …, M'.The reconstruction image of the normalized edge map Γ is then obtained by The vehicle space is the subspace spanned by the eigenvehicles, and the reconstruction image is the closest point to Γ on the vehicle space.The reconstruction error of Γ could be easily calculated by Euclidean distance (ED), The reconstruction images are usually dusted by a lot of noises.For more accurate comparison between Γ and Γ , the comparing method should be more robust to noise and neighboring variations.The images could be regarded as 3D models by taking the intensity at the image coordinate (x, y) to be the height of Γ ˆ World Journal of Engineering and Technology the model bin (x, y).The 3D models can be compared to each other by a crossbin comparison measurement such as the quadratic-form distance [8], Earth Mover's Distance (EMD) [9], and the diffused distance (DD) method [10].Earth Mover's Distance is calculated as ( ) where H 1 and H 2 are two compared histograms, f ij is the work flow from bin i to bin j, and d ij is the ground distance between bins i and j.The diffused distance is calculated by ( )  number of used eigenvehicles.The distance between the original image and reconstruction image decreases with the increment of the eigenvehicle number; however, the execution time of projecting and reconstructing linearly increases with the increment of the eigenvehicle number.According to our analysis, the distance decrement is obviously slowdown when the eigenvehicle number is larger than 50.It is important to consider that is the time cost of using eigenvehicles more than 50 worth enough for the PCA-based verification.The verification ability of the PCA-based method can be analyzed by ROC curves.The variation of the area ratio under the ROC curves shows the limitation of improving reconstruction difference by increasing the eigenvehicle number.Depending on the application, a suggestion of the eigenvehicle number is 40 -80.

Experiment and Discussions
The proposed FCW system has been tested with the videos captured under various weather conditions and environment situations: sunny day (S), cloudy day (C), long shadow (LS), facing sun ray (FS), rainy day (R), heavily rainy day (HR), etc.The comparison of detecting vehicles without and with verification scheme is show in Table 1.The symbol "d*" and "d" represent the trials of detecting vehicles using the idealized constrains and the less rigorous constrains, respectively.The "d*" and "d" trials are both implemented without verification scheme.The geometry constrains being relaxed in "d" trail include the pre-trained learning threshold models, the thresholds of the length of small noise horizontal edge, the ratio width/height of the bounding box for extracting a significant horizontal edge, the searching areas of upper significant vertical edges and symmetric vertical border, the threshold of symmetry, etc.The symbol "v" represents the trial of detecting vehicles using the less rigorous constrains with the verification scheme.
To illustrate the effect of the proposed learning thresholding of edge maps, the system accuracies of using three constant thresholds and the learning threshold to each environment situation are compared in Table 2.The threshold for vertical edge map is dynamically calculated by the proposed learning thresholding in all case for the single-variable analysis.The proposed learning thresholding makes our FCW system having the ability to adjust an appropriate threshold for each frame of any environment situation.As shown in Table 2, applying the learning threshold produces higher accuracy with a lower false detection rate than applying the constant thresholds.
The detecting performances of various driving situation are given in Table 3.The average accuracy of all environment situations could reach 91%.The results

Conclusions
The proposed system possesses several special and unique properties: 1) The appropriate amount of edge points are reserved by a smart learning thresholding method for less influenced by weather conditions; 2) the vehicle detection is accomplished by applying the horizontal edges with the consideration of the location of vertical edges and edge symmetries instead of that each feature is individually used; 3) the vehicle detection is robust to partial occlusions and lighting variations by using local features; 4) the vehicle verification reliably determines the detected candidates by evaluating the probability that the candidate has vehicle appearance.The characteristics of local and global features are complementary to each other.However, most related studies apply only one of the features in their system.The proposed system efficiently uses the merits of local and global features to attain the stability and reliability.

Figure 1 .
Figure 1.Two examples of ROI setting based on the detected lane marks.

Figure 2 .
Figure 2.An example of the vertical characteristics in a long underneath shadow area.
where I ep (x, y) is the value of the edge point map I ep at (x, y) and I ep is one of the vertical, the positive horizontal, and the negative horizontal edge point maps which are I vep , I p_hep , and I n_hep .The horizontal edges in Case-3 are the edges of the vehicle body, though the components which they belong to are unknown.The y-positions of the horizontal edges can't indicate the y-positions of the candidate bottoms, which only gives a hint of the vehicle's location.In the image, two vertical edges could be detected at vehicle sides, which are similar to each other in length and y-position.Therefore, two characteristic vertical edges are extracted from the areas around two endpoints of the horizontal edge, respectively.The distance and the y-position overlapping between two vertical edges are used to determine candidate bottom.

Figure 3 .
Figure 3.The searching areas for vertical edges and the longest vertical edges in each area.

Figure 4 .
Figure 4.The top 16 eigenvehicles of the training set.

7 )Figure 6 .Figure 5 .
Figure 6.The x-axis is the measured distance and the y-axis is the accumulation of samples.The threshold of measured distance is changed in the statistics from minimum to maximum distance values to calculate the ratio of positive and negative samples being recognized as a vehicle.The verification based on PCA reconstruction is deeply influenced by the

Figure 6 .
Figure 6.The measurement statistics of positive and negative samples using three methods.(a) ED; (b) EMD; (c) DD.

Figure 7 .Figure 7 .
Figure 7.The left and right of a pair of images are the results of vehicle detection and vehicle verification, respectively.
are collected with various vehicle types, in various distance far from the own vehicle, and at the own/adjacent lane.The cropped images of preceding vehicles are firstly resized to 32 × 32.For each resized image, the edge map is produced and then normalized by letting the sum of edge values being equal to 1.The normalized edge maps as the training images are represented as vectors Γ 1 , Γ 2 , …,and Γ M of size N, where M is the number of training images and N = 32 × 32.
To generate a set of eigenvehicles, a training set of preceding vehicle images cropped from on-road driving videos are prepared.The on-road driving videos are captured under various weather conditions; the World Journal of Engineering and Technology training images

Table 1 .
The Comparison of System Performance without and with Verification Procedure.

Table 2 .
The Comparison between Constant Thresholds and the Learning Threshold.