Using a Visible Vision System for On-Line Determination of Quality Parameters of Olive Fruits

The increased expectations for food products of high quality and safety standards and the need for accurate, fast and objective quality determination of these characteristics in food products continue to grow. In this situation, new techniques are necessary to enable on-line control of quality parameters. Computer vision provides one alternative for an automated, non-destructive and cost-effective technique in order to accomplish these requirements. The maturity index and sanitary conditions were objectively assessed on-line by image analysis obtained through machine vision, in which algorithms of colour-based segmentation, as well as the main operators to detect edges were used. The proposed methodology is able to estimate the maturity index and the percentage of defects in olives. In addition, this system can be potentially used on-line in the classification of olives, which means that it could help to improve the quality control of olive oil in factories.


Introduction
There are many factors which can affect the quality of olive oil, such as inherent agronomical factors or factors related to the various steps in the olive oil processing, i.e. from extraction to bottling.
Sensory characteristics and chemical composition greatly influence the quality of a product.Traditionally, quality inspection of agricultural and food products has been performed by human graders.However, in most cases these manual inspections are time-consuming and labourintensive.Controlling the oil at the time of production, thus avoiding mixing oils of different qualities and thereby improving the quality of extra virgin oil is essential.
The appearance and colour of food are important factors which influence consumer preferences.During the ripeness time, olives follow different phases.Firstly, they are of a deep green colour with predominantly high levels of chlorophyll and give rise to bitter oil.In their final phase, they present a black colour paste, their chlorophyll is degraded and replaced by anthocyanin and they are more sensitive to external damage and infections [1][2][3].
The maturity index is useful for producers to identify the optimal time for harvesting the olives.Using this index may also help to increase the quantitative and quailtative characteristics of olive oil production.There are many systems used to determine the state of maturity of olives depending on various factors, such as fruit firmness [4,5], the calculation of dry matter [6], the calculation of the respiration rate of fruit [7] or non-destructive methods, such as the use of the hydrometer [8][9][10] or by measuring the transmission of acoustic waves through the fruit [11].
The most common method to evaluate this index in olives is called maturity index (MI), which considers changes in the colour of the skin and flesh during the ripening of fruit and gives them their colour according to a class (classes 1 to 7) [12][13][14].
However, this method is subjective and depends on environmental conditions, which can affect the colour appearance of olives.At times, the evaluators have to decide between colours which are difficult to distinguish.Consequently, in order to provide a successful complement for the current industry, we need a method able to assess the maturity index accurately, which is, additionally, easy to be used, inexpensive, preferably non-destructive, and which can be coupled in the process line for the rapid determination of the maturity index.
Techniques such as infrared spectroscopy have been studied and successfully used for rapid characterization of some olives and some olive oil parameters [15][16][17][18] and, for quality control and for the determination of the geographic origin of virgin olive oils [19].Studies using Raman spectroscopy have also been carried out for authentication purposes, for the classification of PDO olive oils [20,21] and for the detection of adulteration with other low price oils [22][23][24].Moreover, this technique has been applied for the determination of the oxidative degradation [25], free acidity [26,27], the characterization of antioxidant olive oil biophenols [28] and, finally, to provide some contributions dealing with the evaluation of fruit quality [29] .
At present, the olive oil sector is experiencing some changes in technology in the production lines allowing a continuous process control.The olive oil production focuses on producing high quality products.This new approach determines a transition from mass production of olive oil to lower production characterized by a higher quality of marketable products.Consequently, many scholars and scientists have focused on improving production processes as shown in the literature [30,31].This grade of oil production can be optimized with automatic control of the mechanical extraction lines.This would be capable of controlling the cycle of continuous extraction by measuring various technical parameters of olives and by determining the influence of these parameters on the quality of olive oil [32].Some of the parameters which have been studied include, for instance, the oil temperature in the mixer, the mixing time and temperature of the oil centrifugal extractor.Some recent publications have explored the possibility of a specific software system, which is able to control the extraction process and the parameters involved.Artificial neural networks were used to estimate some parameters for the extraction of olive oil in real time [33][34][35], as well as the use of NIR sensors for the on-line characterization of olive oils [36].Finally, other works have been carried out for the determination of parameters of oil and acid value (AV), the bitter taste (K225) and fatty acid composition (FAME) [37], for the monitoring of carotenoid and chlorophyll pigments in virgin olive oil [38], and the measurement of oil content and humidity in olive cakes [39].
In the food industry, artificial vision has found many applications for controlling product quality.This nondestructive method is fast, inexpensive, consistent and objective, and has expanded in many different industries.Its speed and accuracy satisfy ever-increasing production and quality requirements, hence aiding in the development of totally automated processes [40].Some methods based on image processing have been designed over the last five years in order to conduct real-time control and automated inspection thanks to different instrumentation, such as cameras and spectrometers, which allow the detection of features in fruit.By interposing appropriate filters, we will achieve the analysis of non-visible areas which let us understand some characteristics of products, which are not always detected visually by the operators.
Over the previous years, studies have shown the ability to sort the fruit by its colour [50][51][52][53].There are also some algorithm classification of the fruit on the basis of the shape and size of cucumbers [54] and watermelon [55].Some significant correlations between data about the shape, colour and trend of aging, and the age of apples have been found [56].
In the literature there are works related to the correlation between the image and the degree of ripeness of different fruit [57,58] in order to remove products which do not meet the required standards of classification.
In previous works, we have used spectroscopy as a rapid non-destructive method to determine the quality of olives in the process [59].This paper aims to improve the application of techniques for fast on-line analysis of olive oil and proposes a direct method to determine on-line the maturity of olives.This non-destructive method offers many advantages in the quality control of the process, since it is an easy and quick way of obtaining data, which is difficult to obtain manually.The final methodology could be useful for evaluating the performance of commercial systems and better control of the extraction process of olive oil.

Samples
Olive samples of picual variety (2 kg) were collected in several olive mills in the province of Jaen (Spain) during November, 2011.The colour of these olives ranged from dark green to completely black.These samples were used for the acquisition of images; synthetic samples were also carried out so as to be included in the analysis of samples of olive with a full range of maturity index.

Visual Determination of Maturity Index (MI)
The MI was determined by an experienced evaluator; every sampling day an assessment of skin, flesh and colour of olives was performed [60].The procedure involved the distribution of olives into eight groups according to the characteristics summarized in Table 1.
The official method uses 100 olives and calculates a global value of maturity index for each sample or group of olives.The MI is calculated by using the following equation: where i is the group number and ni the number of olives in it.This method has to separate the olives manually, cut the pulp to examine them and count and identify the group to which they belong.

Image System
Images were captured with a JAI AD-080CL multi-spectral camera, combining a visible colour channel (Bayer Mosaic CCD), which can generate 24-bit RGB images and a NIR (Near Infrared) channel (monochrome CCD).The major advantage of this camera is the capability of capturing both channels simultaneously though the same optical path.
This camera uses a standard Camera Link interface, whereby each channel can output images with 8 or 10-bit and 24-bits, and a resolution of 1024 × 768 active pixels per channel.This camera was installed in an enclosed cabin equipped with a controlled lighting in order to achieve a consistent image in all captures.The lighting was a halogen lamp with some filters to get a diffused light simulating the illuminant D65 (6500 K).This is considered the standard by the International Commission on Illumination, which describes the ordinary lighting conditions of a cloudy day at midday.Direct lighting on olive fruit was avoided by means of a reflecting surface placed between the scene and the illuminant.The images were taken at a distance of approximately 45 cm in a neutral white background.

Methodology
The algorithms used were developed with the image processing toolbox, version 7.4.0 of Matlab (The Math-Works, Inc., Natick, MA, USA), together with the software Image Pro-Plus version 6.0 (Media Cybernetics, Inc).
The images were taken in duplicate through both channels, namely, IR (monochromatic) and visible (RGB image) (Figure 1).This means that they are images whose pixels are specified by three values, one for each colour component (red, green and blue), which are used for MI determination.
IR images are used to distinguish damaged olives which are healthy in appearance and which cannot be distinguished in the visible systems.The difference between damaged skins can be clearly seen at near-infrared bands, which offered good contrast between the defect  and the sound skin and were well suited for detecting internal skin damage [8].

Estimation of Maturity Index and External Defects
The technique used was the segmentation, which is based on the identification of regions and edges using clusters of pixels selected according to different criteria (colour, boundary, grey levels, etc.) [61,62].
In order to develop a method for the on-line prediction of maturity index and sanitary conditions, the following steps have been carried out:

1) Identification and separation of objects in the NIR image
The procedure of separation of objects consists in the use of the NIR monochrome image in order to segregate every single olive fruit.To identify objects in the NIR monochrome image, the first step performed is a contrast equalization followed by a spatial filter enhancement "Flatten", which reduces the intensity variations in the background pixels.Spatial filtering operation was applied to the images to enhance or attenuate spatial detail in order to enhance visual performance or to facilitate further processing.The spatial filtering operation can be considered as a "local" procedure in image processing, in the sense of changing the value of each pixel according to the values of the surrounding pixels.From a practical point of view, this step mainly consists in classifying the original grey levels in the original NIR image according to the value in the neighbouring pixels.This task was carried out thanks to the use of two algorithms: on the one hand, Border-4 Neighbour, which determines the edges of the olive, taking into account the intensity values of pixels in the histogram and considering the darker pixels as edges of objects; on the other hand, a connected components algorithm which is used to identify each olive with a value.This edge-based algorithm enumerates each set of pixels which represent an olive [63].
2) Creation of a mask and application of the mask to the original image From the data in the array of connected components, a mask can be created.In addition, various size filters can be applied, so that we can measure complete objects or automatically remove the objects which overlap with an area smaller or bigger than those with a standard size olive [64].Based on these objects, the skeleton of image is constructed.The skeleton is intended to represent the shape of an object with a relatively small number of pixels.Thus, all pixels of the skeleton are structurally necessary.The position, orientation and length of the skeleton lines correspond to those equivalents to the original image; this skeleton simplifies the task of extracting features of an image.
This mask reduces the image to a skeleton and eliminates protrusions.The original image is added to the mask and the colour-segmentation and grey-segmentation are performed in separated objects. 3

) Segmentation based on colour/grey and classification
This segmentation based on colour can be performed by supervised or unsupervised methods in order to get and quantify the predominant colour in the olives.This step requires transforming the RGB image using various functions to transform RGB format in L*a*b colour space.The CIE L*a*b* colour notation system was applied to determine the parameters L*, a* and b*; where L* indicates the lightness, a* means the colour axis from green to red and b* refers to the blue-yellow one.
On the one hand, in the supervised methods a region containing the colour of interest (colour markers) is selected and then averaged.This classification is performed by using the method of the k-nearest neighbours (KNN) [65], where each pixel is classified in the same group as the colour markers with a similar intensity.The k nearest neighbours assign a value of "a" and "b" for each marker and, therefore, it is possible to classify each pixel of the image so as to calculate the Euclidean distance between pixels and colour markers.
On the other hand, unsupervised methods for colour segmentation techniques use clustering algorithms, which essentially perform the same functions as the classifying methods, but without using data training [66,67].In order to compensate for the lack of data training, clustering methods iterate between image segmentation and characterize the properties of each class.In this sense, clustering methods are trained with the available data.The basic idea is to assume that image pixels are points in a three dimensional space (RGB).Therefore, the points of a region of similar colour are grouped together.Clustering techniques allow us to obtain a representative point of each group, based on measures of similarity or distance between these points.In this algorithm, a point xi is assigned to a group R, whose centroid C is closer to it.The Euclidean distance is usually used as a measure for the similarity between the point and the centroid.The results of this classification are normally expressed in terms of the total percentage of pixels coloured in each object.Thanks to these results, fruits can be classified according to their level of maturity (Table 2).
In the case of the determination of external defects, the skeleton is added to the original image of infrared and marks the position of the defects in the original image.To that end, we have a gradient-based segmentation of grey levels, which indicates that the values with the highest values of pixels represent the area of pixel defects and the lowest values represent the healthy areas of the fruit.
This allows us to measure some geometric properties of the defects, specifically, the defect area in each olive.

Class
Olive skin colour 0 >50% bright green 1 >50% green-yellowish 2 %green-yellowish with black and/or reddish spots >%reddish-brown 3 %green-yellowish with black and/or reddish spots <%reddish-brown 4 100% blackish purple or black After an evaluation of the area of defect, the results are collected in percentages as the sum of all pixels in areas occupied by the defects and the sum of all pixels in areas occupied by the sound areas of the olives.

Results and Discussion
Different sets of olives were analysed aiming at developing the proposed approach.The procedure described in the previous section was developed for each batch of olives.
The procedure was performed for the first time in fifty images captured directly on the olive production belt.These algorithms were applied to get the maturity index and quality state of each fruit.The results were obtained as the percentage of each colour and percentage of defects present in the object.These are classified according to different groups.Figure 2 shows an example of analysed on-line images.The results display the classification of olives for colour segmentation.
The results of prediction of the maturity index of fruit by the proposed vision method for images with individual olives are strongly related to visual RI measured.
The errors of prediction have been almost negligible or zero values for other samples, (see results in Table 3).These values are acceptable given the subjectivity which characterizes the MI on-line evaluation by the traditional method.
The determination of defects detection was carried out as described in the previous section.The results of some on-line images were analysed and they were obtained as a percentage of sound areas and defects in each olive.
Each pixel of the fruit is classified into the different groups: sound olives or defective olives, according to the values of its spectral components.For each olive, results were obtained as the total percentage of defects and the total percentage of sound skin.
In order to classify olives according to their degree of defect, five different categories have been agreed.Table 4 defines those categories as follows: sound olives, olives with minor defects, olives with moderate defects, olives with several defects and defective olives.
Figure 3 shows an on-line image analysed with its corresponding results of classification.The results of classification of the olives to various images of olives analysed are shown in Table 5.
According to these results, the proposed methodology can potentially be used for the classification of fruit in the quality control in the on-line extraction process of    olive oil, which consists in the capture of an on-line image and its analysis.This process can occur at acceptable speeds and, therefore, this shall be the future aim of studies.
The described method provides an estimate of the MI value.This procedure requires a negligible amount of time; it is possible to state that the only time required is that needed to collect the olive samples from the trees.It is important to bear that in mind, given the fact that experts working in oil mills perform a real-time monitoring of the agronomical and technological parameters.In some cases, however, the purpose of oil mills is to increase the quality of the extracted olive oil by optimizing some technological parameters.
The development of classification systems for nondestructive methods is applied to each fruit and ensures that their characteristics meet the standards.Concretely, optical techniques offer possibilities in the assessment of surface characteristics and inner qualities or composition.This system can be potentially used on-line in the classification of olives, which means that it would help to improve the quality control of olive oil in factories.Additionally, the systems used to measure optical properties provide significant information about the characteristics of each fruit, which is useful for the assessment of products.
The future of spectral systems applied to food inspection is promising, as both industry and consumers are increasingly aware of the need to ensure quality and food safety.Consequently, this technology is an essential tool for automatic inspection and control of these parameters.

4 5 7 Figure 1 .
Figure 1.Sample representation in RGB and infrared space obtained in the on-line assay conditions.

Figure 2 .
Figure 2. Original on-line image and image with analysed results with pixel index of different classes in objects (yellow = %reddish-brown; red = %green-yellowish).

Figure 3 .
Figure 3. Results of classification for an on-line image analysed.