Shape Retrieval Using Fourier Descriptors Applied to Industrial Process

Julio-Alejandro Romero-González^*, Rosamaría Romero-González, Marcela Herrera-Navarro, Diana-Margarita Córdova-Esparza, Hugo Jiménez-Hernández

Faculty of Informatics, Autonomous University of Queretaro, Queretaro, Mexico.

DOI: 10.4236/jcc.2020.83005                

Abstract

Nowadays, manufacturing processes are carried out at speeds that they themselves demand and subject to rigorous standards to maintain the quality of materials. An important step to define the quality of products in metalworking is the casting process, which principal focus is seeking control and monitoring of properties of materials. Nevertheless, it is not easy due to the high temperatures and gas produced in the vessel. Although some researchers have been attempting to solve these problems, it is difficult to carry out due to hard conditions. This article proposes the analysis of the surface of the liquid metal, that is, the slag on the surface, which is considered as connected spaces characterized by the topology of their discrete surface. These spaces are described through Fast Fourier Transform, associating changes of intensities to the frequency domain and obtaining main features of these frequencies, these features are used to define an enveloping shape that represents the liquid metal. Finally, the results obtained are presented, which, according to them shows that it is possible to characterize the slag, and by which it is possible to spatially locate the molten metal liquid in the refractory. Therefore, this research serves as the basis for the development of new algorithms for level detection and measurement, preventing overflows and damage to refractories.

Keywords

Convex Hull, Shape Retrieval, Features Detection, Fourier Transform, Vision Computing

1. Introduction

The principal problems on a smelting furnace in metalworking are caused by high temperatures and the gases produced by the slag. For the industry, the casting process is important because the product quality depends on it [1], if the process is slow. Then the molten steel will cool and if it is a quick casting then it will provoke turbulences, its effect produces corrosion and erosion in the refractory [2]. It exists several methods for detecting slags. There are several methods to detect and segment slags, in [3] the authors report a method to segment slag during the steelmaking process, in which they propose obtaining X-ray tomography (HRXCT) of the steel and by applying the Otsu model obtaining slag segmentation, however, the limited resolution of HRXCT [4] and the sensitivity of the Otsu method to variations in brightness [5] are the main disadvantages of this method. In the investigation reported in [6] and [7], a CCD camera is used to capture images of the process and slag is manually segmented by selecting a region of interest, this being the main limitation for its implementation in production due to the speeds that the process demands. Other developments are based on the use of a camera for thermography, taking advantage of the emissivity of objects to segment and detect slag [8], however, research using this method focuses on temperature measurement [9], measurement of slag [10], estimation of radioactive parameters in slag [11] or the determination of slag properties.

In addition, private software has been developed for the segmentation and detection of slag based on infrared images [12]. However, in the literature, an automatic method for slag segmentation has not been reported, which allows other vision algorithms to develop a more robust method, for the detection of the slag and measurement of the level of liquid metal, by which they can avoid spillage, mold damage and improve product quality. The motivation of this article stems from this need, which seeks to associate the light changes of the slag in terms of texture, to find a representative form of the slag and to spatially locate the liquid metal in the refractory. To carry out this proposal, this document is organized as follows: Section I describes the introduction, Section II describes the mathematical concepts required for the preprocessing, segmentation, features extraction and the shape retrieval. In Section III, the methodology is presented. The experiments and results are obtained by this method are shown in Section IV. Finally, the conclusions of the methodology are exposed in Section V.

2. Foundations

According to [13], the trichromatic theory, colors are separated into three parameters. They are characterized according to the color space used. Although there are several color spaces, in which, the model RGB (Red, Green and Blue components) is the most used because it does not require transformation for its display [14]. However, other alternatives like the representative model HSV (Hue, Saturation and Value components), may be provided as well for more information on its components. This model separates the image intensity, from chroma or the color information [15]. The transformation from RGB to HSV is defined by [16] like follows

$H=\arccos \frac{\frac{1}{2}\left(R+G+B\right)}{\sqrt{{\left(R-G\right)}^2-\left(R-B\right)\left(G-B\right)}}$ (1)

$S=\frac{\max \left(R,G,B\right)-\min \left(R,G,B\right)}{\max \left(R,G,B\right)}$ (2)

$V=\max \left(R,G,B\right)$ (3)

where R represents the intensity value of red channel, G represents the intensity of green channel and B represents the blue channel of the RGB color model.

2.1. Fourier Transform

The Fourier Transform is applied in a lot of vision systems for transformation, image analysis, filtering, reconstruction, compression, smoothing, and sharpening images, among others [17]. The aim of this preprocessing is the noise attenuation, the abrupt intensities as texture can be distinguished. Fourier theorem states that a function can be represented like a summation of a series of sine and cosine terms as in (4).

$F\left(x\left(t\right)\right)={\displaystyle \underset{-\infty }{\overset{\infty }{\int }}x\left(t\right){\text{e}}^{-2i\pi ft}\text{d}t}$ (4)

where F is the Fourier Transform of a signal x in time domain, f is the frequency of the signal and i is a complex term.

These terms sinusoids show the image behavior in high and low frequencies but to facilitate the image analysis, it was used the Discrete Fourier transform (DFT) (5) and Inverse Discrete Fourier Transform (IDFT) (6).

$X_{k,l}={\displaystyle \underset{i=0}{\overset{N-1}{\sum }}{\displaystyle \underset{j=0}{\overset{N-1}{\sum }}f\left(i,j\right){\text{e}}^{-2i\pi \left(\frac{k_i}{N}+\frac{l_j}{N}\right)}}}$ (5)

where f is the distribution of grayscale image, N is the number of sample of images, k_i and l_j are samples of discrete space of f and X is the DFT.

$f\left(i,j\right)=\frac{1}{N^2}{\displaystyle \underset{k=0}{\overset{N-1}{\sum }}{\displaystyle \underset{l=0}{\overset{N-1}{\sum }}{X}_{k,l}{\text{e}}^{2i\pi \left(\frac{k_a}{N}+\frac{l_b}{N}\right)}}}$ (6)

where X is frequency spectrum, N is the number of sample of images, k_a and l_b are samples of discrete space of X and f is the IDFT.

2.2. Segmentation

Segmentation is an important part of the image processing. It allows extracting a region of interest and reduces the analysis of time. Its objective is to convert an image from grayscale to a binary one, there are several methods that according to their application can be used. These methods are classified in discontinuity that they are based on intensity changes, and they are similarly partitioned into regions with certain criteria of similarity [18]. This investigation focuses on segmentation by histogram thresholding. In [19] the following steps are defined for this method:

1) Estimate an initial threshold to segment the image for creating two sets: object and background.

2) Calculate the average value of each set.

3) Obtain a new threshold from the average values calculated:

$T=\frac{{\mu }_1+{\mu }_2}{2}$ (7)

where, µ₁ and µ₂ represent the average value of each set and T is the threshold calculated.

4) Repeat previous steps using the new threshold calculated until convergence has been reached.

2.3. Convex Hull Method

The convex hull method is used in several applications like interferometric measurements [20], object tracking [21], disaster forecasting [22], gesture recognition [23], rendering model [24], in others. The objective of applying this method in this stage is to generate a unique binary object. For that purpose, a quick hull is going to be used that is then based on a divided and conquered approach which is similar to a quicksort [25]. The steps to apply are shown below.

1) Determinate the points P1 and P2 with minimum and maximum coordinates.

2) Draw the line between P1 and P2 to divide the set into two subsets points.

3) Determine a point (P3) of one of the subsets with maximum distance in respect from the segment P1 and P2.

4) Draw new segments from P3 points to P2 and P1 points.

5) Eliminate the points inside of the triangle formed by P1, P2, and P3.

6) Repeat the above steps using the new segments formed by P1 and P3, P1 and P2 until no points are left.

3. Methodology

In the first step, the image is acquired by a thermographic camera in which a high rainbow palette was used, and the image was transformed from RGB color space to HSV color space (see Figure 1). In which, the gray intensities tend to be clustered in the lower and upper classes, which are translated into low and high temperatures around of 200˚C and 1500˚C respectively. The problem to use the red, and blue components of RGB model is that they reflect high temperatures in the inner and external walls of refractory or combine the liquid metal and the inner wall with minimal values of intensity.

In case of the green component, the difference between the furnace wall and the liquid metal can be seen, but when it is observed in the histogram of Figure 2. It shows minimal values in high temperatures, in comparison with the Hue

Figure 1. Channel decomposition of the RGB and HSV color spaces. Source: Own elaboration.

Figure 2. Distribution of frequencies in: (a) Green component, (b) the Hue component. Source: Own elaboration.

component of HSV. For this reason, HSV model is used, also this step is going to allow a simpler process segmentation.

In the next step, the image is processed with the Discrete Fourier Transform (DFT), in which a low-pass filter is applied to intensify the gray levels and the Inverse Discrete Fourier Transform (IDFT). This technique helps to visualize the texture of slag of the other textures, also, it helps to select an adequate threshold to segment the image (see Figure 3).

As the next step, the histogram is obtained to establish a threshold and binarize the image. The threshold selection using histogram allows to segment the slag from the liquid metal. After that, the binary large objects (blobs) are removed using mathematical morphology (MM) in this case the erosion. The resulting objects are features from slag that allow retrieval of the shape of the liquid metal. Now, these features will be processed to generate a unique binary object, in this step the convex hull is applied, although the blob got does belong to the partial area of liquid metal.

4. Experiments and Results

For validating this methodology, images were captured by a FLIR A320 thermographic camera with a resolution of 24-bits of 640 pixels × 480 pixels, from which, 180 images out of 25 casting operations were obtained.

Figure 4 shows the results after applying the methodology of this proposal, as can be seen, the recovery of the shape is carried out in the area where the hopper does not interfere with the visualization of the slag, in some cases, it is almost the complete shape of the liquid metal section has been recovered in the area of vision. Nevertheless, in others, the algorithm fails, this is mainly due to two things: 1) the first is because when slag falls on the metal, this causes gas to escape, preventing the camera from having visibility of the area under observation; 2) the size of the structuring element when small objects are removed is very

Figure 3. Distribution of frequencies after that DFT is applied. Source: Own elaboration.

Figure 4. Shape retrieval. Images on column a show features from DFT, images on column b are created after convex hull algorithm is applied and images on column c show the original images. Source: Own elaboration.

large, so the size window should be adjusted.

5. Conclusions

The motivation of this paper is due to according to literature, an automated system for slag detection using vision algorithms is not reported yet. In this research, the analysis of the topology of the liquid metal surface is presented, which was characterized through the Discrete Fourier Transform, associating changes of intensities to the frequency domain and obtaining the main features of these frequencies, these features were used to represent molten metal, building its shape by using the convex envelope. Advantages of this proposal are that through the discrete Fourier transform, the molten steel in the furnace has been spatially located, so future work can focus on:

1) Detect the percentage of slag in the refractory.

2) Measure the level of the liquid metal to avoid spillage and mold damage.

3) Develop algorithms to solve camera vision loss problems due to the gas generated by the slag.

Nevertheless, the proposal has limitations, the main is that there are a few features obtained by the Fourier descriptor, which limits the segmentation and complete shape retrieval of the slag in the metal, so it is recommended to use another texture analysis method that is sensitive to subtle changes in lighting to detect slag, but that can omit the edges of the refractory and hope to build a better representation. In addition, for future research, you can think of an algorithm that knowing the location of the slag and supported by holography and calibration of the camera, the level of molten metal can be measured. Another observation for future work is that the slag percentage can be detected, which is done at the moment when the metal begins to fall into the refractory, by means of optical flow analysis.

Acknowledgements

This work has been partially carried out thanks to the support of the scholarship granted by Consejo Nacional de Ciencia y Tecnología (CONACyT).

Conflicts of Interest

The authors declare no conflicts of interest regarding the publication of this paper.

References

[1]	Kim, Y.H., Lee, D.S., Joo, M.G., Kang, T. and Paek, K.N. (2001) Control Problem and Solution in the Steady State in the Strip Casting Process, ISIE 2001. IEEE International Symposium on Industrial Electronics Proceedings, Pusan, Vol. 1, 520-525. https://doi.org/10.1109/ISIE.2001.931847
[2]	Lee, W.E. and Zhang, S. (1999) Melt Corrosion of Oxide and Oxide-Carbon Refractories. International Materials Reviews, 44, 77-104. https://doi.org/10.1179/095066099101528234
[3]	Rout, B.K., Brooks, G., Rhamdhani, M.A., et al. (2018) Dynamic Model of Basic Oxygen Steelmaking Process Based on Multizone Reaction Kinetics: Modeling of Manganese Removal. Metallurgical and Materials Transactions B, 49, 2191. https://doi.org/10.1007/s11663-018-1306-8
[4]	Boone, M.A., Nielsen, P., De Kock, T., Boone, M.N., et al. (2014) Monitoring of Stainless-Steel Slag Carbonation Using X-Ray Computed Microtomography. Environmental Science & Technology, 48, 674-680. https://doi.org/10.1021/es402767q
[5]	Goh, T.Y., Basah, S.N., Yazid, H., Safar, M.J.A. and Saad, F.S.A. (2018) Performance Analysis of Image Thresholding: Otsu Technique. Measurement, 114, 298-307. https://doi.org/10.1016/j.measurement.2017.09.052
[6]	Chakraborty, B. and Sinha, B. (2011) Development of Caster Slag Detection System through Imaging Technique. International Journal of Instrumentation Technology, 1, 84-91. https://doi.org/10.1504/IJIT.2011.043599
[7]	Matthes, J., Waibel, P., Keller, H.B. and Gröll, L. (2018) Identification and Control of the Waelz Process Using Infrared Image Processing. In: Madani, K., Peaucelle, D. and Gusikhin, O., Eds., Informatics in Control, Automation and Robotics, Lecture Notes in Electrical Engineering, Vol. 430, Springer, Cham, 323-341. https://doi.org/10.1007/978-3-319-55011-4_16
[8]	Zhang, Z.M., Bin, L. and Jiang, Y.X. (2014) Slag Detection System Based on Infrared Temperature Measurement. Optik, 125, 1412-1416. https://doi.org/10.1016/j.ijleo.2013.08.016
[9]	Viale, M., Martin, O., Muratori, F., Bertezzolo, U., Perez, J. and Usart, J. (2007) Application of On-Line Infrared Thermography in Steel Making Industry. Proceedings SPIE, Vol. 6541, 9 April, Thermosense XXIX, 65410H. https://doi.org/10.1117/12.721225
[10]	Strakowski, R., Pacholski, K., Wiecek, B., Olbrycht, R., Wittchen, W. and Borecki, M. (2014) Radiative Parameters of Steel Slag for FeO Content Estimation Using Multispectral Thermography System. Quantitative InfraRed Thermography Journal, 11, 222-232. https://doi.org/10.1080/17686733.2014.970754
[11]	Liu, Q., Li, B., Schlangen, E., Sun, Y. and Wu, S. (2017) Research on the Mechanical, Thermal, Induction Heating and Healing Properties of Steel Slag/Steel Fibers Composite Asphalt Mixture. Applied Sciences, 7, 1088. https://doi.org/10.3390/app7101088
[12]	Zahorszki, F. and Lyons, A.R.A. (2000) Online Slag Detection in Steelmaking. Proceedings SPIE, Vol. 4020, 30 March 2000, Thermosense XXII, 11-14. https://doi.org/10.1117/12.381544
[13]	Bora, D.J., Gupta, A.K. and Khan, F.A. (2015) Comparing the Performance of l*a*b* and HSV Color Spaces with Respect to Color Image Segmentation.
[14]	Khattab, D., Hala, E., Ashraf, H. and Mohamed, T. (2014) Color Image Segmentation Based on Different Color Space Models Using Automatic GrabCut. The Scientific World Journal, 2014, Article ID: 126025. https://doi.org/10.1155/2014/126025
[15]	Hernández-Hernández, J.L., García-Mateos, G., González-Esquiva, J.M., Escarabajal-Henarejos, D., Ruiz-Canales, A. and Molina-Martínez, J.M. (2016) Optimal Color Space Selection Method for Plant/Soil Segmentation in Agriculture. Computers and Electronics in Agriculture, 122, 124-132. https://doi.org/10.1016/j.compag.2016.01.020
[16]	Shaik, K.B., Ganesan, P., Kalist, V., Sathish, B.S. and Jenitha, J. (2015) Comparative Study of Skin Color Detection and Segmentation in Hsv and Ycbcr Color Space. Procedia Computer Science, 57, 41-48. https://doi.org/10.1016/j.procs.2015.07.362
[17]	Miao, Z.J., et al. (2002) Fourier Transform Based Image Shape Analysis and Its Application to Ower Recognition. 6th International Conference on Signal Processing, Volume 2, 1087-1090. https://doi.org/10.1109/ICOSP.2002.1179978
[18]	Zaitoun, N.M. and Aqel, M.J. (2015) Survey on Image Segmentation Techniques. Procedia Computer Science, 65, 797-806. https://doi.org/10.1016/j.procs.2015.09.027
[19]	Demanet, L. and Jugnon, V. (2017) Convex Recovery from Interferometric Measurements. IEEE Transactions on Computational Imaging, 3, 282-295. https://doi.org/10.1109/TCI.2017.2688923
[20]	Bo, C. and Wang, D. (2016) Online Object Tracking Based on Convex Hull Representation. IEEE 22nd International Conference on Parallel and Distributed Systems, Wuhan, 13-16 December 2016, 1221-1224. https://doi.org/10.1109/ICPADS.2016.0164
[21]	Mukherjee, D., Nath, R., Ghosh, A., Mallick, S., Roy, M. and Dey, R. (2016) Disaster Forecasting Using Convex Hull & K-Median Approach. IEEE 7th Annual Information Technology, Electronics and Mobile Communication Conference, Vancouver, 13-15 October 2016, 1-4. https://doi.org/10.1109/IEMCON.2016.7746254
[22]	Agrawal, R. and Gupta, N. (2016) Real Time Hand Gesture Recognition for Human Computer Interaction. IEEE 6th International Conference on Advanced Computing, Bhimavaram, 27-28 February 2016, 470-475. https://doi.org/10.1109/IACC.2016.93
[23]	Fei, H. and Xia, L. (2010) A Improved Volume Rendering Algorithm Based on Convex Hull. The 2nd International Conference on Information Science and Engineering, Hangzhou, 3-5 December 2010, 1-3.
[24]	Liu, S. and Carpin, S. (2015) Fast Grasp Quality Evaluation with Partial Convex Hull Computation. IEEE International Conference on Robotics and Automation, Seattle, 26-30 May 2015, 4279-4285. https://doi.org/10.1109/ICRA.2015.7139789
[25]	Mücke, E. (2009) Quickhull: Computing Convex Hulls Quickly. Computing in Science & Engineering, 11, 54-57. https://doi.org/10.1109/MCSE.2009.136