Recognition of Bangla Handwritten Number Using Combination of PCA and FIS with the Aid of DWT

The structure of any Bangla numerical character is more complex compared to English numerical character. Two pairs of numerical character in Bangla resembles to be closed and they are: “one and nine” and “five and six”. We found that, handwritten Bangla numerical character cannot be recognized using single machine learning algorithm or discrete wavelet transform (DWT). Above phenomenon motivated us to use combination of DWT, Fuzzy Inference System (FIS) and Principal Component Analysis (PCA) to recognize numerical characters of Bangla in handwritten format. The four lowest spectral components of a preprocessed image are taken using DWT, which is considered as the feature vector to recognize the digits in first phase. The feature vector is then applied to FIS and PCA separately. The combined method provides recognition accuracy of 95.8% whereas application of individual method gives less rate of accuracy. Instead of storing the images itself in a folder, if we can store the feature vector of images achieved from DWT in tabular form. The records of table can be applied in FIS, PCA or other object detection algorithm. Although the technique used in the paper can detect objects with moderate rate of accuracy but can save huge storage against a benchmark database of images. If a tradeoff is made between storage requirements and accuracy of recognition, the model of the paper is preferable compared to other present state-of-art. Another finding of the paper is that, the spectral components of images acquired by DWT only matched with FIS and PCA for classification but do not match properly with unsupervised (K-mean clustering) and supervised (support vector machine) learning.


Introduction
Huge number of works relevant to object detection and recognition using machine learning is found in recent literature. In this section, we will search some works pertinent to DWT, FIS and PCA in image classification or identification.
Nawaf Hazim Barnouti et al., discussed about combination of DWT and DCT that has been used for embedding and extraction copyright protection by using digital watermarking method in [1]. This two method DWT + DCT applied on two-dimensional images and works in frequency domain which seems to be more robust as found in its result section. Pooja B. Minajagi et al., proposed a method about segmentation of brain MRI image using Fuzzy c means clustering (FCM) and DWT in [2]. The paper provides level set segmentation using fuzzy c means based on special features (SFCM) and segmentation of brain MRI images using DWT algorithm. The performance evaluation is done by computing mean square error, peak signal to noise ratio (PSNR), maximum difference, absolute mean error etc.
Another work in combination of DWT and DCT in digital watermarking of color image is found in [3] by Ravinder Singh et al. The watermarking technique of the paper selected one color component from RGB image, is applicable in embedded watermarking since it requires only red component as discussed in [3]. T. Sridevi et al., implemented a robust watermarking using fuzzy logic approach based on DWT and SVD algorithm in [4]. The fuzzy logic decided how much of watermark has to be added to the cover image, which is based on the image properties as shown clearly in [4].
A classifier based on fuzzy if-then rules that allows the incorporation of weighted training patterns is proposed in [5]. The antecedent part of fuzzy if-then rules are specified by partitioning each attributes into fuzzy sets while the consequent class and the degree of certainty are determined from the compatibility and weights of training patterns. A learning method which adjusts the degree of certainty to improve performance of classification and reduce costs as introduced in [5]. In [6] fuzzy logic-based image processing is used for accurate and noise free edge detection and Cellular Learning Automata (CLA) is used to enhance the previously-detected edges with the help of the repeatable and neighborhood considering nature of CLA. The different results of edge detection technique are compared with fuzzy edge detected and resulting edge is enhanced using CLA. The authors in [7] deal with Fuzzy logic for the automatic analysis of X-ray images of industrial products for defect detection. A to stage algorithm is presented based on the feature analysis of the radiographic images obtained from the inspected product.
In [8], authors developed a semi-automated fuzzy inference system to detect the internal architecture of a mass transport complex (MTC) in seismic images. tering fails to identify image from the spectral data of an image. We found that DWT + FIS combination only works to achieve reasonable rate of accuracy of recognition. Application of PCA in object detection is found in recent literature, for example upper part of human body is detected by PCA in [11]. The performance of PCA is compared with HOG, BDPCA and Haar cascade, where no combined scheme is used. In [12], authors claim that CNN has inherent problem of over-fitting and to overcome the problem they combined PCA with CNN for object detection and recognition in robot-aided visual system. Analyzing all the previous works mentioned here, we found two research gaps.
None of above papers finds the matching of spectrum of DWT with FIS and PCA in object recognition. Another finding of the paper is that, DWT has mismatch with K-mean clustering and SVM in object recognition. This paper for the first time applies DWT + FIS + PCA to recognize handwritten Bangla digits including image enhancement and morphological operation. We successfully recognize Bangla handwritten digits and compare our results with some previous works and got better accuracy of recognition, mentioned at the end of the result section.
The rest of the paper is organized as: Section 2 deals with basic theory of DWT, FIS and PCA along with experimental setup of object recognition, Section 3 provides results based on analysis of Section 2. Finally Section 4 concludes entire analysis.

Methodology
In this paper image recognition is done using DWT, FIS and PCA. This section will deal with basic theory of wavelet transform, FIS and PCA then the experimental setup to combine above three methods.

Wavelet Transform
Wavelet is an oscillatory function of finite duration. If the sinusoidal wave is expressed as [13] [14], where a and b are real (scaling and shifting parameter) and * denotes conjuga- Here d(m, n) is equivalent to continuous wavelet transform W(a, b) when a = 2 m and b = n2 m .

Fuzzy Inference System
FIS is a nonlinear mapping by means of fuzzy logic, from a given set of input value to one or more output values. To produce the expected outputs, it takes inputs and processes them based on the pre-specified rules. Fuzzy rules and fuzzy arithmetic is used in the internal processing. In the fuzzy inference system, real value is used in both the input and output units. The basic structure of a fuzzy inference system consists of a set of conceptual components as shown in Figure 1 as mentioned in [15] [16].

Principal Component Analysis
PCA is widely used in objection recognition or detection with reduction of variable. In this paper the feature vector derived from DWT is applied in PCA to enhance accuracy of object recognition. The steps of PCA algorithm is given below as [17] [18]. 3) The covariance matrix is evaluated as:

Implementation
The steps of preprocessing of image consists of RGB to grayscale conversion, de-noising of image using filter, enhancement of image and finally thinning scheme as shown in Figure 2. The signal vector of the image is extracted using row and column wise DWT, the corresponding algorithm is given in subsection 2.5. Actually each row of the preprocessed image is applied in a filter bank of Figure 3 consists of lowpass (LP) filter of impulse response h(n) and highpass (HP) filter of impulse response g(n) like [19]. Each of the filtered signal is down sampled by a factor of two hence the length of the signal vector of output of the sampler is half of its input. The HP filter generates detail component and the LP provides the approximate component. The approximate component is further split into approximate and detail components.
One dimensional DWT is applied on each row of the preprocessed image until reducing to one element. The single element from each row forms a signal vector, which is again applied to one dimensional DWT until getting a column Journal of Computer and Communications   Table 1 for nine "0", "1" and "2". The scatterplot of a-b and c-d are shown in Figure 4(a) and Figure 4(b). The region of digits "0", "1" and "2" on a-b and c-d using k-means clustering are shown in Figure 4  We got better matching using FIS and PCA, which is highlighted in next section.

Results
Few images of Bangla handwritten numerical character are shown in Figure 5 (before preprocessing) taken from benchmark Indian database (Character Databases of Indic Scripts). The URL of the database is: (https://www.isical.ac.in/~ujjwal/download/database.html) taken on 30 th December 2018. The original image, enhanced image, image with thinning scheme and the result of the proposed algorithm is shown in Figure 6 for four image of each character taken randomly from the database. Here we resize each grayscale image as 256 × 256 and apply DWT on each row of the image until getting a single value against each row. The output of DWT now will be a column vector of size 256 × 1. Next we apply DWT on the final column vector 3 times, therefore the size of the feature vector becomes, 256/2 3 = 32 as mention in subsection 2.5. Journal of Computer and Communications Taking the one "dimensional DWT vector" of co-efficient of length 16, we get the profile like Figure 7. Here we consider only five image of digit 1, 2, 3, 4 and 5. Each digit reveals distinct feature. Reducing the length of vector of length four we get the following data (Table 1)     The combination of DWT and PCA are also properly matched in object detection as found in this paper. Taking the DWT co-efficient of Table 1 against three digits: 0 (object-1), 1 (object-2) and 3 (object-3), we determine four principal components of each object as shown in Figures 11(a)-(d) separately. Each of the four principal components are widely separated and shows better separation compared to Figure 7 hence combination of DWT and PCA works better than DWT alone.
The impact of size of vector V of DWT and the size of preprocessed image on accuracy of recognition is shown in Table 2. The accuracy of recognition of ten objects (Bangla digits) are determined by four techniques as: DWT of [21], PCA + DWT under the concept of [22] [23], FIS + DWT using the technique of [24] and FIS + PCA + DWT as the proposed method. The accuracy increases with increase in size of vector V and that of image for all four cases. The PCA + DWT shows better result compared to FIS + DWT for larger V or size of image. The combination of three schemes outperforms compared to other three cases of Table 2. Three techniques of object recognition are combined using entropy

Conclusion
In this paper, we recognize Bangla handwritten digits using combination of PCA and FIS, taking the feature vector of DWT. We compare the results of our proposed method with some previous works of object recognition and we get better accuracy of recognition. One limitation of the paper is that we did not compare the process time or complexity of algorithms. In future, we will combine more object recognition algorithm to recognize Bangla vowels, consonants and digits all together. The concept of the paper is applicable in any kind of object detection/recognition, although the accuracy of recognition may vary for different type of objects and quality of image. Inclusion of DWT will save the memory against storing the database of images. Still we have the scope to use other object recognition algorithms like, PCA, LDA, SURF, HOG and CNN for comparison in context of accuracy of recognition and process time so that we can select appropriate algorithm for real time operation of computer vision.