Face Recognition Based on Manifold Learning and Rényi Entropy

Though manifold learning has been successfully applied in wide areas, such as data visu-alization, dimension reduction and speech recognition ; few researches have been done with the combination of the information theory and the geometrical learning. In this paper, we carry out a bold exploration in this field, raise a new approach on face recognition, the intrinsic α-Rényi entropy of the face image attained from manifold learning is used as the characteristic measure during recognition. The new algorithm is tested on ORL face database, and the experiments obtain the satisfying results.


INTRODUCTION
Face recognition has becoming a research hotspot in the fields of image processing, pattern recognition and artificial intelligence in recent years.Numerous research papers appear on the famous international publications, a great deal of capital and manpower is invested to this research and its relevant application system developments.However, the performance of the face recognition could be influenced by many factors, such as illumination, gesture, age, facial expressions, image resolution, and noise, etc; which cause difficulties for the computer processing of face recognition, and turn it into a challenging task at the same time.
The existing face recognition methods can be roughly classified into two categories [1]: local feature based, and global feature based.A local feature based method symbolizes a human face with the extracted feature vectors (eyes, nose, mouth, hair, and face contours), designs certain kinds of classifiers to make recognition.On the other hand, a global feature based method employs the whole images as the input feature vectors, and then low-dimensional features are extracted by some learning algorithms.The main difference between the two categories lies in the way how the features are extracted.In a local feature based method, features are designed completely by the algorithm designers; while in global feature based method, features are automatically extracted or learned by some self-learning algorithms.
In recent years, nonlinear dimensionality reduction (NLDR) methods have attracted great attentions due to their capability to deal with nonlinear and curved data [1].All these algorithms rely on an assumption that the image data lie on or close to a smooth low-dimensional manifold in a high-dimensional input image space.A big limitation of NLDR algorithms is the way how to estimate the intrinsic dimension of the manifold.LLE and Laplacian Eigenmaps do not give method to estimate the intrinsic dimension; Isomap shows a simple way to estimate the intrinsic dimension by searching for the "elbow point" where the residual error decreases significantly.However, for some real data, it is difficult to find an obvious "elbow point" to indicate the intrinsic dimension.
The intrinsic dimensionality estimation of a data set is a classical problem of pattern recognition.From the math point, the intrinsic dimension of a manifold is the dimension of the vector space that is homeomorphic to local neighborhoods of the manifold; in other words, intrinsic dimension describes how many "degrees of freedom" are necessary to generate the observed data.When the samples are drawn from a large population of signals one can interpret them as realizations from a multivariate distribution supported on the manifold.The intrinsic entropy of random samples obtained from a manifold is an information theoretic measure of the complexity of this distribution.The entropy is a finite value when the distribution satisfies the restriction of Lebesgue integral on the lower dimensional manifold.
In 2003, Costa and Hero proposed an algorithm that can jointly estimate both the intrinsic dimension and intrinsic entropy of a dataset randomly distributed on a smooth manifold [17,18].The algorithm constructs the Euclidean K-NN graph over all the sample points and use its growth rate to estimate the intrinsic dimension and entropy by simple linear least squares and method of moments procedure.This approach allows for the estimation of the desired quantities using algorithms with low computational complexity that avoid reconstructing the manifold or estimating multivariate distributions.
Considering the fact that on the various conditions of distance, direction, illumination or gestures, one object can form different face images; the corresponding highdimensional dataset of those images can be viewed as a nonlinear low-dimensional embedded manifold determined by factors of illumination, location, scale, gesture, etc.Based on Costa's algorithm, we take Rényi entropy as the characteristic vector; present a novel face recognition method-Face Recognition Based on Manifold Learning and Rényi Entropy.

MANIFOLD LEARNING USING K-NEAREST NEIGHBOR GRAPHS
, where γ is a power weighting constant,   is the Euclidean distance between X and X i .The k-NN graph exhibits an asymptotic behavior of their total edge functional: is the extrinsic Rényi α-entropy of the multivariate Lebesgue density f.So the asymptotic unbiased and strongly consistent estimator of the α-entropy is: The convergence results for k-NN graphs are extended from Euclidean spaces to general Riemannian manifolds.Let (M, g) be a compact smooth Riemann m-dimensional manifold.Suppose is a constant independent of f and M. Accordingly, the asymptotic unbiased and strongly consistent estimator of the α-entropy , the convergence theorem (5) suggests the log-linear model below: The specific steps of the algorithm are described as: 1) Using entire database of signals 2) Estimate the dimension and α-Rényi entropy of the manifold that the sample sets lie in.a) Set parameters M, N, 1 , , The parameters M and N are used to provide a tradeoff between the bias and variance performance of the estimators for finite n.

FACE RECOGNITION ALGORITHM BASED ON MANIFOLD'S RÉNYI ENTROPY
We applied the method to a real high-dimensional dataset with unknown manifold structure and intrinsic dimension and entropy---face images.We chose "ORL Face Database" [19] of AT&T Laboratories Cambridge.There are 10 different images of each of 40 distinct people (40 sample sets).The images were taken at different times, varying the lighting, facial expressions (open / closed eyes, smiling / not smiling) and facial details (glasses / no glasses).The original image is in PGM format, 92*112 pixels, with 256 grey levels per pixel.We first processed each image into BMP format, 64*64 pixels, normalized the pixel values between [0, 1].Then we arranged each im-age in a 4096*1 matrix using the common lexicographic order.
From each of the 40 sample sets X would be classified to the set where its combined entropy is nearest to the origin entropy of the training set.

EXPERIMENT RESULTS
The parameters used in the experiment were set as follows:    3, our Rényi Entropy method gives a weakly higher recognition accuracy compared with PCA and 2DPCA methods for ORL face database in Figure 3.
Table 1 shows a part of estimate results of the training set from ORL face database.
Using the Face recognition algorithm in section3, the results of our method are showed in Table 2.
The results indicate that the face recognition algorithm based on Manifold's Rényi Entropy we present in this paper works effectively.The K-NN graph is employed so that we avoid the complex estimation of geodesic distances on the manifold.
An important factor influenced the recognition efficiency is the sample number of the same set we used is a little small (only 10), which induce the lack of information needed for training samples.Besides, there must be certain inevitable information loss during the procedure of image pretreatment.Further work includes add bootstrap confidence intervals for the estimators and apply other classification method to minimize the error.

CONCLUSIONS
We have presented a new face recognition algorithm based on Manifold's Rényi Entropy.The algorithm is applied to ORL face database and the experiments obtain the satisfying results.At present we are studying the theoretic prove of this method, and trying to make further utilizations in other research fields such as pattern recognition and artificial intelligence.
i.d.random elements of M with bounded density f relative to the volume element g  .Let L be the total edge leng- th of the k-NN graph.Assume 2

.
is an error residual that goes to zero as n   .Bootstrap resampling is used here to estimate mean graph length After that, the following estimates of dimension m and entropy Ĥ are obtained by inversion of the relations:

1 m
, m L   is the limit of the normalized length functional of the corresponding Euclidean entropic graph for a uniform distribution on the unit cube   0,1 m .It can be determined by performing Monte Carlo simulations of the entropic graph length on the unit cube   0,for uniform random samples.Unlike previous solutions, Costa's algorithm can prove statistical consistency of the obtained estimators under weak assumptions of compactness of the manifold and boundedness of the (Lebesgue) sampling density supported on the manifold.This approach allows for the estimation of the desired quantities using algorithms with low computational complexity (   log O n n ) that avoid reconstructing the manifold or estimating multivariate distributions.

Figure 1
shows the 5 training images of face 1; and Figure 2 displays the results of running 20 simulations of the Costa's algorithm
Randomly select a subset of P signals p y from

Table 1 .
Dimension estimate m and entropy Ĥ for some

Table 2 .
The results of Face recognition algorithm based on Manifold's Rényi Entropy.