Linear Dimension Reduction for Multiple Heteroscedastic Multivariate Normal Populations

Songthip T. Ounpraseuth; Phil D. Young; Johanna S. van Zyl; Tyler W. Nelson; Dean M. Young

doi:10.4236/ojs.2015.54033

Home > Journals > Physics & Mathematics > OJS

OJS> Vol.5 No.4, June 2015

Share This Article:

Open Access

Linear Dimension Reduction for Multiple Heteroscedastic Multivariate Normal Populations

Abstract Full-Text HTML XML

Download as PDF (Size:953KB) PP. 311-333

DOI: 10.4236/ojs.2015.54033 2,883 Downloads 3,312 Views

Author(s) Leave a comment

Songthip T. Ounpraseuth¹, Phil D. Young², Johanna S. van Zyl², Tyler W. Nelson², Dean M. Young²

Affiliation(s)

¹Department of Biostatistics, University of Arkansas for Medical Sciences, Little Rock, AK, USA.
²Department of Statistical Science, Baylor University, Waco, TX, USA.

ABSTRACT

For the case where all multivariate normal parameters are known, we derive a new linear dimension reduction (LDR) method to determine a low-dimensional subspace that preserves or nearly preserves the original feature-space separation of the individual populations and the Bayes probability of misclassification. We also give necessary and sufficient conditions which provide the smallest reduced dimension that essentially retains the Bayes probability of misclassification from the original full-dimensional space in the reduced space. Moreover, our new LDR procedure requires no computationally expensive optimization procedure. Finally, for the case where parameters are unknown, we devise a LDR method based on our new theorem and compare our LDR method with three competing LDR methods using Monte Carlo simulations and a parametric bootstrap based on real data.

KEYWORDS

Linear Transformation, Bayes Classification, Feature Extraction, Probability of Misclassification

Conflicts of Interest

The authors declare no conflicts of interest.

Cite this paper

Ounpraseuth, S. , Young, P. , van Zyl, J. , Nelson, T. and Young, D. (2015) Linear Dimension Reduction for Multiple Heteroscedastic Multivariate Normal Populations. Open Journal of Statistics, 5, 311-333. doi: 10.4236/ojs.2015.54033.

References

[1]	Bellman, R. (1961) Adaptive Control Processes: A Guide Tour. John Wiley, Princeton University, Princeton.

[2]	Jain, A.K. and Chandrasekaran, B. (1982) Dimensionality and Sample Size Considerations in Pattern Recognition Practice. In: Krishnaiah, P.R. and Kanal, L.N., Eds., Handbook of Statistics, Volume 2, Elsevier, Amsterdam, 201-213. http://dx.doi.org/10.1016/s0169-7161(82)02042-2

[3]	Rao, C.R. (1948) The Utilization of Multiple Measurements in Problems of Biological Classification. Journal of the Royal Statistical Society: Series B, 10, 159-203.

[4]	Fisher, R.A. (1936) The Use of Multiple Measurements in Taxonomic Problems. Annals of Eugenics, 7, 179-188. http://dx.doi.org/10.1111/j.1469-1809.1936.tb02137.x

[5]	McLachlan, G.J. (1992) Discriminant Analysis and Statistical Pattern Recognition. John Wiley, New York. http://dx.doi.org/10.1002/0471725293

[6]	Decell, H.P. and Mayekar, S.M. (1977) Maximum Likelihood from Incomplete Data via the EM Algorithm. Journal of the Royal Statistical Society, Series B, 39, 1-38.

[7]	Okada, T. and Tomita, S. (1984) An Extended Fisher Criterion for Feature Extraction—Malina’s Method and Its Problems. Electronics and Communications in Japan, 67, 10-16. http://dx.doi.org/10.1002/ecja.4400670603

[8]	Loog, M. and Duin, P.W. (2004) Linear Dimensionality Reduction via a Heteroscedastic Extension of LDA: The Chernoff Criterion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26, 732-739. http://dx.doi.org/10.1109/TPAMI.2004.13

[9]	Fukunaga, K. (1990) Introduction to Statistical Pattern Recognition. 2nd Edition, Academic Press, Boston.

[10]	Hennig, C. (2004) Asymmetric Linear Dimension Reduction for Classification. Journal of Computational and Graphical Statistics, 13, 930-945. http://dx.doi.org/10.1198/106186004X12740

[11]	Kumar, N. and Andreou, A.G. (1996) A Generalization of Linear Discriminant Analysis in a Maximum Likelihood Framework. Proceedings of the Joint Statistical Meeting.

[12]	Young, D.M., Marco, V.R. and Odell, P.L. (1987) Quadratic Discrimination: Some Results on Optimal Low-Dimen-sional Representation. The Journal of Statistical Planning and Inference, 17, 307-319. http://dx.doi.org/10.1016/0378-3758(87)90122-4

[13]	Peters, B.C., Redner, R. and Decell, H.P. (1978) Characterization of Linear Sufficient Statistics. Sankhya, 40, 303-309.

[14]	Brunzell, H. and Eriksson, J. (2000) Feature Reduction for Classification of Multidimensional Data. Pattern Recognition, 33, 1741-1748. http://dx.doi.org/10.1016/S0031-3203(99)00142-9

[15]	Cook, R.D. and Weisberg, S. (1991) Discussion of “Sliced Inverse Regression for Dimension Reduction” by K.-C. Li. Journal of the American Statistical Association, 86, 328-332.

[16]	Li, K.-C. (1991) Sliced Inverse Regression for Dimension Reduction. Journal of the American Statistical Association, 86, 316-327. http://dx.doi.org/10.1080/01621459.1991.10475035

[17]	Tubbs, J.D., Coberly, W.A. and Young, D.M. (1982) Linear Dimension Reduction and Bayes Classification with Unknown Population Parameters. Pattern Recognition, 15, 167-172. http://dx.doi.org/10.1016/0031-3203(82)90068-1

[18]	Schervish, M.J. (19884) Linear Discrimination for Three Known Normal Populations. Journal of Statistical Planning and Inference, 10, 167-175. http://dx.doi.org/10.1016/0378-3758(84)90068-5

[19]	Cook, R.D. (2000) SAVE: A Method for Dimension Reduction and Graphics in Regression. Communications in Statistics—Theory and Methods, 29, 2109-2121. http://dx.doi.org/10.1080/03610920008832598

[20]	Cook, R.D. and Yin, X. (1991) Dimension Reduction and Visualization in Discriminant Analysis. Australian and New Zealand Journal of Statistics, 43, 147-199. http://dx.doi.org/10.1111/1467-842X.00164

[21]	Vellila, S. (2012) A Note on the Structure of the Quadratic Subspace in Discriminant Analysis. Statistics and Probability Letters, 82, 739-747. http://dx.doi.org/10.1016/j.spl.2011.12.020