An Actual Survey of Dimensionality Reduction


Dimension reduction is defined as the processes of projecting high-dimensional data to a much lower-dimensional space. Dimension reduction methods variously applied in regression, classification, feature analysis and visualization. In this paper, we review in details the last and most new version of methods that extensively developed in the past decade.

Share and Cite:

Sarveniazi, A. (2014) An Actual Survey of Dimensionality Reduction. American Journal of Computational Mathematics, 4, 55-72. doi: 10.4236/ajcm.2014.42006.

Conflicts of Interest

The authors declare no conflicts of interest.


[1] Donoho, D.L. (2000) High-Dimensional Data Analysis: The Curses and Blessings of Dimensionality. Lecture Delivered at the “Mathematical Challenges of the 21st Century” Conference of the American Math. Society, Los Angeles.
[2] Diamantaras, K.I. and Kung, S.Y. (1996) Principal Component Neural Networks: Theory and Applications. John Wiley, NY.
[3] Person, K. (1901) On Lines and Planes of Closest Fit to System of Points in Space. Philiosophical Magazine, 2, 559-572.
[4] Jenkins, O.C. and Mataric, M.J. (2002) Deriving Acion and Behavior Primitives from Human Motion Data. International Conference n Robots and Systems, 3, 2551-2556.
[5] Jain, A.K. and Dubes, R.C. (1962) Algorithms for Clastering Data. Prentice Hall, Upper Saddle River.
[6] Mardia, K.V., Kent, J.T. and Bibby, J.M. (1995) Multivariate Analysis Probability and Mathematical Statistics. Academic Press, Waltham.
[7] (2002) Francesco Camastra Data Dimensionality Estimation Methods, a Survey INFM-DISI, University of Genova, Genova.
[8] Fukunaga, K. (1982) Intrinsic Dimensionality Extraction, in Classification, Pattern Recognition and Reduction of Dimensionality, Vol. 2 of Handbook of Statistics, North Holland, 347-362.
[9] Torgerson, W.S. (1952) Multidimenmsional Scaling I: Theory and Methode. Psychometrika, 17, 401-419.
[10] Teng, L., Li, H., Fu, X., Chen, W. and Shen, I-F. (2005) Dimension Reduction of Microarrey Data Based on Local Tangent Space Aligment. Proceedings of the 4th IEEE international Conference on Cogenitive Informatics, 154-159.
[11] Williams, C.K.I. (2002) On a Connection between Kernel PCA and Metric Multidimensional Scaling. Machine Learning, 46, 11-19.
[12] Chatfield, C. and Collins, A.J. (1980) Introduction to Multivariate Analysis. Chapman and Hill.
[13] Platt, J.C. (2005) FastMap, MetricMap, and Landmark MDS are all Nystrom algorithms. Proceddings of the 10th International Workshop on Artificial Intelligence and Statistics, 15, 261-268.
[14] Roweis, S.T. (1997) EM Algorithms for PCA and SPCA. Advances in Neural Information Processing Systems, 10, 626-632.
[15] Lawrence, N.D. (2005) Probabilistic Non-Linear Proncipal Component Analysis with Gaussian Process Latent Variable Models. Journal of Machine Learning Research, 6, 1783-1816.
[16] Welling, M., Rosen-Zvi, M. and Hinton, G. (2004) Exponential Family Harmoniums with an Application to Information Retrieval. Advances in Neural Information Processing Systems, 17, 1481-1488.
[17] Turk, M.A. and Pentland, A.P. (1991) Face Recognition Using Eigenfaces. Proceedings of the Computer Vision and Pattern Recognition 1991, Maui, 586-591.
[18] Huber, R., Ramoser, H., Mayer, K., Penz, H. and Rubik, M. (2005) Classification of Coins Using an Eigenspace Approach. Pattern Recognition Letters, 26, 61-75.
[19] Posadas, A.M., Vidal, F., de Miguel, F., Alguacil, G., Pena, J., Ibanez, J.M. and Morales, J. (1993) Spatialtemporal Analysis of a Seismic Series Using the Principal Components Method. Journal of Geophysical Research, 98, 1923-1932.
[20] Partridge, M. and Calvo, R. (1997) Fast Dimensionality Reduction and Simple PCA. Intelligent Data Analysis, 2, 292-298.
[21] Scholkopf, B., Smola, A. and Müller, K.R. (1998) Nonlinear Component Analysis as a Kernel Eigenvalue Problem. Neural Computation, 10, 1299-1319.
[22] Shawe-Taylor, J. and Christianini, N. (2004) Kernel Methods for Pattern Analysis. Cambridge University Press, Cambridge.
[23] Tipping, M.E. (2000) Sparse Kernel Principal Component Analysis. Advances in Neural Information Processing Systems, 13, 633-639.
[24] Kim, K.I., Jung, K. and Kim, H.J. (2002) Face Recognition Using Kernel Principal Component Analysis. IEEE Signal Processing Letters, 9, 40-42.
[25] Hoffmann. H. (2007) Kernel PCA for Novelty Detection. Pattern Recognition, 40, 863-874.
[26] Lima, A., Zen, H. Nankaku, Y. Miyajima, C. Tokuda, K. and Kitamura. T. (2004) On the Use of Kernel PCA for Feature Extraction in Speech Recognition. IEICE Transactions on Information Systems, E87-D, 2802-2811.
[27] Duda, R.O., Hart, P.E. and Stork, D.G. (2001) Pattern Classification, Wiley Interscience, New York.
[28] Shin, Y.J. and Park, C.H. (2011) Analysis of Correlation Based Dimension Reduction Methods. International Journal of Applied Mathematics and Computer Science, 21, 549-558.
[29] Fukunaga, K. (1990) Introduction to Statistical Pattern Recognition. 2nd Edition, Academic Press, San Diego.
[30] Hotelling, H. (1936) Relations between Two Sets of Vertices. Biometrika, 28, 321-377.
[31] Sun, Q., Zeng, S., Liu, Y., Heng, P. and Xia, D. (2005) A New Methode of Feature Fusion and Its Application in Image Recognition. Pattern Recognition, 38, 2437-2448.
[32] Hastie, T. and Stuezle, W. (1989) Principal Curves. Journal of the American Statistical Association, 84, 502-516.
[33] Kegl, B. and Linder, T. (2000) Learning and Design of Principal Curves. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22, 281-297.
[34] Ozertem, U. and Erdogmus, D. (2011) Locally Defined Principal Curves and Surfaces. Journal of Machine Learning Research, 12, 1249-1286.
[35] Malthouse, E. (1996) Some Theoretical Results on Nonlinear Principal Component Analysis.
[36] Carreira-Perpinan, M.A. (1997) A Review of Dimension Reduction Tecniques. Technical Report CS-96-09. Department of Computer Science, University of Sheffield, Sheffield.
[37] Tibshirani, R. (1992) Principal Curves Revisited. Statistics and Computing, 2,183-190.
[38] Bishop, C.M. (1995) Neural Networks for Pattern Recognition. Oxford University Press, New York.
[39] Ripley, B.D. (1996) Pattern Recognition and Neural Networks. Cambridge University Press, Cambridge.
[40] Spierenburg, J.A. (1997) Dimension Reduction of Images Using Neural Networks. Master’s Thesis, Leiden University, Leiden.
[41] Kramer, M.A. (1991) Non-Linear Principal Component Analysis Using Associative Neural Networks. AIChE Journal, 37, 233-243.
[42] Press, W.H., Flannery, B.P., Teukolsky, S.A. and Vettering, W.T. (1992) Numerical Recips in C: The Art of Scientific Computing. 2nd Edition, Cambridge University Press, Cambridge.
[43] Fowlkers, C., Belongie, S., Chung, F. and Malik, J. (2004) Specral Grouping Using the Nysroem Method. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26, 214-225.
[44] Marida, K.V., Kent, J.T. and Bibby, J.M. (1995) Multivariate Analysis. Probability and Mathematical Statistics. Academic Press, Waltham.
[45] Faloutsos, C. and Lin, K.I. (1995) FastMap: A Fast Algorithm for Indexing, Data-Mining and Visualization of Traditional and Multimedia Datasets. In: Carey, M.J. and Schneider, D.A., Eds., Proceedings of the 1995 ACM SIGMOD International Conference on Management of Data, San Jose, 163-174.
[46] Fodor, I.K. (2002) A Survey of Dimension Reduction Techniques. Center for Applied Scientific Computing, Livermore National Laborary, Livermore.
[47] Chung, F.R.K. (1997) Spectral Graph Theory. American Mathematical Society. CBMS Regional Conference Series in Mathematics in American Mathematical Society, 212, 92.
[48] Belkin, M. and Niyogi, P. (2003) Laplacian Eigenmaps for Dimensionality Reduction and Data Representation. Neural Computation, 15, 1373-1396.
[49] Rivest, R., Cormen, T., Leiserson, C. and Stein, C. (2001) Introduction to Algorithms. MIT Press, Cambridge.
[50] Kumar, V., Grama, A., Gupta, A. and Karypis, G. (1994) Introduction to Parallel Computing. Benjamin-Cummings, Redwood City.
[51] Donoho, D. and Grimes, C. Hessian Eigenmaps: Locally Linear Embedding Techniques for High-Dimensional Data. Proceedings of National Academy of Sciences, 100.
[52] Ye, Q. and Zhi, W.F. (2003) Discrete Hessian Eigenmaps Method for Dimensionality Reduction.
[53] Kamhaltla, N. and Leen, T.K. (1994) Fast Non-Linear Dimension Reduction. In: Advances in Neural Information Processing Systems, Morgan Kaufmann Publishers, Inc., Burlington, 152-159.
[54] Goldberg, D.E. (1989) Genetic Algorithms in Search, Optimization and Machin Learning. Addisn Wesley, Reading.
[55] Raymer, M.L., Goodman, E.D., Kuhn, L.A. and Jain, A.K. (2000) Dimensionality Reduction Using Genetic Algorithms. IEEE Transactions on Evolutionary Computation, 4, 164-171.
[56] Jones, G. (2002) Published Online: 15 APR. University of Sheffield, Sheffield.
[57] Kohavi, R. and John, G. (1998) The Wrapper Approach. In: Liu, H. and Motoda, H., Eds., Feature Extraction, Construction and Selection: A Data Mining Perspective, Springer Verlag, Berlin, 33-50.
[58] Huber, P.J. (1985) Projection Persuit. Annals of Statistics, 13, 435-475.
[59] McCullagh, P. and Nelder, J.A. (1989) Generalized Linear Models. Chapman and Hall, Boca Raton.
[60] Dobson, A.J. (1990) An Introduction to Generalized Linear Models. Chapman and Hall, London.
[61] Leathwick, J.R. Elith, J. and Hastie, T. (2006) Comparative Performance of Generalized Additive Models and Multivariate Adaptive Regression Splines for Statistical Modelling of Species Distributions. Ecological Modelling, 188-196.
[62] Li, K.C. (2000) High Dimensional Data Analysis via SIR/PHD Approach. Lecture Note in Progress.
[63] Dennis Cook, R. and Li, B. (2002) Dimension Reduction for Conditional Mean in Regression. Annals of Statistics, 30, 455-474.
[64] Zhang, Z. and Zha, H. (2002) Principal Manifolds and Nonlinear Dimension Reduction via Local Tangent Space Alignment.
[65] Wang, H. and Xia, Y. (2008) Sliced Regression for Dimension Reduction. Peking University & National University of Singapore, Journal of the American Statistical Association, 103, 811-821.
[66] Feng, W.K., He, X. and Shi, P. (2002) Dimension Reduction Based on Canonical Correlation. Statistica Sinica, 12, 1093-1113.
[67] Lectures on Fractals and Dimension Theory.

Copyright © 2023 by authors and Scientific Research Publishing Inc.

Creative Commons License

This work and the related PDF file are licensed under a Creative Commons Attribution 4.0 International License.