Estimation of Regression Model Using a Two Stage Nonparametric Approach

Desale Habtzghi, Jin-Hong Park

Department of Mathematics, College of Charleston, Charleston, USA.
Department of Statistics, University of Akron, Akron, USA.

DOI: 10.4236/am.2013.48159                

Abstract

Based on the empirical or theoretical qualitative information about the relationship between response variable and covariates, we propose a new approach to model polynomial regression using a shape restricted regression after estimating the direction by sufficient dimension reduction. The purpose of this paper is to illustrate that in the absence of prior information other than the shape constraints, our approach provides a flexible fit to the data and improves regression predictions. We use central subspace to estimate the directions and fit a final model by shape restricted regression, when the shape is known or is stipulated from empirical inspection. Comparisons with an alternative nonparametric regression are included. Simulated and real data analyses are conducted to illustrate the performance of our approach.

Keywords

Dimension Reduction; Central Subspace; Polynomial Regression; Shape Restricted

Conflicts of Interest

The authors declare no conflicts of interest.

References

[1]	R. Luss, S. Rosset and M. Shahar, “Decomposing Isotonic Regression for Efficiently Solving Large Problems,” Proceedings of the Neural Information Processing Systems Conference, Vancouver, 6-9 December 2010, pp. 1513-1521.
[2]	W. Maxwell and J. Muckstadt, “Establishing Consistent and Realistic Reorder Intervals in Production-Distribution Systems,” Operations Research, Vol. 33, No. 6, 1985, pp. 1316-1341. doi:10.1287/opre.33.6.1316
[3]	X. Yin and R. D. Cook, “Direction Estimation in SingleIndex Regressions,” Biometrika, Vol. 92, No. 2, 2005, pp. 371-384. doi:10.1093/biomet/92.2.371
[4]	X. Yin, B. Li and R. D. Cook, “Successive Direction Extraction for Estimating the Central Subspace in a Multiple-Index Regression,” Journal of Multivariate Analysis, Vol. 99, No. 8, 2008, pp. 1733-1757. doi:10.1016/j.jmva.2008.01.006
[5]	G. Obozinski, G. Lanckriet, C. Grant, M. Jordan and W. Noble, “Consistent Probabilistic Outputs for Protein Function Prediction,” Genome Biology, Vol. 9, No. 1, 2008, pp. 247-254. doi:10.1186/gb-2008-9-s1-s6
[6]	Z. Zheng, H. Zha and G. Sun, “Query-Level Learning to Rank Using Isotonic Regression,” 46th Annual Allerton Conference on Communication, Control, and Computing, Allerton House, 24-26 September 2008, pp. 1108-1115.
[7]	M. J. Schell and B. Singh, “The Reduced Monotonic Regression Method,” Journal of the American Statistical Association, Vol. 92, No. 437, 1997, pp. 128-35. doi:10.1080/01621459.1997.10473609
[8]	R. Barlow and H. Brunk, “The Isotonic Regression Problem and Its Dual,” Journal of the American Statistical Association, Vol. 49, No. 5, 1972, pp. 784-789.
[9]	J. Kruskal, “Multidimensional Scaling by Optimizing Goodness of Fit to a Nonmetric Hypothesis,” Psychometrika, Vol. 29, No. 1, 1964, pp. 1-27. doi:10.1007/BF02289565
[10]	S. Weisberg, “Applied Linear Regression,” Wiley, Hoboken, 2005.
[11]	T. Robertson, F. T. Wright and R. L. Dykstra, “Order Restricted Statistical Inference,” John Wiley & Sons, New York, 1988.
[12]	Y. Xia, “A Constructive Approach to the Estimation of Dimension Reduction Directions,” The Annals of Statistics, Vol. 35, No. 6, 2007, pp. 2654-2690. doi:10.1214/009053607000000352
[13]	D. W. Scott, “Multivariate Density Estimation: Theory, Practice, and Visualization,” John Wiley & Sons, New York, 1992. doi:10.1002/9780470316849
[14]	P. Gill, W. Murray and M. H. Wright, “Practical Optimization,” Academic Press, New York, 1981.
[15]	K. C. Li, “On Principal Hessian Directions for Data Visualization and Dimension Reduction: Another Application of Stein’s Lemma,” Annals of Statistics, Vol. 87, No. 420, 1992, pp. 1025-1039.
[16]	J. R. Schott, “Determining the Dimensionality in Sliced Inverse Regression,” Journal of the American Statistical Association, Vol. 89, No. 425, 1994, pp. 141-148. doi:10.1080/01621459.1994.10476455
[17]	Y. Xia, H. Tong, W. K. Li and L. X. Zhu, “An Adaptive Estimation of Dimension Reduction,” Journal of the Royal Statistical Society, Ser. B, Vol. 64, No. 3, 2002, pp. 363-410. doi:10.1111/1467-9868.03411
[18]	D. A. S. Fraser and H. Massam, “A Mixed Primal-Dual Bases Algorithm for Regression under Inequality Constraints. Application to Convex Regression,” Scandinavian Journal of Statistics, Vol. 16, 1989, pp. 65-74.
[19]	R. T. Rockafellar, “Convex Analysis,” Princeton University Press, 1970.
[20]	E. Seijo and B. Sen, “Nonparametric Least Squares Estimation of a Multivariate Convex Regression Function,” Annals of Statistics, Vol. 39, No. 2, 2011, pp. 1633-1657. doi:10.1214/10-AOS852
[21]	M. J. Silvapulle and P. K. Sen, “Constrained Statistical Inference, Inequality, Order, and Shape Restrictions,” Wiley, New York, 2005.
[22]	M. C. Meyer, “Inference for Multiple Isotonic Regression,” Technical Report, Colorado State Uinversity, 2010.
[23]	M. C. Meyer, “An Extension of the Mixed Primal-Dual Bases Algorithm to the Case of More Constraints than Dimensions,” Journal of Statistical Planning and Inference, Vol. 81, No. 1, 1999, pp. 13-31. doi:10.1016/S0378-3758(99)00025-7
[24]	J.-H. Park, T. Sriram and X. Yin, “Dimension Reduction in Time Series,” Statistica Sinica, Vol. 20, 2010, pp. 747770.
[25]	H.-G. Muller and J.-L. Wang, “Hazard Rate Estimation under Random Censoring with Varying Kernels and Bandwidths,” Biometrics, Vol. 50, No. 1, 1994, pp. 61-76. doi:10.2307/2533197