Robust Service Time Measurement Using Comparison Sequential Test

Yefim Haim Michlin^1*, Genady Ya. Grabarnik², Larisa Shwartz³, Ofer Shaham¹

¹Technion-Israel Institute of Technology, Haifa, Israel.
²Department of Math and Computer Science, St. John’s University, Queens, NY, USA.
³IBM T.J. Watson Research Center, Yorktown, NY, USA.

DOI: 10.4236/jssm.2015.85072                

Abstract

The sequential comparison test is a tool for evaluation of the operational innovation in information technology service delivery processes. Due to the strong variability of these processes, the evaluation is done in comparison with the parallel running servers taken as reference. We consider the streams of service-completion events. When the time between events (TBE) is exponentially distributed, the binomial sequential probability ratio test (SPRT) can be used for evaluation. The effect of deviations from the exponential distribution on the characteristics of the test is analysed. We suggest a novel criterion that allows analysing robustness of the test. We show that the main factor influencing these characteristics is the coefficients of variation (CV) of the TBEs. Thus just by using CV of the TBEs, we may conclude whether the test is robust or not. We also suggest approach of handling the case when test for pair of single TBEs is not robust (case of CV > 1). Transition from a single server to a group of servers and from a single stream to a superposed stream of events improves robustness, since superposition of event streams brings the TBEs’ distribution closer to the exponential. Superposition makes it possible to deal with the problem for CV > 1. The analytical dependency of the fixed sample size test (FSST) robustness vs. CV permits simple estimation of robustness of the test in question. The advantage of the test is shown vs. the FSST, and illustrated on a real-life case.

Keywords

SPRT, Service Time, Events Stream, Superposition, Information Technology

Conflicts of Interest

The authors declare no conflicts of interest.

References

[1]	Grabarnik, G.Y., Michlin, Y.H. and Shwartz, L. (2012) Designing Pilot for Operational Innovation in IT Service Delivery. Network Operations and Management Symposium (NOMS), Maui, 16-20 April 2012, 1343-1351.
[2]	Gans, N., Koole, G. and Mandelbaum, A. (2003) Telephone Call Centers: Tutorial, Review, and Research Prospects. Manufacturing & Service Operations Management, 5, 79-141. http://dx.doi.org/10.1287/msom.5.2.79.16071
[3]	Michlin, Y.H. and Grabarnik, G. (2007) Sequential Testing for Comparison of the Mean Time between Failures for Two Systems. IEEE Transactions on Reliability, 56, 321-331. http://dx.doi.org/10.1109/TR.2007.896679
[4]	Brown, L., Gans, N., Mandelbaum, A., Sakov, A., Zeltyn, S., Zhao, L. and Haipeng, S. (2005) Statistical Analysis of a Telephone Call Center: A Queueing-Science Perspective. Journal of the American Statistical Association, 100, 36-50. http://dx.doi.org/10.1198/016214504000001808
[5]	Daley, D.J. and Vere-Jones, D. (2008) An Introduction to the Theory of Point Processes. Vol. II. Springer, New York. http://dx.doi.org/10.1007/978-0-387-49835-5
[6]	Kella, O. and Stadje, W. (2006) Superposition of Renewal Processes and an Application to Multi-Server Queues. Statistics & Probability Letters, 76, 1914-1924. http://dx.doi.org/10.1016/j.spl.2006.04.041
[7]	Khinchine, A.Ya. (1955) Mathematical Methods in the Theory of' Queueing, Moscow. Translation (1960), Griffin, London.
[8]	Palm, C. (1943) Intensity Variations in Telephone Traffic. Ericsson Technics, 44, 1-189. (English Translation by North-Holland, Amsterdam, 1988.)
[9]	Wald, A. (1947) Sequential Analysis. John Wiley & Sons, New York.
[10]	Wald, A. and Wolfowitz, J. (1948) Optimum Character of the Sequential Probability Ratio Test. The Annals of Mathematical Statistics, 19, 326-339. http://dx.doi.org/10.1214/aoms/1177730197
[11]	Hollander, M. and Proschan, F. (1972) Testing Whether New Is Better Than Used. The Annals of Mathematical Statistics, 43, 1136-1146. http://dx.doi.org/10.1214/aoms/1177692466
[12]	IEC 61703-2001. Mathematical Expressions for Reliability, Availability, Maintainability and Maintenance Support Terms.
[13]	Michlin, Y.H., Grabarnik, G. and Leshchenko, E. (2009) Comparison of the Mean Time between Failures for Two Systems under Short Tests. IEEE Transactions on Reliability, 58, 589-596. http://dx.doi.org/10.1109/TR.2009.2020102
[14]	Michlin, Y.H. and Grabarnik, G. (2010) Search Boundaries of Truncated Discrete Sequential Test. Journal of Applied Statistics, 37, 707-724. http://dx.doi.org/10.1080/02664760903254078
[15]	Michlin, Y.H. and Grabarnik, G. (2011) Comparison Sequential Test for Mean Times between Failures. In: Eldin, A.B., Ed., Modern Approaches to Quality Control, InTech Pub., 453-476. http://www.intechopen.com/source/pdfs/22149/InTech-Comparison_sequential_test_for_mean_times_between_failures.pdf
[16]	Michlin, Y.H., Ingman, D. and Dayan, Y. (2011) Sequential Test for Arbitrary Ratio of Mean Times between Failures. International Journal of Operations Research and Information Systems, 2, 66-81. http://dx.doi.org/10.4018/joris.2011010103
[17]	Michlin, Y.H., Ingman, D. and Levin-David, L. (2012) Sequential Test for Reliability under Allowance for Target Uncertainty. IEEE Transactions on Reliability, 61, 1019-1029. http://dx.doi.org/10.1109/TR.2012.2220911
[18]	Michlin, Y.H., Kaplunov, V. and Ingman, D. (2012) Sequential Testing for Two Exponential Distributions at Arbitrary Risks. International Journal of Quality & Reliability Management, 29, 451-468. http://dx.doi.org/10.1108/02656711211224884
[19]	Huber, P.J. (1981) Robust Statistics. Wiley, Hoboken. http://dx.doi.org/10.1002/0471725250
[20]	Staudte, R.G. (1990) Robust Estimation and Testing. Wiley, New York. http://dx.doi.org/10.1002/9781118165485
[21]	Stigler, S.M. (2010) The Changing History of Robustness. The American Statistician, 64, 277-281. http://dx.doi.org/10.1198/tast.2010.10159
[22]	Wilcox, R. (1997) Introduction to Robust Estimation and Hypothesis Testing. Academic Press, San Diego.
[23]	Quang, P.X. (1985) Robust Sequential Testing. The Annals of Statistics, 13, 638-649. http://dx.doi.org/10.1214/aos/1176349544
[24]	Harter, L. and Moore, A.H. (1976) An Evaluation of Exponential and Weibull Test Plans. IEEE Transactions on Reliability, 25, 100-104. http://dx.doi.org/10.1109/TR.1976.5214992
[25]	Montagne, E.R. and Singpurwalla, N.D. (1985) Robustness of Sequential Exponential Life-Testing Procedures. Journal of the American Statistical Association, 80, 715-719. http://dx.doi.org/10.1080/01621459.1985.10478174
[26]	Chaturvedi, A., Tiwari, N. and Tomer, S.K. (2002) Robustness of the Sequential Testing Procedures for the Generalized Life Distributions. Brazilian Journal of Probability and Statistics, 16, 7-24.
[27]	Gordienko, E., Novikov, A. and Zaitseva, E. (2009) Stability Estimating in Optimal Sequential Hypotheses Testing. Kybernetika, 45, 331-344.
[28]	Mace, A.E. (1974) Sample Size Determination. Robert E. Krieger Pub. Co., New York.
[29]	Ghosh, B.K. (1991) Brief History of Sequential Analysis. In: Ghosh, B.K. and Sen, P.K., Eds., Handbook of Sequential Analysis, Marcel Dekker, New York, 1-19.
[30]	Eisenberg, B. and Ghosh, B.K. (1991) The Sequential Probability Ratio Test. In: Ghosh, B.K. and Sen, P.K., Eds., Handbook of Sequential Analysis, Marcel Dekker, New York, 47-66.
[31]	Mandelbaum, A., Sakov, A. and Zeltyn, S. (2001) Empirical Analysis of a Call Center. Technical Report, Technion, Israel Institute of Technology. http://iew3.technion.ac.il/serveng/References/ccdata.pdf
[32]	Koole, G. and Mandelbaum, A. (2002) Queueing Models of Call Centers: An Introduction. Annals of Operations Research, 113, 41-59. http://dx.doi.org/10.1023/A:1020949626017
[33]	Koole, G. (2009) Optimization of Business Processes: An Introduction to Applied Stochastic Modeling. Lecture Notes.
[34]	Albin, S.L. (1986) Delays for Customers From Different Arrival Streams to a Queue. Management Science, 32, 329-340. http://dx.doi.org/10.1287/mnsc.32.3.329
[35]	Davis, J.L., Massey, W.A. and Whitt, W. (1995) Sensitivity to the Service-Time Distribution in the Nonstationary Erlang Loss Model. Management Science, 41, 1107-1116. http://dx.doi.org/10.1287/mnsc.41.6.1107
[36]	Michlin, Y.H. and Migdali, R. (2004) Test Duration in Choice of Helicopter Maintenance Policy. Reliability Engineering & System Safety, 86, 317-321. http://dx.doi.org/10.1016/j.ress.2004.01.006
[37]	Michlin, Y.H. and Shaham, O. (2013) Planning of Truncated Sequential Binomial Test via Relative Efficiency. Quality and Reliability Engineering International, 29, 369-383. http://dx.doi.org/10.1002/qre.1387