Quantitative Security Evaluation for Software System from Vulnerability Database

Hiroyuki Okamura, Masataka Tokuzane, Tadashi Dohi

Department of Information Engineering, Graduate School of Engineering, Hiroshima University, Hiroshima, Japan.
Japan Ground Self-Defense Force, Kagoshima, Japan..

DOI: 10.4236/jsea.2013.64A003                

Abstract

This paper proposes a quantitative security evaluation for software system from the vulnerability data consisting of discovery date, solution date and exploit publish date based on a stochastic model. More precisely, our model considers a vulnerability life-cycle model and represents the vulnerability discovery process as a non-homogeneous Poisson process. In a numerical example, we show the quantitative measures for contents management system of an open source project.

Keywords

Quantitative Security Evaluation; Vulnerability Database; Non-Homogeneous Poisson Process, Contents Management System

1. Introduction

From the latter half of 1990s, many security incidents have been reported in enterprise systems and personal computers, such as the denial-of-service attack via computer viruses and the data leak caused by unauthorized accesses.

Generally, most of security incidents are caused by software flaws and bugs called security holes and vulnerabilities. The effective counter measure against security incidents is to validate there is no flaw in the software during design and testing phases. Nowadays, for these purpose, model verification techniques are enhanced to validate the software design. For example, the model checking ensures that the software behaves according to its specification mathematically [1], and several testing techniques are developed to remove software faults as many as possible in the testing phase [2]. However, even if such techniques are applied, it is difficult to remove all the flaws before releasing the software to the market due to external circumstances of software development; development cost, delivery date and unexpected specification changes. For such software systems, a security patching is one of the feasible solutions that do not allow an attacker to exploit vulnerabilities.

A security patch is a small program to fix the software faults causing security holes and vulnerabilities, and is distributed to the end-users through the Internet or other means after the software release. The user can remove a vulnerability by applying a corresponding security patch which is distributed from the vendor. Ideally, the security patch should be distributed whenever one discovers a vulnerability of the software product. However, the development and distribution of security patches incur expenses for the vendor, and a short development time might cause the distribution of a poorly designed patch causing a new problem. Thus, many of the software vendors design a plan to distribute a security patch at a specified period of time, e.g., quarterly distribution, and the patch fixes all the vulnerabilities which have been discovered until the distribution time. On the other hand, from the user perspective, applying a patch involves not only a tedious task but also a risk that the patch causes an error like misconfiguration. Therefore, in practice, users, especially enterprises and firms, also make a plan of what patches are applied at a specified period of time. These strategies for the software patch are called patch management. In [3], Okamura et al. discussed the optimal patch release timing to help the patch management for enterprise based on the stochastic model.

Essentially, it is important to quantify degree of security for the software system to discuss the patch management. In general, there are two perspectives on the quantitative evaluation of security: vendor’s and user’s perspective. From the vendor’s perspective, the risk is that vendor is to release exploitation of a vulnerability before a patch is distributed. On the other hand, users should consider the risk caused by the delay of applying patches as well as the risk of software system itself. In fact, Okamura et al. [4] tried to evaluate the degree of security from user’s perspective by considering user profile of the system. In this paper, we focus on the security risk for vendors.

In the past literature, many researches considered the risk of security in software system from the vendor’s perspective. Wang et al. [5] presented a continuous-time Markov model to evaluate the security in the intrusiontolerant database system. Jonsson et al. [6] discussed the security model based on the analysis of attacker’s behavior. In these papers, they considered the quantitative security for specific systems and it cannot always be applied to any kind of software system. Also Kimura [7] proposed a stochastic model, which is similar to the classical software reliability growth model, and presented a quantitative evaluation for the security of software system. His method focused on the vulnerability discovery process only and therefore it can be applied to many kinds of software system. However, the model derived in [7] is essentially equivalent to testing-domain dependent software reliability growth model [8]. Thus, it cannot represent a variety of patterns for the vulnerability discovery process.

In this paper, we refine the quantitative software security model based on the vulnerability discovery process by using general distributions. Although the model presented here does not exactly include the model in [7], we adopt the similar situation where vendors and attackers compete to make a patch and to find an exploit. In addition, we present an illustrative example of the quantitative security evaluation of contents management system from the vulnerability data.

The rest of this paper is organized as follows. In Section 2, we describe the vulnerability model with respect to its discovery process. Section 3 presents the formulation of a quantitative security measure based on the vulnerability discovery process, patch release distribution and exploitation time distribution. Section 4 is devoted to the experiment for our quantitative security evaluation based on the vulnerability data.

2. Vulnerability Discovery Model

2.1. Vulnerability Life Cycle

Vulnerability is defined as a fault on system requirements or a program that allows an attacker to violate the system integrity. A vulnerability is often caused by flaws on software requirements as well as software bugs, and thus it is more difficult to find vulnerabilities by software testing than to detect usual software bugs.

Arbaugh et al. [9] presented a vulnerability life-cycle model which consists of the following seven states:

• Birth: The birth of a vulnerability, strictly speaking a flaw, occurs at software requirement or software design.

• Discovery: Someone discovers a flaw on software security, and then the flaw becomes a vulnerability.

• Disclosure: The vulnerability is disclosed when the discoverer reveals details of the problem.

• Correction: The vulnerability is correctable by developing and releasing a security patch.

• Publicity: The vulnerability and its problem become known by disclosing them to public medias.

• Scripting: An exploitation of the vulnerability is released. In this state, crackers with little or no skill can exploit the vulnerability to violate the integrity of system.

• Death: The vulnerability dies when one applies a security patch to all the vulnerable systems.

Figure 1 illustrates the state transition of a typical vulnerability in the life-cycle model.

Conflicts of Interest

The authors declare no conflicts of interest.

References

[1]	E. M. Clarke Jr., O. Grumberg and D. A. Peled, “Model Checking,” MIT Press, Cambridge, 1999.
[2]	G. J. Myers and C. Sandler, “The Art of Software Testing,” John Wiley & Sons, Hoboken, 2004.
[3]	H. Okamura, M. Tokuzane and T. Dohi, “Optimal Security Patch Release Timing under Non-Homogeneous Vulnerability-Discovery Processes,” Proceedings of 20th International Symposium on Software Reliability Engineering (ISSRE’09), Mysuru, 16-19 November 2009, pp. 120-128.
[4]	H. Okamura, M. Tokuzane and T. Dohi, “Security Evaluation for Software System with Vulnerability Life Cycleand User Profiles,” Proceedings of 2012 Workshop on Dependable Transportation/Recent Advances in Software Dependability (WDTS-RASD 2012), Niigata, 18-19 No-vember 2012, pp. 39-44.
[5]	H. Wang and P. Liu, “Modeling and Evaluating the Survivability of an Intrusion Tolerant Database System,” ESORICS 2006, LNCS 4189, Hamburg, 18-20 September 2006, pp. 207-224.
[6]	E. Jonsson and T. Olovsson, “A Quantitative Model of the Security Intrusion Process Based Onattacker Behavior,” IEEE Transactions on Software Engineering, Vol. 23, No. 4, 1997, pp. 235-245. doi:10.1109/32.588541
[7]	M. Kimura, “A Study on Software Vulnerability Assessment Modeling and Its Application to E-Mail Distribution Software System,” The Journal of Reliability Engineering Association of Japan (Japanese), Vol. 25, No. 3, 2003, pp. 279-283.
[8]	T. Fujiwara and S. Yamada, “Testing-Domain Dependent Software Reliability Growth Models and Their Comparisons of Goodness-of-Fit,” Proceedings of the 7th ISSAT International Conference on Reliability and Quality in Design, Washington DC, 8-10 August 2001, pp. 36-40.
[9]	W. A. Arbaugh, W. L. Fithen and J. McHugh “`Windows of Vulnerability: A Case Study Analysis,” IEEE Computer, Vol. 33, No. 12, 2000, pp. 52-59. doi:10.1109/2.889093
[10]	J. D. Musa, “Software Reliability Engineering,” McGrawHill, New York, 1999.
[11]	M. R. Lyu, “Handbook of Software Reliability Engineering,” McGraw-Hill, New York, 1996.
[12]	M. Ohba, “Inflection S-Shaped Software Reliability Growth Model,” In: S. Osaki and Y. Hatoyama, Eds., Stochastic Models in Reliability Theory, Springer-Varlag, Berlin, 1984, pp. 144-165. doi:10.1007/978- 3-642-45587-2_10
[13]	H. Okamura, T. Dohi and S. Osaki, “EM Algorithms for Logistic Software Reliability Models,” Proceedings of 7th IASTED International Conference on Software Engineering, Innsbruck, 17-19 February 2004, pp. 14-22.
[14]	O. H. Alhazmi and Y. K. Malaiya, “Application of Vulnerability Discovery Models to Major Operating Systems,” IEEE Transactions on Reliability, Vol. 57, No. 1, 2008, pp. 14-22. doi:10.1109/TR. 2008.916872
[15]	O. H. Alhazmi and Y. K. Malaiya, “Measuring and Enhancing Prediction Capabilities of Vulnerability Discovery Models for Apache and IIS HTTP Servers,” Proceedings of 17th International Symposium on Software Reliability Engineering, Raleigh, 7-10 November 2006, pp. 343-352.
[16]	S.-W. Woo, O. H. Alhazmi and Y. K. Malaiya, “Assessing Vulnerabilities in Apache and IIS HTTP Servers,” Proceedings of 2nd IEEE International Symposium on Dependable, Autonomic and Secure Computing, Indianapolis, 29 September-1 October 2006, pp. 103-110.
[17]	O. H. Alhazmi and Y. K. Malaiya, “Modeling the Vulnerability Discovery Process,” Proceedings of 16th International Symposium on Software Reliability Engineering, Chicago, 8-11 November 2005, pp. 129-138.
[18]	H. Tijms, “A First Course in Stochastic Models,” John Wiley & Sons, Hoboken, 2003. doi:10.1002/047 001363X
[19]	H. Okamura, Y. Watanabe and T. Dohi, “An Iterative Scheme for Maximum Likelihood Estimation in Software Reliability Modeling,” Proceedings of 14th International Symposium on Software Reliability Engineering, Denver, 17-20 November 2003, pp. 246-256.
[20]	H. Okamura, A. Murayama and T. Dohi, “EM Algorithm for Discrete Software Reliability Models: A Unified Parameter Estimation Method,” Proceedings of 8th IEEE International Symposium on High Assurance Systems Engineering, Tampa, 25-26 March 2004, pp. 219-228.
[21]	K. Ohishi, H. Okamura and T. Dohi, “Gompertz Software Reliability Model: Estimation Algorithm and Empirical Validation,” Journal of Systems and Software, Vol. 82, No. 3, 2009, pp. 535-543. doi:10.1016/j.jss.2008.11.840
[22]	H. Akaike, “Information Theory and an Extension of the Maximum Likelihood Principle,” Proceedings of 2nd International Symposium on Information Theory, 1973, pp. 267-281.
[23]	A. L. Goel and K. Okumoto, “Time-Dependent ErrorDetection Rate Model for Software Reliability and Other Performance Measures,” IEEE Transactions on Reliability, Vol. R-28, No. 3, 1979, pp. 206-211. doi:10.1109/TR.1979.5220566
[24]	S. Yamada, M. Ohba and S. Osaki, “S-Shaped Reliability Growth Modeling for Software Error Detection,” IEEE Transactions on Reliability, Vol. R-32, No. 5, 1983, pp. 475-478. doi:10.1109/TR. 1983.5221735
[25]	B. Littlewood, “Rationale for a Modified Duane Model,” IEEE Transactions on Reliability, Vol. R-33, No. 2, 1984, pp. 157-159. doi:10.1109/TR.1984.5221762
[26]	H. Okamura, Y. Watanabe and T. Dohi, “Estimating Mixed Software Reliability Models Based on the EM Algorithms,” Proceedings of 2002 International Symposium on Empirical Software Engineering, Napa, 3-4 October 2002, pp. 69-78.
[27]	H. Okamura, T. Dohi and S. Osaki, “Software Reliability Growth Models with Normal Failure Time Distributions,” Reliability Engineering and System Safety, 2013 (in Press).
[28]	J. A. Achcar, D. K. Dey and M. Niverthi, “A Bayesian Approach Using Nonhomogeneous Poisson Processes for Software Reliability Models,” In: A. P. Basu, K. S. Basu and S. Mukhopadhyay, Eds., Frontiers in Reliability, World Scientific, Singapore City, 1998, pp. 1-18.
[29]	S. S. Gokhale and K. S. Trivedi, “Log-Logistic Software Reliability Growth Model,” Proceedings of 3rd IEEE International High-Assurance Systems Engineering Symposium, Washington DC, 13-14 November 1998, pp. 34-41.
[30]	A. L. Goel, “Software Reliability Models: Assumptions, Limitations and Applicability,” IEEE Transactions on Software Engineering, Vol. SE-11, No. 12, 1985, pp. 1411-1423. doi:10.1109/TSE.1985.232177
[31]	H. Okamura, T. Dohi and K. S. Trivedi, “A Refined EM Algorithm for PH Distributions,” Performance Evaluation, Vol. 68, No. 10, 2011, pp. 938-954. doi:10.1016/j.peva.2011.04.001
[32]	Q.-M. He and H. Zhang, “On Matrix Exponential Distributions,” Advances in Applied Probability, Vol. 39, No. 1, 2007, pp. 271-292. doi:10.1239/aap/1175266478