Towards Semantic Mutation Testing of Aspect-Oriented Programs

Abdul Azim Abdul Ghani

Department of Software Engineering and Information System, Faculty of Computer Science and Information Technology, Universiti Putra Malaysia, Serdang, Malaysia.

DOI: 10.4236/jsea.2013.610A002                

Abstract

Aspect-oriented programs have received much attention from software testing researchers. Various testing techniques and approaches have been proposed to tackle issues and challenges when testing aspect-oriented programs including traditional mutation testing. In traditional mutation testing of aspect-oriented programs, mutants are generated by making small changes to the syntax of the aspect-oriented language. Recently, a new approach known as semantic mutation testing has been proposed. This approach mutates the semantics of the language in which the program is written. The mutants generated misunderstandings of the language which are different classes of faults. Aspect-oriented programming presents itself with different properties that can be further explored with respect to semantic mutation testing. This paper describes various possible scenarios that semantic mutation testing strategy might have particular value in testing aspect-oriented programs.

Keywords

Aspect-Oriented Program Testing; Mutation Testing; Semantic Mutation Testing

Conflicts of Interest

The authors declare no conflicts of interest.

References

[1]	G. A. Colyer and A. Clement, “Aspect-Oriented Programming with AspectJ,” IBM Systems Journal, Vol. 44, No. 2, 2005, pp. 301-308. http://dx.doi.org/10.1147/sj.442.0301
[2]	G. Kiczales, J. Lamping, C. V. Lopes, J. J. Hugunin, E. A. Hilsdale and C. Boyapati, “Aspect-Oriented Programming,” US Patent No. 6467086, 2002.
[3]	R. T. Alexander, J. M. Bieman and A. A. Andrews, “Towards the Systematic Testing of Aspect-Oriented Programs,” Technical Report, Colorado State University, Fort Collins, 2004.
[4]	F. C. Ferrari, J. C. Maldonado and A. Rashid, “Mutation Testing for Aspect-Oriented Programs,” Proceedings of the 1st International Conference on Software Testing, Verification, and Validation, Norway, 9-11 April 2008, pp. 52-61.
[5]	M. Harman, F. Islam, T. Xie and S. Wrappler, “Automated Test Data Generation for Aspect-Oriented Programs,” Proceedings of the 8th International Conference on Aspect-Oriented Software Development, Charlottesville, 2-6 March 2009, pp. 185-196.
[6]	J. Zhao, “Dataflow-Based Unit Testing of Aspect-Oriented Programs,” 27th Annual IEEE International Computer Software and Applications Conference (COMPSAC’2003), Dallas, 3-6 November 2003, pp. 188-197.
[7]	O. A. L. Lemos, A. M. R. Vincenzi, J. C. Maldonado and P. C. Masiero, “Control and Data Flow Structural Testing Criteria for Aspect-Oriented Programs,” Journal of Systems and Software, Vol. 80, No. 6, 2007, pp. 862-882. http://dx.doi.org/10.1016/j.jss.2006.08.022
[8]	D. Xu, W. Xu and W. E. Wong, “Testing Aspect-Oriented Programs with UML Design Models,” International Journal of Software Engineering and Knowledge Engineering, Vol. 18, No. 3, 2008, pp. 413-437. http://dx.doi.org/10.1142/S0218194008003672
[9]	D. Xu and W. Xu, “State-Based Incremental Testing of Aspect-Oriented Programs,” Proceedings of the 5th International Conference on Aspect Oriented Software Development (AOSD’06), Bonn, 20-24 March 2006, pp. 180-189.
[10]	W. Xu and D. Xu, “State-Based Testing of Integration Aspects,” Workshop on Testing Aspect-Oriented Programs (WTAOP’06), Portland, 17-20 July 2006, pp. 7-14.
[11]	R. M. Parizi, A. A. Abdul Ghani, R. Abdullah and R. Atan, “On the Applicability of Random Testing of Aspect-Oriented Programs,” International Journal of Software Engineering and Its Application, Vol. 3, No. 3, 2009, pp. 1-19.
[12]	F. C. Ferrari, A. Rashid and J. C. Maldonado, “Towards the Practical Mutation Testing of AspectJ Programs,” Science of Computer Programming, Vol. 78, No. 9, 2013, pp. 1639-1662. http://dx.doi.org/10.1016/ j.scico.2013.02.011
[13]	S. A. Ali Naqvi, S. Ali and M. Uzair Khan, “An Evaluation of Aspect-Oriented Testing Techniques,” Proceedings of the 2005 International Conference on Emerging Technologies, Islamabad, 17-18 September 2005, pp. 461-466.
[14]	R. M. Parizi and A. A. Abdul Ghani, “A Survey on Aspect-Oriented Testing Approaches,” Proceedings of the 5th International Conference on Computational Science and Applications, Malaysia, 26-29 August 2007, pp. 78-85.
[15]	A. A. Abdul Ghani and R. M. Parizi, “Aspect-Oriented Program Testing: An Annotated Bibliography,” Journal of Software, Vol. 8, No. 6, 2013, pp. 1281-1300.
[16]	R. A. DeMillo, R. J. Lipton and F. G. Sayward, “Hints on Test Data Selection: Help for the Practicing Programmer,” IEEE Computer, Vol. 11, No, 4, 1978, pp. 34-41. http://dx.doi.org/10.1109/C-M.1978.218136
[17]	J. A. Clark, H. Dan and R. M. Hieron, “Semantic Mutation Testing,” Proceedings of the 3rd International Conference on Software Testing, Verification, and Validation Workshops, Paris, 6-10 April 2010, pp. 100-109. http://dx.doi.org/10.1109/ICSTW.2010.8
[18]	J. A. Clark, H. Dan and R. M. Hieron, “Semantic Mutation Testing,” Science of Computer Programming, Vol. 78, No. 4, 2013, pp. 345-363. http://dx.doi.org/10.1016/j.scico.2011.03.011
[19]	A. T. Acree, T. A. Budd, R. A. DeMillo, R. J. Lipton and F. G. Sayward, “Mutation Analysis,” Technical Report GIT-ICS-79/08, Georgia Institute of Technology, Atlanta, 1979.
[20]	K. N. King and J. Offutt, “A Fortran Language System for Mutation-Based Software Testing,” Software: Practice and Experience, Vol. 21, No. 7, 1991, pp. 685-718. http://dx.doi.org/10.1002/ spe.4380210704
[21]	H. Agrawal, R. A. DeMillo, B. Hathaway, W. Hsu, E. W. Krauser, R. J. Martin, A. P. Mathur and E. Spafford, “Design of Mutant Operators for the C Programming Language,” Technical Report SERC-TR-41-P, Purdue University, Lafayette, 1989.
[22]	P. Chevalley and P. Thevenod-Fosse, “A Mutation Analysis Tool for Java Programs,” International Journal of Software Tools for Technology Transfer, Vol. 5, No. 1, 2002, pp. 90-103. http://dx.doi.org/10.1007/ s10009-002-0099-9
[23]	P. Anbalagan and T. Xie, “Efficient Mutant Generation Testing of Pointcuts in Aspect-Oriented Programs,” Proceedings of the 2nd workshop on Mutation Analysis, 2006, pp. 51-56.
[24]	S. P. F. Fabbri, M. E. Delamaro, J. C. Maldonado and P. Masiero, “Mutation Analysis Testing for Finite State Machine,” Proceedings of the 5th International Symposium on Software Reliability Engineering, Monterey, 6-9 November 1994, pp. 220-229.
[25]	S. P. F. Fabbri, J. C. Maldonado, P. C. Masiero, M. E. Delamaro and W. E. Wong, “Mutation Testing Applied to Validate Specification Based on Petri Nets,” Proceedings of IFIP TC6 8th International Conference on Formal Description Techniques VIII, Vol. 43, 1995, pp. 329-337.
[26]	E. E. Martin and T. Xie, “A Fault Model and Mutation Testing of Access Control Policies,” Proceedings of the 16th International Conference on World Wide Web, May 2007, pp. 667-676. http://dx.doi.org/10.1145/1242572. 1242663
[27]	Y. Jia and M. Harman, “An Analysis and Survey of the Development of Mutation Testing,” IEEE Transactions on Software Engineering, Vol. 37, No. 5, 2011, pp. 649-678. http://dx.doi.org/10.1109/ TSE.2010.62
[28]	J. Offut, “A Mutation Carol: Past, Present and Future,” Information and Software Technology, Vol. 53, No. 10, 2011, pp. 1098-1107. http://dx.doi.org/10.1016/j.infsof.2011.03.007
[29]	J. Offut and R. H. Untch, “Mutation 2000: Uniting the Orthogonal,” Proceedings of Mutation 2000: Mutation Testing in the Twentieth and Twenty First Centuries, San Jose, October 2000, pp. 45-55.
[30]	R. Laddad, “AspectJ in Action: A Practical Aspect-Oriented Programming,” Manning Publications Co., New York, 2003.
[31]	F. C. Ferrari, A. Rashid and J. C. Maldonado, “Design of Mutant Operators for the AspectJ Language,” Technical Report Version 1.0, University of Sao Carlos, Sao Carlos, 2011.
[32]	F. C. Ferrari, R. Burrows, O. A. L. Lemos, A. Garcia and J. C. Maldonado, “Characterising Faults in Aspect-Oriented Programs: Towards Filling the Gap between Theory and Practice,” Proceedings of the 2010 Brazilian Symposium on Software Engineering, Salvador, 27 September-1 October, 2010, pp. 50-59.
[33]	M. Ceccato, P. Tonella and F. Ricca, “Is AOP Code Easier or Harder to Test than OOP Code?” Proceedings of the Workshop on Testing Aspect-Oriented Programs, Chicago, March 2005.
[34]	N. McEachen and R. T. Alexander, “Distributing Classes with Woven Concerns—An Exploration of Potential Fault Scenarios,” Proceedings of the 4th International Conference on Aspect-oriented Software Development (AOSD’ 05), Chicago, 14-18 March, 2005, pp. 192-200. http://dx.doi.org/10.1145/1052898.1052915
[35]	V. Deursen, M. Marin and L. Moonen, “A Systematic Aspect-Oriented Refactoring and Testing Strategy, and its Application to JHotDraw,” arXiv:cs/0503015v1 [cs.SE], 2005.
[36]	J. S. Baekken and R. T. Alexander, “Towards a Fault Model for AspectJ Programs: Step 1—Pointcut Faults,” Proceedings of the 2nd Workshop on Testing AspectOriented Programs, Portland, 20 July 2006, pp. 1-6. http://dx.doi.org/10.1145/1146374.1146375
[37]	J. S. Baekken and R. T. Alexander, “A Candidate Fault Model for AspectJ Pointcuts,” Proceedings of the 17th International Symposium on Software Reliability Engineering (ISSRE’06), Raleigh, 7-10 November 2006, pp. 169-178.
[38]	J. S. Baekken, “A Fault Model for Pointcuts and Advice in AspectJ Programs,” Master Thesis, School of Electrical Engineering and Computer Science, Washington State University, Pullman, 2006.
[39]	C. Zhao and R. Alexander, “Testing AspectJ Programs Using Fault-Based Testing,” Proceedings of the 3rd Workshop on Testing Aspect-Oriented Programs, Vancouver, 12-13 March 2007, pp. 13-16.
[40]	M. L. Bernardi and G. A. Di Lucca, “Testing AspectOriented Programs: An Approach Based on the Coverage of the Interactions among Advices and Methods,” Proceedings of the 6th International Conference on the Quality of Information and Communication Technology, Lisbon, 12-14 September 2007, pp. 65-76.
[41]	C. Babu and H. R. Krishnan, “Fault Model and Test-Case Generation for the Composition of Aspects,” SIGSOFT Software Engineering Notes, Vol. 34, No. 1, 2009, pp. 1-6. http://dx.doi.org/10.1145/ 1457516.1457521
[42]	N. Kumar, A. Rathi, D. Sosale and S. N. Konuganti, “Enabling the Adoption of Aspects—Testing Aspects: A Risk Model, Fault Model and Patterns,” Proceedings of the 8th International Conference on Aspect-Oriented Software Development (AOSD’09), Charlottesville, 2-6 March 2009, pp. 197-206.
[43]	O. A. Lemos, F. C. Ferrari, P. C. Masiero and C. V. Lopes, “Testing Aspect-Oriented Programming Pointcut Descriptors,” Proceedings of the 2nd Workshop on Testing Aspect-Oriented Programs, New York, 20 July 2006, pp. 33-38. http://dx.doi.org/10.1145/1146374.1146380
[44]	S. Zhang and J. Zhao, “On Identifying Bug Patterns in Aspect-Oriented Programs,” Proceedings of the 31st Annual International Computer Software and Applications Conference (COMPSAC 2007), Beijing, 24-27 July 2007, pp. 431-438. http://dx.doi.org/10.1109/COMPSAC.2007.159
[45]	M. Mortensen and R. T. Alexander, “Adequate Testing of Aspect-Oriented Programs,” Technical Report CS04-110, Department of Computer Science, Colorado State University, Fort Collins, 2004.
[46]	M. Mortensen, and R. T. Alexander, “An Approach for Adequate Testing of AspectJ Programs,” Proceedings of the 1st Workshop on Testing Aspect-Oriented Programs — In Conjunction with AOSD’2005, Chicago, 14-18 March 2005.
[47]	M. Singh and S. Mishra, “Mutant Generation for AspectOriented Programs,” Indian Journal of Computer Science and Engineering, Vol. 1, No. 4, 2010, pp. 409-415.
[48]	P. Anbalagan and T. Xie, “Automated Generation of Pointcut Mutants for Testing Pointcuts in AspectJ Programs,” Proceedings of the 19th International Symposium on Software Reliability Engineering, Seattle, 10-14 November 2008, pp. 239-248.
[49]	R. Delamare, B. Baudry and Y. Le Traon, “AjMutator: A Tool for the Mutation Analysis of AspectJ Pointcut Descriptors,” Proceedings of the IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW 2009), Denver, 1-4 April 2009, pp. 200-204. http://dx.doi.org/10.1109/ICSTW.2009.41
[50]	A. Jackson and S. Clarke, “MuAspectJ: Mutant Generation to Support Measuring the Testability of AspectJ Programs,” Technical Report (TCD-CS-2009-38), ACM, 2009.
[51]	F. C. Ferrari, A. Rashid, E. Y. Nakagawa and J. C. Maldonado, “Automating the Mutation Testing of Aspect-Oriented Java Programs,” Proceedings of the 5th Workshop on Automation of Software Test (AST’10), Cape Town, 3-4 May 2010, pp. 51-58. http://dx.doi.org/10.1145/1808266.1808274
[52]	H. Dan and R. M. Hierons, “Semantic Mutation Analysis of Floating-Point Comparison,” Proceedings of IEEE 5th International Conference on Software Testing, Verification and Validation, Montreal, 17-21 April 2012, pp. 290-299.
[53]	H. Dan and R. M. Hierons, “SMT-C: A Semantic Mutation Testing Tool for C,” Proceedings of IEEE 5th International Conference on Software Testing, Verification and Validation, Montreal, 17-21 April 2012, pp. 654-663.
[54]	M. Amar and K. Shabbir, “Systematic Review on Testing Aspect-Oriented Programs: Challenges, Techniques and Their Effectiveness,” Master Thesis, Software Engineering, School of Engineering, Blekinge Institute of Technology, Sweden, 2008.
[55]	R. M. L. M. Moreira, A. C. R. Paiva and A. Aguiar, “Testing Aspect-Oriented Programs,” Proceedings of the 5th Iberian Conference on Information Systems and Technologies, Santiago de Compostela, 16-19 June 2010, pp. 1-6.
[56]	A. Restivo and A. Aguiar, “Towards Detecting and Solving Aspect Conflicts and Interferences Using Unit Tests,” Workshop SPLAT’07, Vancouver, 12-13 March 2007.
[57]	L. Bergmans, “Towards Detection of Semantic Conflicts between Crosscutting Concerns,” AAOS 2003 (Analysis of Aspect-Oriented Software), Darmstadt, 21 July 2003.
[58]	P. Durr, T. Staijen, L. Bergmans and M. Aksit, “Reasoning about Semantic Conflicts between Aspects,” European Interactive Workshop on Aspects in Software, EIWAS, Brussels, 1-2 September 2005.
[59]	L. Bussard, L. Carver, E. Ernst, M. Jung, M. Robillard and A. Speck, “Safe Aspect Composition,” Workshop on Aspects and Dimensions of Concern at ECOOP’2000, Cannes, June 2000.
[60]	W. Havinga, I. Nagy and L. Bergmans, “An Analysis of Aspect Composition Problems,” Third European Workshop on Aspects in Software 2006, Enschede, 31 August 2006, pp. 1-8.
[61]	F. Tessier, M. Badri and L. Badri, “A Model-Based Detection of Conflicts between Crosscutting Concerns: Towards a Formal Approach,” International Workshop on Aspect-Oriented Software Development, China, September 2004.
[62]	H. Chengwan, L. Zheng and H. Keqing, “Towards Trusted Aspect Composition,” Proceedings of IEEE 8th International Conference on Computer and Information Technology Workshops, Sydney, 8-11 July 2008, pp. 643-648.
[63]	K. Tian, K. Cooper, K. Zhang and H. Yu, “A Classification of Aspect Composition Problems,” Proceedings of the Third IEEE International Conference on Secure Software Integration and Reliability Improvement, Shanghai, 8-10 July 2009, pp. 101-109. http://dx.doi.org/10.1109/ SSIRI.2009.33
[64]	W. Xu and D. Xu, “A Model-Based Approach to Test Generation for Aspect-Oriented Programs,” Workshop on Testing Aspect-Oriented Programs (AOSD2005), Chicago, 14-18 March 2005.
[65]	G. Mussbacher, “Aspect-Oriented User Requirements Notation,” Ph.D. Thesis, SITE, University of Ottawa, Canada, 2010.
[66]	G. Mussbacher, D. Amyot, J. Araujo and A. Moreira, “Requirements Modeling with the Aspect-Oriented User Requirements Notation (AoURN): A Case Study,” In: S. Katz, M. Mezini and J. Kienzle, Eds., Transactions on Aspect-Oriented Software Development VII, Springer, Berlin, 2010, pp. 23-68.
[67]	S. Mosser, G. Mussbacher, M. Blay-Fornarino and D. Amyot, “From Aspect-Oriented Requirements Models to Aspect-Oriented Business Process Design Models—An Iterative and Concern-Driven Approach for Software Engineering,” Proceedings of 10th International Conference on Aspect-Oriented Software Development (AOSD 2011), Porto de Galinhas, 21-25 March 2011, pp. 31-42 .
[68]	J. M. Mottu, B. Baudry and Y. Le Traon, “Mutation Analysis Testing for Model Transformations,” In: A. Rensink and J. Warmer, Eds., ECMDA-FA 2006, LNCS 4066, 2006, pp. 376-390.
[69]	P. Alves, A. Santos, E. Figueiredo and F. Ferrari, “How Do Programmers Learn AOP? An Exploratory Study of Recurring Mistakes,” Proceedings of 5th Latin-American Workshop on Aspect-Oriented Software Development (LA-WASP.11), Sao Paolo, 26 September 2011, pp. 131-140.
[70]	L. Cole and P. Borba, “Deriving Refactorings for AspectJ,” Proceedings of the 4th International Conference on Aspect-Oriented Software Development (AOSD’05), Chicago, 14-18 March 2005, pp. 123-134.