Agile Software Development Methodologies: Survey of Surveys

Agile software design and development methodologies have been gaining rigorous attention in the software engineering research community since their early introduction in the mid-nineties in addition to being highly adopted by the software development industry. In the last 15 years, an excessive number of research studies have been conducted on agile methods, a great number of notable methods have been proposed and various surveys have been presented by many researchers. In this study, the authors intend to conduct a literature survey study of the surveys of the different agile methodologies ranging from January 2000 to December 2015 using an intuitive research methodology called “Compare and Review” (CR). Furthermore, these survey papers were classified into four major categories according to their area of study. Additionally, the newly proposed agile methodologies that have not been addressed yet in any other literature review were reviewed and compared in terms of where the changes that they proposed lay on the SDLC.


Introduction
Agile in essence is an iterative, lightweight and lean software design and development methodology that was born in the late 1990s to be highly compatible with the rapid development of the WWW (World Wide Web) [1]. Similar to climbing a well-designed ladder where length of all steps and distance between each step is equivalent, agile methods divides a task into small-length iterations that have the same interval size and distance making the transition between iterations much smoother with much higher pace. Agile methodologies try to find an equilibrium point between no process and too much process, allowing it to survive in dynamic environments where requirements frequently change while striving high quality software product [2]. Unlike other methods, agile methods rely on feedback as control mechanism which ensures greater customer satisfaction [3].
The agile methodology is based on the "iterative enhancement" [13] technique [14]. As a iteration based methodology, each iteration in the agile methodology represents a small scale and self-contained Software Development Life Cycle (SDLC) by itself [1]. Unlike the Spiral model [15], agile methods assume simplicity in all practices [14].
In this research, the authors identify the following contributions: 1) a new research methodology called Compare and Review (CR) is used in this paper; 2) A survey of the surveys on agile methodologies were conducted, in which the survey papers were classified into four categories: "Agile Requirements Engineering", "Agile Methods", "Hybrid Agile Methods" and "Miscellaneous"; 3) Several new agile methods that have not been surveyed yet were reviewed and compared in terms of the changes that they proposed on the SDLC. Moreover, 26 surveys and 4 articles were selected for evaluation out of more than 92 studies. The results prove agile capability as a strong software design and development methodology.
The rest of the paper is organized as follows: Section 2 describes the research methodologies. Section 3 explores the surveys on agile methods in the literature and Section 4 reviews the most recent researches on XP, Scrum and FDD agile methodologies that, up-to-our-knowledge, have not been addressed in any other literature review. Finally, in Section 5, the evaluation of the survey studies and the new agile methods is presented, and then the conclusion is presented in Section 6.

Research Methodology
The authors followed a two-stage research methodology called "Compare and Review" (CR), where the first stage aims to compare the survey studies on agile methodologies. While, the second stage intends to review the most recent research studies on XP, Scrum and FDD agile methods that have not been addressed in any previous literature reviews. In the following subsections, the research methodology will be explained. Figure 1 shows the flowchart for the CR research methodology.

Research Requirements
In this subsection, the research requirements that were used to govern this study • Formulate a comprehensive comparison between the newly proposed agile methods in terms of the changes that they might have made in SDLC phases.

Selection Criteria
In this subsection, the search and selection criteria that was applied is listed. The research must be: • Indexed in one of the following digital databases: 1) ACM Digital Library; 2) IEEE Digital Library; 3) Science Direct; 4) Springer or 5) Wiley Online Library.
• Either a Conference Paper or Journal Article.
• Published between January 2000 and December 2015.
• In English language (the entire paper and not just the abstract).
• Has one of the following key words in the paper Title, Keywords list, or Ab-

Stage 1: Survey of Surveys on Agile Methods
This stage aims to provide a comprehensive comparison between the various survey studies that addresses the different agile methodologies, which were published between January 2000 and December 2015. A strict search and selection criteria were applied, more than 32 studies were identified and 26 studies were selected for the final review.

Stage 2: Literature Review of New Agile Methods
In this stage, the authors reviewed 12 new research papers that have not been addressed yet in any previous literature review. Notably, all the papers were published in 2015 and the same selection criteria that was applied at stage 1 was applied in this stage too.

Survey of Surveys on Agile Methods
In this section, the authors will explore the different surveys that have addressed the agile methodologies in the past 15 years starting from January 2000 to December 2015. The surveys were classified into four categories as illustrated in Figure 2: "Agile Requirements Engineering", "Agile Methods", "Hybrid Agile Methods" and "Miscellaneous".

Agile Requirements Engineering
One the most attractive features of agile methodology is that it accepts changes to requirements during any phase of the SDLC, making it more flexible and highly adaptable to dynamic environments where requirements change frequently. In this subsection, the authors review the literature reviews on Agile Requirements Engineering (RE).
Baruah presented in [16] a comparative study about the different ways each of the agile methodologies manage requirements. Table 1 summarizes the differences between the eight agile software development methods in terms of requirements management as presented in [16].
Inayat, et al. presented in [17] a comprehensive systematic review about resolving traditional requirements engineering issues using agile requirements engineering for studies published between January 2002 and June 2013 in which 21 papers out of 531 were studied. The main results of the study can be summarized as follow: • Distribution of the publication sources is 57% conferences, 19% journals, and 5% magazines.
• The studies can be classified into four main categories according to the specific topic that they address, which are: 1) agile RE practices (29%); 2) new methods (28%); 3) traditional RE vs. agile RE (5%), and 4) agile RE in general (38%).  On-site customer is required to participate in requirements definition, estimation and prioritize.

Scrum
User stories are used to represent requirements. The actual requirements are defied based on the discussion of user stories between software owner and software developers.
Software owner plays the lead role in defining the requirements.

FDD
Requirements are represented using UML diagrams. List of features are used to manage the functional requirements. Requirements are first represented in a high-level context. For each modeling area, the requirements are modeled per domain. After requirements are modeled, it should be peer reviewed.
Not specified.

Lean Software Development
Just-In-Time methodology is applied in requirements gathering. User stories (cards) are also used by the customer to specify initial requirements and sample screens by the developers. Developers then provide a time estimate for each card.
Provide input on sample screens and initial user stories.

ASD
Requirements gathering is part of the speculation phase. Not specified.

Kanban
User stories are used to define each sprint main goal. Each sprint handles a single user story. Each story is divided into server-side and client-side task. Each task is further divided into subtasks.
Not specified.

AUP
Requirement phase consists of the following activities: (1) identifying stakeholders, Not specified.

DSDM
Requirements are gathered and prioritized during the feasibility phase. Not specified.
• The majority of studies are affiliated to authors from North America and EU.
• The majority of studies followed an exploratory research methodology.
• Seventeen common RE practices among the different agile methods were identified as shown in Table 2, which also indicates whether this practice is part of the original XP method or not.
Ramesh, et al. provided an empirical study on agile RE practices in [18]. The study-analyzed data collected from sixteen software companies (in USA) in order to explain the differences between traditional RE and agile RE. The data was solicited through systematic interviews. Moreover, a risk-based framework was proposed to evaluate the influence of utilizing agile RE through factoring the associated risks. The major findings of the study are presented as follow: • Six agile RE practices were identified, which are: 1) Face-to-face communication instead of written specifications; 2) Iterative RE; 3) Using extreme practices in requirement prioritization; 4) Using constant planning to manage requirements changes; 5) Prototyping, and 6) Using review meetings with acceptance tests.

Agile Methods
Stavru surveyed the usage of agile methods through industrial survey studies published between 2011 and 2012 [19]. They determined the papers, which could be trusted and recommend that the quality level of researches could be improved. The author concluded that the majority of the surveyed studies were incomprehensive in addition to being not trustworthy. On the other hand, the authors provided some recommendations in order to raise the research quality.
The recommendations include examining the rate of agile method usage as compared to alternative methods; examine the rate of agile method usage in an organizational level. Also, conducting researches about using agile method by academics beside industry in order to decrease the gap between industry and academia in addition to increase the trustworthiness in widespread adoption of agile method usage. Additionally, providing highly detailed reports in the future; thus, raising the level of confidence and trustworthiness in the reported studies.
Campanelli and Parreiras presented aspects of research on agile methods tailoring in [20]. The term agile method tailoring refers to the problem of selecting an agile method to be adopted in the organization. The authors analyze the total number of papers published in the area and proposed Figure 3  In addition to concern about the quality of studies provided [26].
Abrahamsson, et al. presented a comparative analysis of agile software development methods, including the method's life cycle coverage, project management support, practical guidance type, fitness-for-use, and empirical evidence as the analytical lenses in [27]. Moreover, the authors concluded that the majority of the studied methods did not concern about project management nor about improving the development team member's skills and capabilities. Finally, the authors recommend to concern about the quality of new methods instead of quantity of existing methods.

Hybrid Agile Methods
Selleri Silva, et al. presented a review study on the agile methodologies that integrate the Capability Maturity Model Integration (CMMI) where 3193 studies were identified and 81 were selected for evaluation and classified into two main classes; benefits to the organization in general and benefits to the development process. The results shows that using agile methods was helpful in reaching level 2 and level 3 of CMMI and in some cases even level 5 [28].
Torrecilla-Salinas et al. studied the applicability of complying with the CMMI-DEV model for Web development companies that have followed one of the agile methods [29]. The study surveyed the current state-of-the-art on this topic to answer five questions that were later used to analyze and evaluate the selected studies. Six papers were selected for evaluation out of more than 1453 studies.
The results have shown that in the last 5 years more and more Web development companies are moving towards adopting agile methods in order to help be certified against CMMI-DEV.
Santana, et al. provided a literature review to identify software process improvement (SPI) in agile environment [30]. The authors classified the reviewed papers according to SPI aspects. Additionally, they identified new distinct approaches to Agile SPI, and they have suggested using one of the following three agile SPI approaches: top-down approach, the agile SPI based on improving behavior, and the agile SPI based on improving practices. The authors also identified the difference between traditional and agile SPI especially in their goals. The goal of traditional SPI is to elaborate a repeatable process that could be improved with lessons learned. On the other hand, the process in agile SPI should be ready for changes and improve its capability for changes. Furthermore, the author An overview of the use of software metrics in the industrial agile context is provided by [32]. This study makes three contributions: First, the authors categorize the metrics found in empirical agile studies and compares them with the metrics suggested by agile literature, and they conclude that agile teams use many metrics suggested by the agile literature. Second, the study highlights the reasons for and in GSD [34]. Three main ideas were identified in this review paper on GSD; performance of global software development, governance related issues, and software engineering process related issues. More distant analysis of literature on both agile and GSD showed that various types of agile methods were applied and tailored appropriately to produce optimal performance in the context of GSD, while empirical studies addressing GSD-Agile fit were not found. The most common agile method used in GSD is SCRUM as the authors conclude.
Sohaib and Khan presented a literature review to describe how usability fit with agile software development in order to gain stronger and effective usable software system [35]. The authors mentioned that the usability could be adapted to agile software development by doing some steps such as: using more iterations and concern about testing at each phase of SDLC.
Using agile methods in the embedded software projects is reviewed systematically by Shen, et al. in [36]. The authors concluded that there is a need for more effective research is this area. Although, there are many difficulties, Xie, et al.
concluded that using agile in such projects holds a positive impression [37]. The authors presented many challenges of applying agile in embedded software projects, such as: development team challenges, time constraints and budget. The authors recommended choosing the appropriate agile method according to the challenges derived from the embedded system and how can the selected agile method tackle these challenges. The findings of these comparisons indicated that scientific software development projects that have adopted agile have an improved testing process to compare to the traditional methods [38].

Miscellaneous
A new analytical framework developed by Qumer and Henderson-Sellerscalled (4-DAT) [39]. The proposed framework was applied on six agile selected methods in addition to two traditional methods for comparison purposes. The evaluation approach is done in this analytical framework to evaluate these methods at the process level and practice level from four perspectives. The evaluation aimed to select an appropriate agile method for a particular development.
Rauf and Al Ghafees provided an industrial survey to study the use of scrum and XP agile practices in computer application development, the authors posted a questionnaire to investigate the benefits of using agile, 79 responses were collected from 45 companies. Survey results showed that 57% respondents use Scrum and in turn it is the most common agile method used, while 27% respondents use both XP and Scrum and only 5% respondents have reported to use XP solely. In addition, the authors investigated the strength of benefits against each agile practice, and explored the challenges against each agile practice [40].

Literature Review
In the following subsections, the authors will briefly introduce three agile methods (i.e. XP, Scrum and FDD), then review the most recent researches that, up-to-our-knowledge, have not been addressed in any other literature review yet.

Extreme Programming (XP)
Among all of the agile methods, Extreme Programming (XP) is the most popular and well-documented method [14] [23]. The XP method was first proposed by Beck as a last resort to rescue a project that had been declared a failure [8]. In XP, simplicity is the driving factor that applies to the software development practices including the communications with the customer. Beck has defined twelve rules that govern the XP method [14] [42] that are: • Planning Game: the planning starts at the beginning of each iteration where the stakeholders of the project meet to define, estimate and prioritize the "User Stories" (i.e. requirements) for the next release.
• Small Releases: there are two types of version releases: initial version and working version. The initial version is produced after a few iterations and it does not implement all features, but only essential ones. While, a working version is produced after a few weeks and contains most of the features.
• Metaphor: it is used in the modelling of the software system and is constructed by all the stakeholders. • Simple Design: It is the base of the XP methodology and applies to the requirements gathering, system design, coding, and communications with the customer.
• Tests: In XP methodology, testing is considered one of the major activities to ensure high quality product in addition to high customer satisfaction. Testing begins before the coding phase, where developers are required to prepare the test functions prior writing the code itself. While, customers are required to prepare the functional test scenarios for each iteration.
• Refactoring: It means that any changes made on the system must uphold the simplicity feature.
• Pair Programming: Coding of the software system is carried out in a group of two developers. • Continuous Integration: New parts of the software system are integrated as soon as they pass both the unit and functional test cases.
• Collective Ownership: The ownership of the code produced belongs to all the developers.
• On-Site Customer: Someone from the customer side must work with the development team at all times. • 40-Hour Weeks: The maximum working hours per week for developers must not exceed 40-hours, which implies that the requirements must be revised to adhere to this rule. • Open Workspace: All work, including coding and development must take place in a common environment.
Following, the review of the most recent works on XP that, up-to-our-knowledge, have not been addressed in any other literature review yet.
Chen and Wu proposed a modified method of XP consisting of 11 steps called "my Agile" in order to allow more integration between XP methodology and other computer science topics such as data structure, Object-Oriented Programming (OOP), algorithms, etc. The 11 steps proposed method ("my Agile") shares 4 steps with the original XP method, which are step 5: "Dispatching and Scheduling", step 6: "Unit Test Code", step 9: "Coding" and step 10: "Unit Testing and Acceptance Testing". Although, the authors identify steps 0, 1, 2 and 3 ("Exploring Requirements", "Scenarios", "Acceptance Test Cases and User Manual", and "CRC Session" accordingly) as adopted from the Software Engineering discipline, steps 0 and 1 can be easily mapped to the first rule (i.e. planning the game) of the original XP method. While, step 3 can be partly mapped to the fifth rule (i.e. testing). Although, the remaining three steps 4, 7 and 8 ("Reverse Engineering Tool", "Data Structure Design", and "Algorithm Design" accordingly) are identified as novel steps designed specifically for "my Agile", step 4 can be seen as merely using Computer-Aided Software Engineering (CASE) tools to automate some of the design and verification steps. A project aiming to develop a student grading system was designed to evaluate the proposed methodology and it was applied in four universities in Taiwan [43].
Haryono presented a case study where XP methodology was applied in a project for developing a Financial Management System (FMS) as a part of the E-Government (e-Gov) in Indonesia [44]. The project followed the core practices of XP methodology as shown in Figure 5. The project was evaluated using a satisfaction questionnaire. The results showed that 100% of respondents were satisfied with the method and indicated that it has helped with communications.
Moreover, it showed an increase in the sense of belonging to the system. Nonetheless, 2% of respondents declared it less fit to the project type.

Scrum
Scrum is a management and control process used for developing and sustaining complex products in order to build software that meets business needs, incrementally and empirically. It is considered a widely used agile method, first Figure 5. Development methodology of the Indonesian E-gov FMS project. Adopted from [44].
described in the year 1996 [12]. Scrum is also considered a lightweight, simple to understand and difficult to master method.
Scrum starts by splitting the project into iterations (sprints). Before each sprint, in the planning phase, all tasks to be done are kept in a list called "release backlog". During the planning process a next-sprint goal is identified and announced to the developers in order to show them the tasks are being performed and at which level of detail to implement them, in addition, a prioritized collection of tasks are selected from release backlog to be completed in the next sprint.
When the planning phase is complete, each development team carries their tasks.
The tasks in the sprint backlog remain unchanged until the end of the sprint phase. While the development teams develop their tasks, the project should be tracked and monitored through daily meetings and track tasks status in order to enhance communications between teams and keep focus on the overall project goals. After every sprint, an analysis and evaluation process progress through pre-sprint meeting.
Following, the review of the most recent works on Scrum that, up-to-ourknowledge, have not been addressed in any other literature review yet.
Chandana Ranasinghe and Perera demonstrated how the challenges and issues related to offshore development (OSD) in Sri Lankan context can be overcome using the scrum method, and to achieve success in OSD it is important to combine engineering practices with scrum [46].  [50]. A case study is also presented that illustrates the adoption of Scrum to manage the capstone project, which represented a direct and objective approach in order to have an environment similar to the real one. The authors' proposal was to determine, discuss and quantify how flexible and collaborative Scrum becomes when teaching Software Engineering.
Pauly, et al. presented a study to assess the adoption or adaption of Scrum principles at an e-commerce company. The authors presented an in-depth single case study, which in turn revealed that not all scrum principles are suitable in each context [51].
The relationship between SCRUM and SDLC practices was studied in [52].
The authors proposed a framework to utilize the user experience design in SDLC in organizations that use SCRUM in association with Capability Maturity Model Integration (CMMI) practices. Moreover, the authors utilized the user experience design dimensions recommended by the Human Factors Institute, which include training of professionals, creating and managing metrics to evaluate the usability, and establishing a successful cases database for training purposes.
Many studies focus on using the scrum agile method in software companies, such as [53] that study using of SCRUM in Brazilian small business.
Raj, et al. proposed a modified scrum process that focuses more on testing, using Test-as-a-Service (TAAS) implementation in order to get the results faster without increasing the cost of the project [54].

Feature-Driven Development (FDD)
FDD methodology is one of the AGILE methods for software development. FDD is an iterative and incremental method based on dividing the software into many different features (models), and then builds each model separately. The development process for each model (feature) consists of five activities: develop general model, build feature list, plan for feature, design for feature and build by feature.
In the first activity, develop overall model, general high-level overview of the project is set to better understand the problem domain, this step is im-portant to the team in order to manage the relationship and interaction between the team members and the customer. In build feature list activity, the team uses the knowledge extracted from the first activity to build a list of features (or functionalities) required, and categorize them according to business subjects.
The next activity is to plan for feature, in this stage, the development plan set by the project manager and development manager, the plan contains the ownership of feature, the time schedule and the responsibilities of the feature development team. In the next activity, design the feature, the focus moves to the features itself according to the programming tasks, the classes defined, the sequence diagrams provided and methods prologue defined. The final activity is to build by feature where each feature developed.
FDD is used for large projects, because it can be divided into many small-size tasks, which increase the possibility to complete the project successfully. This also gives the management the feasibility to change the team while the project running is without affecting the project time schedule and the overall quality.
On the other hand, the documentation task is more complex, and the overhead of the chief programmer is high, because he acts as a coordinator, mentor and lead designer.
Mahdavi-Hezave and Ramsin proposed the Feature-Driven Methodology Development (FDMD) for Situational Method Engineering (SME), where the object-oriented features used to specify the requirement of the target methodology of SME in order to facilitate the development of the target methodology, and enhance maintainability and reusability [55].
ISMAIL, et al. compared the difference between using FDD and SCRUM methodologies to gather requirements for Open Source Software (OSS) projects [56]. The authors concluded that FDD team gets more time than SCRUM to accept changes in requirements, because FDD divided changes according to its severity while SCRUM accepted changes in requirement at any stage of the development process. The authors conclude that both the methodologies set the tasks priorities with different roles, in FDD the domain expert does this task, while in SCRUM it is done be scrum master. The authors compare the two methodologies according to customer interaction too. They conclude that the customer interact with teams at every phase in SCRUM, and in the initial phase in FDD.

Evaluation
In this section, the evaluation of the reviewed studies is categorized into two main categories; survey of surveys and new agile methods.

Evaluation of the Survey of Surveys
In this survey, the authors study several surveys related to agile software development. Table 3 summarizes twenty-sex survey papers related to agile software  The table indicates the reference number for each paper, the paper published year, the paper type (whether the paper is published in either a conference (C) or a journal (J)). Additionally, Table 3 demonstrates the publisher and shows the survey category used in the article according to classification illustrated in Figure 2. Furthermore, it illustrates the period covered by the survey paper in addition to the number of articles surveyed in each reference. Finally, it lists the methods used in each survey.
As illustrated in Table 3, the number of surveys in agile software development increased in the last three years, the year 2015 alone has eight papers, six journal articles and two conference papers were published in reputable databases. The reason of this increase refers to the increasing number of methods and enhancements of the Agile methods; thus, the need to compare, analyze and summarize the increasing amount of researches in Agile methods become a very important topic to the researcher and to the software development industry in order to improve their performance to gain better output quality. Figure 6 shows the number of survey papers published per year.
Interestingly, the distribution of paper type across conference papers and journal articles was divided equally as shown in Figure 7. Notably, "Hybrid Agile Methods" category was the most of surveyed in the literature showing the importance of agile method compared to the other development methodologies. Figure 9 shows the percentage of survey papers based on the four classification categories.
Lastly, XP and Scrum Agile methods were the most surveyed agile methods.
Over 5529 papers were addressed in total in all of the 26 survey papers.       As explained in Section 4(C), "FDMD" utilized common OOP practices in the requirements elicitation.

Conclusion
Agile is considered one of the most popular software design and development