User Experience Design and Agile Development: From Theory to Practice

We used existing studies on the integration of user experience design and agile methods as a basis to develop a framework for integrating UX and Agile. We performed a field study in an ongoing project of a medium-sized company in order to check if the proposed framework fits in the real world, and how some aspects of the integration of UX and Agile work in a real project. This led us to some conclusions situating contributions from practice to theory and back again. The framework is briefly described in this paper and consists of a set of practices, artifacts, and techniques derived from the literature. By combining theory and practice we were able to confirm some thoughts and identify some gaps—both in the company process and in our proposed framework—and drive our attention to new issues that need to be addressed. We believe that the most important issues in our case study are: UX designers cannot collaborate closely with developers because UX designers are working on multiple projects and that UX designers cannot work up front because they are too busy with too many projects at the same time.


Introduction
Agile development has become a mainstream regarding software development processes.At the same time, an increasing understanding of the importance of good UX came along and the need to integrate these two areas emerged.Agile methods as well as User Experience (UX) design methods aim to build quality software, but despite this common concern, each approaches development from a different perspective [1].According to the authors, while Agile methods mainly describe activities addressing code creation or project management, UX design methods describe activities for designing the product's interactions and/or interface with a user.
These two methodologies traditionally use different approaches for resource allocation in a project [2].Agile methods strive to deliver small sets of software features to customers as quickly as possible in short iterations while, on the other hand, User-Centered Design (UCD) advocated spending considerable effort on research and analysis before development begins.
Up until a few years ago, little attention had been given to the integration of UX and Agile.This could be due to historical issues such as not understanding the need for a good user experience and by not knowing Agile methods in detail, or just by thinking that they could not work together due to their differences in focus.However, nowadays there are a reasonable number of studies addressing this integration, as can be seen in [3].
Thus, our aim in this paper is to create a better understanding of Agile and UX design in theory and practice.In this paper, we present similarities and differences between the findings from a previous theoretical study [3] and a framework proposed for integrating UX and Agile development that emerged from that theoretical study.We also describe in detail the work involved in integrating UX design and Agile development in a in a world leading technology company that develops and manufactures collaboration products. 1As the result, we will be able to see the theory in practice and the contributions from the practice to the theory.This paper is structured as follows: Section 2 presents some related work regarding this topic; Section 3 briefly presents the proposed framework; Section 4 describes the case study and its findings; Section 5 presents a discussion and Section 6 presents some conclusions and upcoming steps for this research.

UX and Agile Integration
In the following section, we will summarize the existing literature regarding UX and Agile integration.Most of the work is based on case studies and leads to similar conclusions-which we use as the basis of the proposed framework.
Chamberlain et al. [4] present a framework for use by teams trying to integrate UCD practices with Agile development.They present some similarities between UCD and Agile based on the literature and an observational study too.Based on their results, they suggest five principles for integrating User-Centered Design and agile development, such as: 1) The user should be involved in the development process; 2) Designers and developers must be willing to communicate and work together extremely closely; 3) Designers must be willing to feed the developer with prototypes and user feedback; 4) UCD practitioners must be given ample time in order to discover the basic users' needs before any code; 5) Agile/UCD integration must exist within a cohesive project management framework.
Ferreira et al. [5] state that the integration of UI design and agile development is not well understood and report a qualitative grounded theory study of Agile projects involving significant UI design.Some results of their study were that using iterative development-an Agile practicefacilitates usability testing, allows software developers to incorporate results of those tests into subsequent iterations, and can significantly improve the quality of the relationship between UI designers and software developers.
Mcinerney and Maurer [6] performed a set of interviews with UCD designers involved with Agile projects and concluded that all of the UCD practitioners' reports were positive.These authors also report that the current literature indicates that improving our understanding of how to coordinate and integrate the work of UX designers and Agile developers helps bridge the gap between the Software Engineering and HCI.
Still, according to Ferreira et al. [1], the problem of having both Agile developers and UX designers contributing their skills to a software development project has typically been characterized as a problem of merging one method with another.For example, Patton [7] explains how Usage-Centered Design can be combined with extreme Programming; Obendorf and Finck [8] explain how Scenario-Based Usability Engineering was combined with extreme Programming; and Miller [9] explains how User-Centered Design techniques were integrated with an Agile process that was a combination of Adaptive Software Development and extreme Programming.
Sy [10] describes the adaptations at her company, which includes adjusting timing and granularity of usability investigations and the way that usability findings were reported.She also concluded that the new Agile UCD methods produce better-designed products than versions designed using a waterfall approach.Also, Singh [11] proposes a process, U-Scrum, which adapts Scrum to promote usability.Beyer [12] presents a process proposal in which he describes the need for UX designers to understand Agile principles and presents best practices for this integration.
Another indication of the importance of the integration of these two techniques is the number of studies addressing UX and Agile found in the literature over the last ten years.More details and deeper analysis on the existing literature can be found at [3].

Proposed Framework
Jokela and Abrahamsson [13] and Sohaib and Khan [14] commented that Interaction Design and Agile methods fit well, and that the challenge is not to make Agile less agile but in adapting the methods of UCD so they can be "light" and efficient at the same time.
Hussain et al. [15] pointed out some beneficial similarities between UCD and Agile, e.g., having the client on-site, continued testing and iterative development.Moreover, as noted in [16], the two methods have much to offer when they share iterations because the iterations used in Agile facilitate usability testing and allow developers to incorporate results of these tests in subsequent iterations.However, [17] commented that improving the usability of a product does not come without costs or risks even when the methods are rationalized.
In order to integrate Agile and UX Design and at the same time minimize these costs and risks, the proposed framework suggests the use of usability artifacts and practices in a condensed form.This is indicated by Agile principles and we expect that it will improve the usability of products while at the same time minimally impacting the activities of normal Agile development.
The structure of our framework is similar to the processes described by [5,10,18].The difference is that we propose a combination of the most common practices, artifacts and processes identified in the systematic review [3] previously cited, i.e. our framework is more concrete in its recommendations than previous work.
The flexibility and adaptability offered by Agile methods are widely discussed, so the intent of our framework is not to stiffen these methods, but, as suggested by [14], to adapt usability practices to an Agile setting in order to improve the usability of products developed using these methods.
The framework we propose is presented at a high level in Figure 1 and it is organized according to the activities of the Interaction Design lifecycle model [19], as follows: User Research, (Re)Design, and Evaluation.

User Research
methods, the Interaction Designers should focus on a small number of tasks at a time instead of on trying to design the entire product at the beginning of the project.One of the major concepts in Agile is LDUF (Little Design up Front 2 ); that is, it is recommended to perform only a little (but not all) design work before implementation begins.We propose the adoption of Sprint 0: an iteration that takes place before the start of implementation or even before the official kick off of the project.It is important to include Interaction Designers in Sprint 0 due to their skills in gathering and analyzing data from UX because, according to [20], developers tend to listen to what customers want instead of looking at what they do.
Contextual investigation, task analysis, user interviews, prototyping, and prototype validation could be performed during each iteration.To facilitate the process, the Interaction Design team should work on user research one sprint ahead of the development team.This allows the results of the user research to be fed into the iteration plan in form of user stories on time for the development team to start implementing the design.Sprint 0 encompasses activities such as context research (Contextual Inquiry [21]), Observations, Task Analysis, and Interviews.Such activities should generate paper prototypes and/or Design Cards3 that will help to define the User Stories.To facilitate LDUF, activities related to requirements gathering and analysis must be performed throughout the development process in order to avoid concentrating these tasks towards the beginning of the project.The result of this should be just-in-time design.In other words, as with the developers using Agile Once User Stories have been defined, the Interaction Design team should design the user interface and validate these stories, adding usability aspects as acceptance criteria for each User Story.The Interaction Design team should deliver the Story Cards to the development team.These activities should be done before the planning game, so they can be discussed during that activity.

(Re)Design
Prototypes, Feature Cards 4 , Design Cards, and Issue Cards 5 could be used to report both designs and evaluation results to guide the redesign of the interface.It is important to mention that we do not want to suggest specific types of prototypes because each team or product has its particularities regarding prototypes and diagrams.
A light way of reporting such issues is very important in an Agile context.Using simple artifacts helps in communication between all stakeholders and adds value to the development process.We recommend the use of a User Experience Board for maintaining a shared vision where possible.
We can use prototypes and/or Design Cards for communication between UX Designers and developers.When delivering designs to the development team, it may also help to make use of Feature Cards and prototypes-both low and high fidelity-this will depend on the teams' characteristics.
Problems and/or modifications can be reported in daily meetings using Oral Storytelling 6 and Issue Cards.This can help the development team incorporate design improvements into their work.

Evaluation
We suggest evaluating the entire implementation one sprint after development.This is because evaluating the implementation of the current sprint would require the development team to deliver before the end of the sprint.This would reduce staff time for development and leave them idle while the Interaction Design team conducts evaluations.
Usability evaluations should be performed regularly.Performing user testing during the sprints is hard given the time required to prepare, schedule, conduct, and analyze this type of test.So, while it may not be possible to do these evaluations for each sprint, we recommend that they be done, at the least, before the delivery of the release.It is worthwhile to mention that these tests should be conducted with real users.An alternative is the inclusion of activities related to data capture and usability when performing acceptance tests.
The UX Team should work closely with the development team to support them in terms of designing and conducting inspection evaluations on the implementation of the current sprint and provide feedback.This needs to be done without blocking the development team.However, the UX Team must analyze the problems they identify and determine if there is time to fix these issues in the same sprint or if they should be documented and carried over into the next sprint.

Case Description
We performed a field study in an ongoing project of a medium-sized company in order to evaluate if the proposed framework fits in a real project and how some aspects of the integration of UX and Agile work in the real world.
We begin by describing our field study in terms of the people, the project, the research site, how we collected the data and performed the data analysis.We then relate our findings to the aspects highlighted by our proposed framework.
A study of UX designers and their interactions with an Agile team working on the same project was carried out over three iterations-45 days.In the following section, we describe the team, their project and their organization setting then we discuss how we collected and analyzed the data.

The People
Our study involved seven members of an Agile team and one UX designer.The developers were part of the Development Team and the designer was part of the UX Team.The developers had been developing software using Scrum for approximately two years.Although they are called developers, individuals in the team have their own role according to their background and skills.The roles were Project Manager/Scrum Master, Product Owner, Developer, Technical Leader and Tester as can be seen at Table 1.
Information architects, graphic designers, and interaction designers compose the UX team.Each project has one UX designer, but a UX designer usually works on more than one development team at a time.The same goes for Project Managers.Interestingly, Project Managers are known as Scrum Masters in this organization.

Role Individuals
Project manager/Scrum master 1

Product owner 1
Technical leader 1 Developer 2 Tester 2 4 Describes the acceptance criteria that determine when that feature is complete, and also includes a time estimate for completion [10]. 5Physical cards used to communicate information from observations and interviews with users.This is a way of communicating to persuade [10]. 6It is a technique that can take rational ideas and bring them more fully into the world by giving them a human context to affect people.So one of the best things about stories is that they inspire other stories [29].
It is worthwhile to mention that despite we observed one team and one UX Designer, as presented in Table 1, we interviewed three UX Designers of the company.

The Project
Due to confidentiality constraints, we cannot provide much information about the projects.As already mentioned, we observed two projects that we will refer to as Project X and Project Y:  Project X: development of new features for an existing product of the company;  Project Y: development of an existing product of the company for a mobile/tablet device.
The UX member's role in Project X was to help software engineers envision new features for the product.In Project Y, the UX members' roles were to prototype and design the User Interface and the User Interaction flow for the product.

The Research Site
The developers and designers were part of a technology company that develops and manufactures collaboration products, as already mentioned.
The team of developers was one of several Scrum teams in the company working on software development.The developers and designers were seated in an openplan office space located in the same building.However, they were not seated together.They were spread in the building, although the UX team members were seated close to each other.The researcher was seated with a UX member that was observing these projects.

Data Collection
We used two first-degree data collection techniques: observations and interviews.
Regarding observations, due to the characteristics of invoking the least amount of interference in the work environment and the least expensive method to implement and, most importantly, because the company did not permit video or audio recording of the meetings, we choose to manually record our observations of the meetings described below.
We shadowed a UX person during his activities for three iterations-45 days-and observed meetings in which he was involved, such as meetings of the UX Team and meetings for the two different projects:  Project X: two requirements7 meetings, one retrospective meeting;  Project Y: one demo meeting, three planning meetings, three retrospective meetings and two user testing sessions;  UX group meetings: four meetings.
For our interviews, we interviewed three members of the UX group that work in different projects and one project manager.
The Project Manager was interviewed in order to better understand which Agile methods the company uses and how this integration of UX and Agile is or is not successful from his point of view.The UX people were interviewed in order to understand UX work on the different projects of the company.

Data Analysis
We performed Open and Focused Coding [22] to analyze the data.Initial memos were extracted by the researcher from the field notes produced during the observations and from the interviews performed with members of the teams.
After generating memos, Open Coding was performed in order to generate new insights and themes.Focused Coding was also performed and this coding consisted of linking the memos generated to the key aspects identified in the systematic review [3].Also, some new aspects emerged from the analysis of the observations and interviews.Later, some integrative memos were also written in order to relate the field notes, the key aspects, and these new codes.
We classified our findings according to the key aspects used for Focused Coding.We also presented our findings to the company in order to validate these insights and the teams' members confirmed them.
In the following section, we relate our findings in terms of the activities we observed and place these activities in the proposed framework context, establishing a relationship between theory and practice.

Findings
Here we take a deeper look at the work of the UX designers and Agile developers and how they coordinated their work on a day-to-day basis.The work of interest during our observations and interviews was all the activities, tools, and artifacts used from requirements elicitation to the product evaluation.In this section, we link between the memos extracted from field notes and interviews and the codes-key aspects-that emerged in the Systematic Review.

Little Design Up Front
-There is no design up front.
-There is no collaboration between the UX team and the Marketing Team.We noticed that the company does not perform any design up front.We also noticed that there is no collaboration between the UX Team and the Marketing Team, based on comments like: "Some User Research is performed by the Marketing Team.In general, the Marketing Team knows what they say they need, not what they really need.It's a not a target effort to gather what the user need… It's a sell visit."-UX1 8.Additionally, UX3 remarked that "We absolutely are not close to the Marketing Team."The accomplishment of LDUF by the UX Team is compromised because the Marketing Team does not have the background to perform real user studies.Consequently, the results from their studies are not that good for design.Only some information comes from the Marketing to the UX team.Since there is almost no User Research before the project is launched, the UX Team has to try to think from the user's point of view.But we observed that in some projects there is an effort from the UX Team to participate of the Requirements Meetings to better understand the needs of the Customer; however, it is worthwhile to remember that the Customer is not always the User.Thus, we notice that the UX Team understands the concept of LDUF ("We don't need to design everything up front."-UX3)although they do not make use of it.

Prototyping
-UX Team makes really good use of prototypes.
-The use of a certain type of prototype depends on the UX Designer's background.We noticed that the UX Team makes really good use of prototypes.Depending on the situation they use, e.g."Paper prototypes, high fidelity prototypes, the product… sometimes prototyping tools, sometimes high-fi prototypes, sometimes low-fi."-UX1.Some members of the UX Team cannot design/draw interfaces and they complain about that, as follows: "We UX people all can do some stuff, for example, User Testing and finding usability iss-ues… but not everyone can be designers, for example" -UX1."It's tricky for UX people to code."-UX2.The term "code" here is not about being a developer, but is about creating a UI using HTML, for instance.There is an issue because some of team members are UX Designers and not Graphic Designers. 9We also noticed that since the members of the UX Team have different backgrounds, some of them can code some functional-highfidelity-prototypes and some of them cannot.
User Testing -User Tests with real users are rarely performed even when the project is in its final stages.
-User Tests used to be performed with internal users.
This led to problems after the product release.When they perform some User Testing with real users, the pro-bability of a product being released with fewer problems increases.User Testing used to be performed with internal users based on the justification that there are always new and old employees with different profiles and backgrounds: "[For] internal studies… [we use] new people and old people from inside the Company..."-UX2.
User Stories -There are not User Stories specific to UX.
-There is no standard on reporting UX issues.Sometimes when UX User Stories are written, these stories are then broken into smaller stories with technical development criteria.Hence, in general, User Stories do not have usability issues as acceptance criteria as suggested by the literature, e.g.[11].We also notice that sometimes User Stories are used to report usability issues identified during usability tests: "Sometimes we add new user stories based on the results of the User Testing.But it depends on the problem.We can also put them as a bug."-UX2.The different ways that teams and UX members use User Stories lead to another observation: a lack of standard on how to report UX issues.

Usability Inspection
-UX Team performs peer reviews on the UI design.
-Peer reviews work well for them.We observed that the UX Team performs some inspection evaluations, e.g. a member of the UX Team designs a prototype and then another member perform some evaluations ("We perform some experts evaluations, peer review."-UX1).Even though they do not follow a specific method of inspection evaluation, e.g.Heuristic Evaluation [23], it seems to work very well for them.This peer review is a practice like pair programming, and it seems to provide the necessary feedback to the teams.
One Sprint Ahead -UX Team does not work one sprint ahead.
-UX Designers work on more than one project at the same time.
It is clear that the UX Team does not work one sprint ahead from the development team as suggested by the literature.Although the UX Team knows the benefits of designing one sprint ahead, it is not possible for now because they did not find the time yet ("We should work at least one sprint ahead the development team."-UX3).One of the problems is that the UX Team members do not have time to design one sprint ahead of the development team, probably because they are busy with other projects.It is consistent with the fact that there is not only One Team as Agile principles suggest.We observe that the UX is almost outsourced.Even if there is a UX Team within the company, UX members are not full members of the development team, they are always working on many projects at the same time: "We used to work on multiple projects, but it's really easier to work on only one product.It is so much easier to get involved, you know what's going on.Your attention is right directed."-UX2.UX people cannot have the same level of inclusion in all the projects they are involved concomitantly ("I'm working on 7 to 10 projects at the same time.With different levels of inclusion."-UX3).We suggest that UX members should be dedicated to one team or a very small number of teams in order to avoid issues.This might be related to the issues with LDUF discussed above in that if no research or design is carried out up front, the UX Team member does not know what do in the current sprint based on the upcoming sprints.

Close Collaboration
-There is no close connection/collaboration between UX and Marketing.
-UX Designers cannot be deeper involved in a project because they are working on too many projects.We observed that there is no collaboration between the UX and the Marketing Team, but there is a good communication between the UX Team and the Development Team.However, as already mentioned, sometimes UX members block the Development Team because they are working on other projects.Sometimes even the daily meetings block the UX member, because since they are working in many projects, they have a lot of meetings to attend.For example: "UX people should be Pigs 10 ."-PM.UX people should be more committed to the projects.But it is not that they do not want to, they just cannot because they have too many commitments.("Sometimes, the member of the team doesn't know who to answer to, to the Project Manager or to the Function Manager."-PM.)This last quote seems to be a problem due to a confusion of roles inside the company, which compounds an already difficult situation, described below.We could observe that the company uses an adapted Scrum.In Scrum by the book, there are: Product Owner, Scrum Master and Team.In the company observed there are: Project Manager, Product Manager, Function Manager and Team.Each team member has a Function Manager based on her skills as well as a Project Manager based on the project(s) she is assigned to.A project team consists of team members with different skills reporting to their Function Manager.The Product Manager acts more like the Product Owner.The Project Manager acts more like the Scrum Master in terms of being a facilitator, because he is not a developer and he also manages more than one project.The Function Manager is in charge of the teams' members' time.For example, the UX Team has a Function Manager who defines which project has to be prioritized.That is why we said that UX is almost outsourced: the UX member is not always available to the rest of the team.It is extremely important to note that all of the observations and findings are related to new projects or new products.We notice that for products that are being developed for a long time, the UX members that work for a long time in the same project already have some standards that they follow.But these standards are too specific to be used in another product or by another member of the UX Team.
Big Picture -UX Designers have no problem to maintain the project Big Picture.We did not observe problems about the maintenance of the Big Picture of the company's projects.However, we heard some complaints from the UX Team members about how to communicate their ideas or design decisions to higher levels of management in the company.They said that sometimes Directors, for instance, do not understand their concepts and prototypes are not enough.
We performed a thematic analysis of the data from the observations and interviews in order to identify patterns or themes.Then we relate the findings to the three activities that compose interaction design [19]: User Research, Design, and Evaluation.

Discussion
Our thematic analysis led us to some conclusions situating contributions from practice to theory and from theory to practice.They are briefly presented as follows.

User Research
Regarding User Research, we notice that the UX Team does not perform any-at least in the projects observed.The Marketing Team performs some studies, e.g., some observations in context, however this is not always the case.Based on the literature, we recommend the use of techniques such as Contextual Inquiry in order to benefit from User Research because it is important that there is some design up front, but not all.

(Re)Design
While the UX team made good use of prototypes, we notice that these artifacts were not used for the communication between the UX Team and the Development team.Based on the literature, low-fidelity prototypes could be used to facilitate communication between UX and development teams and presentations of these prototypes and concepts could be used to facilitate the communication between UX teams and Management/Stakeholders in the early stages.The use of these practices would not require significant changes to the company's process and could result in strong benefits.

Evaluation
The use of peer review evaluations and validations by members of the Development Team and other UX Team members was perceived by team members to be a quick way to get feedback about their work.However, we maintain some reservations about the effectiveness of this practice-especially given that our investigation uncovered that issues were likely to be found earlier in an application if evaluations using real users were performed.

Conclusions
The theory on which we based this work-the results from a Systematic Literature Review-led us to a framework for integrating UX and Agile development.This framework briefly described in this paper consists of a set of practices, artifacts, and techniques derived from the literature.
The practice on which we based this work-a field study carried out in a company that tries to combine UX and Agile-showed us some aspects that work and some that do not work in this specific real-world environment.
By combining theory and practice we were able to confirm some thoughts and identify some gaps-both in the company process and in our proposed framework-and drive our attention to new issues that need to be addressed.
We believe that the most important issues in our case study are: UX Designers cannot collaborate closely with Developers because UX Designers are working on multiple projects and that UX Designers cannot work up front because they are too busy with too many projects at the same time.
According to our observations, this is translated to a cycle.Once a UX Designer is working on more than one project at the same time, he is busy all the time and he cannot perform research or a little design up front.Without any research or design up front, he does not know what to design afterwards.Consequently, the UX Designer is always working at the same iteration of the Development Team, or even worse, he is working one sprint behind them.The issue is that the UX people do not have the information needed to make informed decisions about UX design at a point when they can easily impact the development effort.
This study carried out inside this company is part of a larger Action Research initiative that we are performing within this company.So, we are constantly analyzing the data and refining the framework.
As we could notice in the literature, Mcinerney and Maurer [24] reported results of interviews with UX Designers, just mentioning the necessity of adding UX Design into Agile development without a specific proposal.Chamberlain et al. [25] presented general principles for the integration of UX and Agile that should guide UX Designers on how to take advantage of the Agile process structure.Hussain et al. [26] present some of the artifacts most used by UX Designers working on Agile teams, whereas Adikari et al. [27] introduce the concept of Little Design Up Front.Sy [28] reports experiences on integrating UX Design and Agile development, suggesting adaptations of the UCD (User-Centered Design) in terms of tasks granularity, reporting and timing.
As can be noticed, a lot of studies have been undertaken, however each of them addresses different aspects of the integration.Moreover, in general, the studies provide experience reports or propose an approach without any kind of verification or validation.The framework proposed aims at addressing different aspects of this integration, providing alternatives to the UX Designer inserted in the Agile context.
One of our next steps is to perform field studies in other companies with ongoing projects, instead of new ones.We are aiming at refining our framework and making it suitable to different stages of projects and to different companies.

Figure 1 .
Figure 1.High-level representation of the proposed framework.