QualiTeam: A Support Tool When Learning Software Quality and Testing Concepts

QualiTeam is a web application to support the teaching-learning process on Software Quality Assurance, Quality Control and Testing introductory concepts. It has two main objectives: to facilitate the understanding of concepts learned in theory and to facilitate the monitoring of SW projects that students develop. The system gives the teacher control and the students a guide on the activities that must be carried out throughout a software project development. QualiTeam is a tool conceived to help in the challenge of providing students with concrete examples with which they can practice and clarify the topics taught in the classroom. With it, students can apply concepts that, in the ini-tial training of a software engineer, are generally taught only at a theoretical level such as: review process, change requests, trouble reports, document version control and testing documentation management. QualiTeam is free and available online. It has been in operation for 5 years, through which improvements have been made until achieving a quite stable version.


Introduction
With the expansion of the use of computers throughout society and the consequent growth in the size of programs, Quality Assurance (QA), Quality Control (QC) and Testing have become fundamental aspects of Software (SW) development. There is no SW product, with a minimum degree of complexity, which is exempt from defects that, at least potentially, generate failures. Since SW systems are used more and more in devices in which the consequences of failures are extremely serious (death, failure in important missions, security breaches, financial al exercises and group projects for the teaching of quality assurance. Although they specify the standards on which they are based to teach each of the topics, they do not specify the way in which theoretical concepts are put into practice. On the other hand, they do not use a specialized SW tool for practical exercises. Regarding to tools to support activities related to these topics, Ibarra and Muñoz [5] made a review of tools and practices implemented for SQA and they found that more frameworks than tools have been developed. Muñoz reported in [6] that most of the tools for SQA are focused on the execution of dynamic tests to solve code problems, as well as on the continuous integration of software components. However, as it was already noticed by [7], quality assurance isn't just testing but also methods and approaches. The tool proposed in [5] is meant to be used by companies to support their SQA activities, while QualiTeam is meant to be used by undergraduate students in order to reinforce specific concepts included in syllabus.
As far as we know, there are no SW tools similar to QualiTeam, which is devoted to being used for students in order to put into practice what was learned in theory.
It is worth mentioning that in this work we explain some of the SQA and Testing subjects and their relationship with QualiTeam, however we will not give a detailed explanation of each of the subjects. The structure is as follows: Journal of Software Engineering and Applications Section 2 contains a brief QualiTeam description. Section 3 has an explanation of the QualiTeam basic operation. Section 4 explains how to work with the Qua-liTeam subsystems. Sections 5 and 6 have results and conclusions respectively.

QualiTeam Scope
In the SW industry, Quality Control (QC) means looking for defects and correcting them, at the lowest possible cost, and before they produce failures [8]. In general, this is achieved with tests specifically designed to verify that the SW does not fail, that is, that it meets the requirements. This verification is carried out throughout the development process and, therefore, on all generated intermediate products (milestones) are verified. The idea is to detect defects at the earliest possible stage to prevent them from spreading to the following stages, since the costs derived from their correction would be greatly increased [9]. Qu-aliTeam is a tool to help during the process of software developers' training on SW QA, QC and Testing concepts. It supports control of documents associated with different projects. Documents are not tied to a specific software development model, and they can be intermediate products like: requirements specification, design document, code modules and also the documents that indirectly affect them: documentation rules, coding rules, procedure descriptions, test documents, etc.
(To use QualiTeam, users must first register).
QualiTeam was developed in NetBeans and JavaServer Faces combined with Rich Faces libraries for the front-end. It uses MySQL for the database. QualiTeam has the following subsystems: Project Manager, Security, Document Manager, Change Requests, Error Reports and Tests.
In order to upload QualiTeam and making it available on the network, we configured a server with GlassFish, running on a Centos operating system, and MySQL server is used for the database.
Currently QualiTeam is in Spanish, however, if any non-Spanish-speaking University requests it, we will proceed to make the English version.

The QualiTeam Sub-Systems
The QualiTeam subsystems are illustrated in Figure 1.
Security module, users' access to the system, Project Manager manages the projects and allows access to the following subsystems: Error Reports, Change Requests, and Tests. The QualiTeam modules, except Security, use the Document Manager subsystem, which is a service subsystem. The main functions of Journal of Software Engineering and Applications each of the subsystems are briefly mentioned below.
• Security: this subsystem is in charge of managing users and validating that the person who wants to enter the system correctly provides their user ID and password.
• Project Manager: manages the project members (users) and the main project documents. Once a project has been selected, it allows the access to: Error Reports, Change Requests and Tests subsystems.
• Tests: this subsystem is used to manage the different test documents, which are: Test Plan, Test Procedures, Test Logs, Test Designs and Test Cases.
• Change Requests (CR): allows creating Change Requests on base documents (from previous projects) to build documents of a new project.
• Error Reports: it is used to manage the error reports generated on deliverable products (documents or the code of a project).
• Document Manager: provides services to the other subsystems. It is responsible for creating new documents and new versions of documents in the system, download and upload documents to and from the server, and displaying their properties. It is also supports the document review process, that is, distribution lists, comments and change of status.

The QualiTeam Software Development Process
QualiTeam was developed following the waterfall process with subprojects. To connect the requirements phase with the design phase, the User Interface (UI) mockups were made and also we defined the UIs flow with the User Interface Transition Diagrams (UITD) [10]. Then, we defined the system architecture and continued with the detailed design, which was done through Detailed Sequence Journal of Software Engineering and Applications Diagrams (DSD) [11]. During the design of the UIs, DTIUs, system architecture and DSDs, a peer review process was done in order to improve their quality. The Requirements Specification, Design, Tests and User Manual documents were generated.
The implementation began with the coding of the system skeleton. With this skeleton, it was possible to distribute the work to different people, each one had to code one or two subsystems. Module tests were carried out in each subsystem and later they were integrated into the whole system, where integration tests were performed. As a final stage, the server was configured and QualiTeam, with its database, were uploaded.

Project Management
Once the user registers or enters with his/her correct account and password, a list is displayed with the registered projects in the system. The user can select a project from the existing ones or create a new one. When he/she creates a new project, he/she has the role of "leader" of that project. A leader can edit his/her project to modify its name and the members list. The following main documents are automatically generated: Project Plan, Requirements Specification, Design and Test Plan. Project members can also generate new project documents. The user who generates a new document has the role of its author. Users who are not project members can view the documents but they cannot generate new ones neither participate in any of the project's activities.

Document Management
It is important to mention that whenever a document is generated, its related data are registered in the database. These document data are called the "Document Details". However, the content of the document must be uploaded to the server in a PDF format. The "Document Details" interface contains the following data: identity, title, author, version, creation date, status and the effort (in hours) that the author invested in its elaboration along with some action buttons. The "Download Content" button transfers the document content from the server to the user's computer. The author of a document and the project leader are authorized to edit the document details, so they have the "Edit Properties" button active. This button triggers an interface in which the title, the effort, and the distribution list can be modified. So, the editor can change the "author name" to transfer the authorship of the document to another project member. There is also the "Upload File" section which has an "Add" button in this User Interface to send the content of the document to the server. It is necessary for the document to be uploaded using the PDF name extension but the true format of the file can be just anyone. We suggest that other types of files should be named with their original extension but with an additional pdf extension. For instance, a filenamed "code.java" should be renamed to "code.java.pdf" before uploading it to QualiTeam.
The list of comments to the document can be accessed through the Document Detail interface with the "View Comments" button.

Working with QualiTeam Subsystems
In order to work with all the QualiTeam features, the user must know: the peer review process, generation of new document versions, testing documentation management, change requests and error reports. These topics are addressed below.

The QualiTeam Peer Review Process
The SW peer review process in IEEE 1028 standard [12] is applicable not only for SW modules but also for the other documents of a SW Project. The procedure for conducting a document peer review is implemented in QualiTeam and it is illustrated in Figure 3. The procedure is as follows: 1) The author uploads a document and edits the list of persons who will review the document (distribution list).
2) The author or the project leader initiates the review process. This action informs all reviewers about their task on the document.  based on their knowledge, experience and skills, find defects. 4) Based on the found defects, reviewers add comments to the document in a separate conversation thread for each defect. Each comment has the reviewer identity. All reviewers, the author, and the leader can add comments to each thread. 5) After all participants in a thread reach an agreement on what has to be done, the thread may be closed by a reviewer, the author or the project leader. 6) Only the author and project leader have access to the "Create Next Version" button which will be available only when all threads have been closed. 7) When a new version of a document is created, the status of the previous one is set to "Reviewed" and the process repeats from step 1. If there is no need to create a new version, then the document may be "accepted" by the project leader. In this case, a new version of the document is created with the status "Accepted" and with an integer version number, opposed to the fraction version numbers used during the review process.  The current document must be changed to "In Review" status in order to be able to create a new version (the "new version" button is displayed). The subsequent version numbers will be automatically increased by 0.01. When a new version is generated, the current document status changes automatically from In Review to Reviewed, as illustrated in Figure 4. A Reviewed version cannot be edited.

The Document Peer Review Cycle
The project leader gives the document the status Accepted/Rejected when he/she decides to finish the peer review process. Any document in Accepted/Rejected status cannot be edited.

Comments on a Document
The QualiTeam comments system is accessed through the "Comments" button in the "View Details" interface. Any user can view the comments of a document by selecting the "View Comments" button, however, only the members of the reviewers list, the author and the leader have permission to add comments. The user interface to view and enter comments is shown in Figure 5. On the left side column, the list of conversation threads is displayed. For each topic to be discussed, it is recommended that a new thread is added with the "New Thread" button. The list of comments associated with a selected conversation thread is displayed by clicking on one of the threads' "See" buttons.
The author, project leader and reviewers can close a conversation thread with the "Close Thread" button. No one can add more comments to a thread that is closed. If someone wants to make more comments he/she will have to create a new thread.

Testing
When a test is run, the observed behavior is contrasted against the expected behavior of the system or component. The IEEE 610.12-1990 standard [13] defines testing as "the process of operating a system or component under specific conditions, observing and recording the results, and performing an evaluation of some aspect of the system or component". In order to execute the tests of a system, a structured process is carried out, this process is illustrated in Figure 6 and consists of test planning, test design, and test execution. Additionally, a control is carried out to supervise the design and execution of the tests. The planning of the tests is documented in the test plan. In it, the techniques and methods to be used are specified, as well as the environment, the necessary tools and the schedule. During the test design, the elements to be tested are identified. Test data and conditions are also specified depending on the elements to be tested. Also, the structure of the tests and the environment they require are designed. The tests are structured, organized by groups and categories, and prioritized based on a risk analysis.
Tests documents are managed in QualiTeam according to the IEEE-829 Standard [2]. To start the QualiTeam test subsystem, the desired project must be first selected and then click on the "tests" button. Figure 7 illustrates the documents that must be generated during a testing process. The test plan is one of the mandatory documents that are created automatically when a new project is generated. Test plan contains the schedule of the test activities, objectives and requirements of the tests, personnel who will do the tests, software and hardware resources, etc. The project leader uploads the content of the test plan document to the project.  In QualiTeam, a test design groups together a set of test cases. Each test design includes the description of the common characteristics of this set's test cases.
Each of the test cases describes only one test. On the other hand, a test procedure describes how the tests should be run. Then, a test case is associated with a test design and can be associated with a test procedure if necessary. There may be several test procedures associated with the test plan, as indicated in Figure 7.
Finally, a test log contains the information about the tests that were carried out and the obtained results such as whether the test was successful or not. Each test log is associated with the project's test plan. There can be several test logs associated with a test plan.

Trouble Reports and New Versions Generation
An event that occurred during the execution of a test or at any time after a Journal of Software Engineering and Applications product (document) has been accepted that requires correction or clarification must be recorded in a Trouble Record (TR) and linked to some Accepted document. Every TR must go through the peer review process. At the end of this process, it has a final status: Accepted or Rejected. A TR is generated for each found defect, and when the project leader authorizes, a new version of the product (document or SW module), which includes attention to all generated TRs for it, is released. This is, the attention to the TR of a product leads to a new version containing modifications. The new version must include the list of attended TRs and their implementation. It is worth mentioning that this new version must go through a revision procedure in order to ensure its completeness and consistency.
QualiTeam's Trouble Records subsystem allows generating and viewing all the TRs of a project. To start the QualiTeam Trouble Records subsystem, the desired project must be first selected and then click on the "Trouble Reports" button.
The TRs subsystem shows the list of Trouble Reports generated for the accepted documents of a particular project. In the example in Figure 8, three TRs for the "Design" document are shown and one for the "Requirements" document. The TRs review process is the same as for any document, that is, it has the status "In Creation" first, and then it is set to "In Review" by the author and so on.
Each time the button "Create TR" is clicked, a list is displayed with all the accepted documents in the project, since it does not make sense to make a TR on a document that has not been accepted yet. The new TR will be associated with the selected "base document". After that version, the subsequent decimal versions are: 1.02, 1.03, etc. When the document is accepted, QualiTeam automatically assigns the next integer number version, which in this example would be 2.0. Figure 9 illustrates the way in which new version numbers, decimal and integer, are generated.

Change Requests
When Requirements of a new project are very similar to those of previously released projects, then the Change Requests (CR) procedure may be used. The idea of a CR is to reuse documents and/or code from previous projects in order to optimize the documentation and configuration of a new project. CRs are documents describing the modifications to be made to an already accepted product, which is called the base product. A product can be a document (requirements specification, design, tests, etc.) or a SW module from a different project. These modifications are not originated from a defect in the base document but, instead, they arise from the need to make changes to the base project to build a new similar project that does not necessarily replace the base project. Through practice, we observed that there is confusion when teaching the CR concept.
With QualiTeam CRs subsystem, students can practice with CRs and clarify its use.
As it is illustrated in Figure 10, the documentation of a new project may comprise CRs to base products along with the corresponding base product,  products generated from the merging of base products and their CR, new products and even unchanged products from previous projects. Figure 11 shows that, when the product is a document (not code), there are two ways to use a CR. In the first option (Figure 11 a1), the base product and the CR are merged in a new product that would replace the both. In the second option ( Figure 11 a2), the documentation includes both: the base document and the accepted CR on that base document. The latter case is useful when the CR describes only a few changes. All CR documents must be reviewed and eventually accepted to be part of the project's documentation.

Implementation of Change Requests
When a CR is created on a software module, a new software module is always generated for the new project, and it is the result of merging the base project module with the CR, as it is shown in Figure 11(b).
The QualiTeam CR subsystem is accessed by selecting a Project and then clicking on the "Change Requests" button. The CR subsystem displays the CR list of the current project. In the example of Figure 12, the CR subsystem shows two CRs, CR 5-5 was made on the "Requirements Specification" document in Project "QualiTeam 1.0" and CR 5-6 was made on the "Design" document also in "QualiTeam 1.0".
Every time the "Create Change Request" button is clicked, an interface is shown as in Figure 13, that shows the list of all projects, except the current project (in our example "QualiTeam 2.0"). When a base project is selected, its documents in accepted status are displayed on the right. To create a new CR, it is necessary to first select a base project and then the document to which the CR will be associated, that is, the base document to which the changes described in the CR must be applied. Journal of Software Engineering and Applications

Results
QualiTeam was launched at the beginning of 2017. Through these 5 operation years the following maintenance activities have been done: • Correction of the non-detected defects during the testing phase-Most defects were corrected during the first months of the system operation. The current version is already quite stable.
• Improved features-Users' feedback allowed us to make improvements on the review process operation, specifically in the conversation threads for the reviewers' comments. Improvements were also made to the information presentation in some of the user interfaces. Journal of Software Engineering and Applications Figure 13. Selecting a base document within the selected base project.
• Code refactoring-The code has been refactored in order to facilitate its maintenance, improve performance and solve some bugs. Current version is 3.0.
Our students have reinforced what they learned in the SQA and Testing lectures by doing the "Practices with QualiTeam" included in [14]. This book is available on internet and it is free.
We have taught the course "Quality and Testing" to undergraduate students of Computer Engineering since 2008. In the first 9 years (2008-2016) QualiTeam was not available to support the classes, while the following 5 years the students had QualiTeam to apply the procedures learned in theory. One of the course assignments is to document the procedures seen in class and apply them during the development of a project. We noticed a general improvement between the procedures' descriptions prepared by the students from 2008 to 2016 and those prepared by the students from 2017 to 2021. In some cases, the procedures were improved by students by introducing adaptations to their particular projects when using QualiTeam. Even during the years 2020 and 2021, in which the courses were taught online due to the Covid-19 pandemic, the procedures' descriptions prepared by the students indicated that they were well understood.
On the other hand, it is important to encourage teamwork among students, since most of the projects in the software industry are done by teams. Quali-Team records the work of each author and each document reviewer. This makes it easier for teachers to detect to what extent students actually collaborate with the project. This helped in the students' evaluation and encouraged them to remain active in producing meaningful work for the team, since they experienced Journal of Software Engineering and Applications in practice getting a bad grade when their participation in the teamwork was not as expected.
QualiTeam is maintained by the authors, at Universidad Autónoma Metropolitana.

Conclusions
The training of students in the area of SW Quality Assurance leads to a challenge which is: providing students with concrete examples with which they can practice and clarify the topics taught in the classroom. Jaccheri [15] proposed an interesting approach which consists of having interactions with the SW local industry. While these interactions are undoubtedly beneficial for future SW developers' training, it is worth mentioning that finding companies willing to collaborate with students can be quite difficult. Moreover, linking companies with students is out of the reach of many teachers/researchers. QualiTeam was conceived to help overcome this challenge.
QualiTeam is a free tool that helps in the teaching-learning process of the following software quality assurance procedures: peer review process, trouble reports, change requests, document versions control and management of tests documentation according to the IEEE-829 Standard [2]. With QualiTeam, the teacher can see the record of the individual contributions to a project of each participant, which helps in guiding students when someone does not fully understand a process. Additionally, the record of each student's work promotes a more equitable participation during the development of team projects and helps the educator in the students' assessments.
According to [16] various studies have demonstrated that blended learning systems, a mixture of face-to-face and online education, have important advantages over fully face-to-face or fully online courses. QualiTeam made remote work much easier during the online classes in the present Covid-19 pandemic suggesting that it could be adopted by online and blended education.
QualiTeam has successfully helped the teaching of "Quality and Testing" at the Universidad Autónoma Metropolitana in Mexico City for five years now and we hope that other universities can also benefit from this teaching tool. Users' comments are always welcome; thanks to them we can keep on improving the tool.