The Development and Evaluation of a Multi-Agent System for Supporting Flipped Classroom

The flipped classroom is an active teaching methodology that alters the organization logic of face-to-face teaching, in which content is studied before students attend class, emphasizing the active participation of the student in the search of knowledge and teamwork during the resolution of tasks, giving opportunities to experience educational practices that allow the simultaneous execution between theory and practice. However, when using this methodology, a problem that occurs during its application is the difficulty for the teacher (facilitator) to follow the whole progress of the activities performed outside the classroom. Thus, this paper presents the development and evaluation process for a multi-agent system to support the flipped classroom according to the needs of users, with the purpose of correcting the problem inherent to the implementation of this learning theory.


Introduction
Distance Education (DE) is a teaching and learning modality that has grown and produced good results. The research of Mansour & Mupinga (2007) demonstrated that practices and concepts of distance education are converging to face-to-face teaching and forming hybrid methodologies and merged with in-formation and communication technologies, in which the simple incorporation of these technologies in the teaching and learning process is not enough.
According to Bennett, Dawson, Bearman, Molloy, & Boud (2017), it is necessary to think of methods that appropriately incorporate these technologies, from a broader and more critical thinking of the entire teaching and learning process, as well as the agents involved in this process. In view of the above, it is necessary to use, in the educational context, methodologies that allow greater interaction between student and teacher, in order to stimulate essential characteristics, such as self-directed study, construction of autonomy and the simultaneous application between theory and practice. In this way, it is highlighted in this paper the flipped classroom method, which constitutes an active methodology that enables the student's collaborative participation in the process of knowledge construction.
However, a problem that occurs during the application of this methodology is the difficulty for the facilitator to follow the whole development process of the activities carried out outside the classroom. Thus, this paper presents the development and evaluation process a multi-agent system for supporting flipped classroom according to the needs of users, with the purpose of correcting the problem inherent to the implementation of this learning theory. This paper is structured as follows: Section 2 describes the main concepts related to the flipped classroom; Section 3 presents the survey of the user requirements; Section 4 presents an explanation of multi-agent systems, Subsection 4.1 describes agent-based approach to support the flipped classroom; Section 5 describes the implementation of the system; Section 6 carries out the evaluation of the developed system; finally, Section 7 presents the final considerations and future work.

Flipped Classroom
The Flipped Classroom is a methodology that inverts the logic of classroom organization, that is, it is a teaching modality in which content and instructions are studied before the student attends the classroom, which is now the place to deepen the contents already studied. In this learning theory, the facilitators do not transfer the knowledge to the students and then test them through assessments, they motivate the students to seek through investigation and research certain knowledge.
According to the Flipped Classroom Field Guide (2014), the basic rules for invert the classroom are: 1) classroom activities involve a significant amount of questioning, problem solving, and other active learning activities, forcing the student to recover, apply and expand the material learned virtually; 2) students receive feedback immediately after performing face-to-face activities; 3) students are encouraged to participate in virtual and face-to-face activities, which are computed in the student's formal assessment, in other words, they score; 4) both the material to be used virtually and in the face-to-face learning environments When applied, the flipped classroom offers some benefits described by Lo, Lie, & Hew (2018) among them can be highlighted the following: • Possibility for students to work at their own rhythm and to develop as much understanding as possible; • Encourage social exchanges among students; • Prior identification of the points that need to be better assimilated and formulation of doubts that can be clarified in the classroom; • Teachers will be able to adapt classroom activities according to the students' needs.

Survey of User Requirements
In order to identify the needs of the users and to elaborate a list of requirements for a flipped classroom computational tool, the following activities were carried out: 1) An initial questionnaire was applied with teachers whose purpose was to analyze their prior knowledge about the use of technologies, the flipped classroom method, and identify their needs to produce a list of requirements; 2) A primary study was conducted based on empirical research on the subject of the flipped classroom, through a systematic review of literature in five databases: IEEE Xplore, ACM Digital Library, Google academic, Capes-Research for subjects, ERIC-Capes journals, seeking to identify the main computational tools used for supporting the application of the method; 3) The existing computational tools were analyzed and the essential requirements for the system were collected in the systematic review; 4) It consisted in gather students from the Computer Science program in the final phase and researchers from the area so that through these collected requirements implemented a computational tool.
According to the requirements collected and to better understand how the application of the method works, a flowchart for the flipped classroom was created, as shown in Figure 1.
According to data collected in the SRL (Systematic review of literature), the relevant characteristics of the main computational tools used to support the application of the Flipped Classroom method were analyzed in Table 1 Conceptual maps x x --Source: Data produced by authors.
functionalities of two or more support tools. Some of them because they were not developed for the educational purpose, end up dispersing students' attention to other resources not related to learning.

Multi-Agent System
According to Russell & Norving (2002), agents are autonomous software entities that perceive their environment through sensors and perform actions in the environment by means of actuators, processing information and knowledge. Multi-agent systems can model complex systems, allowing agents to have common or conflicting goals. Wooldridge (2002) adds that agents must act autonomously in the environment in order to achieve the goals for which they were designed.  (2007), states that intelligent agents are programs that execute a set of operations in instead of a user, using a representation of knowledge that contains the goals of the user. In addition, the author also states that agents are programs that conduct dialogues to negotiate and coordinate transfers of information. In this sense, this research used the multi-agent system-MAS to act in a way that the students and the facilitator do not perceive their actions, carrying out tasks that aim to help the students and the facilitator throughout the proposed cycle for the flipped classroom.

Approach Based on Agents for Supporting Flipped Classroom
In   Input Parameters: Course and task identifier.

A System for Supporting Flipped Classroom
Output Parameters: Sends a request to the adviser agent and the monitor of the type "monitor-task", sending the task identifier.
Activation Condition: When the MAS is started.

Condition of Finalization: When the MAS closes.
Associated Information: It acts by checking, among the existing courses, if there is any recently created task. If a task in these conditions is found, it sends a message to the adviser agent and the monitor, requesting the execution of their behaviors.
Description: A single agent of this type is created in the MAS. This agent is responsible for finding new tasks to be monitored.
Source: Data produced by authors. Goals: This agent is responsible for suggesting the formation of equivalent groups by accessing the information added in the student's access in the "My profile" tab.
Input Parameters: Information about the availability of day and time for study and the level of knowledge in the requirements described by the teacher, assigning values from 0 to 10 for each requirement.
Output Parameters: Sends a request to the monitor by sending the task identifier.
Activation Condition: When triggered by the manager agent.
Condition of Finalization: When the suggestion of group formation is finished.
Associated Information: It acts by verifying the data registered by the users, and through this information suggests the formation of the groups for the development of the task. Description: For each task an agent of this type is created in the MAS. This agent is responsible for suggesting the formation of groups for a given task.
Source: Data produced by authors. Goals: This agent is responsible for verify the accomplishment of the steps and the complete task deadlines, besides monitoring the participation of each student through the chat and forum of the tool.
Input Parameters: Task, groups and student identifier.
Output Parameters: Sends messages to the student's email.
Activation Condition: When triggered by the manager agent.
Condition of Finalization: When the agent realizes that the task is finished.
Associated Information: This agent has two periodic behaviors. When requested by the manager agent, it performs monitor behavior that consists of detecting passive students, checking daily the accomplishment of the steps and the complete task deadlines, and weekly monitoring of the participation of each student through their interaction in the chat and forum of the tool, based on this information the monitor agent sends messages of success or failure to the student's email. Description: For each task an agent of this type is created in the MAS. This agent is responsible for monitoring deadlines and student interaction.
Source: Data produced by authors.
students during their application. The module operates through two types of access: teacher and student.
In teacher access, as shown in the Figures 3-5, after creating the course and adding the activity, it should describe the complete task and define its duration.
Then it add necessary requirements of the task, in other words, what subjects students should be aware of for resolution and divide the task into steps by describing and assigning initial and final deadlines for each of them. In the module references tab it will indicate references and/or attach materials for research.
This information will be important for the action of the agents when triggered.
When the teacher finishes filling out these data, students registered in the course should access the module through the student access, inform their availability of schedules and the level of knowledge in each task requirement assigning  a value from 0 to 10. After all students add this information, the managing agent will send a message triggering the adviser agent that will analyze the reported characteristics and will suggest to the teacher the formation of proportional groups, as shown in the Figure 6 and Figure 7.
It is important to emphasize that when the teacher links or changes the   suggested groups, according Figure 8, all the development of the task will be supervised by the monitor agent through the chat and forum checking daily the accomplishment of the steps and the complete task deadlines, besides monitoring weekly the participation of each student sending success or failure messages to his email.

Evaluation of the Flipped Classroom Computational Tool
The representative users (Teachers and students) were invited to carry out the evaluation directly in the developed system. In this stage was used the Technology Acceptance Model (TAM), an evaluation framework proposed by Davis, Bagozzi, & Warshaw (1989)   After the period of use, a questionnaire was applied in the two classes using the Likert scale, which is a type of psychometric response scale commonly used in questionnaires, seeking to evaluate the system in the constructs based on the TAM model, in other words, in the following dimensions: perceived utility, perceived facility of use and external variables (intention to use, complexity and applied training). The following levels of affirmation were applied to the questions: 1 and 2 disagree with the statement, 3 represent indecision and the values 4 and 5 agree with the statement.
In Graph 1, it is possible to observe that most users of UFERSA considered the tool easy to use, while IFRN users were undecided about its facility of use.
This result can be considered as a consequence of the short period of use of the tool in this institution. However, it can be stated that the tool proved to be easy to handle.
Another variable that had a very positive return in the two institutions studied was with regard to perceived utility, in which most users considered the tool useful for applying the flipped classroom methodology giving the necessary support to its users, as shown in Graph 2.
Based on Graph 3, it can be observed that half of the users of the two institutions showed interest in using the tool, with a percentage of 34% of UFERSA being undecided regarding intention to use and 28% of IFRN. Only 16% of UFERSA users and 25% of IFRN users did not show intention to use at other times. Graph 2. Perceived utility. Source: Data produced by authors.
Graph 3. External variables (intention to use). Source: Data produced by authors.
The complexity related to the flipped classroom methodology was a rather complicated issue to be dealt with the IFRN users, since most of them did not know this methodology and had their first experience in the evaluation period of the system. This difficulty is confirmed by the results obtained in the questions related to this variable, where some users demonstrated a slight difficulty in assimilating the methodology while the others were undecided regarding the complexity.
The UFERSA users have demonstrated knowledge about the methodology and a greater facility in assimilating their goals, because these are graduate students with a greater academic experience, as shown in Graph 4.
In relation to the training applied in the two educational institutions, all the functionalities of the Invert Class system were presented in-person and all the architecture also provides support for group creation.
As a contribution of this work, we highlight the development of an agent-based architecture that supports during the application of the flipped classroom. As a future work, we intend to improve the chat approach by allowing the creation of separate chats by groups to discuss the resolution of the task and add the option to change the dates of the general task steps after registration. Finally, we intend to carry out a case study as a way of validating the solution through the analysis of the academic achievement reached with the use of invert class.