The Development and Evaluation of a Multi-Agent System for Supporting Flipped Classroom


The flipped classroom is an active teaching methodology that alters the organization logic of face-to-face teaching, in which content is studied before students attend class, emphasizing the active participation of the student in the search of knowledge and teamwork during the resolution of tasks, giving opportunities to experience educational practices that allow the simultaneous execution between theory and practice. However, when using this methodology, a problem that occurs during its application is the difficulty for the teacher (facilitator) to follow the whole progress of the activities performed outside the classroom. Thus, this paper presents the development and evaluation process for a multi-agent system to support the flipped classroom according to the needs of users, with the purpose of correcting the problem inherent to the implementation of this learning theory.

Share and Cite:

Mendes Neto, F. , de Almeida, L. , de Freitas Lopes, E. , de Araújo Pontes, V. , Chagas, J. and Alves, F. (2018) The Development and Evaluation of a Multi-Agent System for Supporting Flipped Classroom. Creative Education, 9, 1667-1679. doi: 10.4236/ce.2018.911121.

1. Introduction

Distance Education (DE) is a teaching and learning modality that has grown and produced good results. The research of Mansour & Mupinga (2007) demonstrated that practices and concepts of distance education are converging to face-to-face teaching and forming hybrid methodologies and merged with information and communication technologies, in which the simple incorporation of these technologies in the teaching and learning process is not enough.

According to Bennett, Dawson, Bearman, Molloy, & Boud (2017) , it is necessary to think of methods that appropriately incorporate these technologies, from a broader and more critical thinking of the entire teaching and learning process, as well as the agents involved in this process. In view of the above, it is necessary to use, in the educational context, methodologies that allow greater interaction between student and teacher, in order to stimulate essential characteristics, such as self-directed study, construction of autonomy and the simultaneous application between theory and practice. In this way, it is highlighted in this paper the flipped classroom method, which constitutes an active methodology that enables the student’s collaborative participation in the process of knowledge construction.

However, a problem that occurs during the application of this methodology is the difficulty for the facilitator to follow the whole development process of the activities carried out outside the classroom. Thus, this paper presents the development and evaluation process a multi-agent system for supporting flipped classroom according to the needs of users, with the purpose of correcting the problem inherent to the implementation of this learning theory.

This paper is structured as follows: Section 2 describes the main concepts related to the flipped classroom; Section 3 presents the survey of the user requirements; Section 4 presents an explanation of multi-agent systems, Subsection 4.1 describes agent-based approach to support the flipped classroom; Section 5 describes the implementation of the system; Section 6 carries out the evaluation of the developed system; finally, Section 7 presents the final considerations and future work.

2. Flipped Classroom

The Flipped Classroom is a methodology that inverts the logic of classroom organization, that is, it is a teaching modality in which content and instructions are studied before the student attends the classroom, which is now the place to deepen the contents already studied. In this learning theory, the facilitators do not transfer the knowledge to the students and then test them through assessments, they motivate the students to seek through investigation and research certain knowledge.

According to the Flipped Classroom Field Guide (2014) , the basic rules for invert the classroom are: 1) classroom activities involve a significant amount of questioning, problem solving, and other active learning activities, forcing the student to recover, apply and expand the material learned virtually; 2) students receive feedback immediately after performing face-to-face activities; 3) students are encouraged to participate in virtual and face-to-face activities, which are computed in the student’s formal assessment, in other words, they score; 4) both the material to be used virtually and in the face-to-face learning environments are highly structured and well-planned.

When applied, the flipped classroom offers some benefits described by Lo, Lie, & Hew (2018) among them can be highlighted the following:

・ Possibility for students to work at their own rhythm and to develop as much understanding as possible;

・ Encourage social exchanges among students;

・ Prior identification of the points that need to be better assimilated and formulation of doubts that can be clarified in the classroom;

・ Teachers will be able to adapt classroom activities according to the students’ needs.

3. Survey of User Requirements

In order to identify the needs of the users and to elaborate a list of requirements for a flipped classroom computational tool, the following activities were carried out:

1) An initial questionnaire was applied with teachers whose purpose was to analyze their prior knowledge about the use of technologies, the flipped classroom method, and identify their needs to produce a list of requirements;

2) A primary study was conducted based on empirical research on the subject of the flipped classroom, through a systematic review of literature in five databases: IEEE Xplore, ACM Digital Library, Google academic, Capes―Research for subjects, ERIC―Capes journals, seeking to identify the main computational tools used for supporting the application of the method;

3) The existing computational tools were analyzed and the essential requirements for the system were collected in the systematic review;

4) It consisted in gather students from the Computer Science program in the final phase and researchers from the area so that through these collected requirements implemented a computational tool.

According to the requirements collected and to better understand how the application of the method works, a flowchart for the flipped classroom was created, as shown in Figure 1.

According to data collected in the SRL (Systematic review of literature), the relevant characteristics of the main computational tools used to support the application of the Flipped Classroom method were analyzed in Table 1. In this way, the following aspects of the tools were observed: FSC―Function of synchronous communication (AC: audio conference, Chat, VC: video conference); FAC―Function of asynchronous communication (EM: e-mail, DG: discussion group, F: forum); RT―Research tools (C: collaboration, DO: downloads, L: library); AT―Aid tools (A: activities, DS: day schedule, T: tips); SQS―Student questions and suggestions (AQ: allows questions); SA―Student assessment (QLA: qualitative assessment, QTA: quantitative assessment).

It was observed that the analyzed tools do not offer all the support to accomplishment, effectively, of the flipped classroom, being necessary to use

Figure 1. Flipped classroom flowchart. Source: Data produced by authors.

Table 1. Existing tools.

Source: Data produced by authors.

functionalities of two or more support tools. Some of them because they were not developed for the educational purpose, end up dispersing students’ attention to other resources not related to learning.

4. Multi-Agent System

According to Russell & Norving (2002) , agents are autonomous software entities that perceive their environment through sensors and perform actions in the environment by means of actuators, processing information and knowledge. Multi-agent systems can model complex systems, allowing agents to have common or conflicting goals. Wooldridge (2002) adds that agents must act autonomously in the environment in order to achieve the goals for which they were designed.

According Bellifemine, Caire, & Greenwood (2007) , states that intelligent agents are programs that execute a set of operations in instead of a user, using a representation of knowledge that contains the goals of the user. In addition, the author also states that agents are programs that conduct dialogues to negotiate and coordinate transfers of information. In this sense, this research used the multi-agent system―MAS to act in a way that the students and the facilitator do not perceive their actions, carrying out tasks that aim to help the students and the facilitator throughout the proposed cycle for the flipped classroom.

Approach Based on Agents for Supporting Flipped Classroom

In order to support the implementation of the flipped classroom in a virtual learning environment (VLE), it was used Moodle―Modular Object Oriented Distance Learning, a free learning support software, in which a module was created specifically to meet the main requirements for the accomplishment of the flipped classroom. Three agent types were created and added to the module: The Manager Agent (MaAg), the Adviser Agent (AAg) and the Monitor Agent (MoAg). The organization of the Multi-agent System (MAS) is a community-type of expert, since each of the three agents created for this research project is at the same level, each of them specialized in a certain task, as described in the Tables 2-4 below.

5. A System for Supporting Flipped Classroom

From the perspective presented in previous sections, it was developed and integrated to the moodle a module called Invert Class, according Figure 2, developed based on the data collected through the users’ requirements to meet the main characteristics of the flipped classroom, supporting the facilitator and the

Table 2. Textual template of the manager agent.

Source: Data produced by authors.

Table 3. Textual template of the adviser agent.

Source: Data produced by authors.

Table 4. Textual template of the monitor agent.

Source: Data produced by authors.

students during their application. The module operates through two types of access: teacher and student.

In teacher access, as shown in the Figures 3-5, after creating the course and adding the activity, it should describe the complete task and define its duration. Then it add necessary requirements of the task, in other words, what subjects students should be aware of for resolution and divide the task into steps by describing and assigning initial and final deadlines for each of them. In the module references tab it will indicate references and/or attach materials for research. This information will be important for the action of the agents when triggered.

When the teacher finishes filling out these data, students registered in the course should access the module through the student access, inform their availability of schedules and the level of knowledge in each task requirement assigning

Figure 2. Invert class module. Source: Data produced by authors.

Figure 3. Job data. Source: Data produced by authors.

a value from 0 to 10. After all students add this information, the managing agent will send a message triggering the adviser agent that will analyze the reported characteristics and will suggest to the teacher the formation of proportional groups, as shown in the Figure 6 and Figure 7.

It is important to emphasize that when the teacher links or changes the

Figure 4. Job requirements. Source: Data produced by authors.

Figure 5. Task steps. Source: Data produced by authors.

suggested groups, according Figure 8, all the development of the task will be supervised by the monitor agent through the chat and forum checking daily the accomplishment of the steps and the complete task deadlines, besides monitoring weekly the participation of each student sending success or failure messages to his email.

6. Evaluation of the Flipped Classroom Computational Tool

The representative users (Teachers and students) were invited to carry out the evaluation directly in the developed system. In this stage was used the Technology Acceptance Model (TAM), an evaluation framework proposed by Davis, Bagozzi, & Warshaw (1989) to understand the external variables of user acceptance and the current use of technology. This framework seeks to understand the behavior of these users through the knowledge of utility and the facility of use

Figure 6. Time availability. Source: Data produced by authors.

Figure 7. Knowledge level. Source: Data produced by authors.

perceived by them.

In this way, the system was used by two classes of different teaching institutions, being a class composed by 12 students of the Graduation Program in Computer Science, a broad association between the Universidade Federal Rural do Semi-Árido (UFERSA) and the Universidade do Estado do Rio Grande do Norte (UERN), which used for a period of three weeks. The other group was

Figure 8. Groups suggestion. Source: Data produced by authors.

composed by 32 students of the technical course of the high school in Electrotechnology, in the integral form of the Instituto Federal do Rio Grande do Norte (IFRN) that due to the academic calendar used the system for only one week.

After the period of use, a questionnaire was applied in the two classes using the Likert scale, which is a type of psychometric response scale commonly used in questionnaires, seeking to evaluate the system in the constructs based on the TAM model, in other words, in the following dimensions: perceived utility, perceived facility of use and external variables (intention to use, complexity and applied training). The following levels of affirmation were applied to the questions: 1 and 2 disagree with the statement, 3 represent indecision and the values 4 and 5 agree with the statement.

In Graph 1, it is possible to observe that most users of UFERSA considered the tool easy to use, while IFRN users were undecided about its facility of use. This result can be considered as a consequence of the short period of use of the tool in this institution. However, it can be stated that the tool proved to be easy to handle.

Another variable that had a very positive return in the two institutions studied was with regard to perceived utility, in which most users considered the tool useful for applying the flipped classroom methodology giving the necessary support to its users, as shown in Graph 2.

Based on Graph 3, it can be observed that half of the users of the two institutions showed interest in using the tool, with a percentage of 34% of UFERSA being undecided regarding intention to use and 28% of IFRN. Only 16% of UFERSA users and 25% of IFRN users did not show intention to use at other times.

Graph 1. Perceived facility of use. Source: Data produced by authors.

Graph 2. Perceived utility. Source: Data produced by authors.

Graph 3. External variables (intention to use). Source: Data produced by authors.

The complexity related to the flipped classroom methodology was a rather complicated issue to be dealt with the IFRN users, since most of them did not know this methodology and had their first experience in the evaluation period of the system. This difficulty is confirmed by the results obtained in the questions related to this variable, where some users demonstrated a slight difficulty in assimilating the methodology while the others were undecided regarding the complexity.

The UFERSA users have demonstrated knowledge about the methodology and a greater facility in assimilating their goals, because these are graduate students with a greater academic experience, as shown in Graph 4.

In relation to the training applied in the two educational institutions, all the functionalities of the Invert Class system were presented in-person and all the doubts that emerged at the time were clarified. Finally, an email was provided for contact in case of doubts during the use, as well as through the forum of discussions of the system users could interact. With this, the most of users agreed that the training performed was adequate, as shown in Graph 5.

7. Final Considerations and Future Work

The flipped classroom is a teaching methodology successfully applied through the use of various virtual learning environments. This method emphasizes students’ active participation, motivating them to seek knowledge, teamwork, and collaboration during the task resolution.

However, it was observed that the tools collected during the development of this research do not offer all the support to the accomplishment, effectively, of the flipped classroom, being necessary to use functionalities of two or more support tools. Some of them, because they were not developed for the educational purpose, end up dispersing students’ attention to other non-learning resources.

This work presented an approach that uses software agents to manage, recommend group formation, and monitor the entire task development by providing full support to the facilitator. Using the proposed approach, it is possible to achieve a reduction in student dispersion because, upon detecting that the focus has been lost, it notifies the facilitator, who can take appropriate action. The

Graph 4. External variables (methodology complexity). Source: Data produced by authors.

Graph 5. External variables (applied training). Source: Data produced by authors.

architecture also provides support for group creation.

As a contribution of this work, we highlight the development of an agent-based architecture that supports during the application of the flipped classroom. As a future work, we intend to improve the chat approach by allowing the creation of separate chats by groups to discuss the resolution of the task and add the option to change the dates of the general task steps after registration. Finally, we intend to carry out a case study as a way of validating the solution through the analysis of the academic achievement reached with the use of invert class.

Conflicts of Interest

The authors declare no conflicts of interest.


[1] (2014). Flipped Classroom Field Guide. Portal Flipped Classroom Field Guide.
[2] Bellifemine, F. L., Caire, G., & Greenwood, D. (2007). Developing Multiagent Systems with JADE (p. 5). Hoboken: Wiley.
[3] Bennett, S., Dawson, P., Bearman, M., Molloy, E., & Boud, D. (2017). How Technology Shapes Assessment Design: Findings from a Study of University Teachers. British Journal of Educational Technology, 48, 672-682.
[4] Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989). User Acceptance of Computer Technology: A Comparison of Two Theoretical Models. Management Science, 35, 982-1003.
[5] Lo, C. K., Lie, C. W., & Hew, K. F. (2018). Applying “First Principles of Instruction” as a Design Theory of the Flipped Classroom: Findings from a Collective Study of Four Secondary School Subjects. Computers & Education, 118, 150-165.
[6] Mansour, E. B., & Mupinga, D. M. (2007). Students’ Positive and Negative Experiences in Hybrid and Online Classes. College Student Journal, 41, 242-248.
[7] Russell, S., & Norving, P. (2002). Artificial Intelligence: A Modern Approach (p. 1132). Upper Saddle River: Prentice Hall.
[8] Wooldridge, M. (2002). An Introduction to Multiagent Systems. New York: John Wiley and Sons.

Copyright © 2023 by authors and Scientific Research Publishing Inc.

Creative Commons License

This work and the related PDF file are licensed under a Creative Commons Attribution 4.0 International License.