ABSTRACT

In this article, we present a formative evaluation of an ongoing NSF-sponsored research project in classroom innovation using augmented reality (AR) to enhance STEM education. Exposing students to advances in digital modeling, data visualization and performative software prepares them for new pathways for decision-making in the AEC professions. Recent research shows that Technology Mediated Learning Environments (interacting with computer-based tools) can enhance learning. Augmented Reality (AR) or the ability to augment the real-world environment with computer-generated information is bringing a new dimension to learning and designing using multiple data streams.

Keywords:

Cyberlearning, STEM Education, Virtual and Augmented Reality, Sustainability

1. Introduction

Introducing new tools into a familiar environment is always a challenge. Introducing new teaching tools involves working with two distinct groups, instructors and students. Students lack familiarity with any previous models of the teaching environment as the course and the course materials are assumed new. General learning theory assumes the critical issue for learners is that they actively seek to integrate new knowledge with knowledge already in their cognitive structures (Novak, 2015) . However, introducing the additional complexity of cyberlearning tools including augmented, mixed and virtual reality or any digital device creating an immersive virtual experience is challenging for both students and teachers. Recent research indicates that Technology Mediated Learning Environments (interacting with computer-based tools) can enhance learning (FN). Augmented reality (AR) or the ability to augment the real-world environment with computer-generated information experienced by multiple users in real-time is bringing a new dimension to learning (Kamarainen et al., 2013) . Integrating AR with other simulation technologies has the promise of leading the next generation of computer-based learning environments.

Our initial research questions were: 1) how would engaging with digital tools impact students problem solving skills and collaborative learning interactions? 2) how would interaction with the project change students’ understanding of building science principles in their respective disciplines? 3) would using AR enhance the ability of students to successfully transfer the principles they have learned to new situations? and 4) what impact could the project approach have on student motivation of further collaborative learning, remain relevant and continue to inform our next steps?

Over the past two years, the University of Arkansas collaborated with Florida International University and Missouri State University on an NSF―Improving Undergraduate STE < Education (NSF-IUSE) grant “Collaborative Research: Strategies for Learning, Augmented Reality and Collaborative Problem-Solving for Building Sciences” (NSF #1504898). This project developed a new teaching and learning environment using integrated Building Information Modeling (BIM) with augmented reality (AR) in order to provide three-dimensional, interactive, annotated models of buildings for visual learning. In order to quantify and qualify the influence of AR on students learning, interdisciplinary courses were leveraged between architecture, mechanical and civil engineering. The faculty collaborated to teach a joint course offered as an elective in all three disciplines on advanced topics in sustainability. The teaching staff believed this was an area with adequate duplication in all three disciplines for meaningful content to be created for a shared course.

The collaboration not only provided a platform for investigating AR in the classroom, but also exposed students to known professional domain affiliations in industry between architecture and engineering. In general, the strength of the architecture disciplines is design and performance visualization while engineering focuses on design and performance analysis. Often, professional students do not encounter real-world professional situation in the Architecture, Engineering and Construction (AES) disciplines until they are in the professional world. Introducing the opportunity for collaboration in a shared setting to them in academia better prepares them to be leaders in their respective fields and advances student success by promoting innovation in teaching and learning in multi-disciplinary classroom environment (Messadi & Newman, 2017) .

2. Methodology

The multidisciplinary collaboration at the University of Arkansas (UA) was pursued through three coordinated courses and group projects. The other participating schools, Florida International University (FIU) and Missouri State University (MSU) followed a similar structure. During the spring 2016 semester, the UA control group classes participated in the project. These included undergraduate level courses with students from 1) architecture (ARCH 303v Advanced Topics in Sustainability taught by Dr. Tahar Messadi, 2) Civil Engineering (CVEG 4863 Sustainability in Civil Engineering, taught by Dr. Andrew Braham) and 3) Mechanical Engineering (MEEG 4473 Indoor Environmental Design, taught by Dr. Darin Nutter). The UA team included Dr. Newman (PI), Drs. Nutter, Messadi and Braham and Dr. Fredrick, Director of Tesseract. The content of courses, testing and implementation process in the control group at UA, followed a similar path to FIU’s Control Group. These courses ran autonomously but included interdisciplinary lectures by the participating faculty, combined with in-class presentation of student research. In addition, guest speakers provided additional insights into the topics examined in class. The courses comprised 45 students working in teams of 5 - 6 students with each discipline represented on the team.

In Fall 2016 semester the three groups of students tested traditional learning tools to understand sustainable concepts in building envelopes, heating, and ventilation systems, and structural elements of an addition to Vol Walker Hall on the UA campus In Fall 2017 three instructors, one from each respective department continued to the grant study by co-teaching a combined course introducing augmented or mixed realty into the students learning process. This was our experimental group.

The UA team used the HoloLens^TM a head-mounted display (HMD) with a holographic computer that creates a blended environment where the user can view reality while also “seeing” overlaid holographic data. This allows the user to interact with digital content as part of the real world. Figure 1 is an example of the way we employed AR (technically referred to as “mixed reality”)¹ using the HoloLens to identify heating and ventilation (bright yellow), fire suppression (red), and support elements (dark yellow) in Vol Walker Hall. By overlaying these images on reality, students were able to “see” the different components of the building while standing in the building space. Based on our experimental data our formative evaluation informed refinements in the approach to the development of content for the HoloLens. Students indicated they became overwhelmed trying to work through too large an area in a building while managing quantitative analysis for three systems; we therefore focused on a single classroom area, a large studio, on the three flow of the building.

¹Mixed reality is a general term used to indicate the use of a (typically) digital device creating a virtual or simulated representation of either 2-d or 3-d information. Augmented reality indicates the user is aware of both a real-world condition and a virtual one at the same time. For example, a museum visitor using headphones while viewing an exhibit is augmenting their experience. Hence, it is an augmented reality.

For the content development of the digital information in the HoloLens the team collaborated with the UA Tesseract Lab using previously generated BIM information from Vol Walker Hall. We anticipate continuing this affiliation moving forward. The summative evaluation of the IUSE-project uses pre- and

post-surveys, video of collaborative student interaction and testing to compare control and experimental groups from the 2016 to 2017 semester to measure the influence of AR on learning.

The FIU team using a similar protocol managed the cyberlearning components employing a different approach to augments reality, AR-Skope and VR-Skope. The device used was a mobile tablet with software design to give students just-in-time data and knowledge for building systems related to a specific campus building. Students were expected to go to the building and use the handheld device in situ while control and experimental group for the FIU Team are given in this paper. A brief overview of their AR approach follows here, and further steps are discussed in the conclusion.

The FIU team included Shahin Vassigh (FIU PI), Ali Mostafavi, Deborah Davis and Albert Elias. Three courses were identified in architecture, construction management, and mechanical engineering. Rather than teach a unified course with students registered for the same class, students registered for unique courses, but worked together in three projects. The semester work was divided into three five-week units organized around the 3 units of: 1) Building Siting and Foundation, 2) Building Envelop and Mechanical Systems, and 3) Construction and Post-Occupancy Evaluation. In cases that the content was not a part of the traditional course, it was added with an additional guest lecture, video recordings, or posted materials online. The course structure for the experimental Group courses was almost identical to the Control Group course work, with the expectation that the Experimental group of students had access to the project instruments. The Fall 2016 semester did not include AR and was designated the control group. The spring 2017 semester included the AR-Skope software used on a mobile tablet device and was designated the experimental group.

3. Results

As with our prior work, a formative evaluation and assessment continued centered around observing and understanding the interaction, collaboration and group approaches used when completing the Technical Reports, with a particular focus on improvements across our two experimental groups from fall 2016 and spring 2017. A key component of our formative evaluation was to provide just-in-time, constructive and informative feedback to the groups regarding their collaborative efforts, comments on their questions with respect to the Technical Report, as well as timely process-based evaluations. An important aspect involved communicating and enabling subsequent process changes to help improve group interactions and collaborative learning. The following preliminary results are based on responses from the control and experimental groups given on a pre- and post- Attitude Survey Questionnaire, Table 1.

Pre-/Post-Attitude Survey on Collaborative Learning

A mixed model repeated measures ANOVA was conducted to compare the effect of groups (Control Group, Experimental Group 1, and Experimental Group 2) and majors (Architecture, Construction Management and Mechanical Engineering) (IVs) on tests (Pre-/Post-test course learning and content knowledge) (DV). A main effect of Groups was found, F (2256) = 7.24 p < 0.01, η²_partial = 0.54 as well as a significant interaction between Group and Test, F (2256) = 10.33 p < 0.01, η²_partial = 0.075. Follow up ANOVAs found a significant difference between Groups for the Posttest F (2262) = 3.35 p = 0.04, η²_partial = 0.025, but not the Pretest. Post hoc tests using the Bonferroni correction revealed a significant difference between the Control Group 1 (M = 4.29, SD = 0.23) and Experimental Group 2 (M = 5.15, SD = 0.18). Posttests (p = 0.03) (see Figure 2). There was no significant difference between the posttest of Experimental Group 2 and the other two groups.

Pretest scores for all three groups were statistically the same, students in Experimental Group 2 who employed the use of AR performed significantly better on the Posttests than students in the Control Group, who did not use any digital tools. This indicated that use of our digital tools, AR positively impacts and increases on student learning and content knowledge of building science principles. This is important for two reasons. First, it provides evidence interactive digital VR technologies can improve learning across Management and Mechanical Engineering. Second, the successful implementation of our AR application demonstrated these technologies can further provide additional content otherwise might be more difficult and time consuming to provide this type of tool.

Students in all three groups were also administered Pre- and Post-Attitude Surveys on their views of collaborative learning. Although engagement in collaborative learning was a focus of the formative assessment process, we were also interested in investigating whether use of the technologies in combination with formative collaborative learning approaches would impact student attitudes towards and motivations for engaging in further collaborative learning. This is of particular interest in this project as it involved students from three different domains where, some would argue, collaboration would be more difficult as a result of the need to collaboration across these divergent domains.

For all three groups, Pre-Attitude surveys were administered at the beginning of the semester, prior to the commencement of any group work. Post-Attitude surveys were administered at the end of the semester, once all group projects were completed. The survey consisted of 13 questions using a 5-point Likert scale, where responses ranged from Strongly Disagree to Strongly Agree. An overall score was calculated for each question to an integer ranging from 1-5 and adding together the individual scores from each question.²

Overall, results indicate students in all three groups indicated a decrease in their attitude towards collaborative learning form the Pre-Attitude (M = 48.57, SD = 0.66) to Post-Attitude (M = 40.24, SD = 1.08) surveys. As can be seen in Table 1, this pattern is consistent across each of the questions of the survey. This finding is not surprising given the cross-domain nature of our study, but also does not tell the whole story.

²A mixed model repeated measures ANOVA conducted compared the effect of Group (Control Group, Experimental Group 1 and Experimental Group 2) and Major (Architecture, Construction Management and Mechanical Engineering) (IVs) on Test (Pre-/Post-Collaborative Learning Attitude Score) (DV). Main effects of Text, F (1255) = 61.24 p < 0.01, η²_partial = 0.194, Group, F (2255) = 5.43 p = 0.005, η²_partial = 0.041, and Major, F (2255) = 5.79 p = 0.003, η²_partial = −0.023, and Test, major and Semester, F (4255) = 6.43 p < 0.001, η²_partial = 0.092, along with Semester and Major, F (4255) = 2.53 p = 0.04, η²_partial = 0.038.

To better understand the formative influences on participants’ attitudes, the significant interactions discussed above found should be explored more closely. Follow up ANOVAs found significant differences for the Posttest between Groups, F (2255) = 5.71 p = 00.4, η²_partial = 0.043, and Major, F (2255) = 6.05 p = 0.001, η²_partial = 0.045, and a significant interaction between Group and Major, F (4255) = 5.02 p = 0.001, η²_partial = 0.073. There was no significant difference found for the Pretest. Post hoc tests using the Least Significant Difference revealed difference between the Control Group (M = 34.77, SD = 2.21) and both Experimental Group 1 (M = 43.91, SD = 1.66 Posttests (p = 0.01) and Experimental Group 2 (M = 42.03, SD = 1.70), Posttests (p = 0.011). A significant difference for Posttests was found between the Mechanical Engineering Majors (M = 44.95, SD = 1.60) and both Architecture Majors (M = 36.10, SD = 2.10) (p = 0.001) and Construction Management Majors (M = 39.66, SD = 1.89) (p = 0.03) (see Figure 3). No significant difference was found between Architecture and Construction Management Majors. As can be seen in Figure 3, Architecture and Construction Management Majors in our Control Group had the steepest decreases in collaborative learning, our formative assessments and improvements in subsequent semester does appear to have had a mitigating impact on attitudes towards collaborative learning. Thus, although in Experimental Group 1 and Experimental Group 2 a decrease in Pre- to Post-Collaborative Learning Attitudes is present, the decrease is not as marked as with the Control Group.

4. Conclusion

Based on our statistical analysis of preliminary results, we are hopeful, but cautious about the value of cyberlearning in the STEM classroom. Results are not

consistent and may reflect cultural differences across the majors participating in the study of other factors including the impact of the initial learning process with the technology, the ease of use of the AR technology offered in the FIU experimental group, or the overall organization of the course requiring students meeting additional time outside class to complete the collaborative assignments. There are various factors potentially impacting the attitudes of students in the project. The UA faculty completed the Experimental Group course in Fall, 2017 and will be able to compare their results to the FIU study. We anticipate some variation in outcomes due to the difference in technology.

Designing learning models is a dynamic interaction between instructors, subject matter, and students. Adding cyberlearning tools in our case of a digital tablet giving students just-in-time access to critical data or knowledge, complicates the evaluation of results twofold. However, based on a faculty de-brief and indicators of productivity for the university such as publications and reports, there are several collaboration benefits worth mentioning. First, faculty developed curriculum across departmental boundaries. This was significant for developing new courses, especially elective courses as second; giving student in the AEC disciplines the opportunity to work together during their university experience prior to their professional life is an important strategy for education professionals able to navigate complex real-world problems. Third, differences in discipline approaches to similar problems helped faculty in the respective disciplines identify areas of improvement for courses specific to their disciplinary curriculum. For example, engineering students had more difficulty reading plans, elevation and sections where architecture students were not familiar with useful software for building performance analysis. And finally, faculty productivity was affected as the study supported numerous publications, proceedings, and grant applications disseminating findings over a wider array of journals and publications. Faculty published in journals typically outside of their sphere of influence as results pertained to students and classes in multiple disciplines.

It is our intention to continue to pursue this line of questioning regarding the value of cyberlearning in the STEM classroom. Our initial research questions about collaborative learning and technology integration in the classroom remain relevant continuing to inform our next steps.

Acknowledgements

The work described in this paper was partially supported by the National Science Foundation under award No. 1504898. The Hololens-AR development work is underway at the Tesseract Center at the University of Arkansas under the direction of Dr. David Fredrick with Keenan Cole, Chloe Costello and undergraduate student Corey Booth. Documentation of the Vol Walker construction was done by the Center of Advanced Spatial Technologies (CAST) at the University of Arkansas.

Conflicts of Interest

The authors declare no conflicts of interest regarding the publication of this paper.

Cite this paper

Messadi, T., Newman, W. E., Braham, A., & Nutter, D. (2018). Cyber-Innovation in the STEM Classroom, a Mixed Reality Approach. Creative Education, 9, 2385-2393. https://doi.org/10.4236/ce.2018.915179

References