Reliability and Validity of Online Individualized Multimedia Instruction Instrument for Engineering Communication Skills

Abstract

The utilization of Multimedia Instruction (MI) in teaching and learning is growing rapidly. The combination of various media assists educational reform, and is important to the improvement of education outputs. The MIuse has been a challenge to educators especially in Jordan. This study aimed to re-calculate the reliability and validity of online individualized MI instrument in a new Online Individualized Multimedia Instruction (OIMI) framework for engineering communication skills. In this study, this model designates the multimedia instruction as one of the latent variables, to be measured by six observed variables, which are modality, contiguity, personalization, coherence, redundancy, and signaling. Data collected and tested from 166engineering learners. Confirmatory factor analysis using AMOS was conducted to obtain three best-fit measurement models. The results showed evidence of a five-dimension measurement model for MI except forcoherence. This result enlightens the model, which includes explanations of Mayer’s Cognitive Theory of MI and multimedia instructional in Practice.

Share and Cite:

Mashagbh, A. , Din, R. , Nasir, M. , Halim, L. and Al-Batainah, R. (2019) Reliability and Validity of Online Individualized Multimedia Instruction Instrument for Engineering Communication Skills. Creative Education, 10, 3041-3048. doi: 10.4236/ce.2019.1012228.

1. Introduction

Engineering field is accountable for major industrial, technological, and economical progress in human history. The engineer who design and build things plays a major role in keeping the society running smoothly. Well qualified engineers are always needed in a growing society. The format of engineering communications can vary widely, from summaries of calculations, to short technical messages, to oral presentations, to drawings describing data or machinery.

In Jordan, a number of engineering branches and specializations are taught by the public and private universities. Each university tries to offer unique specializations to attract students enrolment. Nevertheless, studying abroad continued even after the establishment of the engineering colleges in Jordan (Aqlan et al., 2010). These issues due to the reality where engineering education traditionally relies more on technical skills and less communication skills (Corrello 2012). Engineering instructors seem to conduct inappropriate teaching techniques in order to develop engineering student communication skills (Nasir et al., 2018; Baharudin et al., 2018). The study sought to examine the reliability and validity of an instrument use to measure Multimedia Instruction (MI) constructs. MI is used to deliver communication skills course for engineers. A research question of whether the measurement scale for MI is construct-valid was formed to guide the study.

2. Past Study

2.1. Multimedia Instruction (MI)

Multimedia learning is the learning that “occurs when people build mental representations from words (such as spoken text or printed text) and pictures (such as illustrations, photos, animation, or video)” (Mayer, 2009). Mayer had investigated a number of instructional design principles, which had suggested ways of creating multimedia presentations intended to promote multimedia learning. Mayer’s Cognitive Theory of MI acknowledged that for successful learning to occur, information must be synthesized into a coherent model of knowledge and integrated into long-term memory all within the working memory store.

Multimedia has remarkable impact on learning (Zainal et al., 2018; Gabarre et al. 2018; Ahmad et al. 2016; Azizul & Din, 2016). It has emerged in various form of recourses and equipment which can be use to aid the instructor and learner effort in ensuring effective learning environment (Ahmad et al., 2016). When meeting all these conditions, there would be an opportunity to increase efficiency and effectiveness of engineering communication skills specifically in Al al-Bayt University in Jordan.

The usage of MI in engineering education has changed the practice of structured engineering. MI in the circumstance of structured engineering enhances the engineering process. It can be used in the design and construction of a building including safety management in construction work, and computer-aided design and construction. The MI through video conferencing, shared-screen computing and remote multimedia links on construction projects could have a significant impact on inter-professional communication in engineering education (Di Gironimo et al., 2013). In short, this approach for learning with high-quality instructional materials can reduce lecture time and learners as well as instructors efforts (Khalid & Quick, 2016).

2.2. Learning Styles

According to Romanelli at el., (2009) “learning styles” are “characteristic cognitive, effective, and psychosocial behaviors that serve as relatively stable indicators of how learners perceive, interact with, and respond to the learning environment”. Mismatches exist between common learning styles of engineering students and traditional teaching styles of engineering professors. In consequence, students become bored and inattentive in class, and do poorly on tests. According to Felder (2002), learning-style model classifies students according to the ways they receive and process information on a number of scales pertaining. A model intended to be particularly applicable to engineering education. Sensing and intuitive learners, visual and verbal learners, active and reflective learners sequential and global learners are describe in engineering student learning styles model (Felder & Silverman, 1988; 2002). Din (2010, 2017) also acknowledged that learning style can be divided into social and sensory learning style.

3. Methodology

This study uses a quantitative research approach via a survey distributed at a University in Jordan. It is about measuring engineering students’ level of MI. It was constructed based on reviewed literature specifically grounded on Mayer’s Cognitive Theory of Multimedia Learning (Mayer 2010; 2009).

3.1. Respondents

The sample was 166 engineering learners from Al-Bayt University in Jordan who enrolled in the first semester of an academic year. They were selected purposively from four major engineering courses: communication skills course; the provisions of the building; skills practice of the profession course; technical skills course. The sample size was still within the acceptable range (Hoyle, 1995).

3.2. Instrument

The MI item was developed specifically based on Mayer’s Cognitive Theory of Multimedia Learning (Mayer 2010; 2009). The instrument consists of thirty-one items; five Items for each indicator (modality, contiguity, personalization, redundancy, signaling), while six items were for coherence measure. The measurements scale is a Likert-type scale, which has 1 to 5 scales; 1 equals “strongly disagree” and 5 equals “strongly agree.” 1 represents the lowest and most negative impression on the scale, 3 represents an adequate impression, and 5 represents the highest and most positive impression. In addition, a response category for “Not Applicable” was added for each Likert item. Table 1 shows the contents of the MI measure after the content validation.

CFA was conducted on the hypothesized six-factor structure model using AMOS model-fitting program. The program adopted maximum likelihood estimation to generate estimates in the full-fledged measurement model. At the beginning, this model indicates the latent variable, MI, to be measured by six

Table 1. MI factors.

observed variables (modality, contiguity, personalization, coherence, redundancy, and signaling). The construct MI was indicated by six measured indicators and was identified. It had more degrees of freedom than the paths to be estimated.

To assess the fit of the measurement model, the analysis relied on a number of descriptive fit indices, which included the 1) relative chi-square (χ2/df), 2) comparative fit index (CFI), 3) Tucker-Lewis coefficient (TLI), and 4) root mean square error approximation (RMSEA). Hair et al. (2006) suggest the use of relative chi-square (chi-square/df) as a fit measure.

4. Results

The estimated six-factor model for MI using the data drawn from 153usable and completed responses. A value of approximately .08 or less for RMSEA shows a reasonable error of estimation. The items from each scale were assumed to load only on the respective latent variable. The results indicate that the parameters ranging from .11 to .97. In the MI case, all of the coefficients were acceptable (>.7) except for the Coherence indicator which was .11 ( Hair et al. 2006; Arbuckle 1997; James et al. 2006). The CFI (.922) exceed the threshold of .90 indicating a good fit ( Hair et al. 2006, Arbuckle 1997, James et al. 2006), while the TLI (.870) fit indicators did not exceed the threshold of .90, indicating a poor fit ( Hair et al. 2006; Arbuckle 1997; James et al. 2006). The root-mean square error of approximation (RMSEA = .160) was >.08, Chi-square (χ2) was 44.083 with degree of freedom (9) and p value = 0 (normally acceptable at p > .05) reflecting a possible fit problem ( Hair et al. 2006; Arbuckle 1997; James et al. 2006).

Since the hypothesized model was found to be contaminated (p value = 0 and TLI (.870) is less than .9), the model was revised. The revised model was achieved after examining the modification indices in order to correlate the measurement errors of the signaling with contiguity, as well as correlate the measurement errors of the redundancy with personalization, and contiguity and modality factors. After a number of iteration to fit the model the results reflect a possible fit problem, yet no possible modifications could be made. As suggested by Aryee and Lee (2005) the researcher decided to delete a factor with the lowest loading where the “coherence” factor was dropped.

To validate the likelihood of the revised five-factor model, another CFA was applied on the same sample. The magnitude of the factor loadings in the revised model was substantially significant with CFI = .997, TLI =. 986 and chi-square = 3.971. The parameters were free from offending estimates, ranging from .72 to. 99. The CFI (.998) and TLI (.983) fit indicators exceeded the threshold of .90, indicating a very good fit ( Hair et al. 2006, Arbuckle 1997). The root-mean square error of approximation (RMSEA = .72) also indicated a good fit (Hair et al. 2006).

In this revised model, the chi-square (χ2) with a value of 1.784, with degree of freedom (1) successfully met the required threshold of <5, indicating a high goodness-of-fit value. The p value of .182 (acceptable at p > .05) hence indicates that the test failed to reject the hypothesized model. The procedures established the model in Figure 1 as the validated confirmatory measurement model. The cronbach alphas for the five sub-constructs after CFA are range from .814 to .879 (Modality = .839, Contiguity = .814, Personalization = .879, Redundancy = .873 and Signaling = .813) while cronbach alpha for the whole section measures = .927. It is worth to mention cronbach alpha for coherence” factor which was dropped was .651. Overall result indicates that the test failed to reject the hypothesized model. Thus, the procedures established that the model in Figure 1 was a validated confirmatory measurement model.

5. Discussion

The study was able to validate the MI model, which is measured by five observed variables: modality, contiguity, personalization, redundancy, and signaling. As

Figure 1. Revised CFA measurement model for MI.

proposed in literature, the study offered evidence that (five out of the six dimensions excluding the coherence indicator) the five-dimension measurement model did generate from data collected from Al al-Bayt university engineering learners. The result did not establish any basis, which can be used to claim that the MI model is incorrect. Thus, MI measurement model can be explained by five factors namely, redundancy, contiguity, personalization, modality, and signaling. This result was consistent with several literatures on MI (Gerjets at el. 2004; Ibrahim & Callaway, 2012a; 2012b; Sorden, 2012; Clark & Mayer, 2011).

This conception represents a major adjustment in the way engineering faculties have usually developed engineering communication skills. The results of the present study are relevant to give insights for theorists, learners, academic staff and knowledge management system designers and developers towards the goal of achieving effective learning and teaching environment for engineering communication skills. In addition, these findings would assist engineering learners with differentiated learning style preferences to learn and practice engineering communication skills knowledge by integrating MI theories into the learning environment via Blackboard Course Management System especially in Jordan higher learning education.

Acknowledgements

This publication and research were funded by the Faculty of Education and Universiti Kebangsaan Malaysia under the project code GG-2019-046 and PP-FPEND-2019, DCP-2017-202/3, GG2018-011 respectively. The authors would also like to express special thanks to all personalized education research team members, students, researchers, graduate and research assistants for their help and contribution in conducting this project.

Conflicts of Interest

The authors declare no conflicts of interest regarding the publication of this paper.

References

[1] Ahmad, M., Badusah, J., Mansor, A. Z., Abdul Karim, A., Khalid, F., Daud, M. Y., Din, R., & Zulkefle, D. F. (2016). The Application of 21st Century ICT Literacy Model among Teacher Trainees. Turkish Online Journal of Educational Technology, 15, 151-161.
[2] Aqlan, F., Al-Araidah, O., & Al-Hawari, T. (2010). Quality Assurance and Accreditation of Engineering Education in Jordan. European Journal of Engineering Education, 35, 311-323.
https://doi.org/10.1080/03043797.2010.483608
[3] Azizul, S. M. J., & Din, R. (2016). Teaching and Learning Geometry Using Geogebra Software via Mooc. Journal of Personalized Learning, 2, 39-50.
[4] Baharudin, H., Nasir, M. K. M., Yusoff, N. M. R. N., & Surat, S. (2018). Assessing Students’ Course Satisfaction with Online Arabic Language Hybrid Course. Advanced Science Letters, 24, 350-352.
https://doi.org/10.1166/asl.2018.12005
[5] Clark, R. C., & Mayer, R. M. (2011). E-Learning and the Science of Instruction (3rd ed.). San Francisco, CA: Pfeiffer.
https://doi.org/10.1002/9781118255971
[6] Di Gironimo, G., Mozzillo, R., & Tarallo, A. (2013). From Virtual Reality to Web-Based Multimedia Maintenance Manuals. International Journal on Interactive Design and Manufacturing, 7, 183-190.
https://doi.org/10.1007/s12008-013-0185-0
[7] Din, R. (2010). Development and Validation of an Integrated Meaningful Hybrid E-Training (I-Met) for Computer Science: Theoretical-Empirical Based Design and Development Approach. Bangi, Malaysia: Faculty of Technology and Information Science, Universiti Kebangsaan Malaysia.
[8] Din, R. (2017). Asas Pendidikan dan Kejurulatihan ICT: Integrasi Teori, Media, Teknologi dan Reka Bentuk Pembelajaran. Rangi: Penerbit UKM.
[9] Felder, R. M. (2002). Designing Tests to Maximize Learning. Professional Issues in Engineering Education and Practice, 128, 1-3.
https://doi.org/10.1061/(ASCE)1052-3928(2002)128:1(1)
[10] Felder, R. M., & Silverman, L. K. (1988). Learning and Teaching Styles in Engineering Education. Engineering Education, 78, 674-681.
[11] Gabarre, S., Gabarre, C., & Din, R. (2018). Personalizing Learning: A Critical Review of Language Learning with Mobile Phones and Social Networking Sites. Journal of Advanced Research in Dynamical and Control Systems, 10, 1782-1786.
[12] Gerjets, P., Scheiter, K., & Catrambone, R. (2004). Designing Instructional Examples to Reduce Intrinsic Cognitive Load: Molar versus Modular Presentation of Solution Procedures. Instructional Science, 32, 33-58.
https://doi.org/10.1023/B:TRUC.0000021809.10236.71
[13] Hair Jr., J. F., Black, W. C., Babin, B. J., Anderson, R. E., & Tatham, R. L. (2006). Multivariate Data Analysis (Vol. 6). Prentice, NJ: Prentice Hall.
[14] Hoyle, R. H. (1995). Structural Equation Modeling: Concepts, Issues, and Applications, an Introduction Focusing on AMOS. Thousand Oaks, CA: Sage Publications.
[15] Ibrahim, M., & Callaway, R. (2012a). Assessing the Correlations among Cognitive Overload, Online Course Design and Student Self-Efficacy. In Society for Information Technology & Teacher Education International Conference.
[16] Ibrahim, M., & Callaway, R. (2012b). Implications of Designing Instructional Video Using Cognitive Theory of Multimedia Learning. Vancouver, Canada: American Educational Research Association (AERA).
[17] Khalid, N. M., & Quick, D. (2016). Teaching Presence Influencing Online Students’ Course Satisfaction at an Institution of Higher Education. International Education Studies, 9, 62.
https://doi.org/10.5539/ies.v9n3p62
[18] Mayer, R. E. (2009). Multimedia Learning (Vol. 2). New York: Cambridge University.
https://doi.org/10.1017/CBO9780511811678
[19] Mayer, R. E. (2010). Applying the Science of Learning. Upper Saddle River, NJ: Pearson.
[20] Nasir, M. K. M., Surat, S., Maat, S. M., Abd Karim, A., & Daud, M. Y. (2018). Confirmatory Factor Analysis on the Sub-Construct of Teaching Presence’s in the Community of Inquiry. Creative Education, 9, 2245-2253.
https://doi.org/10.4236/ce.2018.914165
[21] Romanelli, F., Bird, E., & Ryan, M. (2009). Learning Styles: A Review of Theory, Application, and Best Practices. American Journal of Pharmaceutical Education, 73, 1-5.
https://doi.org/10.5688/aj730109
[22] Sorden, S. (2012). The Cognitive Theory of Multimedia Learning, Handbook of Educational Theories.
[23] Zainal, N. F. A., Din, R., Abd Majid, N. A., Nasrudin, M. F., & Abd Rahman, A. H. (2018). Primary and Secondary School Students Perspective on Kolb-Based STEM Module and Robotic Prototype. International Journal on Advanced Science, Engineering and Information Technology, 8, 1394-1401.
https://doi.org/10.18517/ijaseit.8.4-2.6794

Copyright © 2021 by authors and Scientific Research Publishing Inc.

Creative Commons License

This work and the related PDF file are licensed under a Creative Commons Attribution 4.0 International License.