A Systematic Review of Empirically Based Universal Design for Learning: Implementation and Effectiveness of Universal Design in Education for Students with and without Disabilities at the Postsecondary Level

This systematic review explored methods of UDL implementation for postsecondary students with and without disabilities and the degree to which these methods are effective. The authors examined 17 empirically based studies published across 12 journals focused on the application of UDL principles. The studies were analyzed with regard to 1) participant information, 2) courses and delivery mode, 3) independent and dependent variables, 4) implementation strategies, and 5) effectiveness of implementation. The analysis revealed that 15 of the studies reported effective outcomes, one study resulted in blended effects, and one did not discuss implementation. Two studies used a blended delivery mode for special education courses, and four studies used online delivery modes for a teacher education course and three professional development programs. Other studies used face-to-face instruction for teacher education, general courses, and workshops. The most common independent variables were UDL principle-based course design and implementation, followed by hands-on activities, training of instructors, peer-led team learning, and a collaborative professional development model. The dependent variables included course evaluation, learning outcomes, such as revision of lesson plans and technology use, and level of confidence or acquisition of knowledge about UDL and disabilities. Finally, multiple instructional strategies focusing on the UDL principles were utilized, to include web-based computer-mediated communication, web-based class management systems, interactions with technolHow to cite this paper: Seok, S., DaCosta, B. and Hodges, R. (2018) A Systematic Review of Empirically Based Universal Design for Learning: Implementation and Effectiveness of Universal Design in Education for Students with and without Disabilities at the Postsecondary Level. Open Journal of Social Sciences, 6, 171-189. https://doi.org/10.4236/jss.2018.65014 Received: March 23, 2018 Accepted: May 21, 2018 Published: May 24, 2018 Copyright © 2018 by authors and Scientific Research Publishing Inc. This work is licensed under the Creative Commons Attribution International License (CC BY 4.0). http://creativecommons.org/licenses/by/4.0/


Method
The authors systematically and independently searched three databases: Educational Resources Information Center (ERIC), PsycINFO, and Professional Developmental Center. The searches were limited to peer-reviewed, empirically based studies published in English. The terms universal design for learning, universal design for instruction, and postsecondary education were used as the descriptors. A total of 102 studies were identified that met these search criteria.

Inclusion Criteria
Each of the 102 studies were independently reviewed for inclusion based on the following criteria: the study had to 1) be empirically based; 2) contain the terms universal design and/or UD, universal design for learning and/or UDL, and/or universal design for instruction and/or UDI in the title or abstract; 3) have identified the participants; and 4) either implement UDL practices or conduct a survey or interview on UDL intended to enhance knowledge about or implementation of UDL practices for diverse, postsecondary-level learning populations.
Of the 102 studies examined, the authors independently identified 16 studies, among which 14 were common among the authors. This amounted to an 88% agreement rate on the inclusion criteria use. Some of the 16 studies had been published in the same journals. That is, some of the journals had published two or more of the 16 studies. Upon reviewing these journals, the authors identified two additional studies that met the aforementioned inclusion criteria. The result, therefore, was a total of 18 studies.
The authors together scrutinized the 18 studies in a second review using the same inclusion criteria. This ended in an agreed-upon removal of three studies.
Two additional studies were also reviewed because they had been cited by two or more of the remaining studies and met the inclusion criteria. In sum, the removal of the three studies coupled with the addition of the two studies resulted in a final count of 17 studies for review.

Data Extraction
Information was next extracted from the 17 studies in three passes. In the first pass, the studies were summarized for 1) participant information, 2) courses and delivery mode, 3) independent and dependent variables, 4) implementation outcomes, and 5) elements of UDL in the implementation. In the second pass, implementations were further fleshed out, summarizing 6) research methods, 7) implementation strategies, 8) technology implementation, and 9) class activities S. Seok  for studies in which the course or instruction for the workshop was designed, implemented, and had higher course evaluation and significant improvement; or blended, if the outcome was not better than or the same as that of the control group, but still showed improvement in the learning assessment.

Inter-Coder Agreement
Inter-coder agreement was assessed on the extracted information using a check-  Table 1 depicts the information extracted from the 17 studies, published in 12 journals. That is, 1) participant information, 2) courses and delivery mode, 3) independent and dependent variables, 4) findings, 5) effectiveness of implementation, and 6) elements of UDL in the implementation.

Results
As shown in the table, six studies used mixed methods (see [20]- [25]), seven used quantitative methods (see [26]- [32]), and four used qualitative methods (see [33] [34] [35] [36]). Two studies conducted experimental and control group comparisons (see [26] [31]), while three used pre-and posttests (see [25] [29] [31]). (Table 1 does not call out the implementation strategies in a separate   column; rather, the strategies are nested within the other table columns as part of implementation, delivery modes, or elements of UDL.) -Evaluation on the effectiveness of web-based on-demand curricula (called FAME).

Participants
-Preferred subjects listed in order from most to least preference: UDL, web accessibility, distance education, adaptive technology, computer lab accessibility, and accommodations.
-Preferred training modes listed in order from most to least preference: On demand, web-based, two-or three-hour workshop, one-hour or daylong workshop, handouts, training or resources available anywhere and anytime.
-Instructional method in order from most to least use: Lecture, class discussion, critical thinking, or problem solving.
-Faculty's evaluation of FAME: An average of 94% of faculty agreement on the appropriateness of contents and information on professional development and the needs of SDs.
-92% agreement on the enhanced comfort as a result of FAME implementation.
-Categorized as effective.
-Multiple means of representation: Representing concepts with multimedia; participants' ideas shared using video clips that were captioned and included transcripts used for strategic engagement.
-Multiple means of expression: Applied case scenarios with feedback, pre-and post-assessment and practice.
- -Overall benefits from the course for students.
-Reduction in design accommodation for SDs manually performed by university office of disabilities.
-97% of participants reported the following to be helpful: Access to text descriptions of images in PowerPoint, detailed topic outline, and lists of key concepts in study guideline.
-Course components that more than 90% of participants perceived to be very impactful on their learning: Choice of completing elective activities or taking a final exam, individual or group assignment types and paper due dates; posting instructional materials before class; consistent format of instructional material organization on WebCT pages.
-High flexibility of resources.
-Critical factors of social presence: Instructor's availability outside of class session and his/her immediacy.
-Shared information using discussion board.
-Stress reduction: Course design and organization, study guidelines, flexibility of deadlines.
-Students' success: The attributes of UDL design.
-Categorized as effective. -Changes in revised lesson plans were quantified and categorized.
-Revisions should include students' special needs in their learning and UDL instructional strategies using the three UDL principles.
-Participants' reflections on their assignment were coded using a code book.
-Significant revisions using UDL approaches made by both pre-and inservice teachers; preservice teachers made more revisions using UDL principles than practicing teachers.
-Preservice teachers made more revisions in multiple means of expression than in systematic engagement.
-Analysis of participants' reflection: Perception of UDL was a transformative practice that served as an effective model for inclusion for all students.
-Categorized as effective.
[24] -114 students -F2F special education class with eLearning elements lasting for 1 hour 40 minutes per week -Blended Independent variables: Implementing a web-enabled F2F class focusing on online team collaboration (online teams of nine each) to complete eight assignments during the semester; a different case of a student with special academic needs (e.g., LD, ADHD, and gifted) was assigned to each team.
The design of the course using adult learning tenets and UDI principles: -Categorized as effective.
-Equitable use: Enhanced and easy access by all students with multiple learning abilities (e.g., lecture notes, links to Internet resources, peer discussion thread).
-Tolerance of error: CMS used; students studied at their own pace.
-Minimizing physical efforts: Online and in-person availability of the course on an "anytime, anyplace basis." -Perceptible information: The design of CMS to easily access the instructional materials (e.g., all germane information located in a place).
-Community of learners: Course made use of team interactions and communications between students and between students and professors.
-Simple and intuitive: Design of the blended course resulted in effective interactions. -Acquisition of new terms and concepts.
-Faculty's confidence and willingness to implement UDI principles in their instruction: More than 86% of participants rating themselves as confident about disability laws, legal definition of disability, UDI, faculty responsibilities, making adequate accommodations, creating accessible documents, types of campus services available, and finding additional support.
-Categorized as effective.
-Text-, audio-, and video-based materials of the online program that allow faculty to access the program anywhere and anytime.
-Beside the implementation, to implement the effective workshop, Technology Needs Survey was conducted prior to the workshop to plan it. -Effectiveness of UDL environment on the last day of class.
-Perceived effectiveness, use of various instructional tools, and perceived learning.
-Actual learning: An average of student non-cumulative scores from three exams over the course. -Categorized as effective.
- The Inclusive Teaching Strategies Inventory used.
-Measurement dimensions: Accommodations, accessible course materials, course modification, inclusive lecture strategies, inclusive classroom, inclusive assessment, and disability laws and concepts.
-Attitude: The components that U.S. faculty believed significant were the provision of accessible course materials, inclusive classroom, inclusive lecture strategies, inclusive assessment, and course mediation; the components that Canadian faculty believed significant were the provision of accommodations and knowing laws and concepts of disability. -The ratings of dimension of inclusive classrooms were similar across the three countries.
-Categorized as effective.
-Pre-and post-surveys contained the same questions.
-The instructors reported significant improvement in UDL strategy implementation.
-Areas with significant enhancement: Information presentation in multiple formats -Categorized as effective.
-Multiple means of representation: Instructors presented information using lecture, text, graphics, audio, and video.
-Multiple means of expression: Students expressed their knowledge using traditional tests, written essays, projects, and portfolios.
-Multiple means of engagement: Investigation of students' motivation and challenges and instructors' enthusiasm for the subject matter. Strategies used, instructional consistency with the principles of UDL, and students' perceptions of the degree and ways of student engagement were surveyed.
-The instructors' self-reported use of strategies is listed in the column of Elements of UDL Curriculum.
-The strategies students most frequently used for UDL recognition learning network: [31] -72 graduate and undergraduate students -Two special education and two general education classes -F2F Intervention: One-hour in-class lecture to the experimental group on how to change lesson plans for students with mild and severe disabilities incorporating the ULD principles (experimental group design).

Dependent variables (Data collection: Quantitative method):
-Professor-developed scoring rubric with zero-to-two points used.
-Pre/posttest scores on students' modified lesson plans.
-Information needed about UDL approaches for the teachers to design lesson plans for diverse learners. -Categorized as effective. -Persistence in STEM course.
-Growth in using learning strategies measured by LASSI pre-and post-group tests: Significant improvement shown with 0.05% level in the group means of skill cluster (pretest; posttest = 42 nd ; 66 th percentile with a probability value of 0.005), will cluster (pretest; posttest = 42 nd ; 63 rd percentile with a probability value of 0.005) and self-regulation cluster (pretest; posttest = 29 th ; 56 th percentile with a probability 0.001).
For three clusters, single-tailed test was applied.
-Lower academic performance of PLTL SDs than non-PLTL SDs in the STEM courses: PLTL SDs might be the most at-risk subgroup on campus.
-In peer mentors' training: Instructions on UDI principles and matching them with learning characteristics of students with LD and/or ADHD.
-Multiple means of representation (from peer mentors): Written templates, such as paper-based charts, lists, or diagrams, video templates, and color codes from red to blue on the whiteboard.
-Multiple means of engagement (in PLTL): Building the learning community with peers and mentors. Open Journal of Social Sciences Common phenomenon between SDs and SwoDs called a phenomenological approach.
Dependent variables (Data collection: Structured interview using an interview protocol): -SDs' perceptions of instructional methodologies and strategies to enhance their learning.
-Perceptions of instructional methodologies and strategies found to enhance learning: SDs addressed issue more than SwoDs; organization of physical environment (11 SDs vs. 2 SwoDs), equity (9 SDs vs. 1 SwoD), the degree of professors' familiarity with disabilities (11 SDs), frustration with accommodations and policies (9 SDs), stigma associated with disabilities (7 SDs), and stress from dependency on others, need for extra time for study, and challenges with transportation.
-Accommodation issues: Satisfaction from nine SDs; Students' concerns: Inadequate time given for tests; faculty members' lack of understanding SDs; lack of provision of built-in accommodations (e.g., human reader vs. built-in computer screen reader); functional mismatch between SDs' needs and technology (e.g., voice recognition software for the students with communication impairment [CI]).
-Learning preference: Varied between learners, but the flexible combination of learning preferences leading to the best learning outcomes.
-Matching SDs' perception of UDL principles: Effective strategies based on the SDs' perception: Class and small-group discussion, clarity of class expectations, dissemination of class outlines and instructional materials before class, frequent feedback before the final grades of the projects, writing center, general feedback, screen readers, use of computer, transition from class to class, tutoring, and counseling services.
-Equitable use: Access to the same instructional materials.
-Flexibility of use: Class discussion or small-group discussion.
-Simple and intuitive: Clarity of class expectations.
-Perceptible information: Dissemination of class outlines ahead of time.
-Tolerance for error: Frequent feedback before final grades of the projects.
-Low physical effort: Use of screen reader.
-Size and space for approach and use: Transitions between classes.
-Community of learners: Tutors, counseling services, and class and small-group discussion.
-Multiple means of representation: Equitable use, flexibility of use, perceptible information, tolerance for error.
-Multiple means of expression: Flexibility of use, perceptible information, tolerance for error, low physical effort, instructional climate.
-Multiple means of engagement: Flexibility of use, simple intuitive (clarification of class expectations), tolerance for error, low physical effort, size and space of approach and use, a community of learners, and instructional climate.
[34] -16 disability service providers at postsecondary institutions; two focus groups of eight -F2F Independent variables: -Keynote presentation addressing UDI principles, construct, and application for instruction for student with LD (SLD), and inclusion using UDI.
-Focus group discussion.
Dependent variables (Data collection: Qualitative method): -Responses to open-ended questionnaires of the protocol developed by UDI project team with foci on experiences with or teaching SLD/ADHD and analysis of discussion.
-Perceptions of the strengths and weaknesses of UDI as a strategy for faculty to increase inclusion in their teaching.
-Perceptions of the participants' responsibility for enhancing UDI on campus.
-Perceptions on office for students with disabilities (OSD) supports to incorporate UDI as strategies of faculty development.
-Focus group discussion audiotaped, transcribed, and analyzed as data: Code book and multiple processes of interrater reliability were used.
-Benefits of UDI implementation: Increased instances of enrollment and graduation of students with different cultural backgrounds, adequate instructional approaches to all students, support for data-based teaching practices, and decrease in stigma related to disabilities.
-Weakness of UDI implementation: Faculty resistance, training issues, technology requirements, lack of students' self-advocacy, lack of instructional knowledge of service providers, and legal void of UDI implementation.
-Perceptions of participants' responsibility: Widely spreading knowledge about UDI to campus leadership, collecting data and supporting faculty who incorporate UDI in their teaching, and facilitating faculty members' UDI implementation, different appreciation from a campus based on culture and disciplines.
-Perceptions of OSD supports: Supports needed from campus leaders to incorporate UDI; provision of knowledge about UDI, including information about instructional technology and research on the efficacy of UDI.
-Findings from the interviews: Data collection as a facilitator of UDI implementation; service providers' expertise -learning strategies vs. faculty members' expertise -the content areas.
-Categorized as effective. -Interview using grounded theory.
-Preservice teachers' understanding and implementation of UDL in their teaching.
-The effectiveness of UDL strategies in enhancing their teaching in terms of class structure, learning, engagement and accessibility for every student in class.
-Preservice teachers' confidence in their knowledge about UDL and its application.
-Categorized as effective.
- [36] -46: 6 faculty and 11 teachers taking a UDL online course; 17 faculty and teachers attending the first two-day F2F summer institute; 12 attending F2F event -Online or F2F summer course Independent variables: -Instruction using the collaborative professional development model and UDL strategies (e.g., computer-aided instruction, productivity software, and instruction management software).
-Learning topics taught: UDL principles and networks, significance of electronic texts, learning barriers associated with students with special needs, and hands-on technology implementation (e.g., Thinking Reader, voice recognition, IntelliMathics, computer voice, other hands-on technology tools).
-Building learning team in which university faculty and teachers grouped as a team while taking online Blackboard class.
Dependent variables (Data collection: Qualitative method): Written reflections (feedback) from teachers.
-Participants' inefficient use of multiple technology devices for instruction due to a lack of skills in technology.
-Collaboration essential to produce progress in teaching practices.
-UDL approaches valued: Teachers' appreciation of the assistive and adaptive technology to benefit their students with different learning backgrounds; adequate amount of practice needed to generalize their acquired technology knowledge in different contexts with continuous supports.
Categorized as effective.
was offered regarding the number of participants in each of these disability categories.
Implementation for faculty members took the form of professional development programs, with each study using a different type of program. The independent variables included keynote presentation on UDL and focus group discussion (see [34]); implementation of five instructional modules, including accommodation, UDL, web accessibility, college writing, and climate assessment (see [21]); intensive workshops (see [28]); an online disability awareness program (see [25]); and instruction using the collaborative professional development model and UDL strategies (see [36]).

Dependent Variables
Multiple dependent variables were also measured. Six studies evaluated UDL-implemented courses using course evaluations (see [20] [22] [24] [27] [29] [30]), which included measures of effectiveness, flexibility, learning outcomes, course accessibility, and accommodations. Other measures focused on the degree of the instructor's social presence (see [22]). These included interactions with the instructor in and outside the class with immediacy; the number of class interactions using technology and discussion (see [20] [24] [27]); the relationship between use of technology and grades or learning (e.g., the relationships between the frequency of using tools and academic scores, see [27]) and products. The products were made by the students as the effect of the UDL implementations. These products included the revision of lesson plans (see [23]), development of the modified lesson plans (see [31]), GPA (see [32]), and preservice teachers' UDL implementation in their own classes (see [35]). Studies also collected and analyzed preservice teachers' reflections on their perceived level of knowledge acquisition on UDL implementation from the courses as a transfor- Finally, preferred training subjects and program delivery modes were also explored (see [21]).

Implementation Strategies
Regarding strategies, nine studies delineated or listed UDL approaches matching UD or UDL principles (see [20] [35]). Furthermore, group or individual assignments and hands-on activities were employed to enhance the principles of multiple expression and engagement (see [22] [23] [35]). Course modifications in some studies were made based on student feedback (see [29] [30]), while for professional development programs, "anywhere and anytime" web-based instruction, web-based workshops, and web-based information were generally utilized (see [21] [25] [28]).

Effectiveness of Implementation
Implementation of the interventions was generally reported to be effective. The outcomes of captioned conditioning (see [26]) and revision of the lesson plans (see [31]) were reported effective with statistically significant differences for effect sizes from the control/experimental group comparison. The outcomes from the studies using pre-and posttests (see [25] [29] [31]) were also reported effective with statistical significance. Improvement was reported in the following areas: application of learning strategies of students with LD and ADHD across two semesters (see [32]); participation in discussion, interaction with technology, and reflection skills (see [20] [40]. The studies also offered different perspectives and learning outcomes for both professionals and students at the postsecondary level (faculty, teachers, service providers, or administrative staff and students). In this context, effectiveness of UDL implementations was found among both groups. The studies that used group comparison between experimental and control groups found statistically significant differences between them, consistent with the larger body of research [39] [41].

Discussion
Furthermore, the studies using pre-and posttests found significant improvements between training with and without implementation, likewise aligned with the group comparison studies (see [25] [29] [31]). Finally, the results from the comparison studies addressing the importance of teacher education were consistent with those of existing studies [39] and across the studies included in this review (see [20] [23] [24] [31] [35]), further indicating the overall value of UDL at the postsecondary level.
Of particular importance was the finding that participants with LD and/or ADHD did not perform better in their STEM courses than their peers with other disabilities who did not participate in PLTL. As a result, further research should address these findings with a particular focus on examining different disabilities, at-risk groups on campus, and other identified variables [42].
Also, of importance, three studies discussed students' use of products to evaluate the effectiveness of the UDL implementation. The results suggested that the best approach was comprised of professors' instruction with examples, students' practice and products using hands-on activities, and feedback and assessments (see [23] [27] [31]). These findings are aligned with the other studies in this review, showing student preference for in-house teacher-made instructional materials over commercial materials [27] and the effectiveness of hands-on activities [23] [31] [33]. Overall, processes of instruction were aligned with one of the evidence-based practices in other disciplines [43] [44]. That is, behavior skills training comprised of instruction, rehearsal, and feedback. Behavior skills training is generally used for teachers and students in the training of individuals with developmental disabilities.
It is also important to emphasize that less than 5% of the participants sampled in the 17 studies were reported to have disabilities. This might explain the discrepancy in the number of enrollments and class attendance rates for these students (see [21] [29]), which was aligned with existing reports [2] [3] [45]. Although SDs were not representative, by and large, information from the participants suggests that UDL application was effective for all students sampled, whether they had disabilities or not. These are findings consistent with existing research [46].
The review also revealed that teacher education courses were described in some of the studies as involving students with and without special needs (see [20] [24] [31]), in practicum (see [35]), and in teacher education courses (see [23]  about UDL and related strategies should be taught to pre-and inservice teachers to improve instruction and learning outcomes as part of their teaching practices-a conclusion that is supported by and aligned with findings reported in the literature [47] [48].
Another finding is the importance of considering students' needs when providing instruction and accommodations. In Smith [30], students listened to in-class lectures more than using other accommodations to learn and study. At the same time, they least utilized a speech-to-text device to learn and study. This finding is aligned with other investigations, such as that of Black et al. [33], in which SDs reported inappropriate accommodations and mismatches between their needs and accommodations or modifications provided. While this may be viewed as a finding emerging from this investigation at the postsecondary level, such a discovery has been addressed in secondary education [49], revealing an important implication for the provision of accommodations and modifications.
The instructional strategies were found to match UD or UDL principles across the included studies. The strategies were nested in eLearning (e.g., web-based asynchronous vs. synchronous discussion; distance learning for two different classrooms at two different institutions; anytime-, any place-based), adult learning (e.g., learning autonomy, self-discipline), and traditional learning (e.g., in-class lecture). One of the strengths of web-based instruction is its flexibility in terms of time and space to promote learning as well as its inherent ability to handle built-in accommodations and modifications [50] [51]. Most of the courses described in the 17 studies utilized web-based management systems, although some also included F2F instruction. It is, therefore, reasonable to conclude that web-based instruction or eLearning could be applied as an effective delivery mode of UDL implementation to enhance built-in accessibility. This may be viewed as a new finding resulting from this investigation. As stated, delivery modes were viewed in this review as one of the built-in mechanisms of UDL practice but have been mostly excluded as such in the existing research.
Finally, the most frequent independent variables were design and implementation of UDL-based courses. Other independent variables included strategies to facilitate UDL implementation, such as revisions and modifications of lesson plans; training instructors to enhance learning outcomes and interaction; use of technology; and learning community. All in all, the findings reinforce the value of UDL-based blended instruction, focusing on the three UDL principles of multiple means of representation, expression, and engagement.

Conclusion
This systematic review examined 17 empirically based studies focused on UDL implementation, presenting a discussion on the value of UDL supported by existing research. Altogether, the analysis revealed that these studies, conducted on