Assessing Clinical and Academic Performance in a Master ’ s Level Speech Language Pathology Program : A Path Analysis

This study assessed a path model to determine the direct and indirect effects on clinical and academic success from students’ undergraduate and graduate performance. Astin’s I-E-O model served as the theoretical foundation for the hypothesized model. Input (I) data included students’ undergraduate information (GPA, major, and GRE scores); Environment (E) data included grades in required graduate courses and formative clinical evaluation; and Output (O) data included PRAXIS score and summative clinical evaluation. The sample was 122 students who completed a SLP Program at a New England graduate school. Results of the path analysis supported the efficacy of the proposed model in determining the direct and indirect effects on professional competence. Graduate students with an undergraduate degree in SLP were at a disadvantage upon entry to the program relative to students from other undergraduate majors. Implications of the study are discussed.


Introduction
The field of Speech and Language Pathology (SLP) has become increasingly diverse in recent years.The current range of settings, clinical populations and specialized areas of diagnosis and treatment are widely varied.In 2011, the American Speech, Language and Hearing Association (ASHA) website describes 18 independent divisions in the field (www.asha.org).With the increasing number of special interests, graduate programs are required to teach theory and practice on a wide variety of clinical issues.Because of the breadth of competence necessary to practice, many states require a minimum of a master's degree to practice.Therefore, a graduate program in SLP must admit and then graduate highly competent practitioners.The graduate program evaluated in the study, for example, requires six very demanding semesters of course-work and at least three clinical placements for degree completion.
Selecting students for admission is encumbered by the limited information available the applicant's admissions file.In addition to this limited information, it has been argued that, for example, undergraduate GPA's suffer from grade inflation since the 1960's and are, as a result, compressed at the upper end of the scale (Dostoevsky & Hartley, 2002).Letters of recommendation likewise, tend to be inflated (Rosovsky & Hartley, 2002).It has also been shown that undergraduate GPA is less accurate at predicting success in graduate school than standardized GRE scores (Kuncel & Hezlett, 2007).GRE scores have themselves, demonstrated mixed validity in predicting graduate student success (Kuncel & Hezlet, 2007;Sacks, 1999;Sternberg & Williams, 1997;Zwick, 2002).
Anecdotally, the authors have experienced surprise by the students who excel as well as those who have difficulty, based on information from their admissions file.For example, some students with very high GRE scores and superior undergraduate GPAs have failed to achieve to expectations, especially with regard to clinical competence.Conversely, other students with marginal scores have exceeded faculty expectations by demonstrating superior clinical competence.The hypothesized model in this study aimed to identify the direct and indirect effects on clinical and academic success from students' undergraduate and graduate performance.
Prior research on the topic of academic success is rather limited with regard to SLP programs.One study by Forrest and Naremore (1998) analyzed admissions data using a discriminant analysis to categorize students as either "top of the class students" or "bottom of the class students".These categories were defined by scores on board exams and graduate school GPA.Their results indicated that undergraduate GPA had a 93% overall accuracy rate, yet, GRE failed to achieve any statistically significant contribution.The limitations of this study include a small number of students (N = 30) and few predictor variables.
Other researchers (Astin, 1991(Astin, , 1993;;Bean, 1983;Tinto, 1993) have proposed that understanding success in any academic program requires assessing the interaction between the student and the institution's environment.The present study utilized Astin's (1991Astin's ( , 1993) ) I-E-O model.The premise of the I-E-O model is that educational assessments must include information on student inputs (I), the educational environment (E), and student outcomes (O).Inputs include the students' demographics, academic background, and previous experiences; Environment refers to the range of experiences students encountered during their college tenure; and Outcomes are the knowledge, beliefs, and values the student achieved at their institution.The Inputs are presumed to shape outcomes directly and indirectly with the institutional environment.Thus, academic success (the Outcome), is a function of the interaction between the student characteristics (Inputs) and the characteristics of the academic setting (Environment).

Purpose
The primary purpose of this study was to construct and assess a path model to determine the direct and indirect effects on two Outcome (O) variables (i.e., PRAXIS scores and Summative Clinical Evaluation).Input (I) data included the student's undergraduate information: Undergraduate major (Speech Major), Undergraduate GPA (UG GPA), and GRE (GRE Q and GRE V) while the Environment (E) data were grades in required graduate courses (Course Average) and Formative Clinical Evaluation.The hypothesized model is presented in

Procedure
This study analyzed the following students' Input variables, 1) major (SLP or other); 2) undergraduate GPA; and 3) GRE scores (verbal and quantitative).The Environment variables were the Average GPA from required graduate courses and Formative Clinical Evaluation.The Formative Clinical Evaluation was calculated from scores obtained from the 42 items on the Clinical Practicum Competency Checklist.This checklist is composed of eight subscales, 1) Clinical Excellence Across Contexts; 2) Assessment Data Analysis; 3) Assessment Procedures; 4) Intervention Planning; 5) Intervention Execution; 6) Clinical Excellence in Writing; 7) Clinical Excellence as a Professional; and 8) Self Evaluation.All items are scored on the following scale: 1 = fails to demonstrate behavior consistently regardless of amount of supervision or needs excessive and repetitive instructions, 2 = demonstrates behavior with specific instruction from supervisor, 3 = demonstrates behavior with general guidance from supervisor, 4 = demonstrates behavior with minimal guidance from supervisor, and 5 = demonstrates behavior independently.The Output variables were PRAXIS score and Summative Clinical Evaluation.The PRAXIS is designed by experts from The American Speech Language Hearing Association (ASHA), with the goal of testing the skills necessary to enter professional practice as a new graduate.The scores on the exam range from 250 -990 with a passing score of 600 (Educational Testing Service, 2009).The Summative Clinical Evaluation was comprised of performance scores ranging 1 (poor) to 5 (superior) by the department's clinical practicum coordinator.The practicum coordinator oversees all students in at least three settings outside of the academic setting.This process involves multiple site visits, counseling, and consulting between the site supervisors and the students and therefore the coordinator has a comprehensive understanding of how students perform in professional clinical settings.The Summative Clinical Evaluation was based on the written and verbal assessments of the supervisors in outplacement settings, students' academic performance, the level of independence exhibited by the student, as well as commonly-identified characteristics of good clinicians (e.g., flexibility, proactivity, and critical thinking).The proposed model was comprised of seven structural equations.In the first two equations, PRAXIS and Summative Clinical Evaluation was regressed on Course Average and Formative Clinical Evaluation.The next two structural equations had GRE-Quantitative, GRE-Verbal, and Undergraduate GPA predict Course Average and Clinical Evaluation.The last three equations had GRE-Quantitative, GRE-Verbal, and Undergraduate GPA regressing on Undergraduate SLP Major (coded: "1" as "Yes", "0" as "No").The proposed model illustrates the linear causal relationships among variables from the three time points.This allows for understanding the variables from the admissions file most related to course and clinic performance during the program and which variables from graduate school performance are most related to scores at graduation.

Data Analysis
A Hotelling's MANOVA was conducted to detected differences between SLP majors and non-SLP majors on 1) undergraduate GPA; 2) GRE verbal and quantitative scores; 3) graduate GPA; and 4) formative clinical evaluation.Independent samples t-tests were conducted as univariate follow-ups.A path analysis, an application of multiple regression in conjunction with causal theory, was used to analyze the causal models in this inquiry.
Figure 2 presents the path coefficients as well as the coefficient of determination for each of the endogenous variables.Undergraduate Speech Major was negatively associated with both GRE Q and GRE V, but was positively related to UG GPA.All three predictors for Course Average (GRE Q, GRE V, and UG GPA) achieved significant coefficients, while GRE Q and UG GPA were moderately correlated to Formative Clinical Evaluation; GRE V failed to predict Formative Clinical Evaluation.Course Average achieved a statistically significant path coefficient for PRAXIS; however, Formative Clinical Evaluation did not.Both Course Average and Formative Clinical Evaluation achieved significant path coefficients for Summative Clinical Evaluation.

Discussion
Findings from the present study have implications for future theoretical work in the area of clinical and academic performance in graduate SLP programs.The results indicated a better understanding of success can be derived by assessing the inter- for this study was 122 students who completed the SLP Program in the 2007-2009 academic cycles at a New England graduate school.The sample was comprised of mostly women ranging in age from 23 to 65, (M = 27.67,SD = 5.37).The GRE-Q scores ranged from 250 to 790 (M = 587.64,SD = 97.44)while the GRE-V scores ranged from 290 to 800 (M = 516.75,SD = 103.47).Just under half (40.3%) of the participants were SLP Majors as undergraduates with the remaining 58.9% representing a wide range of majors.