The Impact of Implementing a Portfolio Assessment System on Pre-Service Teachers ’ Daily Teaching Reflections on Improvement , Performance and Professionalism

Two groups of pre-service teachers (portfolio and non-portfolio) were compared on three factors, Improvement, Performance, and Professionalism. The Student Teaching Reflection Survey (STRS) (Robichaux, 2001) was developed to assess these traits. It was hypothesized that the portfolio group would reflect more on their teaching skills resulting in greater scores on the STRS. Results indicated that the portfolio group scored statistically significantly greater on Professionalism than the non-portfolio group. However, the two groups scored equivalently on both Improvement and Performance. Thus, it appears that requiring pre-service teachers to develop a portfolio leads to reflecting more on their professionalism. This professionalism leads to the development of a more effective educator who can handle the complexities of teaching.


Introduction
Regardless of the age of the student and the content being presented, teaching is a complex endeavor.Simultaneously, the teacher must choose the precise words that will maintain interest and convey the intended meaning for a large variety of learners, while maintaining an environment conducive to learning.Countless decisions must be made every second of the lesson in order for it to be successful.Indeed this is no easy task for veteran educators, let alone pre-service teachers taking full responsibility for a classroom for the very first time.Student teaching can be quite stressful for not only the student teacher, but for the supervising teacher as well as she/he observes the daily struggles of the student teacher.To assist pre-service teachers in their own professional growth, it is imperative that pre-service teachers use daily reflection.According to Groce, Henson and Woods (1999), the development of the ability to reflect on one's teaching is one of the underpinnings of the teaching profession.By consciously reflecting on their teaching practices and the resulting student behaviors, pre-service teachers can "construct their own learning through an interaction among their beliefs, their prior knowledge, and their experiences" (Lin, Gorrell, & Porter, 1999).As a result of this learning experience, pre-service teachers become more responsive to changing their behaviors based on the needs of the classroom (Rosen, 2000).As Costa and Kallick (2000) assert, "the ultimate purpose of reflection is to get us into the habit of thinking about our experiences."Through this thinking about their experiences, pre-service teachers learn what best practices are for their current students and become more effective.
Although it has been widely documented that reflection is an effective tool for improving teaching practices (Merryfield, 1993), teacher educators do not always explicitly teach pre-service teachers how to meaningfully reflect on their teaching.Instead, pre-service teachers are often expected to engage in self-reflection even if they have never been provided with the opportunity to learn these skills (Good & Whang, 2002).Furthermore, "definitions that reveal differing theoretical orientations about reflection have resulted in confusion about its meaning and uses" (Mackintosh, as cited in Imel, 1998).According to Loughran (2002), some educators consider reflection to simply be "thinking about something, whereas for others, it is a well-defined and crafted practice that carries very specific meaning and associated action."

Purpose
Thus, the primary purpose of this exploratory investigation was to examine the nature of the content of pre-service teachers' daily teaching reflections.Specifically, the researchers sought answers to the following questions: 1) What do preservice teachers concentrate on when they reflect on their teaching practices?2) Compared to the pre-service teachers who are required to assemble a professional portfolio, how does the content of the reflections differ from those pre-service teachers who are not required to do so?
Previous research reports that pre-service teachers tend to write about classroom management problems, students' social interactions and the social dimension of learning (Draper, 1998;Van, 1991).Beginning pre-service teachers seem to think that if they do not have classroom management dilemmas, then they are being effective teachers.They often fail to give sufficient attention to the content of their lessons as an indicator of their effectiveness.However in instances when portfolio assessment is used with pre-service teachers, it has been shown that the content of their teaching reflections does include a focus on the content and the activities used to teach that content (McFarland, 1998).Additionally, as pre-service teachers gain experience, classroom management issues become less of a concern and curriculum, lesson content and context, and pedagogy become more significant when reflecting on their teaching (Lee, 1999).In general, "reflection continually emerges as a suggested way of helping practitioners better understand what they know and do as they develop their knowledge of practice through reconsidering what they learn in practice" (Lougran, 2002).
Although several studies have examined reflection and reflective teaching in teacher education, the overwhelming majority of these studies have been qualitative in design, focusing on a small number of cases.Furthermore, the paucity of research comparing the reflective practices of pre-service teachers being assessed through an undergraduate professional portfolio with those who were not assessed in this manner makes this study relevant for those universities implementing or currently employing portfolio assessment.

Method Sample
Participants were 510 pre-service teachers, who had completed 180 hour of actual teaching and a minimum of 270 hours of observations, conferencing, and participation in other teacher duties such as grading, writing assessments, and researching future lesson topics, during their student teaching semester.Of the 510 participants, 232 completed their student teaching during the first year of this study prior to the implementation of a portfolio assessment system.Thus, these participants were not required to submit an exit portfolio at the conclusion of their degree programs.The remaining 278 participants completed their student teaching experience during the second year of this study and were required to submit a professional portfolio as a requirement for graduation.
All participants successfully completed the student teaching semester and thus, completed the degree requirements for their major in education.Eighty seven percent were female while 13% were male.Slightly over 90% labeled themselves as Caucasian with the remaining 10% classified themselves as either African American, Asian or "Other".

Procedure
Over the course of four consecutive semesters, participants in this study completed The Student Teaching Reflection Survey (STRS) (Robichaux, 2001) on the last day of their student teaching experience.

Instrumentation
Adopting Costa and Kallick's (2000) theoretical framework, the first author devised a three-domain instrument, The Student Teaching Reflection Survey (STRS) (Robichaux, 2001).STRS is a twelve-item questionnaire assessing three goals of reflection: Improvement, Performance, and Professionalism, scored on a 4-point Likert-type scale.
According to Costa and Kallick, the mental processes that one uses when reflecting include "comparing the results that were anticipated and intended with the results that were achieved, acting on and processing the information by analyzing, synthesizing, and evaluating, and applying learning to con-texts beyond the one in which it was learned and making commitments to plans of action."Thus in the STRS, Improvement was operationalized as whether or not the student teacher compared the outcomes of the lesson taught to what was expected.Items in the Performance domain in the STRS assessed whether or not the student teacher analyzed, synthesized and evaluated specific aspects of the lesson.The final domain, Professionalism, evaluated the student teacher's use of reflection as a tool for professional growth.

Results
A 2 × 3 mixed ANOVA was conducted for this study.The between variable was group (non-portfolio, portfolio) while the within variable was the three goals of reflection (performance, professionalism, and improvement).Results indicated a statistically significant interaction effect, Wilks' Lambda = .639,F (2, 507) = 143.39,p = .001,eta-squared = .027.
A series of independent samples t-tests were conducted as follow-up to the interaction effect to assess differences between the two groups on: 1) Improvement; 2) Performance; and 3) Professionalism.Results indicated differences on Professionalism.Because the equal variance assumption was violated as indicated by a significant Levene's test, a Behrens-Fisher correction was calculated.The portfolio group scored statistically significantly greater than the non-portfolio group, F (403.55) = 3.96, p < .001.See Table 1 for means and standard deviations.
Follow-up repeated measures analysis indicated that that for non-portfolio group, only Improvement was significantly greater than Professionalism or Performance.There was no significant difference between Professionalism and Performance.See Table 2 for means and standard deviations.
A second follow-up repeated measures analysis indicated that for the portfolio group, Improvement was significantly greater than Professionalism and Performance.Professionalism was significantly greater than Performance.See Table 3 for means and standard deviations.

Discussion
The purpose of this investigation was to examine the nature of the content of pre-service teachers' daily teaching reflections.In general, the content of most of the participants' reflections  contained some aspects Improvement and some aspects of Performance.Additionally, a majority of the participants agreed that daily reflection is an indicator of one's Professionalism.
With respect to the first research question concerning what pre-service teachers concentrate on when they reflect on their teaching, the results indicate that the pre-service teachers scored higher on the Improvement domain.Specifically, they appear to write about four aspects of improvement: 1) how the lesson could be changed to make it better if it were to be taught again; 2) what parts of the lesson were successful; 3) what parts of the lesson were unsuccessful; and 4) how the students reacted to the lesson.
For the second research question, which addressed the differences in the reflective practices of the participants required to assemble a professional portfolio as compared to those who were not required to do, the results indicate that there was a difference between the two groups.The non-portfolio group scored statistically significantly higher on the Improvement domain than on the Performance and Professionalism domains.However, the portfolio group scored statistically significantly higher on both the Improvement and Professionalism domains than on the Performance domain.This difference validated an objective of completing a "professional" portfolio.Thus, the portfolio group appears to be more reflective which is in agreement with previous research that claimed, "Portfolio assessment is valuable for promoting reflective practice in preservice teachers" (McFarland, 1998).
The results of this study seem to indicate that even if preservice teachers are not formally trained on how to reflect upon their teaching and are not given reflection prompts, they still appear to reflect on issues that are important to becoming effective educators.They also appear to learn to value reflection as a tool for professional growth.By requiring pre-service teachers to reflect daily on their teaching, it seems that these pre-service teachers get in the habit of reflecting.By getting into this reflective habit, these pre-service teachers are developing habits of professional growth and improvement (Costa & Kallick, 2000).Furthermore, the authors suggest that teacher educators provide their pre-service teachers with training on how to most effectively reflect on their teaching so that the benefits of reflection are maximized.
The results also indicate that the depth of a participant's reflections was related to whether or not the participant was required to assemble a professional portfolio.Thus, it appears that besides being an effective overall assessment tool, assembling a professional portfolio also leads to being a more reflective practitioner which in turn leads to the development of a more effective educator who can handle the complexities of teaching.

Table 1 .
Group differences on professionalism.