Creative Education 2012. Vol.3, No.7, 1221-1230 Published Online November 2012 in SciRes (http://www.SciRP.org/journal/ce) http://dx.doi.org/10.4236/ce.2012.37181 Copyright © 2012 SciRes. 1221 Effects of Video-Supported Web-Based Peer Assessment on Microteaching Applications: Computer Teacher Candidates Sample Gülten Kavas, Nesrin Özdener Department of Computer and Instractional Technology, University of Marmara, Istanbul, Turkey Email: gultenkavas@hotmail.com, nozdener@marmara.edu.tr Received September 10th, 2012; revised October 8th, 2012; accepted October 22nd, 2012 The aim of this study was to analyze the effects of the web-based system used for the development of teaching skills of the teacher candidates. In the research, the effects of the web-based system on the teaching performances of the teacher candidates and development of their peer assessment skills were tested, and their opinions on the practice were determined. In the study where experimentation model was used, qualitative and quantitative research techniques were employed, and the study was conducted with two working groups, namely those who used the web-based system outside the course hours and those who did not use the web-based system outside the course hours. The results of the research emphasized that the experiment group who took web-supported training was more successful than the control group in terms of teaching performance, and was more consistent in peer assessments. Keywords: Web-Supported Training; Microteaching; Peer Assessment; Teachers’ Training; Video Recording Introduction Owing to the revolutionary developments in the field of con- temporary science and technology, the use of new approaches in learning-teaching processeshas lead the institutions that edu- cate teachers to adopt a student-oriented structure in their gen- eral, occupational, and special field educations for teacher can- didates. From this viewpoint, conventional methods that meas- ure whether the teacher candidates have reached the level of maturity to deserve being awarded with a diploma are insuffi- cient in terms of determining whether this competence has been achieved or not (Darling-Hammond, 2001; Heartle, 1991; Mit- chel et al., 2001; Porter et al., 2001). Within this framework, performance measurements have gained significance. Performance measurements reflecting the levels of develop- ment of students during the process form an alternative to the limited nature of conventional assessment methods. Perform- ance measurements can make us understand the results of learning based on real-life situations, to determine the deficien- cies of the students in a more direct manner. In addition, these measurements can reveal both the strengths and weaknesses of the programs that prepare the teacher candidates and contribute to the improvement in the level of quality of teaching. In earlier research, performance measurements have been characterized as being more successful when compared with the conventional methods, and these measurements have been defined as effec- tive learning experiences (Anderson & Demeulle, 1998; Dar- ling et al., 2000; Mitchel et al., 2001; Porter et al., 2001). Microteaching, which is an innovative method used in per- formance assessments, is a simplified working environment that is used to provide prompt feedback on the performance of the teacher candidates, and resembles the actual system. Earlier research on the subject have demonstrated that when employing this method, which is based on giving the teacher candidates the opportunity to have teaching experience, recording of the practices of the teacher candidates through video-recording, making them watch those recording so as to enable them to analyze their deficiencies, and scoring proved to be more in- fluential in raising the levels of success (Frederiksen et al., 1999; Kpanja, 2001). Mcaleese and Unwin (1997) stated that video-supported microteaching practices improved the analysis capabilities of the teacher candidates, and Fernandez (2005) emphasized that these practices facilitate input of more con- structive feedback and comprehension of innovative education by virtue of the feedback received from the teachers, and peer cooperation. In addition, the results of the research conducted by Ceyhun and Karagölge (2002) revealed that repeated micro- teaching practices could increase the success of the students. However, while it is difficult to implement the performance measurements even once, owing to the crowded classrooms (Çakan & Kılıç, 2006), it is almost impossible to repeat these practices in practice courses at the universities. Furthermore, though self-and peer assessment activities can be exercised with an aim to support microteaching practices, the fact that a part of the insufficient course hours can be used for the assessment increases the pressure of time. This makes it obligatory for the teaching staff of education faculties to use microteaching prac- tices for group studies instead of individual studies. With regard to the above-mentioned difficulties, there is a need for solutions that can eliminate the time limitation, at least partially. A new tendency that has emerged at every level of education is the integration of education with internet, owing to the need to carry the course hours to outside the boundaries of school. Davies (2000) emphasized that the announcement of scores is usually postponed or delayed owing to crowded class- rooms and heavy work load of teachers and that computer-
G. KAVAS, N. ÖZDENER supported evaluation shall facilitate this procedure. Tsai et al. (2001a) underlined that the use of internet for peer assessment can overcome the limitations of time and space, and make learning more efficient. Similarly, Mc Gourty and De Meuse (2000) particularly emphasized that computer-supported or on- line peer assessments can accelerate the scoring process and facilitate anonymity of the scorer. Eventually, Davies (2000) stated that in cases where the names of the students who gave scores were not kept confidential, the students felt themselves under pressure and found it difficult to give scores. In addition, earlier research also revealed that the students made more real- istic evaluations when their names were kept confidential (Bal- lantyne et al., 2002; Lejyk & Wyvill, 2001). When constructivist approach is applied to the classrooms, it becomes necessary for the teachers, who have the required cap- abilities, to create a suitable learning atmosphere for the stu- dents and provide them an active role in teaching. Despite the efforts made for student-oriented teaching, it still has not be- come a natural practice to make the students measure and de- sign their own learning education. Although it is defended that each measurement should be made by forming an association between the student and the teacher (Stefani, 1998), there is no consensus on how this alliance should be established during the process of education. As peer assessment, which is one of the alternative assess- ment methods, can be defined as a process in which individuals assess their peers, this process may involve activities such as peer feedback and peer learning (Falchikov, 2001). According to Falchikov (2001), with the help of this method, members of a classroom can evaluate the homework or performance of their classmates using specified criteria. During this assessment process, it was found that the stu- dents had the chance to observe their friends and have a clearer idea as to how they learned. Sluijsmans et al. (1999) concluded that the assessment process gave the students the duties of judging and acting as arbitrators. During the process of peer assessment, the students learned to develop high-level respon- sibility, focus on the learning itself, and give scores. In addition, this assessment method provided the students with the chance to observe the teacher’s role and the nature of the assessment process. Participation of students in peer assessment helped them to develop their critical assessment skills (Jaques, 1991), to be- come aware of different problem solutions (Gibbs, 1981; Gibbs, Habeshaw, & Habeshaw, 1986), to develop reflective (projec- tive) thinking skills (Schon, 1983, 1987), and to be self-suffi- cient and self-oriented learners (Boud, 1991; Brown & Knight, 1995; Boud & Holmes, 1995). The skills gained at peer as- sessment studies could be used for the development of coopera- tive working ability in the long term (Hanrahan & Isaacs, 2001). As a consequence, more reliable assessment results can be ob- tained through assessment by the learners, and students may also learn by observing the skills that are newly learned from a critical perspective, acting more professionally at the time of giving feedback and during evaluation. Thus, peer assessment activity can contribute to both learning and assessment. Therefore, as video-and peer assessment activities that are beneficial for the efficient use of the microteaching method employed in programs aimed at educating teachers are time- consuming, it becomes a necessity to adopt new approaches. Although it cannot be denied that the internet, which is fre- quently used to solve time and space problems, is helpful in obtaining a solution, there are ongoing research efforts to de- termine at which stages and for what purposes the use of the internet would be most beneficial. Within this context, it would be useful to conduct new experiments to observe how the web- based system may be employed to provide the opportunity for more practice by increasing the number of microteaching prac- tices, to develop the assessment skills of teacher candidates, and to facilitate the establishment of communication among the students. Purpose of the Study The purpose of this study was to analyze the effects of the web-based system on the process of improvement of the skills of the teacher candidates. In this research, the effects of the web-based system on improvement of the teaching perform- ances and peer assessment skills of the teacher candidates were tested, and effort was made to determine their opinions on prac- tice. The following hypotheses were tested within the scope of the research: 1) When the microteaching performances of the teacher can- didates within the scope of the course are evaluated, there will be difference between those in the experimental group who took web-supported training and those in the control group who did not take web-supported training, favoring the experimental group. 2) When the teaching performances of the teacher candidates at schools where they undertook internship are evaluated, there will be difference between those in the experimental group who took web-supported training and those in the control group who did not take web-supported training, favoring the experimental group. 3) When the results of the peer-expert assessment of those in the experimental group who took web-supported training are compared with those of the teacher candidates in the control group who did not take web-supported training, there will be difference in favor of the experimental group in terms of con- sistency. 4) In the section of the research conducted with the experi- mental group, when the comments made in the classroom envi- ronment are compared with those made through the web on the basis of the number of students who made comments, there will be difference in favor of the experimental group. 5) When the comments made through the web-based system in the experimental group are compared with those made in the classroom in terms of their contents, there will be difference among the topics on which the comments were particularly focused on. In addition, in the research, the following questions were ad- dressed with an aim to determine the opinions of the teacher candidates on the practice. What do the teacher candidates think about the peer as- sessment activities after the practice? What do the teacher candidates who took web-supported training (experimental group) think about the preparatory video-recording activity after the practice? What do the teacher candidates who took web-supported training (experimental group) think about the activity of comment-making through the web after the practice? Method In the study where experimentation method was employed, Copyright © 2012 SciRes. 1222
G. KAVAS, N. ÖZDENER quantitative and qualitative research techniques were used. In the research with an aim to test the influence of the web-based system on the development of the peer assessment skills of the teacher candidates, the experimentation model involving post- test control group was used, whereas in the research with an aim to test the influence of the web-based system on the devel- opment of microteaching performances of the teacher candi- dates, experimentation model involving pre-test-post-test con- trol groups was used. In the part of the research that was conducted with the ex- perimental group, videos recorded in the classrooms were used to compare, on the basis of numbers, the comments made in the classroom with those made through the web, and to determine the categories of comments. The numbers of the comments made through the web and content analysis were determined with the help of the database. To find out what the teacher can- didates thought about the practice, they were asked to fill in the opinion forms. The forms that were filled were analyzed through the content analysis method. A systematic illustration of the research conducted in the course of Instructional Meth- ods II is given in Figure 1. The research was conducted with two working groups, namely, the Experiment group who used the web-based system outside the course hours as well, and the Control group who did not use the web-based system outside the course hours. With an aim to determine the working groups, the students were listed alphabetically according to their surnames, and the list was divided into two, such that both groups comprised 21 persons. In addition, the teaching methods used in the practice were determined, and a chart was prepared so as to demonstrate the methods that would be used each week. The students were in- cluded in the chart in an alphabetical order of their surnames. In this manner, the methods that would be used by the students in their performances were randomly determined. To find out whether the working groups were identical groups, the grade- Figure 1. Data resources and research pattern. point averages calculated by using the entire occupational, field, and special field courses that the teacher candidates took before the course of Instructional Methods II when the practice was conducted, were taken into account. The grade-point averages that were used were significant as they were related to the knowledge and skills of the teacher candidates in their own fields as well as their professional teachership knowledge and skills. As it can be seen in Figure 1, the teacher candidates in both groups exhibited 20-min long microteaching performances within the scope of the course. However, the teachers in the experimental group made trial recordings, broadcasted the im- ages in the web-based system, and reviewed their studies in light of the comments made by their friends, before performing in the classroom. This procedure continued throughout the se- mester, and each week, 2 - 3 teacher candidates performed microteaching. While the number of teacher candidates in the experimental group was 21, the total number of videos up- loaded into the website was 20, because the student who per- formed microteaching by using the method of field trips was not requested to make video-recording. In the section of the research conducted with the post-test control group experimentation model, microteaching perform- ances of the teacher candidates in both groups during their own course hours were evaluated by their group friends and the course lecturer. As an assessment tool, the Teaching Practice Assessment Form, which is commonly used in the faculty, was used. The teacher candidates both in the experimental and con- trol groups used the web-based system for the purpose of scor- ing at the time of peer assessment. The assessment results were used to determine the improvement in the microteaching per- formances of the teacher candidates as well as to examine the relationship between peer-expert assessments according to the groups. In the section of the research that was conducted with the pre-test-post-test control group experimentation model, the per- formances of the teacher candidates in both groups during the course (20 min) were compared with their teaching perform- ances at schools where they underwent internship for one day each week. The performances of the teacher candidates at schools where they underwent internship were realized in the actual school environment for a whole course hour (40 min). The assessments (peer and course lecturer) within the scope of the research were made online through the web-based system. In the section of the research in which the students’ opinions were determined, an opinion form that included various ques- tions to teacher candidates regarding peer assessment activities was developed by the researchers and applied. The responses of the teacher candidates who responded to the questions in writ- ing were analyzed through the method of content analysis. The research was conducted with the fourth-grade students who studied at Marmara University, Faculty of Atatürk Educa- tion, Computer and Educational Technologies Department. The working group of the research conducted with 42 students is given in Table 1. Data Gathering Tools With an aim to gather the data in the research, web-based system and portfolios prepared by the teacher candidates were used. Copyright © 2012 SciRes. 1223
G. KAVAS, N. ÖZDENER Copyright © 2012 SciRes. 1224 Web- Based System user accounts, enquiring about the scores achieved by the students, and observing the assessment results at any time. Figure 2 shows the transactions that can be made with the teacher’s module in the web-based system. The course lecturer carries out transactions, such as giving scores, determining the teacher candidates who will be scored, monitoring the videos uploaded to the system, and registering new users through this module. While Figure 2(a) shows the students who have per- formed and who will be assessed, Figure 2(b) presents a part of the criteria in the assessment form used during scoring and Figure 2(c) shows a section from the videos uploaded to the internet site. The web-based system was prepared to store the expert- and peer assessment results, as well as to broadcast and monitor the preparatory video recordings made by the teacher candidates in the experimental group for trial purposes, and to write com- ments to the videos. When the teacher candidates wanted to view the scores that they achieved after the assessment of their microteaching performances, they were able to access the peer assessment average and the score given by the course lecturer. Based on the studies that emphasized the importance of crea- tion of an environment in which the students do not feel them- selves under pressure (Ballantyne et al., 2002; Lejyk & Wyvill, 2001), only the course lecturer was allowed to access the scores given by the group candidates, to obtain more realistic results. Assessments were made in the classroom following the micro- teaching performances, and the assessment procedure ended with the assessment of the course lecturer. The assessments made by the course lecturer were used to compare the teaching performances of the group members in the classroom with those at the schools where they were undergoing internship for one day each week. The course lecturer’s assessments obtained through the system were also used for comparing the peer as- sessment results of the group members. Peer Assessment Module It is the module in which the students carry out the peer as- sessment procedures, upload their prepared course videos to the system, and write comments regarding the uploaded videos. Teacher candidates and the course lecturer enter the system by using their user names and passwords. During the first week of the course, the students were introduced to the system and valid user names and passwords were identified. The system had three main modules: User and teacher inter- face, web server application module, and database server. The web-based system was prepared by using ASP. All the students who participated in the research were provided with the user names and passwords for entering into the system, during the first week of the course. Table 1. Table type styles. Group Method n Experimental GroupUsing web-based system out of class 21 Control Group Not using web-based system out of class21 Total 42 Teacher’s Module It is the module used for transactions, such as arranging the Figure 2. Teacher interface.
G. KAVAS, N. ÖZDENER Teachership Practice Assessment Form It is an online observation form prepared by using the assess- ment form employed with the whole education faculty in com- mon, to assess the teacher candidates on the basis of subject field and field knowledge, learning teaching process, assess- ment, and communication skills. Portfolio The teacher candidates were asked to prepare a portfolio in which their activities throughout one semester could be as- sessed. The contents of the portfolio were formed by using the course schedules, course videos, activities during the courses (presentations, training materials, videos, homework, etc.) used in the classroom and at the schools where underwent internship, and the opinion forms. The portfolios were used to evaluate the teachership performances demonstrated by the teacher candi- dates at schools where they underwent internship. Experimental-Group and Control-Group Videos Microteaching videos were obtained by recording the micro- teaching performances of all the teacher candidates in the ex- perimental and control groups in the classroom during the course hours, and the teachership performances at schools where they underwent internship. These recordings were used in the assessments made by the course lecturer, and Teacher- ship Practice Assessment Form was used at the time of assess- ment of the microteaching videos. Preparatory Videos These were the videos that the teacher candidates in the ex- perimental group prepared before their microteaching perform- ances in the classroom environment and shared them through the website with their group friends. The teacher candidates in the group gave feedback to these videos through the website, and helped in improvement of their group friends before the course. They used the images that they watched and the com- ments that they read, and tried improving their preparations before coming to the course. The comments made to the videos that were watched through the website were compared with those made to the microteaching performances in the classroom in terms of the number and contents. The comments that were obtained were grouped according to categories that were formed by reviewing the literature (Wu & Kao, 2008; Andrade & Du, 2007; Fernandez, 2005) and the comments made. Opinion Forms In the opinion form prepared by the researchers with an aim to gather data, there were 7 questions aimed at determining the opinions of the teacher candidates on the peer assessment ac- tivities, 7 questions aimed at finding their opinions on prepara- tory video recording, and 4 questions aimed at learning about their opinions on making comments through the web. At the time of gathering the data, the opinion forms were distributed to all 42 teachers who were asked to respond to the questions in writing. Findings In the research, effort was primarily made to determine whe- ther the experimental and control groups were identical. The Shapiro-Wilk test conducted with an aim to determine whether the field, special field, and occupational course scores of both groups showed a normal distribution or not, showed that the scores of the students of both working groups had a normal distribution (p > .05). The results of the Independent Samples T-test used to determine whether the two groups were identical are given in Table 2. As it can be seen in Table 2, no meaningful difference was observed between the scores of the students in both working groups (t(36) = −.327; p > .05). In this case, it can be argued that the groups were equivalent groups in terms of the levels of success relating to field, special field, and occupational course scores before the practice. Hypothesis 1. When the microteaching performances of the teacher candidates during the course are evaluated, there will be difference between those in the experimental group who took web-based training and those in the control group who did not take web-based training, favoring the experimental group. With an aim to test the effect of the web-based system on the microteaching performances of the teacher candidates in the classroom, the scores obtained by the experimental and control groups were compared by using the Mann Whitney U-Test, and the results are given in Table 3. From Table 3, it can be seen that there is a meaningful dif- ference between the success levels of the teacher candidates in the two groups (U = 105.0; p < .05). When the mean ranks of those in the experimental group who took web-supported train- ing and those in the control group who did not take web-sup- ported training were taken into consideration, it was observed that the teacher candidates in the experimental group reached a higher level of success. This finding shows that the first hy- pothesis has been verified, that is, there is a meaningful differ- ence between the microteaching performances of the teacher candidates in favor of the experimental group, and that the web- based assessment system can have influence in raising the level of success. Hypothesis 2. When the teaching performances of the tea- cher candidates at schools where they underwent internship are assessed, there will be difference between those in the experi- mental group who took web-based training and those in the control group who did not take web-based training, favoring the experim en t a l group. With an aim to test the influence of the web-based system on the teaching performances of the teacher candidates at schools where they underwent internship, Mann Whitney U-Test was used, and a comparison was made on the basis of the scores given by the expert, and the results are given in Table 4. Ac- cording to Table 4, there is a meaningful difference between the levels of success of the teacher candidates in the experi- mental and control groups (U = 64; p < .05). When the mean ranks were taken into consideration, it was observed that the teacher candidates in the experimental group reached a higher level of success when compared with those in the control group. Table 2. Independent Sample T-test related to the comparison of the groups. Groups n Mean t sd p Experimental Group 20 69.70 Control Group 18 70.33 −.327 36 .745 Copyright © 2012 SciRes. 1225
G. KAVAS, N. ÖZDENER Table 3. Mann Whitney U-Test Results of success levels of microteaching per- formances. Group n Mean Rank Sum of Ranks U p Experimental 21 26.98 566.50 Control 21 16.02 336.50 105.00.00 Table 4. Mann Whitney U-Test results with respect to the levels of success of teaching performances of teacher candidates. Group n Mean Rank Sum of Ranks U p Experimental 21 27.30 546.50 Control 21 13.70 274.50 64.00 0.00 This finding shows that the second hypothesis has been verified, and that the web-based assessment system is influential on raising the level of success of the teacher candidates. Hypothesis 3. When the peer-expert assessment results of the teacher candidates in the experimental group who took web-supported training are compared with those of the teacher candidates in the control group who did not take web-sup- ported training, there will be difference in favor of the experi- mental group in terms of consistency. With an aim to analyze the influence of the web-based sys- tem on peer assessment skills of the teacher candidates, the scores that the two groups received from the course lecturer and peer assessment scores of both groups were compared on the basis of averages and standard deviations. Mean and standard deviation results of the candidates in the control group who did not use the web-based system outside the course hours: Table 5 shows the mean scores and standard deviations obtained as a result of expert assessments and peer assessments in the control group. Although the mean of peer assessments in the control group was 80.52, the standard devia- tion was ±8.44 (n = 21). The mean of the scores given by the course lecturer was 58.33 and the standard deviation was ±19.2 (n = 21). It can be observed that the expert scores are lower when compared with the results of peer assessments made by the teacher candidates and that the peer assessment scores of the teacher candidates are listed in a narrower range when compared with the scores of the course lecturer, as they have a lower standard deviation. The first group results of the protocol by Kwan and Leung (1996), used with an aim to make a de- tailed analysis of the consistency (agreement) between the peer assessment results and the scores given by the expert, are shown in Table 6. In the control group, whose cross-table results related to the scores obtained as a result of expert and peer assessments are given in Table 6, there were 21 teacher candidates who par- ticipated in the peer assessment activities. Out of the 21 teacher candidates, 10 received scores that were in the acceptable range (course lecturer’s score ±1 sd), in peer assessments. According to the results, there were 11 candidates who received 1 sd more than the expert score, and there were none who received 1 sd less than the expert score in the peer assessment activities. Mean and standard deviation results of the experimental group candidates who used the web-based system outside the course hours: Table 7 shows the mean scores and standard de- viations achieved as a result of expert assessments and peer assessments in the experimental group. Although the mean of the peer assessments in the group of teacher candidates who used the web-based system was 80.14, standard deviation was ±8.30 (n = 21). The mean of the scores given by the expert was 75.28 and the standard deviation of these scores was ±16.95 (n = 21). It can be observed that the expert scores are lower than the results of peer assessments made by teacher candidates, and that peer assessment scores of teacher candidates are listed in a narrower range when compared with the scores of the course lecturer as they have a lower standard deviation. With an aim to analyze the consistency (agreement) between the peer assess- ment results and expert scores in detail, the protocol developed by Kwan and Leung (1996) was used. The results of the second group are shown in Table 8. In the experiment group, whose cross-table results of expert and peer assessments are given in Table 8, there were 21 teacher candidates who participated in the peer assessment activities. Out of the 21 teacher candidates, 18 achieved scores that were in the acceptable range (expert score ±1 sd) in peer assessments. According to the results, there were three candidates who received scores that were 1 sd more than the expert score in the peer assessment activity, and there were none who received 1 sd less than the expert score. Hypothesis 4. In the part of the research that was conducted with the experimental group, the comments made in the class- room environment and those made through the web may show difference in terms of the number of students who made com- ments. Table 5. Comparison of the expert assessments and peer assessments in the control group on the basis of means and standard deviations. Assessment Type n Mean sd Expert assessment 21 58.33 19.20 Peer assessment 21 80.52 8.44 Table 6. Expert and peer assessments of the control group: Cross-Table. Range of expert points d* > 19.2 f (%) 19.2 > d* > 19.2 f (%) d* < 19.2 f (%) Total f (%) >77.54 0 3 (14) 0 3 (14.3) 39.12 - 77.548 (38) 7 (33) 0 15 (71.4) <39.12 3 (14) 0 0 3 (14.3) Total 11 (52) 10 (48) 0 21 (100) Note: d = the difference between the means of peer assessment results and expert scores. Table 7. Comparison of the expert and peer assessments of the group of teacher candidates used the web-based system on the basis of means and stan- dard deviations. Assessment Type n Mean sd Expert assessment 21 75.28 16.95 Peer assessment 21 80.14 8.30 Copyright © 2012 SciRes. 1226
G. KAVAS, N. ÖZDENER In the part of the research that was conducted with the ex- perimental group, with an aim to test the hypothesis that the comments made in the classroom environment and those made through the web may show difference in terms of the number of students who made comments, the video recordings related to the comments made by the teacher candidates in the classroom environment were watched, and the number of persons who made the comments was determined. Furthermore, the number of comments made by the teacher candidates in the group through the web-based system was determined and the results are given in Table 9. While the number of comments made by the students in the classroom during a semester was 162, those made in the inter- net site was 380. When the comments that were made were classified into categories, the comments made in the classroom were listed under 374 items, while those made through the web were listed under 795 items. Based on these results, it can be argued that through the web-based system, more conversations can be generated, and that more students participate in conver- sations. Hypothesis 5. There will be difference among the topics on which the comments are particularly concentrated, when the comments made through the web-based system and in the class- room are analyzed in terms of the contents in the experimental group. The contents of the comments made through the web and in the classroom were significant in terms of the quality of the assessments made by the teacher candidates. The comments made to the 20 videos uploaded to the internet site and the 20 microteaching performances demonstrated in the classroom du- ring the semester were grouped under seven categories, namely, teaching method, communication with students, body language teaching materials and their use, subject field knowledge, dress, and other comments. Table 10 shows the analysis of the com- ments made by the teacher candidates through the web and in the classroom. From Table 10, it can be observed that the category in which most of the comments were made both through the web and in the classroom was the teaching method (web: 27%, classroom environment: 31%). Among the comments made through the web, the second category that received most of the comments Table 8. Expert and peer assessments of the experimental group: Cross-Table. Range of expert points d* > 16.95 f (%) 16.95 > d* > 16.95 f (%) d* < 16.95 f (%) Total f (%) >92.23 0 1 (5) 0 1 (5) 58.83 - 92.23 0 18 (86) 0 18 (86) <58.83 3 (14) 0 0 3 (14) Total 3 (14) 18 (86) 0 21 (100) Note: d = the difference between the means of peer assessment results and expert scores. Table 9. Comments made in the classroom environment and through the web. Comments Class Web Number of students who made comments 162 380 Total number of comments 374 795 Table 10. Analysis of the comments made by teacher candidates through the web- based system and in the classroom. Number of Comments Category Class n (%) Web n (%) Teaching Method 116 (31) 215 (27) Communication with Students 72 (19) 104 (12) Body Language 60 (17) 205 (25) Material 75 (20) 190 (24) Subject Field Knowledge 31 (8) 40(6) Dress 1 (0) 30 (4) Other 19 (5) 11 (2) Total 374 (100) 795 (100) was body language, and the third category was the teachingma- terials and their use. On the other hand, the second category that received most of the comments made in the classroom was teaching materials and their use (20% of the total comments), while the third category was communication with the students (19% of the total comments). From Table 10, it is worthwhile to note that in the discus- sions made in the classroom, there were almost no comments regarding the dresses (0%), whereas 4% of the comments made through the web were related to dresses. Similarly, it can be in terms of expression style (e.g., a student recommending anoth- erstudent to urgently take diction courses, or telling a student that he/she acts unskillfully in terms of student control during a course, telling a student that the material used was ordinary). However, among the comments made in the classroom envi- ronment, such bitter expressions were lesser. This suggests that the teacher candidates are more comfortable while they are making comments through the web. Research Question 1. What are the thoughts of the teacher candidates on the peer assessment activities following the prac- tice? In the research, the opinions of the teacher candidates on peer assessment activities were analyzed, and the results are shown in Table 11. Table 11 summarizes the opinions of the teacher candidates in the experimental and control groups on peer assessment activities. According to the findings, 71% of the teacher candidates in the experimental group believed that the scores that they received at peer assessment are fair, 19% believed they received low scores, and 10% believed they re- ceived high scores. Fifty-seven percent of the teacher candi- dates in the control group believed that the scores they received at peer assessment are fair, while 43% believed that they re- ceived low scores. However, there were no students in the con- trol group who believed that they received high scores. While 62% of the teacher candidates in the experimental group be- lieved that they gave scores independent of their friendship relations, this ratio was 48% in the control group. Furthermore, 52% of the teacher candidates in the experimental group stated that there would be no change in the scores that they gave even if the scores were not being included in the evaluation at the end of the semester. Twenty-four percent of the teacher candi- dates in the control group stated that there would be no change in the scores that they gave in the above-specified situation. Copyright © 2012 SciRes. 1227
G. KAVAS, N. ÖZDENER Copyright © 2012 SciRes. 1228 Table 11. Students’ opinions on peer assessment activities. Exp. Group f (%) Control Group f (%) Opinions Y NS N Y NS N While giving scores to my friends, I acted independent of my friendship relations and I gave scores by observing the assessment criteria. 13 (62) 5 (24)3 (14) 10 (48) 2 (10)9 (43) If my friends were seeing what score I gave, the scores I gave would be different. 5 (24) 6 (29)10 (48) 9 (43) 5 (24)7 (33) If the scores I gave were not to be included in the assessment at the end of the semester, then the scores I gave would be different. 8 (38) 2 (10)11 (52) 11 (52) 5 (24)5 (24) I give importance to the scores that I got from my friends. 18 (86) 2 (10)1 (5) 16 (76) 1 (5) 4 (19) I was impatient to learn about the score that I got from my friends after the microteaching performance. 19 (91) 1 (5) 1 (5) 15 (71) 4 (19)2 (10) I think that the peer assessment methods contribute to my professional development. 21 (100)0 (0) 0 (0) 16 (76) 3 (14)2 (10) I believe I got the scores that I deserved at the peer assessments. 15 (71) 2 (10)4 (19) 12 (57) 1 (5) 8 (38) I believe that the scores I received from peer assessments are lower than what I deserved. 4 (19) 0 (0) 17 (81) 9 (43) 5 (24)7 (33) I believe that the scores I received from peer assessments are higher than what I deserved. 2 (10) 0 (0) 17 (81) 0 (0) 0 (0) 21 (100) Note: Y: I agree, NS: I’m not sure, N: I disagree. Research Question 2. What are the opinions of the teacher candidates who took web-based training (experimental group) after the practice on the activity of preparatory video-record- ing? Another question that was posed to the teacher candidates in the experimental group who used the web-based system within the scope of the research was their thoughts regarding the activ- ity of preparatory video-recording. Table 12 summarizes the opinions of the teacher candidates in the experimental group regarding the process of preparatory video-recording. All the teacher candidates stated that the activity of recording preparatory videos contributed to their development, 71% sta- ted that they had the chance, by virtue of the recordings, to pre- pare twice for the subject that they would present, their com- mand over the subject increased, and 71% stated that the re- cordings were in the nature of rehearsal and these rehearsals helped them to overcome their anxiety. Furthermore, none among the teacher candidates, who had to watch a few videos each week and write down comments for each of them, thought that this process put pressure on them. Research Question 3. What are the opinions of the teacher candidates who took web-based training (experimental group) on the activity of making comments through the web after the practice? Another question that was raised to the teacher candidates in the experimental group who took web-based training was their thoughts concerning making comments through the web. Table 13 summarizes the opinions of the teacher candidates in the experimental group regarding the process of making comments to the preparatory videos published through the web. As it can be seen in Table 13, 95% of the teacher candidates stated that the comments made by their friends regarding the videos that they recorded helped them to notice their deficiencies, 67% stated that the comments made through the web helped them to prepare for the comments that could be made in the classroom, 62% stated that the comments made by the other teacher can- didates through the web increased their motivation towards the course, and 86% stated that the process of writing comments to preparatory videos caused a better understanding of the assess- ment criteria. Table 12. Students’ opinions on recording preparatory videos. f (%) Opinions Y NSN The preparatory recordings prior to the course helped in my development. 21 (100)0 (0)0 (0) Making preparations twice on the same subject helped me to have better command of the subject. 15 (71) 5(24)1 (5) The activities prior to the practices in the classroom were in the nature of rehearsal, an they helped me overcome my excitement. 15 (71) 6 (29)0 (0) Watching the videos recorded by my friends through the web developed my perspective and I took lessons from the mistakes that I observed. 21 (100)0 (0)0 (0) Watching a few videos each week and writing comments to each made me to feel the pressure upon me. 0 (0) 2 (10)19 (90) I watched most of the videos uploaded to the system for a few times. 17 (81) 0 (0)4 (19) Note: Y: I agree, NS: I’m not sure, N: I disagree. Table 13. Students’ opinions regarding making comments through the web. f (%) Opinions Y NS N The comments made by my friends helped me realize and cover up my deficiencies. 20 (95) 1 (5)0 (0) The comments made through the web helped me to guess the comments that could be made in the classroom and to become prepared for such comments. 14 (67) 5 (29)2 (10) The comments I received through the web motivated me towards the course. 13 (62) 6 (24)2 (10) Writing comments to the videos recorded by my friends enabled me to better understand the assessment criteria. 18 (86) 2 (10)1 (5) Note: Y: I agree, NS: I’m not sure, N: I disagree.
G. KAVAS, N. ÖZDENER Conclusion and Suggestions With this research, the effects of the web-based system used in the course of development of teaching skills of the teacher candidates on the performance of such teacher candidates and on development of their peer assessment skills were tested, and the opinions of the teacher candidates on the practice were de- termined. The findings obtained from the research led to the following conclusions. Within the scope of the research, it was determined that the activities made through the web influenced the level of success of the teacher candidates positively. It was found that the teacher candidates in the experimental group who took web- supported training showed were more successful than those in the control group, in terms of the microteaching performances (Experimental Group: 75.28; Control Group: 58.33). Similar results were also observed in the teaching performances of the teacher candidates at schools where they underwent internship (Experimental Group: 84.61; Control Group: 73.60). Based on these results, it can be argued that the practices that are con- ducted through the web-supported training contribute positively to the professional development of the teacher candidates. The web-based system helped the teacher candidates in the experimental group who took web-supported training not lim- ited within the course hours, and gave them the chance to con- duct more activities. Moreover, it can be argued that the fact that the teacher candidates who used the web-based system had the chance to study and practice at any time they wished influ- enced their level of success. Although the results of the research indicate a difference in favor of the experimental group, the high level of teaching per- formance of the experimental group and the control group at the schools of internship make it necessary to mention the effects of video use. When only the practice conducted with the control group was evaluated, the importance of video recordings in microteaching performances could be observed. It can be ar- gued that by watching the microteaching performances of the teacher candidates in the classroom, the level of success in- creased. A similar result was obtained in the research conducted by Ceyhun and Karagölge (2002), in which the influence of video recordings on microteaching performances of the teacher candidates was examined. In support of this result, in the re- search that Kpanja (2001) conducted with two working groups, a significant difference was observed between the microteach- ing performances of the experimental group who used video recordings and the control group who did not use video recordings. While both the experimental group and the control group were found to benefit from the video recordings within the scope of the research, activities out of class had a significant impact in favor of the experimental group. The teacher candi- dates in this group had to follow-up the course more regularly and also carry out activities by sparing time outside the course hours. It can be argued that these activities that were performed each week helped the experimental group who took web-based training to be more successful. Another research finding revealed that the scores obtained by the teacher candidates in peer assessment practice were grouped into a more limited range when compared with those given by the course lecturer, and that the standard deviation was smaller. This result is in accordance with those presented in the litera- ture (Hughes & Large, 1993; Stefani, 1994; Kwan & Leung, 1996, Kılıç & Çakan, 2006). Based on these findings, it can be argued that the teacher candidates abstain from giving low scores to their group friends. It is remarkable that there were no scores lower than 50. These findings indicating that the peer assessments were higher than those made by the course lecturer (Magin & Hellmore, 2001; Kılıç & Çakan, 2006) can be ex- plained by the fact that the students had not yet developed ob- jective-assessment responsibilities, do not have the same as- sessment skills as teachers, or lacked expertise on the subject field that was being assessed. Furthermore, results that are con- trary to those observed in this study have also been observed (Stefani, 1994; Freeman, 1995; Kwan & Leung, 1996). The funda- mental reason for such a difference might be the percentage by which the scores obtained from peer assessment activities affect the course-passing score. While the scores obtained as a result of expert and peer assessments within the scope of the research constituted 16.6% of the end-semester score, Stefani (1994) found that the scores obtained from peer assessment activity influenced the end-semester score by 2%. Another reason for the difference among the research results can be attributed to the classroom level of the students who participated in the re- search. In this respect, the differences between the behaviors of the final-year students who were about to graduate within the scope of the research, and the first-year students with whom Stefani (1994) worked may influence the results. It can be argued that the final year students might have given high scores in the assessment owing to their close relationships with each other or owing to their anxiety regarding their ability to graduate. The findings obtained from the opinion forms in the research revealed that the teacher candidates in both the groups have opined that the peer assessment activities influenced their pro- fessional improvement. However, a significant part of the stu- dents in both groups expressed that they were under the influ- ence of friendship relations at the stage of peer assessment, and that if the grades were not being included in the assessment, the grades they gave would be different (Experimental group: 38%, Control group: 52%). These results indicate that the teacher can- didates are not professional enough in the assessment activities. The teachers in the experimental group stated that the activ- ity of making comments through the web caused better com- prehension of the assessment criteria, and that they noticed their deficiencies by virtue of the comments made. Wu and Lee (2005) who encountered similar results established that the course-success levels of teacher candidates who communicated through the forum were increased, and that discussion forums created a new learning environment. Similarly, Wu and Kao (2008) determined that using web-supported marking system and watching videos of friends developed the teachership skills of the students. According to the research findings, the first category in which comments were made both in the classroom and through the web was the teaching method (web: 27%, classroom: 31%), while the second category was the teaching materials and their use (web: 24%, classroom: 20%). Similarly, Wu and Kao (2008), in a study conducted with computer teacher candidates, determined that the comments made through the web were mostly concentrated in categories of communication with the students and the use of blackboard. The basis of the four year graduate studies of teacher candidates studying at BÖTE depart- ment was formed by pedagogic courses, special field courses, and material development practices. Teacher candidates had, throughout their education, taken courses in connection with re- cognizing visual-audio and multimedia materials, understand- ing the principles and methods related to the development and use of these materials, and grasping the methods for selecting and evaluating these materials. Furthermore, as the background Copyright © 2012 SciRes. 1229
G. KAVAS, N. ÖZDENER Copyright © 2012 SciRes. 1230 of the information technologies of the teacher candidates in teaching materials and the course researched was Special Teaching Methods II (as the teaching methods were discussed), it is understandable that the comments were concentrated on these categories. In addition, the fact that 4% of the comments made through the web was in relation to dresses, while almost no comments were made regarding dresses in the discussions made in the classroom (0%), it can be argued that the teacher candidates avoided making certain criticisms face- to-face. When the results of the research in relation to peer assess- ment were scrutinized, we found that the involvement of the students in the process of formation of assessment criteria can provide benefit to the understanding, acceptance, and internali- zation of the criteria. We presume that internalization of the assessment criteria by the students will provide for more con- sistent peer assessment results. In the future research, it will be beneficial to arrange for a brainstorming activity during the first week of the practice with an aim to prepare the assessment criteria together with the students, as it will provide efficient use and comprehension of the assessment criteria employed within the scope of the study. In addition, as performance as- sessment is considered to be more difficult when compared with homework or project assessments, and may require field knowledge, it would be appropriate to conduct pilot studies. Thus, it can be concluded that both the skills of selecting teaching method, material preparation and making effective use of such materials, and peer and self-performance assessment skills by exhibiting an objective attitude are essential capabili- ties that should exist in teacher candidates. While Race (1991) emphasized that self assessment is a talent and becomes more reliable after repetition, Stefani (1994) pointed out that it would be beneficial if the students are introduced to the methods of peer and self assessment at a young age. It can clearly be ob- served in both recommendations that time is an important de- termining factor in improvement of the assessment skills. The results of the research show promise in increasing the reliability of the assessment and in improving the teaching performances through the use of the web-based system. We observed that the group of teacher candidates who carried out studies out of the class hours through the web-based system was more successful in terms of teaching performances and assessment skills. REFERENCES Anderson, R. S., & DeMeulle, L. (1998). Portfolio use in twenty-four teacher education programs. Teacher Education Quarterly, 25, 23- 32. Ballantyne, R., Hughes, K., & Mylonas, A. (2002). Developing proce- dures for implementing peer assessment in large classes using an ac- tion research process. Assessment and Evaluation in Higher Educa- tion, 27, 427-441. doi:10.1080/0260293022000009302 Boud, D. (1991) Implementing student self-assessment. Campbelltown: Higher Education Research and Development Society of Australia Inc. Boud, D., & Holmes, H. (1995) Self and peer marking in a large tech- nical subject. In D. Boud (Ed.), Enhancing learning through self as- sessment. London: Kogan Page Ltd. Ceyhun, I., & Karagölge, Z. (2002) Kimya eğitiminde tezsiz yüksek lisans öğrencileri ile mikroöğretim. Proceedings of V.UFBMEK Conference, URL (last checked 18 May 2012). http://www.fedu.metu.edu.tr/ufbmek-5/b_kitabi/b_kitabi.htm Darling-Hammond, L. (2000). Teacher quality and student achievement: A review of state policy evidence. Education Policy Analysis Archi- ves, 8, 1. http://olam.ed.asu.edu/epaa/v8n1 Darling-Hammond, L., & Snyder, J. (2000). Authentic assessment of teaching in context. Teaching and Teacher Edu c at i o n, 16, 523-545. Davies, P. (2000). Computerized peer assessment. Innovations in Edu- cation and Teaching I nte r n at i o nal, 37, 346-355. Falchikov, N. (2001). Learning together: Peer tutoring in higher edu- cation. London: Routledge Falmer. Fernandez, M. L. (2005). Learning through microteaching lesson study in teacher preparation. Action Teacher Education, 26, 37-47. doi:10.1080/01626620.2005.10463341 Freeman, M. (1995). Peer assessment by groups of group work. As- sessment and Evaluation in High e r Education, 20, 289-299. doi:10.1080/0260293950200305 Frederiksen, J. R., White, B. Y., & Shimoda, T. A. (1999). Enabling students to construct theories of collaborative inquiry and reflective learning: Computer support for metacognitive development. Interna- tional Journal of Artificial Intelligence in Education, 10, 151-182. Gibbs, G. (1981) Teaching students to learn: A student-centred ap- proach. Philadelphia: Open University Press. Gibbs, G., Habeshaw, S., & Habeshaw, T. (1995) 53 interesting ways to assess your students. Bristol: Technical and Educational Services. Hanrahan, S. J., & Isaacs, G. (2001) Assessing self and peer assessment: The students’ views. Higher Education Research and Development, 20, 53-70. doi:10.1080/07294360123776 Hughes, I. E., & Large, B. J. (1993). Staff and peer-group assessment of oral communication skills. Studies in Higher Education, 18, 379- 385. doi:10.1080/03075079312331382281 Jaques, D. (1991) Learning i n G r o ups (2nd ed.). London: Kogan Page. Kilic, G. B., & Cakan, M. (2007). Peer assessment of elementary sci- ence teaching skills. Journal of Science Teacher Education, 18, 91- 107. doi:10.1007/s10972-006-9021-8 Kpanja, E. (2001). A study of the effects of video tape recording in microteaching training. British Journal of Educational Technology, 32, 483-486. doi:10.1111/1467-8535.00215 Kwan, K., & Leung, R. (1996). Tutor versus peer group assessment of student performance in a simulation training exercise. Assessment and Evaluation in Higher E d uc at io n, 21, 205-215. Lejk, M., & Wyvill, M. (2001). The effect of the inclusion of self- assessment with peer assessment of contributions to a group project: A quantitive study of secret and agreed assessments. Assessment & Evaluation in Higher e d u c a ti o n , 26, 551-561. doi:10.1080/02602930120093887 Magin, D., & Helmore, P. (2001). Peer and teacher assessments of oral presentation skills: How reliable are they? Studies in Higher Educa- tion, 26, 287-298. doi:10.1080/03075070120076264 McGourty, J., & De Meuse, K. (2000). The team developer: An as- sessment and skill building program. New York: J. Wiley and Com- pany. Porter, A., Youngs, P., & Odden, A. (2001). Advances in teacher as- sessments and their uses. In V. Richardson (Ed.), Handbook of re- search on teaching (4th ed., pp. 259-297). Washington DC: AERA. Schon, D. A. (1983) The reflective practitioner: How professionals think in action. San Francisco, CA: Jossey-Bass. Schon, D. A. (1987) Educating the reflective practitioner. San Fran- cisco, CA: Jossey-Bass. Sluijsmans, D., Dochy, F., & Moerkerke, G. (1999). Creating a learning environment by using self-peer and co-assessment. Learning Envi- ronment Research, 1, 293-319. doi:10.1023/A:1009932704458 Stefani, L. A. J. (1998), Assessment in partnership with learners. As- sessment & Evaluation in High e r Education, 23, 339-350. Topping, K. (1998). Peer assessment between students in colleges and universities. Review of Educational Research, 68, 249-276. Tsai, C.-C., Liu, E. Z.-F., Lin, S. S. J., & Yuan, S.-M. (2001). A net- worked peer assessment system based on a vee heuristic. Innovations in Education & Teaching International, 38, 220-230. doi:10.1080/14703290110051415 Wu, C.-C., & Lee, G. C. (2005). Use of computer-mediated communi- cation in a teaching practicum course. International Journal of Sci- ence and Mathematics Education, 2, 511-528. doi:10.1007/s10763-004-2892-2 Wu, C.-C., & Kao, H.-C. (2008). Streaming videos in peer assessment to support training pre-service teachers. Educational Technology & Society, 11, 45-55.
|