A Mixed-Method Study of EFL Writers’ Monitoring and Revising Processes in Completing an Integrated Writing Task ()
1. Introduction
Writing is generally considered to be difficult to execute independently of other language abilities; rather, it relies on getting information from other sources by reading, listening, or doing both (Gebril & Plakans, 2013; Golparvar & Khafi, 2021; Hinkel, 2006; Hirvela, 2004; Wang, 2022). Integrated writing tasks have been recommended as a potential task type in language instruction and testing, for instance, TOEFL, has added a revision to the existing independent writing task, in which test-takers are required to listen to and read a text, then summarise it in a written summary.
In a typical reading-to-write task, writers have to use both their reading and writing skills. The monitoring process is one important cognitive process that has been extensively studied in the completion of this type of task. Field (2004) pointed out that writers engage in the monitoring process at different levels throughout the writing process: at a basic level, monitoring involves checking the mechanical accuracy of the text produced, for example, spelling, word use and syntax; while at a more advanced level, it involves monitoring higher-level features of the text produced such as the development of arguments, relevance to and adequacy for the task set (Shaw & Weir, 2007).
Another essential process in completing a reading-to-write task is the revising process, which is highly connected with the process of monitoring and may be conducted at any stage in reading-to-write. When revising, writers return to aspects of the text identified as unsatisfactory and make corrections or adjustments (Shaw & Weir, 2007). Although these aspects identified may not all be revised, “it is very unlikely that revising occurs without monitoring (Chan, 2013: p. 71).” There are two levels of revising, each corresponding to one of the two levels of monitoring process: at the basic level, writers make revisions of issues relating to textual features, for example, spelling and word use; at the advanced level, writers deal with issues such as the development of arguments, coherence, and cohesion of the text.
Many studies have compared the use of revising process between skilled and unskilled writers, finding that skilled writers are more proficient in revising than their counterparts (Graham & Harris, 1996, 2000; Perl, 1979). Perl (1979) found that writers who adopted a knowledge-transforming approach engaged more often in revising writing goals and main ideas of the text. In contrast, novice writers devoted much less attention to revising in writing, and, when revising, they were more likely to revise lower-level features of the text, for example, correcting spelling errors and making small changes in wording (MacArther, Graham, & Harris, 2004).
Most of the previous process studies used self-report methods in which participants are asked to report their cognitive processing activities either concurrently (for example, think-aloud protocols) or retrospectively (for example, retrospective interviews). One major concern of using concurrent self-report methods is the extent of reactivity and potential disruption imposed on test-takers’ actual cognitive processes (Stratman & Hamp-Lyons, 1994). This issue needs to be considered carefully in particular when the test examined is highly demanding in cognitive resources (for example, reading-to-write tasks that involve integration of at least two skills). Retrospective self-reporting methods do not interfere with participants’ actual processes, however, issues such as memory decay and over-reporting may also be detrimental to the accuracy of data collected (Harwood, 2009).
These two types of self-report methods rely largely on participants’ perceptions of their cognitive processes, and on their ability to report or recall the processes (Smagorinsky, 1994). Meanwhile, as there are time costs in collecting and analysing think-aloud or interview protocols, a relatively small number of participants are usually involved in these studies (questionnaire is also a kind of self-reporting technique that can be used in large-scale studies). Other researchers have investigated test-takers’ cognitive processes by using direct observation methods such as video recording (Bosher, 1998), and screen capture software (Chan, 2011). These studies allow participants to focus on their actual cognitive processing, with minimum interruption. However, observations are essentially an “etic” method (based on the researcher’s interpretation of what he/she observes), and if it is not triangulated with participants’ perceptions of their cognitive processing then important information may be lost.
There are pros and cons of using each method mentioned above independently to look at writers’ cognitive processes in reading-to-write, however, studies that combined these methods have been scarce. To fill this gap in the literature, it was decided to use a combination of eye-tracking, stimulated recall methods, and questionnaires to investigate EFL writers’ monitoring and revising processes in responding to a reading-to-write task, with the goal of triangulating data from different sources and learning more about the nature of these processes.
Two research questions were proposed:
1) How do EFL writers monitor the quality of their writings as they read and write? (RQ1)
2) How do EFL writers make revisions of their writings as they read and write? (RQ2)
2. Methods
This research study includes two independent studies: Study I, the eye-tracking and stimulated recall study, and Study II, the questionnaire study. The detailed plans for data collection and analyses of these two data sources are presented in the following sections.
2.1. Participants
16 participants took part in Study I. They were all native Chinese master’s students enrolled at a university in the UK. They were between the ages of 21 and 28 years old, with 11 females (69 percent) and 5 males (31 percent). All of them had taken the International English Language Testing System (IELTS) test before the data of Study I was collected. Their test scores are shown in Table 1. According to the Common European Framework of Reference for Languages (CEFR), these individuals had English proficiency levels ranging from B2 to C1.
![]()
Table 1. Participants’ IELTS test scores.
Study II involved a total of 172 undergraduate students. They were all native Chinese students studying at two public universities in China. Of the participants, 120 were from University A and, at the time of data collection, were in their second year of study; 52 students were from University B and were in the third year of their study. They ranged in age from 20 to 21 years old. 91.4 percent of these participants were female and 8.6 percent were male. Their English proficiency levels were estimated to be between CEFR B2 and C1 (based on their scores on the TEM-4 test, i.e., Test for English Majors-Band four, a national test for English majors in China; for those who had not sat the TEM-4 test, their scores on the end-of-term English test were referenced).
2.2. Instrument and Data Collection
In Study I, the 16 participants’ eye movements were tracked and recorded by a Tobii TX300 eye-tracker while they completed a sample reading-to-write task from the Test for Business English Majors-Band 8 (TBEM-8, developed and administered in China). The task prompt consists of a task instruction and five source materials about Steve Jobs’ resignation from Apple. On the eye-tracker screen, the task was divided into two sections and presented in each half of the screen: the instruction and three of the five source texts were on the left half, while the other two source materials, as well as the answer box (where the participants typed the words) were on the right half. The participants were first asked to complete the reading-to-write task, and then they were instructed to verbalise their thoughts during task completion using the eye traces recorded by the eye-tracker as stimuli for retrospection. The stimulated recall session was done in Mandarin Chinese, with audio and video recordings made for data analysis.
In Study II, a reading-to-write process questionnaire developed by Chan (2013) was administered to another 172 participants. In University A, 120 students first took the TBEM-reading-to-write task as a mid-term classroom test for an English academic writing course. The task was delivered via the computer in a multi-media classroom on campus. Immediately after the participants had completed the task, the questionnaire was used to prompt them to report the extent to which they employed different types of cognitive processes throughout task completion. A total of 120 questionnaires were collected from these participants. In University B, the same reading-to-write task was integrated into an end-of-term English test for a course of integrated English as its writing component to assess the participants’ reading-to-write abilities. This test was administered through the traditional paper and pencil method. It lasted about 120 minutes. After the participants finished the test, they were asked to complete the questionnaire. 52 responded questionnaires were collected from these participants.
2.3. Analysis
In Study I, the participants’ verbal reports were first transcribed by one of the researchers (a native Mandarin speaker). The transcriptions were then coded to identify participants’ monitoring and revising processes as they completed the task (see Table 2 for the working definitions and examples of these processes), and the number of instances of each type of these processes/subprocesses was calculated. Excerpts from the participants’ verbal protocols were also presented to show how they applied these processes to complete the reading-to-write task.
In Study II, 172 questionnaires were collected from the participants. Two of the questionnaires were discarded because of insufficient completion (more than 10 items were left unresponded to), and the remaining 170 valid questionnaires were submitted to SPSS for further statistical analyses. In order to understand the extent to which participants employed the specified cognitive processes while completing the reading-to-write task, a frequency analysis was performed to know the percentage of participants choosing each number (1 to 5) for each question, and the agreement rate for each question was calculated by adding up the percentage of those who agreed and strongly agreed.
![]()
Table 2. Working definitions and examples of monitoring and revising processes.
3. Results and Discussion
3.1. Study I
The participants reported 270 instances of monitoring process during and after writing. First, at a basic level, 155 instances of monitoring processes were found. The participants reported that they engaged in low-level monitoring at different stages of the text production process, either during or after writing a word, a sentence, or a paragraph, or even the first draft. The protocol of Participant 8 provides some good examples of this level of analysis, for example, when thinking about what to write in the next sentence, she said:
“Most of the time, when I finished writing a sentence, I would go back to read the previous sentences to check if there were any grammar mistakes or any more content to be added into them”, and “here I went to reread the first part of this sentence, and I thought the sentence structure was not appropriate, so I decided to delete it and changed to another way of expression.”
One thing to note that is that it was not uncommon for the participants to refer back to the source materials when monitoring (this may be a difference of this process between independent and integrated writing tasks), during which they sought clarification of certain words or information, for example, Participant 4 reported that he went back to source materials to check if he spelt the word “resignation” right. Also, the participants were found to monitor the word count of their essays during writing, for example, Participant 10 recalled that, “I read through the whole essay, and counted the number of words in it, because the instructions say that it is required to write 250 - 280 words, so I checked the word count.”
Second, 115 instances of monitoring process were conducted at a more advanced level. For example, Participant 1, when writing the third paragraph of her essay, went back to reread what she had written in the previous paragraph, and claimed that “I wanted to make sure what I wrote in this paragraph was different from that in the previous one”, and Participant 8 stated that “I was wondering if I can add any more points into the writing, because I thought the opinions were not adequately described, so I wanted to add more analysis to…” It should be noted that, in over half (66 instances) of the 115 instances, the participants reported that they referred back to the instructions to check if what they had written was relevant to the task set. This indicates that these participants paid considerable attention to this aspect of task fulfilment, and it may be because they were trained to do so, as Participant 12 said in her protocols, “Do not go off the topic, this is what I learned from writing Chinese essays. If we go off the topic, there might be a problem, so I kept looking back to the instructions during writing to make sure I was on the right track.” Another function of high-level monitoring that can be found in the participants’ protocols was to examine the cohesion and coherence of the text produced, for example, Participant 1 stated that:
“I was reading through the paragraphs I had written, I was not checking the grammar points, but thinking about the connection between what I was going to write and the previous paragraphs, how can I write it in a logic way and make these two parts coherent…”
As discussed in the introduction, the high cognitive demands on the translation process may hinder the execution of other processes, very likely including the process of monitoring. Field (2004) therefore argued that during the actual process of text production, writers’ attention might be given to the lower-level features such as the accuracy of spelling, word use and syntax, but the higher-level features such as relevance to and adequacy for the task set are more likely to be monitored at a post-production stage. This is in line with the findings in this study with regard to where the instances of monitoring processes occurred, that the participants were found conducting the low-level monitoring literally at any stage of text construction, either during or after writing a word, a sentence, a paragraph, or a first draft, however, the high-level monitoring mostly happened when the participants had just finished writing a sentence or a paragraph (9 instances of high-level monitoring were found after the completion of a first draft). The 270 instances were also further analysed to differentiate the use of monitoring process between different writing phases. The results show that participants engaged in the two levels of monitoring process (low-level monitoring: 32 instances; high-level monitoring: 9 instances) after writing much less frequently than they did during writing (229 instances). This agrees with the results in eye-tracking data that the participants spent, on average, limited time going back to look at what they had written after they finished the first draft of their essays.
111 instances of the revising process were found in the stimulated recalls, most of which were conducted at the basic level (92 instances), for example, Participant 8 deleted the word she wrote and changed it to another one to avoid repetition, “I found that I had already used ‘however’ in the previous sentence, so I deleted ‘however’ and changed it to another word”, and Participant 3 stated that he “deleted this sentence, and put the adverb clause of time in the front, and then added a declarative sentence to make a simple description.” Nineteen instances of revising at the advanced level were found in the protocols, whose number was much less than that of the corresponding monitoring process (115 instances). Ten participants reported that they engaged in this level of analysis, for instance, Participant 10, when writing the first paragraph, stated that, “at this time I was looking at the materials, and I found some important information that I had not described about, so I decided to add it into the first paragraph.”
3.2. Study II
3.2.1. Low-Level Editing
Six items (see Table 3) were designed to measure the process of low-level editing. It is through this process that participants check the accuracy of spelling, punctuation and syntax, etc. Results of the frequency analysis are presented in Table 3. It should be noted that, as the items about participants’ editing (monitoring and revising) processes were in the fourth and fifth sections of the
![]()
Table 3. Agreement with items measuring the process of low-level editing.
questionnaire, the item numbers remained unchanged as they were in the original questionnaire.
Items 4.13, 4.14 and 4.15 are related to the participants’ low-level editing processes while writing the first draft. More than 80 percent of participants checked that they had put the ideas drawn from the source materials into their own words (Item 4.13), indicating that most of them had a good understanding of the task and integrated information from the materials into their own writing. For Item 4.14, the percentage of agreement fell to 59.6, and about 30 percent of participants claimed that they had no idea whether they checked the grammatical accuracy and range of the sentence structures during writing. This suggests that the participants did not necessarily think about the grammatical range and accuracy of their texts as they were writing. Perhaps they were more focused on getting ideas into the text and writing something coherent. A slightly higher level of agreement was found in Item 4.15 where 69.4 percent of participants reported that they checked the spelling, usage and range of the vocabulary whilst writing.
Items 5.6, 5.7 and 5.8 investigate the participants’ low-level editing process after finishing the first draft. For Item 5.6, the figure for agreement rate remains stable, with 81.9 percent of participants agreeing that they checked if they had put ideas from the source materials into their own words. The proportion of agreement in Items 5.7 and 5.8 increased to 71.5 and 75.0 percent, indicating that the participants engaged more often in low-level editing process after they had finished the first draft than they did whilst writing. Again, it is interesting to note that the participants seemed to be a bit more focused on using the source material and integrating it well than they were with achieving formal accuracy in their writing. This suggests that they may prioritise the use of selecting and connecting processes (particularly during writing) in completing an integrated writing task.
3.2.2. High-Level Editing
Ten items were designed to measure the process of high-level editing. At this level of editing, the concern is mainly with the extent to which the text produced so far fits in with participants’ writing goals established in the previous stages, its relevance to the task set and the development of the structure of the text. Results of frequency analysis are presented in Table 4.
Items 4.8 to 4.12 look at the participants’ high-level editing process while writing the first draft. Overall, it seems that editing at both levels exhibit similar extent of agreement (the levels of agreement are all roughly around 60 to 80 percent). The highest levels of agreement were found in Items 4.11 and 4.12, with 81.8 percent of participants claiming that they did check if they included all appropriate main ideas from all the source materials (Item 4.11) and 83.4 percent of participants reporting that they checked if they included their own viewpoint on the topic (Item 4.12). Some slightly lower levels of agreement were found in Items 4.8 to 4.10. 76.2 percent of participants checked if the content of their essays was relevant (Item 4.8) and 70.6 percent checked if the essays were well-organised (Item 4.9). For Item 4.10, 67.5 percent of participants checked the coherence of the essays. It looks like task achievement (e.g., relevance of content, inclusion of viewpoint) was more strongly endorsed than more linguistic foci (e.g. checking topic sentences, connectives).
Items 5.1 to 5.5 investigate the participants’ high-level editing process after finishing the first draft. The proportion of agreement in these items all increased compared with items 4.8 to 4.12. Item 5.5 had the largest increase that 91.0
![]()
Table 4. Agreement with items measuring the process of high-level editing.
percent of participants reported that they had checked if they included their own opinions on the topic. It may be safely concluded that a relatively large number of participants were well aware of the importance of editing at both the basic and advanced levels, and the participants seemed to engage more in the high-level editing process than the low-level editing process, and edit more often after finishing the first draft than whilst writing.
4. Conclusion
There is ample evidence of the use of monitoring and revising processes at either a basic level or a more advanced level in participants’ stimulated recalls and the questionnaire data. One notable difference between the findings from these two sources of data is that: in the eye-tracking and stimulated recall study, most of the reported monitoring processes occurred during writing; while the questionnaire data showed that the participants monitored more often after they finished the first draft than whilst writing. This may be due to the fact that participants (Study I) who were eye-tracked while completing the TBEM-8 reading-to-write task may have spent more time accommodating themselves to the equipment they were working with (e.g., the eye-tracker and the keyboard) than those who (Study II) took the test in a normal classroom setting, and so they (Study I) tended to spend more time on the task and had relatively less time devoted to monitoring after completing the draft.
Another interesting finding in the questionnaire study was that the participants seemed to be more focused on using the source material and integrating it well than they were on achieving formal accuracy in their writing; in other words, task achievement was more strongly endorsed than more linguistic foci. This suggests that the participants may prioritise the use of connecting and other relevant processes such as monitoring in completing an integrated writing task.
The study’s major outcome is a better understanding of how EFL writers monitor the quality of their writings, as well as how they make appropriate revisions or adjustments when completing a reading-to-write task, with which EFL teachers may adapt or improve their lesson plans to better teach monitoring and revising processes in reading-to-write instruction. Furthermore, there are some methodological implications in that a combination of eye-tracking techniques, stimulated recall methods, and questionnaires may provide rich data from participants and provide a solid foundation from which conclusions about their cognitive processing as they read and write can be made. This way of triangulating data among different qualitative and quantitative research methods could also be adopted in test validation studies to improve the validity of data and results.