Pupils ’ Thinking Skills Development across Grade 4-6 : An Investigation of 2096 Pupils in Mainland China Based on APTS

A variety of thinking skills interventions have been implemented in schools and relative assessments emerged. However, due to inconsistencies of assessment techniques and lack of norms from large-scale samples, it remains problematic to compare the effects of various thinking interventions in general. This study aimed to investigate the current situation of thinking skills of 2096 pupils in 4, 5, and 6 grade from six primary schools located in Beijing, Guangzhou, and Xi’an respectively. The “Assessment of Pupils’ Thinking Skills (APTS)” measure developed by Burke and Williams (2012) was translated into Chinese and used as the instrument. Results demonstrated that there were significant improvements in the pupils’ overall thinking skills from 4 to 5 grade and from 5 to 6 grade as well. However, the pupils’ metacognitive reflection did not improve significantly from 4 grade to 5 grade while they increased dramatically from 5 to 6 grade. The pupils’ definition of thinking skills and application of some thinking skills (i.e., Grouping, Finding Reasons and Conclusions, Decision Making and Problem Solving) showed the same trends as metacognitive reflection. Differentiations in thinking skills development were found when compared among schools. Reasons for these differentiations and implications for teaching thinking in primary schools were discussed.

Pupils (age 6 -12) are at critical stages of cognitive development, ranging from Preoperational stage, Concrete Operational stage to Formal Operational stage (Piaget, 1964;Salkind, 2008).At this period, teaching thinking may be critical and most rewarding.Since pupils' thinking skills develop rapidly during this period, it is necessary to provide a relatively objective reference of their development levels across different ages.With such references or norms, certain comparisons will be feasible for researchers, especially for those who have difficulties in finding control groups.Moreover, all thinking skills interventions should be designed in accord with pupils' thinking levels and characteristics.
Thus, a relatively large-scale investigation on pupils' thinking skills, which could provide an insight into pupils' cognitive development and/or serve as a baseline for intervention and comparison, would be necessary and valuable.
In the next three sections, this paper gave a brief literature review from three perspectives: frameworks of thinking skills, assessment of pupils' thinking skills, teaching thinking in Mainland China and its implication for educational equilibrium.Based on the literature review, research questions for this study were proposed.

Frameworks of Thinking Skills
There is no doubt that teaching of thinking aims to improve students' higher order thinking skills rather than simple and rote memorization.However, as to what higher order thinking skills are, different theorists have different opinions.
Bloom identified six fundamental hierarchical cognitive objectives, in which the top three levels (Analyzing, Evaluating, and Creating) are generally regarded as higher order thinking skills (Anderson et al., 2001).Lewis and Smith (1993) defined higher order thinking as the thinking "which occurs when a person takes new information and information stored in memory and interrelates and/ or rearranges and extends this information to achieve a purpose or find possible answers in perplexing situations."They listed some higher thinking skills, which include deciding what to believe, deciding what to do, creating a new idea, etc.
Another attempt was to define core thinking skills.Marzano (2001) identified a detailed list of core thinking skills which are defining problems, setting goals, observing, formulating questions, encoding, recalling, comparing, classifying, ordering, representing, identifying attributes and components, identifying relations and patterns, identifying main ideas, identifying errors, inferring, predicting, elaborating, summarizing, restructuring, establishing criteria, and verifying.
By conducting a meta-analysis of 55 frameworks, Moseley, Elliott, Gregson and Higgins (2005) devised a two-tier framework for learning and teaching thinking.This two-tier model distinguishes strategic/reflective thinking (i.e.engagement with and management of thinking) from cognitive skills (i.e.information gathering, building understanding and productive thinking).Moseley et al. (2005) used the terms "strategic and reflective thinking" here to reflect awareness and control not only of cognitive processes, but also of related motivation and affect.
Though differences exist in scope and in emphasis among theoretical frameworks for understanding thinking during the last half-century, some important common fundamental thinking capacities have been identified.These capacities are core thinking skills, critical thinking, creative thinking, problem solving, decision making and meta-cognitive processes across them (Burke & Williams, 2008).
As explained above, the concepts of thinking skills, core thinking skills, higher order thinking skills, creative thinking, critical thinking, and meta-cognition are highly overlapping.In this study, we only distinguished meta-cognition from thinking skills and consider critical thinking, creative thinking and decision making as some sort of thinking skills.For each thinking skill, there is a corresponding meta-cognitive reflection.
Based on the thinking frameworks of Swartz and Parks (1994) and McGuinness et al. (2007), Burke andWilliams (2008, 2012) designed the "Assessment of Pupils' Thinking Skills (APTS)" for pupils among 9 to 12-year-olds.APTS measures six thinking skills and corresponding meta-cognitive reflections comprehensively.In order to weaken the respective limitations of the multiplechoice tests and the open-ended tests, the APTS used a combination of these two formats.
The APTS measure is suitable for whole class testing (Burke & Williams, 2012).Although articles that introduced the APTS measure had been cited more than 20 times, few studies showed that the measure had been used to investigate a relatively large sample of pupils.It remained a problem to get an overall developmental data of pupils' thinking skills based on the APTS, and it was still unavailable to compare effects of thinking skills inventions in general.years.Most research on educational equilibrium focused on gender, urban-rural and interscholastic differences in terms of educational opportunities, public educational resources allocation, education quality and educational achievement (Zhai & Sun, 2012).Literatures show that there has been a continuous development in equilibrium progress of basic education in China in terms of education opportunities, the distribution of educational resources, the educational quality and the educational attainment (Zhai & Sun, 2012).

Teaching Thinking in
In fact, apart from differences among schools, differences also existed within schools.Controlling the balance within schools is more practical and feasible for principals.The development of pupils' thinking skills is an important part of educational achievement.However, few studies concerned the differences among or within schools in terms of the development of pupils' thinking skills.

The Present Study
This study was part of a larger study funded by MOE (Ministry of Education in China) as a project of Humanities and Social Sciences.The funded larger study was intended to promote educational equilibrium among primary schools through constructing a community of practice on teaching of thinking.In 2014, the community of practice, Alliance of Thinking Schools (ATS), was sponsored by the project team.Though most teachers and students were excited about their growth resulted from the teaching of thinking, obstacles always existed when they tried to ascertain the starting points or evaluate the effects of establishing their thinking skills interventions.
In order to get informed of the overall situation of the pupils' thinking skills and to provide a relatively objective baseline for comparisons, this study investigated 2096 pupils in 4 th , 5 th and 6 th grade from six primary schools in ATS.This study also aimed to explore the differences in pupils' thinking skills development among schools.
The key research questions of this study are: • What was the overall situation of the pupils' thinking skills in these six primary schools?How did pupils' thinking skills developed over grades (i.e., 4 th , 5 th and 6 th )?
• Were there any differences in the pupils' thinking skills among schools?

Participants and Context
The participants of this study were 2096 pupils (850 4 th grade students, 676 5 th grade students and 570 6 th grade students) from six mainstream state-run schools in Beijing, Guangzhou and Xi'an, which located in North, South, and Northwest China respectively (Table 1).All of these six schools were members of the ATS.Since we were going to establish a thinking skills curriculum for students from 4 th to 6 th grades in these six schools, we chose students who were going to take thinking lessons as participants in this study.Before the tests, five of these six schools had not given any thinking skills interventions to their students.One school (#2) in Beijing had taught thinking skills fragmentarily in a school-based curriculum, the main intervention materials were Mind Mapping invented by Buzan and Buzan (1996) and five thinking tools (i.e., PMI, CAF, C&S,FIP, RULES) from CoRT1 designed by De Bono (1983).
According to education policies in China, children under six years old are not allowed to go to primary school.For the school year started on September 1 every year, the ages of pupils in 1 st grade range from 6 years old to 6 years and 11 months old, and the average is 6.5 years old when they enter schools in September.As our investigation was carried out in March, which is the beginning of the second semester in the school year, the average ages of the pupils in 4 th , 5 th and 6 th grades were 10, 11 and 12 respectively.

Instrument
The instrument used in this study was the Chinese version of the APTS measure developed by Burke and Williams (2012) and translated by the research group.
APTS can be used to investigate pupils' thinking skills development with large-scale participants and/or assess effects of thinking skills interventions by monitoring changes in thinking skills over time among 9 to 12-year-olds (Burke & Williams, 2008, 2012).
Six specific thinking skills were incorporated into the APTS (Table 2) (Burke & Williams, 2008, 2012).In the APTS, the respondents are required to define the thinking skills and identify examples of the skills being used.Furthermore, the respondents are required to answer questions assessing how they apply the thinking skills and corresponding meta-cognitive reflection questions to identify

Testing Procedures
The investigation was carried out in March 2014, the beginning of the spring semester (or the second semester) in the 2013-2014 school year.As suggested by Burke & Williams (2012), the tests were conducted in the pupils' classrooms within 60 minutes.In the first five minutes, the printed questionnaires were handed out to the respondents, and respondents were asked to fill out some basic information, such as grades and classes.After that, the questionnaire was read aloud to the respondents by the testers in the classroom.Pupils who had difficulties in comprehending the items could ask for help.

Scoring and Data Analysis
Six junior or senior undergraduate students majored in Educational Technology at Beijing Normal University were trained by the researchers to rate pupils' response papers according to the scoring matrix designed in the APTS (Burke & Williams, 2008, 2012).Each response paper was rated by two of these raters (Rater A and Rater B) independently.A third rater (Rater C) checked the consistency of the scores given by Rater A and Rater B for each item.If the differences were within 1.0, the averages will be used as the final scores.Otherwise, Rater C would re-read the papers and gave the scores synthesizing the scores given by Rater A and Rater B. After finishing rating all the response papers, data were calculated with Excel and analyzed with SPSS18.0.

Reliabilities of the Scoring
To verify the reliabilities of the scoring by Rater A and Rater B, inter-judge reliabilities were calculated for each item using the Pearson product-moment correlation coefficient.It was found that the reliabilities ranged from 0.74 to 1.00 (Table 3).For item #1 and item #1, the inter-judge reliabilities were very high for these two items adopted multiple choice response formats.The reliabilities of others were relatively lower due to they were in the format of open-ended questions.So, a third rater was adopted to improve the accuracy and objectivity of scoring.

Thinking Skills Definition, Application and Metacognitive Reflection
To present an overall situation of the pupils' thinking skills from 4 th to 6 th grade, the average of D_TS, A_TS,M_TS and the total were calculated (Figure 1).
A one-way between-grades ANOVA was conducted to explore whether there were differences in scores of D_TS, A_TS, M_TS and total among grades.Results showed that statistically significant differences existed in all four conditions (Table 4).
Post-hoc comparisons using the Tamhane T2 test highlighted that the means of the total and A_TS of 5 th grade were significantly higher than those of 4 th grade were.Similarly, the means of the total and A_TS of 6 th grade were significantly higher than those of 5 th grade.
Table 3.The inter-judge reliability for each item of APTS.For D_TS and M_TS, post-hoc comparisons indicated that statistically significant differences existed between 4 th grade and 6 th grade, and between 5 th grade and 6 th grade as well.However, no statistically significant differences existed between 4 th grade and 5 th grade.

Individual Thinking Skills
The APTS was broken down to analyze the differences in individual thinking skills among grades.A one-way between-grade ANOVA conducted to identify these differences showed a statistically significant difference in the mean of each skill among grades (see Table 5).Post-hoc comparisons showed that the skill CC and CUI significantly improved over grades.However, for the skill GRP, FRC, DM and PS, there was significant growth from 5 th grade to 6 th grade, while no significant gains were found from 4 th grade to 5 th grade (Table 5).

Meta-Cognitive Reflections on Individual Thinking Skills
The differences in meta-cognitive reflection on individual thinking skills among grades were also analyzed through a one-way between-grades ANOVA.Results showed statistically significant differences in the mean of all six metacognition skills among grades (Table 6).Post-hoc comparisons showed that the skill M_CC significantly improved over grades.However, for the skill M_GRP, M_CUI, M_FRC, M_ DM, and M_PS, though there were significant growthfrom 5 th grade to 6 th grade, no significant gains were found from 4 th to 5 th grade, and some meta-cognitive reflections (i.e., M_GPR, M_DM, and M_PS) even dropped slightly in this period (Table 6).

Differences in Pupils' Thinking Skills among Schools
To discover differentiations in the pupils' thinking skills development among schools, the total scores, D_TS, A_TS and M_TS of each grade were analyzed  through a one-way between-schools ANOVA.For 4 th grade and 6 th grade, results showed statistically significant differences in the mean of the total, D_TS, A_TS and M_TS among all involved schools (Table 7).For 5 th grade, results showed that there were significant differences in the mean of the total, D_TS, and M_TS among these schools, while there were no statistically significant differences in the mean of A_TS among these schools.

School-Characteristics of Pupils' Thinking Skills Development over Grades
In order to explore how pupils' thinking skills developed over grades within schools dynamically, one-way between-grades ANOVAs for five schools except school #1 were conducted respectively.School #1 was excluded for only students at 4 th grade and 5 th grade attended the investigation.Results showed that for all five schools, significant differences existed among all three grades (Table 8).
For school #2, #3 and #6, post-hoc comparisons highlighted that the mean scores of 6 th grade were significantly higher than those of 4 th grade and those of 5 th grade, while there were no statistically significant differences between 4 th grade and 5 th grade.
For school #4, post-hoc comparisons highlighted that the mean score of 5 th grade was significantly higher than that of 4 th grade, while no statistically significant differences existed between 4 th grade and 6 th grade or between 5 th grade and 6 th grade.
For school #5, post-hoc comparisons highlighted that the mean score of 5 th grade was significantly higher than that of 4 th grade, and the mean score of 6 th grade was significantly higher than that of 5 th grade.
To make schools' impacts on students' thinking skills development over grades more explicit, line graphs were drawn to show dynamic trends of thinking skills development from 4 th to 6 th grade for school #2, #3, #4, #5 and #6 (Figure 2).It was obvious that these schools differed a lot from each other.For school #2, #3 and #5, students' thinking skills developed over grades.However, for school #6, a slight decline was found from 4 th grade to 5 th grade.More surprisingly, for school #4, there was a significant decline from 5 th grade to 6 th grade.

Discussion
Firstly, we will discuss the characteristics of the pupils' thinking skills development from 4 th to 6 th grade, and then we will try to discuss the implication for teaching thinking in primary schools and educational equilibrium from the perspective of thinking skills development.

The Pupils' Overall Performance in Thinking Skills
Considering the full score is 72, the pupils' performances in the test were relatively low to some extent.Take 6 th grade for example, the average score (M = 28.96,S.D. = 5.15) did not reach half of the full score (i.e., 36.00),let alone the   and reflections should be blamed for this first.When pupils were asked to describe their thinking processes reflectively, most of them were at a loss.Education in China was widely scolded for its rote indoctrination and insufficient emphasis on students' thinking processes.These results sent a signal that we should provide students with more space and time to think reflectively and to describe their thinking process more explicitly in daily teaching and learning.
In terms of individual thinking skills, students performed best in "Grouping" (M = 2.64, S.D. = 0.68 for 4 th grade; M = 2.67, S.D. = 0.67 for 5 th grade; M = 2.81, S.D. = 0.65 for 6 th grade) and worst in "Coming up with ideas" (M = 1.73,S.D. = 0.62 for 4 th grade; M = 1.83,S.D. = 0.62 for 5 th grade; M = 1.95,S.D. = 0.68 for 6 th grade).These findings were consistent with the characteristics of education in China, which focused much more on absorbing knowledge than discovering something new.As a result, knowledge on "What, Where, Who and When" were well taught in classes and knowledge about "How to generate new ideas" were usually ignored.For "Coming up with ideas" is an essential element of creative thinking, this finding urged that the focus of teaching should be shifted from rote reception to meaningful construction.

Development of Thinking Skills over Grades
Overall, pupils' thinking skills developed dramatically over grades (or ages).
However, the gain between 4 th and 5 th grade was much lower than that between 5 th and 6 th grade.This could be explained by Piaget's theory of cognitive development.Piaget suggested that there are four stages of cognitive development, including Sensorimotor stage(age 0 to 2), Preoperational stage(age 2 to 7), Concrete Operational stage(roughly ages 7 to 11) and Formal Operational stage (roughly ages 11 to approximately 15 -20) (Oakley, 2004;Piaget, 1964;Salkind, 2008).According to Piaget's theory, pupils in 4 th grade (average age = 10) and 5 th grade (average age = 11) were in their final phase of the Concrete Operational Stage, during which their abstract thinking was not developed apparently.Relatively, more pupils in 6 th grade (average age = 12) had entered their early Formal Operational Stage, in which they were advancing from logical reasoning with concrete examples to logical reasoning with abstract examples (Kuhn, 1979;Salkind, 2008).The measure used in the study tested six thinking skills, in most of which abstract thinking or reasoning with abstract examples was required.
Therefore, few gains appeared from 4 th grade to 5 th grade while significant development occurred from 5 th grade to 6 th grade.

Implications for Teaching Thinking in Primary Schools
The results enhanced the necessity and feasibility of teaching of thinking in 4 th , 5 th and 6 th grade.For one thing, the fact that the pupils from School #2 outperformed others demonstrated that teaching of thinking, even fragmentally, was helpful to some extent.For another, the overall relatively lower performance in thinking skills development urged the importance to take effective measures.
Moreover, the results verified that students in 4 th grade, 5 th grade and 6 th grade were in a critical period for thinking development.More specifically, the period from 5 th grade to 6 th grade is more critical than that from 4 th grade to 5 th grade, since there was much more development during the former period.This also reminds us that thinking skills interventions aimed to improve meta-cognitive reflection and abstract thinking may be too early to be accepted by students at lower grades.For example, it might be not advisable to teach Mind Mapping, Concept Mapping or CoRT for students in grade 1 -3 in primary schools.

Thinking Skills Differentiations among Schools
Though similar distributions and trends of thinking skills development were found among these six schools, significant differentiations also emerged.This might result from different school cultures, teachers' conception of teaching or students' family background.Take school #2 for example, students in 6 th grade performed much better than their counterparts in other five schools despite their urban-rural background.In fact, some thinking skills interventions (Mind Mapping and several tools from CoRT1) had been taught fragmentarily in the school since March 2013, which was one year ahead of our investigation.This was one possible reason why students at some grades there performed much better than students at the same grades in other schools did.
The performance of students in 5 th grade from school #4 surprised us most not only for its leading place among these schools, but also for it was higher than that of 6 th grade from the same school.However, the principal of this school was not surprised at all.She explained that teachers of this grade worked very hard and adopted latest educational ideas actively.
Though the differentiations among and/or within schools were comprehensive effects of many factors, it was schools' administration and teachers' teaching, other than economic situation or schools' geographic positions, that played a more decisive role in pupils' thinking skills development.

Implications for Educational Equilibrium
The results of this research revealed various differences of thinking skills development existed among schools.This reminds educators and administrators to pay more attention to educational equilibrium from the perspective of educational outcomes, especially students' thinking development, rather than only in terms of educational opportunities and resources allocation.

Limitations of the Study and Future Research
This study revealed a relative low improvement of students from 4 th grade to 5 th grade and implied that it might be not advisable to teach students under 4 th grade how to think abstractly or reflectively.However, there was not enough empirical evidence for this conclusion.It will be a useful attempt to teach thinking skills at across 1 st grade to 3 rd grade and then observe how students' performances on thinking improve over time.This sort of attempt will make it more clearly in terms of which types of thinking skills interventions to use in lower and middle grades in primary schools.
This study indicated that the gain in meta-cognition and some thinking skills from 4 th grade to 5 th grade was relatively less than that from 5 th grade to 6 th grade.
In order to acquire more details of thinking skills development during this period, it would be worthwhile to reexamine these differences in a qualitative way with the aids of interviews or case studies etc.It would also be interesting and valuable to verify which kind of interventions could be applied to improve these thinking skills, especially meta-cognitive reflections, and to what extent pupils' thinking skills could be improved.
Moreover, as pointed by Burke and Williams (2008), APTS captured thinking skills of students while ignoring other important aspects of thinking, such as thinking dispositions.Future studies could devise or adopt other measures to capture a broader scope of thinking.

Figure 2 .
Figure 2. Thinking skills development over grades in each school.
gested that differentiations existed in pupils' thinking skills development among schools.To explore the characteristics of the pupils' thinking skills development, this study made an important contribution to broaden the scope of application of APTS in Chinese condition, and provided a relative objective baseline for examining effects of various thinking skills interventions and for comparing

Mainland China and Its Implication for Educational Equilibrium
tional opportunities, in which the balanced development of compulsory education is a top priority."Studies on educational equilibrium development have boosted in recent 10

Table 2 .
The structure of APTS.

Table 4 .
Mean performance of all three grades on D_TS, A_TS, M_TS and total.

Table 5 .
Mean performance on individual thinking skills in three grades.

Table 6 .
Mean performance on meta-cognitive refection on individual thinking skills in three grades.

Table 7 .
Mean performance of all six schools on the APTS.

Table 8 .
Mean performance of total scores of all three grades at each school.