Mapping the Knowledge Domain of Meta-Analysis Used in SLA Research: A Scientometric Review (2000-2022)

Abstract

The study examined intellectual domains of meta-analysis used in SLA research based on Chinese core journals and English peer-reviewed journals from 2000 to 2022. Two datasets were created via CSSCI and Web of Science (WoS): 1) a Chinese dataset of core Linguistic and educational journals comprising 50 articles, and 2) a WoS dataset of 167 articles published in the reputed journals within registers of Linguistics and educational research. It employed document co-citation analysis (DCA) and author co-citation analysis (ACA) to capture the underlying intellectual structure and characterize distinct clusters in the WoS dataset. Co-word analysis and burst detection were adopted to discern the research trends, frontiers, and hotspots displayed in the two datasets. Results show that the WoS dataset focuses on the research topics: L2 instruction, corrective feedback, computer-mediated communication, self-motivation, and L2 writing with papers about procedures or amendments to meta-analysis as the knowledge base. The author intellectual groups categorized under specific contexts demonstrate a profound influence. Keywords captured in the WoS dataset can be mainly grouped into three types: research subjects, research topics, and terminologies relevant to meta-analysis. In contrast, research topics and terms related to meta-analysis are the dominant lexical chunks in the Chinese dataset. The emerging research spots, individual factors, language achievements, and knowledge maps provide directions for future research.

Share and Cite:

Xu, F. and Cao, R. (2022) Mapping the Knowledge Domain of Meta-Analysis Used in SLA Research: A Scientometric Review (2000-2022). Open Journal of Social Sciences, 10, 506-528. doi: 10.4236/jss.2022.1011032.

1. Introduction

Second language acquisition (SLA) research established itself as a field of inquiry in the 1960s. The early studies highlighted the universal properties of mental mechanisms, gradually directed to address individual differences, and of late, explored issues pertaining to language pedagogy (Ellis, 1999). The beginning of the 20th century witnessed the increased momentum of L2 research being submitted to meta-analysis, partly motivated by the exponential growth of research on language learning and teaching (Norris & Ortega, 2006). Initially, meta-analysis was used to investigate the “effectiveness of L2 instruction” in the classroom context (Norris & Ortega, 2000). After that, it has sprawled to other subfields of the SLA, such as corrective feedback, learning strategies, motivations, anxieties, and so on. In China, the journal on meta-analysis used in SLA research first appeared in the core journal literature in 2008, which briefly introduced the procedures of conducting meta-analysis and the international research status. Of late, meta-analysis gradually received attention in Chinese SLA research and proved to be a useful method to deal with controversial points in the field. It affords to synthesize the primary research and displays a tremendous statistical power based on summarized larger sample sizes, capable of identifying patterns among the network of study outcomes and compensating for the single studies insufficiency (Plonsky & Oswald, 2012).

Some scholars conducted comparative analyses based on Chinese and international reputed journals and captured the knowledge structures, dynamic development, and research hotspots via scientometric mapping within the SLA knowledge domain across different temporal spans. Xu et al. (2020) mapped the recent 10-year studies of L2 writing published in Chinese and international prestigious journals and concluded that the L2 writing research in China and abroad is diverse in knowledge structures and demonstrates a convergence trend, focusing on writing feedback, writing assessment, academic English writing teaching, etc. Yang & Wang (2022) analyzed the literature on classroom assessment in foreign language teaching published in key international and Chinese journals from 2008 to 2020 and found that Chinese studies highlight the assessment effect, whereas overseas research underlines the assessment process. Chinese studies follow the pace of their international counterparts but slightly fall behind in assessment literacy and focused classroom assessment research. Ren et al. (2020) visualized Chinese and foreign articles concerned with language testing and figured out that the research fields in China are relatively broad, emphasizing the largescale test. In contrast, foreign research weighs on testing students on one specific skill.

To date, no study has been set out to explore the intellectual structure of meta-analysis used in SLA research and pick out the discrepancies between foreign and Chinese core journals. The purpose of the current study is to visually illustrate the intellectual structure of meta-analysis within the SLA research domain, based on peer-reviewed English journals, and depict the research frontiers and hotspots in China and abroad from 2000-2022. The visualization software, Citespace 5.3 is employed to detect the underlying intellectual structure that is interwoven by clusters denoting knowledge groups, research themes, and the dynamic growth of the field across temporal slices (Goswami & Agrawal, 2019). In addition, analytical methods entailing co-citation analysis, co-word analysis, and burst detection are applied to meet the target. The research contributes to SLA research by probing into the underlying knowledge base, the research frontiers, and the emerging research topics so as to provide new ideas and directions for SLA research in China and abroad.

2. Method

Citespace 5.3 supports several types of bibliometric studies, and in this study, the authors focus on document co-citation analysis (DCA), author co-citation analysis (ACA), and co-word analysis between 2000 and 2022.

DCA shows the relation between cited references. A co-citation link can be established by citing two documents in one article (Sooryamoorthy, 2021). Chen (2006) put, “Documents clustered by their co-citation links can represent leading specialties. The abrupt disappearance and emergence of such document clusters indicate rapid shifts in research focus” (p. 149). ACA focuses on interrelationships among influential authors in the literature. Author co-citation is a more rigorous grouping principle because the connectivity is based on repeated and collective views of subject experts expressed in their publications (White & McCain, 1998). As Chen (2003) notes, “With the availability of both ACA and DCA, it becomes possible to compare and contrast messages conveyed through different co-citation networks of the same topic as if we were wearing two pairs of glasses” (p. 151). ACA offers a new perspective to the discovery of knowledge structures in parallel to views of DCA. Co-word analysis produces a network of co-occurrences between different words amassed from the citing publications. Scientific research structure and dynamics can be mapped using this method (Sooryamoorthy, 2021).

2.1. The Procedures of Data Collection

The study examines the literature over the period 2000-2022 to identify the network structure of influential research topics and leading authors involved in the evolution of meta-analysis’s application in SLA research. The procedures of data collection are summed up in Figure 1. A representative dataset of the published literature is firstly generated, and specialized software is subsequently applied to mine and extract the hidden structures in the data (Chen, 2016). The documents are elicited from two databases, CSSCI and Web of Science, representing the mainstream research status and having a high impact worldwide. Keywords “second language research”, “SLA”, and “Meta-analysis” are used to retrieve the research journals within registers of linguistics and education. In order to guarantee the quality of index publications, among all publication types available,

Figure 1. The research procedures.

only those classified as research journals or reviews were selected; others, such as conference papers, notes, and dissertations, were excluded.

A total of 217 bibliographic records of original research articles and reviews were retrieved. The two datasets are created: 1) a Chinese core journal dataset consisting of 50 articles published in Foreign Language World, Modern Foreign Languages, and Foreign Languages and Their Teaching, Foreign Language Education in China, Journal of Xian International Studies University, Journal of Foreign Languages, Foreign Language Research, Foreign Language Learning Theory and Practice, Foreign Languages in China, Language Teaching and Linguistic Studies, and Educational Science Research, and so on., and 2) a WoS dataset comprising 167 prestigious journals, generated from Studies in Second Language Acquisition, Studies in Second Language Learning and Teaching, Recall and Language Teaching Research, Modern Language Journal, Language Learning Technology, Journal of Research in Reading, Journal of Pragmatics, Foreign Language Annuls, Journal of Research in Applied Linguistics and so on.

2.2. Data Analysis

The authors employed a co-citation method that comprises DCA and ACA.to detect the intellectual structure of meta-analysis used in SLA research published in international prestigious linguistic journals. The co-citation network can be decomposed into various clusters of tightly coupled references or authors to embody a topic of research. After that, these clusters are by terms extracted from the titles of the most representative citing papers for each cluster (Chen et al., 2014a). Citespace can identify highly cited references and authors that acted as pivotal transition points within and among research clusters (Chen, 2004).

The research frontiers can be analyzed regarding the references within the co-citation network and keywords generated by citing papers (Chen et al., 2014b). Co-word analysis permits the researcher to collect the static and the dynamic aspects of a particular discipline (Tijssen & Raan, 1989). The emergence of new keywords in the recent period provides the basis on which the authors suggest further research directions (Luc et al., 2020). Research frontiers in China and abroad will be identified by co-word analysis. Burst detection can identify events with much higher frequencies than other events (Kleinberg, 2003). Keyword Burstness reveals research hotspots and their influential period across a particular temporal slice, and the burst strength indicates the quantified magnitude of its influence.

3. Results and Discussions

3.1. Visualizing Document Co-Citation Network

The document co-citation (DCA) technique was adopted to measure the frequency of references co-cited together in the citing papers. The cluster of co-cited documents is considered to represent the knowledge base of a specialty (Small, 1977). CiteSpace generated the network model by synthesizing a time series of annual networks of co-cited references and divided the network into diverse clusters labeled by noun phrases extracted from citing articles’ titles.

The topologies incorporate the top 367 most cited articles woven by 996 links, sorted by one year per slice, and the top 6 clusters are identified, as demonstrated by Figure 2. Each node denotes a published article about information visualization, and each link denotes a co-citation relationship. The size of a node is proportional to the number of times the underlying article is cited by other articles (Chen, 2006: pp. 3-4). The co-cited duration is represented by nodal colors ranging from cooler colors to warmer colors indicating temporal variations

Figure 2. A landscape view of the document co-citation network.

(Chen, 2017). Changes in colors indicate the trends of research topics within the domain. The colder color cluster #0 computer-mediated communication and #1 second language research denotes the earliest time slice where the connection is made, whereas the warmer color #18 motivational self-system and #8 achievement represent the most recent burst. This DCA network has a high Q metric of 0.9042, suggesting a pretty well-structured network, and the average silhouette index is 0.9675, indicating the high homogeneity of the structures.

The cluster information is summarized in Table 1 and ranked by the log-likelihood ratio (LLR) that identifies cluster themes by indexing noun phrases in the abstracts of citing articles since the ranking algorithm displays unique aspects of the cluster (Chen et al., 2010) and is more precise at identifying cluster themes (Aryadoust & Ang, 2019).

3.2. Indexes for Characterizing the Identified Clusters

The citing and cited publications constitute the linked concepts that might result in the emergence of each cluster (Aryadoust et al., 2020). Each cluster will be characterized by the content of the citing and cited articles, which would be measured by different indexes.

The citing publications are measured by the coverage value and Global Citation Score (GCS). The coverage value is the number of references in the cluster

Table 1. Summary list of identified clusters.

that the citing article cited. The high coverage value entitles a high descriptive value of the cluster. GCS shows the total number of citations to a paper in the WoS database despite their inclusion in a connected component of a citation network. Papers with high GCS are recognized as seminal or influential papers in the body of knowledge (Knoke & Yang, 2008). GCS is able to identify the papers that represent the basis of a field (Strozzi et al., 2017).

The cited articles are measured via burst strength, denoting the quantified magnitude of influence. Journals with high burst strength value prove to be the pillar of the cluster-embodied domain. Zhou et al. (2019) note, “A research cluster containing a certain quantity of articles with citation burst can be considered as a new research field” (p. 2317). Bursts can determine the predominant features and serve as an essential index for the knowledge base in every cluster. The frequency value will be taken into account in the absence of the burst value.

The citing articles with high coverage value or GCS and the cited references with burst strength or high-frequency value will be selected for narrative literary reviews to detect the knowledge structure of the research domain. It should be noted that the citing or cited publications’ titles are abbreviated as “the author’s surname plus the publication year” in narration. Full titles will be provided on the condition that the author and the publication year are duplicated.

3.3. Characterizing the Detected Clusters

Figure 3 informs that Cluster # 0 computer-mediated communication contains the top 5 citing articles ranked by coverage value and GCS: Plonsky (2014) “How big is ‘big’ interpreting effect sizes in L2 research”, Liu & Brown (2015), Plonsky (2014) “Study quality in quantitative L2 research (1990-2010) a methodological synthesis and call for reform”, Plonsky & Gonulalb (2015), and Junkyu (2015). In addition, Figure 3 depicts those five cited references in the dataset have a

Figure 3. The list of Cluster #0 citing publications.

burst strength: Oswald & Plonsky (2010) with the burst strength of 4.84, Li (2010) with the burst strength 0f 4.84, Plonsky & Gass (2011) with the burst strength of 3.64, Plonsky (2013) with the burst strength of 3.64, Plonsky (2014) with the burst strength of 3.30.

The citing publications all discuss methodologies used in SLA research, in which statistical analysis and research synthesis are counted as the primary concern. In light of contents, the articles can be roughly divided into two major groups: 1) the procedures of conducting meta-analysis and interpreting the research findings, and 2) the limitations of research synthesis originating from the practical research. Plonsky & Oswald (2014) pick out the deficiencies of the traditional effect sizes, essentially defaulted to Cohen’s three levels, and propose a field-specific scale for interpreting effect sizes that may be affected by the potential moderators: sample size, time point, contexts, publication bias, the variety of statistical methods (Plonsky, 2014; Plonsky & Gonulal, 2015). They provide a basic operational framework for doing meta-analysis within the SLA domain. The practical difficulties derive from the insufficient information offered by the research papers, involving inadequate reporting of the research context, methodology and statistical analysis, designed issues, mixed application of instruments, and a wide array of outcome accuracy measures (Liu & Brown, 2015). The citing papers call for reforming research synthesis or meta-analysis and their application to investigate methodologies in the SLA field.

As shown in Figure 4, the bursts of references in this cluster are categorized into two aspects: 1) suggestions for complementing the methodology of meta-analysis, and 2) assessment of methodologies in SLA research. A series of methodological moves can be taken to improve the weakness of meta-analysis. The moves involve refining the inclusion/exclusion criteria, using both fixed and random effect models, controlling sample size inflation, shifting away from Cohen’s benchmarks, using major substantive indicators of publication bias, and interpreting results in the context (Plonsky & Oswald, 2010; Li, 2010). Study outcome is correlated with methodological quality affected by means-based analysis, missing data, and design preferences (Plonsky & Gass, 2011; Plonsky, 2013). Moreover, the effect of corrective, explicit, and implicit feedback is explored in this cluster.

Cluster #1 second language research captures relatively fewer seminal literature or highly influential publications compared with Cluster #0. Figure 5 delineates that Oswald & Plonsky (2010) and Bowles (2010) have distinct coverage values and GCS, which are essential to describing the clusters’ features. The frequency value is used as the measurement, for no bursts are detected. The journal, Abraham (2008), is most frequently cited compared with other articles within the cluster, as shown in Figure 6. Cited journals tie for second place with a frequency value of 2, comprising Borenstein et al. (2009), Henson (2006), Lee & Huang (2008), Mackey (2014).

The citing publications depicted in Figure 6 mainly provide suggestions for conducting meta-analysis in SLA research, whereas the cited articles are concerned with two aspects: 1) generic textbooks or journals that present the

Figure 4. The list of Cluster #0 cited publications.

Figure 5. The list of Cluster #1 citing publications.

progression of methods in SLA or meta-analysis in general, and 2) factors affecting L2 learners’ reading comprehension. Mackey (2014) notes, “Corpus techniques have the potential to take center stage SLA, and a new direction for future research involves the integration of burgeoning computational tools into SLA methodologies” (pp. 89-90). Computer-mediated vocabulary or grammar

Figure 6. The list of Cluster #1cited publications.

learning shows a medium effect on learners’ reading comprehension (Abraham, 2008). However, visual input enhancement (VIE) has a negative effect on readers’ meaning processing (Lee & Huang, 2008).

Figure 7 summarizes three citing papers constituting the knowledge base of Cluster #3 empirical research. Ranked by coverage value, the three articles are Lin (2015) “A meta-synthesis of empirical research on the effectiveness of computer-mediated communication”, Lin (2015) “Computer-mediated communication in L2 oral proficiency development: A meta-analysis”, and Varela (2017). Figure 8 presents the top 5 of the most frequently cited references: Grqurovic et al. (2013), Alastuey (2011), Yun (2011), Taylor (2013), and Alastuey (2011).

Cluster #3 narrows down two themes: computer-mediated language learning and oral proficiency. The citing and cited publications focus on the two themes, and study outcomes show that computer-supported language learning produced a moderately positive effect on L2 learners’ oral proficiency compared to face-to-face communication (Lin, 2015; Alastuey, 2011). Moreover, CALL L1 glossing can effectively improve L2 vocablary acquisition (Taylor, 2013; Yun, 2011). This cluster delineates the sub-field of oral performance within the SLA domain.

Merely one citing article is displayed in Figure 9. Its GCS indicates that the paper may not contain groundbreaking ideas. However, the coverage value is relatively moderate, suggesting that the article can act as the knowledge structure of Cluster #8 achievement. Figure 10 demonstrates three cited references occupying the first position: Teimouri et al. (2019), Brown et al. (2018), and Al-Hoorie (2018).

Figure 7. The list of Cluster #3 citing publications.

Figure 8. The list of Cluster #3 cited publications.

Figure 9. The list of Cluster #8 citing publications.

Figure 10. The list of Cluster #8 cited publications.

Cluster #8 centers around a particular dimension of calculating effect size-correlation, and the effect of the individuals’ emotions on L2 performance, such as self-efficacy, anxiety, and motivation. Goetze & Driver (2022) conclude, “Self-efficacy has a medium-sized, positive relationship with L2 achievement, and suggest that learning success may indeed be a question of believing in one’s abilities during the learning process” (p. 248). As for motivation, the three components of the L2 motivational self-system, the ideal L2 self, the ought to L2 self, and the L2 learning experience, is significant predictors of intended subjective effort, though weaker predictors of objective measures of achievement (Al-Hoorie, 2018). The overall correlation between anxiety and achievement is moderate negative with the possible moderator: different types of language achievement measures, educational levels, target languages, and anxiety types (Teimouri et al., 2019).

The same is with Cluster # 14 L2 writing where merely one citing publications are qualified for the pillar in this research domain, as shown in Figure 11. The cited references within this category contain fewer documents. Figure 12 unfolds the top 5 articles: Van Beuninge et al. (2012), Bitchener & Knoch (2010a), Evans et al. (2011), Bitchener & Knoch (2010b), and Hartshorn et al. (2010).

Cluster #14 points to the research area of corrective feedback (CF) and L2 writing. Truscott (1996) claims that corrective feedback is ineffective, harmful, and should therefore be abandoned. However, the longitudinal studies demonstrate a positive effect of CF on L2 writing, and the earlier comparative studies show conflicting findings between direct and indirect corrective feedback (Bitchener & Knoch, 2010a; Liu & Brown, 2015). Van Beuningen et al. (2012) find that direct CF contributes to pupils’ grammatical accuracy gains in new writing and that the nongrammatical accuracy benefits most from indirect CF, suggesting that comprehensive written corrective feedback (WCF) is a useful tool that teachers can use to help L2 learners improve their written accuracy across time. WCF proves a positive impact on students’ linguistic accuracy in the context of EFL or L2 students enrolled in an intensive English program (Evans et al., 2011;

Figure 11. The list of Cluster #14 citing publications.

Figure 12. The list of Cluster #14 cited publications.

Hartshorn et al., 2010). Research on WCF proves several methodological limitations, reporting inadequacy, ecological invalidity, and insufficient interpretation of the statistical data, presenting obstacles for meta-analytic research (Liu & Brown, 2015), which can act as moderate variables when operating meta-analysis in future studies.

Figure 13 reveals that Cluster # 18 motivational self-system incorporating one citing paper with a moderate GCS and coverage value and Figure 14 offers six cited papers, among which Al-Hoorie’s articles are essential in this cluster, which is concerned with individual motivations. The L2 motivational self-system (MSM) consists of the ideal L2 self, the ought-to L2 self, and the L2 learning experience. The three components of the L2MSS had positive correlations with intended effort but dropped with achievement. Via meta-analysis, Al-Hoorie (2018) explains that some conflicting results in the literature may originate from the different measures researchers adopted in their studies, where subjective measures show more substantial support for the L2MSS than objective measures. Flipped learning, where learners preside over the classroom and monitor the procedures, proves a clear and substantial effect on most language outcomes. The higher the learners’ proficiency, the higher the effects of flipped learning (Vitta & Al-Hoorie, 2020). The cluster centers around the effect of individual differences.

3.4. Visualizing Author Co-Citation Network

The landscape view represents the author-correlation network, in which the most influential scholars in the knowledge domain appear near the intellectual structure’s center. In contrast, researchers with unique expertise tend to appear in peripheral areas (Chen, 2016). A total of 12 clusters are identified and labeled by LLR, and the most cited authors in each cluster are generalized in Figure 15. The largest five clusters represent the top 5 intellectual groups. Cluster #0 quantitative L2 research consists of 54 members with the most relevant citer being Ziegler; Cluster #1 universal grammar comprises 46 members with Tsai Yu-Ling’s

Figure 13. The list of Cluster #18 citing publications.

Figure 14. The list of Cluster #18 cited publications.

Figure 15. A landscape view of the author co-citation network.

contributions as the basic idea; Cluster #2 technology-supported language has 35 members led by Grgurovic et al. (2013); Cluster #3 gender difference includes 34 members headed by Albert; Cluster #5 computer-mediated communication possesses 27 members with Huifen’s article published in 2015 demonstrates the highest relevance.

The timeline view puts clusters of authors on discrete horizontal axes. Clusters are arranged in a vertical manner descending in size, with the largest cluster at the top (Aryadoust et al., 2020). The colored tree rings denote highly cited authors, and the colored lines are co-citation links pointing to intellectual groups.

Figure 16 presents the timeline visualization revealing three periods of dominant scholars. The first period is from 2000 to 2003. This period was relatively uneventful, with few academicians involved in the research themes concerned with universal grammar, research synthesis, and English. The pioneering academicians are Bangert, Cadierno, Alanen, Chaudron, Andersen, Bresnan, and Dulay. The period of 2003-2016 is full of high-impact scholars’ contributions, especially in the top 6 clusters, as shown by large tree rings. The highly influential scholars include Lipsey, Blis, Norris, Cooper, Hedges, Rosenthal, Cohen, Dornye, Plonsky, and so on. The third period is from 2016-2022. Although no highly impactful academicians are detected in this period, the intellectual groups that thrived in this period shed light on the recent development of the specialty. The top 4 intellectual groups remain vigorous in this period, and the groups characterized by reading strategies demonstrate a new direction. The representative scholars are Revesz, Sagarra, Dixonl, Marsdene, Abbas, Butler, Koda, and so on.

Figure 16. A timeline view of the author’s co-citation network.

3.5. Co-Word Analysis and Burst Detection

Keywords have the potential for effectively describing the contents of a paper and are indicative of the actual partitions of interrelated concepts in the literature (Khasseh et al., 2017). The two datasets are processed using the co-word algorithm to produce a word co-occurrence network. The co-word algorithm first builds networks that can identify areas of intense focus and then identify terms associated with more than one network, thereby indicating overlapping issues (Delecroix & Eppstein, 2004). The nodes represent keywords, and the colored links show their connections. Each cluster formed by the nodes connected by the internal links depicts a particular concept, and the nodes tied by external links across clusters indicate the overlapping part of the two concepts.

Figure 17 shows that clusters are tightly connected and dispersed in all directions, with Cluster #2 2nd language acquisition as the center. Every cluster comprises dense nodes and links, suggesting that the constituted concepts are pretty complete. The keywords and terms can be classified into three categories: subjects, sub-fields within SLA research, and terms related to meta-analysis. The subjects include children, students, and learners; The research fields cover instruction, feedback, academic achievement, phonological awareness, psychometrics, second language pragmatics, and computer-mediated communication; Effect size is the term relevant to meta-analysis.

Few clusters with relatively sparse nodes and links are identified in Figure 18. The connections between clusters are loose, indicating fewer overlapping concepts. Keywords or terms in the Chinese dataset are primarily concerned with

Figure 17. A landscape view of the co-word network generated by the WoS dataset.

Figure 18. A landscape view of the co-word network generated by the Chinese dataset.

sub-fields of SLA research, comprising L2 writing, L2 reading, L2 instruction corrective feedback, and working memory. The terminologies related to meta-analysis cover effect size, moderate, quantitative, and qualitative analysis.

In comparison, the two datasets commonly highlight L2 instruction, reading comprehension, phonological awareness, accuracy, and effect size. However, the Chinese dataset may lay more emphasis on L2 writing and tend to adopt moderate analysis in studies. Conversely, the WoS dataset underlines learners’ academic achievement and the interpretation of effect sizes.

Different keywords with bursts represent the corresponding characteristics in each period (Wei & Zhang, 2020). Bursts of the keywords reveal the research hotspots in different periods. Burst detection will make it possible to find the keywords that receive particular attention in a certain period, and burst strength indicates the red line shows the magnitude of impact and burst duration.

Via burst detection, the burst of keywords in Figure 19 presents clear trends that can be classified into three periods: 2000-2010, 2010-2016, and 2018-2022. During the first period, “2nd language acquisition”, “bilingualism”, and “negotiation” have a robust burst strength, characterizing the research themes of this period. Figure 6 shows that “L2 instruction” becomes the mainstream research topic, followed by “corrective feedback” in the second period, indicating that scholars focus on exploring the effect of teachers’ role in SLA research. The third period witnesses the citation bursts of keywords: “students”, “reading comprehension”, “achievement”, “performance”, and “efl”, and their influences still last to date, indicating the potential research directions and research hotspots. The bursts of those keywords illustrate that the researchers direct to highlight the effect of individual differences on their L2 performance or achievements, implying

Figure 19. Burst detection of keywords and terms in the WoS dataset.

Figure 20. Burst detection of keywords and terms in the Chinese dataset.

that future studies can explore the effect of personal factors, such as self-efficacy, motivations, language aptitude, and learning strategies, on students’ L2 proficiency.

With the same γ vale, the Chinese dataset produces fewer keyword bursts, and the first burst keyword, “Jiu Cuo Fan Kui (corrective feedback)”, started in 2012 and demonstrated an excellent burst strength and a long influential period. Figure 20 indicates the terms with robust burst strength: “Fu Za Xing (complexity)”, “Liu Li Xing (fluency)”, and “Zhun Que Xing (accuracy)”. The collocations of “complexity” includes “task complexity”, “lexical complexity”, and “complex adaptive system”. Fluency is matched with “oral”, and accuracy with “grammar”. Those terms depict the research themes: L2 instruction, L2 speaking, and L2 writing. However, each keyword thrives within an average one-year period, and “Zhi Shi Tu Pu (knowledge map)” is estimated to be a new research spot.

4. Conclusion

The article provides a scientometric review of meta-analysis used in SLA research by Citespace based on two datasets elicited from Web of Science and CSSCI from 2000-2022. Co-citation analysis comprising DCA and ACA is employed to map the underlying intellectual structures. As for DCA, the integration of citation and co-citation patterns provides a rich, ecological representation of a knowledge domain (Chen, 2016). According to DCA, the primary research domains of the WoS dataset cover L2 instruction, corrective feedback, computer-mediated communication, self-motivation, and L2 writing. Articles concerned with reflections on research synthesis or meta-analysis procedures prove to be the knowledge base of research domains. To grasp the big picture of a knowledge domain, ACA is adopted to generate intellectual groups sensitive to contexts. The top 5 influential intellectual groups are identified, and the leading authors, such as Lipsey, Blis, Norris, Rosenthal, Cohen, Dornye, and Plonsky, demonstrate a profound impact on specific concepts. ACA provides a promising augmentation to existing document- and concept-centered approaches to knowledge visualization (Chen, 2003).

The research fronts, trends, and hotspots are captured via co-word analysis and burst detection. The keywords in the WoS dataset include the research subjects: children, learners, and students, research topics: L2 instruction, feedback, computer-mediated communication, academic achievement, phonological awareness, psychometrics, and second language pragmatics, and terminologies relevant to meta-analysis-effect sizes. Via burst detection, those research themes are put on a time shaft. Three periods are identified: 2000-2010, 2010-2016, and 2018-2022. The first period is dominated by research topics: bilingualism, grammar correction, negotiation, and SLA. L2 instruction receives great attention in the second period, apart from corrective feedback. The burst of reading comprehension and students’ performance or L2 achievement depicts the research fronts and hotspots. In contrast, keywords marked in the Chinese dataset are categorized into research themes and terms related to meta-analysis. Specifically, Chinese scholars prefer to research reading comprehension and L2 writing and rely on moderate analysis when conducting meta-analysis. The burst started in 2012, and most of the burst keywords have a short influential duration, except for corrective feedback and L2 writing. “Knowledge map” is estimated to be a new research spot.

The findings indicate a discrepancy between the two core datasets. The new research topic, for instance, the effect of individual factors on L2 achievements, and the knowledge map provide a new research direction for researchers. Future studies can employ meta-analysis to investigate the relationship between individual emotions, self-efficacy, learning strategies, motivations, language aptitude, and the learners’ L2 achievement or to explore the relationship between those elements.

Fund

This research was funded by Beijing Forestry University education and research project #BJFU2019JY074; the grant of New Liberal Arts Research and Reform Project from the Ministry of Education in China (No. 2021070015).

Author Contributions

Fang Xu designed the research, analyzed the data, and prepared the first draft of the paper. Rongping Cao, as the corresponding author checked the data and revised the draft. All authors approved the final draft.

Conflicts of Interest

The authors declare no conflicts of interest regarding the publication of this paper.

References

[1] Abraham, L. B. (2008). Computer-Mediated Glosses in Second Language Reading Comprehension and Vocabulary Learning: A Meta-Analysis. Computer Assisted Language Learning, 21, 199-226.
https://doi.org/10.1080/09588220802090246
[2] Alastuey, M. C. B. (2011). Perceived Benefits and Drawbacks of Synchronous Voice-Based Computer-Mediated Communication in the Foreign Language Classroom. Computer Assisted Language Learning, 24, 419-432.
https://doi.org/10.1080/09588221.2011.574639
[3] Al-Hoorie, A. H. (2018). The L2 Motivational Self System: A Meta-Analysis. Studies in Second Language Learning and Teaching, 8, 721-754.
https://doi.org/10.14746/ssllt.2018.8.4.2
[4] Aryadoust, V., & Ang, B. H. (2019). Exploring the Frontiers of Eye Tracking Research in Language Studies: A Novel Co-Citation Scientometric Review. Computer Assisted Language Learning, 34, 898-933.
[5] Aryadoust, V., Zakaria, A., Lim, H. M., & Chen, C. (2020). An Extensive Knowledge Mapping Review of Measurement and Validity in Language Assessment and SLA Research. Frontiers in Psychology, 11, Article No. 1941.
https://doi.org/10.3389/fpsyg.2020.01941
[6] Bitchener, J., & Knoch, U. (2010a). Raising the Linguistic Accuracy Level of Advanced L2 Writers with Written Corrective Feedback. Journal of Second Language Writing, 19, 207-217.
https://doi.org/10.1016/j.jslw.2010.10.002
[7] Bitchener, J., & Knoch, U. (2010b). The Contribution of Written Corrective Feedback to Language Development: A Ten Month Investigation. Applied Linguistics, 31, 193-214.
https://doi.org/10.1093/applin/amp016
[8] Borenstein, M., Hedges, L. V., Higgins, J. P., & Rothstein, H. R. (2009). Introduction to Meta-Analysis. John Wiley & Sons, Ltd.
https://doi.org/10.1002/9780470743386
[9] Bowles, M. A. (2010). Concurrent Verbal Reports in Second Language Acquisition Research. Annual Review of Applied Linguistics, 30, 111-127.
https://doi.org/10.1017/S0267190510000036
[10] Brown, A. V., Plonsky, L., & Teimouri, Y. (2018). The Use of Course Grades as Metrics in L2 Research: A Systematic Review. Foreign Language Annals, 51, 763-778.
https://doi.org/10.1111/flan.12370
[11] Chen, C. (2003). Mapping Scientific Frontiers: The Quest for Knowledge Visualization. Springer.
https://doi.org/10.1007/978-1-4471-0051-5
[12] Chen, C. (2004). Searching for Intellectual Turning points: Progressive Knowledge Domain Visualization. The National Academy of Sciences of the United States of America, 101, 5303-5310.
https://doi.org/10.1073/pnas.0307513100
[13] Chen, C. (2006). Information Visualization: Beyond the Horizon (2nd Ed.). Springer.
[14] Chen, C. (2016). CiteSpace: A Practical Guide for Mapping Scientific Literature. Nova Science Publishers.
[15] Chen, C. (2017). Science Mapping: A Systematic Review of the Literature. Journal of Data and Information Science, 2, 1-40.
https://doi.org/10.1515/jdis-2017-0006
[16] Chen, C., Dubin, R., & Kim, M. C. (2014a). Emerging Trends and New Developments in Regenerative Medicine: A Scientometric Update (2000-2014). Expert Opinion on Biological Therapy, 14, 1295-1317.
https://doi.org/10.1517/14712598.2014.920813
[17] Chen, C., Ibekwe-SanJuan, F., & Hou, J. (2010). The Structure and Dynamics of Co-Citation Clusters: A Multiple-Perspective Co-Citation Analysis. Journal of the American Society for Information Science and Technology, 61, 1386-1409.
https://doi.org/10.1002/asi.21309
[18] Chen, Y., Chen, C., Hu, Z., & Wang, X. (2014b). Principles and Applications of Analyzing a Citation Space. Science Press.
[19] Delecroix, B., & Eppstein, R. (2004). Co-Word Analysis for the Non-Scientific Information Example of Reuters Business Briefings. Data Science Journal, 3, 80-87.
https://doi.org/10.2481/dsj.3.80
[20] Ellis, R. (1999). The Study of Second Language Acquisition. Shanghai Foreign Language Education Press.
[21] Evans, N. W., Hartshorn, K. J., & Strong-Krause, D. (2011). The Efficacy of Dynamic Written Corrective Feedback for University-Matriculated ESL Learners. System, 39, 229-239.
https://doi.org/10.1016/j.system.2011.04.012
[22] Goetze, J., & Driver, M. (2022). Is Learning Really Just Believing? A Meta-Analysis of Self-Efficacy and Achievement in SLA. Studies in Second Language Learning and Teaching, 12, 233-259.
https://doi.org/10.14746/ssllt.2022.12.2.4
[23] Goswami, A. K., & Agrawal, R. K. (2019). Building Intellectual Structure of Knowledge Sharing. Journal of the American Society for Information Science and Technology, 50, 136-162.
https://doi.org/10.1108/VJIKMS-03-2019-0036
[24] Grgurovic, M., Chapelle, C. A., & Shelley, M. C. (2013). A Meta-Analysis of Effecti-veness Studies on Computer Technology: Supported Language Learning. ReCALL, 25, 165-198.
https://doi.org/10.1017/S0958344013000013
[25] Hartshorn, K. J., Evans, N. W., Merrill, P. F., Sudweeks, R. R., Strong-Krause, D., & Anderson, N. J. (2010). Effects of Dynamic Corrective Feedback on ESL Writing Accuracy. TESOL Quarterly, 44, 84-109.
https://doi.org/10.5054/tq.2010.213781
[26] Henson, R. K. (2006). Effect-Size Measures and Meta-Analytic Thinking in Counse-ling Psychology Research. The Counseling Psychology, 34, 601-629.
https://doi.org/10.1177/0011000005283558
[27] Khasseh, A. A., Soheili, F., Moghaddam, H., & Chelak, A. M. (2017). Intellectual Structure of Knowledge in IMetrics: A Co-Word Analysis. Information Processing and Management, 53, 705-720.
https://doi.org/10.1016/j.ipm.2017.02.001
[28] Kleinberg, J. (2003). Bursty and Hierarchical Structure in Streams. Data Mining & Knowledge Discovery, 7, 373-397.
https://doi.org/10.1023/A:1024940629314
[29] Knoke, D., & Yang, S. (2008). Social Network Analysis (2nd ed.). Sage.
https://doi.org/10.4135/9781412985864
[30] Lee, S. K., & Huang, H. T. (2008). Visual Input Enhancement and Grammar Learning: A Meta-Analytic Review. Studies in Second Language Acquisition, 30, 307-331.
https://doi.org/10.1017/S0272263108080479
[31] Li, S. (2010). The Effectiveness of Corrective Feedback in SLA: A Meta-Analysis. Language Learning 60, 309-365.
https://doi.org/10.1111/j.1467-9922.2010.00561.x
[32] Lin, H. (2015). Computer-Mediated Communication (CMC) in L2 Oral Proficiency Development: A Meta-Analysis. ReCALL, 27, 261-287.
https://doi.org/10.1017/S095834401400041X
[33] Liu, Q., & Brown, D. (2015). Methodological Synthesis of Research on the Effectiveness of Corrective Feedback in L2 Writing. Journal of Second Language Writing, 30, 66-81.
https://doi.org/10.1016/j.jslw.2015.08.011
[34] Luc, P. T., Lan, P. X., Le, A. N. H., & Trang, B. T. (2020). A Co-Citation and Co-Word Analysis of Social Entrepreneurship Research. Journal of Social Entrepreneurship.
https://doi.org/10.1080/19420676.2020.1782971
[35] Mackey, A. (2014). Practice and Progression in Second Language Research Methods. AILA Review, 27, 80-97.
https://doi.org/10.1075/aila.27.04mac
[36] Norris, J., & Ortega, L. (2000). Effectiveness of L2 Instruction: A Research Synthesis and Quantitative Meta-Analysis. Language Learning, 50, 417-528.
https://doi.org/10.1111/0023-8333.00136
[37] Norris, J., & Ortega, L. (2006). Synthesizing Research on Language Learning and Teaching. John Benjamins Publishing Company.
https://doi.org/10.1075/lllt.13
[38] Oswald, F. L., & Plonsky, L. (2010). Meta-Analysis in Second Language Research: Choices and Challenges. Annual Review of Applied Linguistics, 30, 85-110.
https://doi.org/10.1017/S0267190510000115
[39] Plonsky, L. (2013). Study Quality in SLA: An Assessment of Designs, Analyses, and Reporting Practices in Quantitative L2. Studies in Second Language Acquisition, 35, 655-687.
https://doi.org/10.1017/S0272263113000399
[40] Plonsky, L. (2014). Study Quality in Quantitative L2 Research (1999-2010): A Methodological Synthesis and Call for Reform. The Modern Language Journal, 98, 450-470.
https://doi.org/10.1111/j.1540-4781.2014.12058.x
[41] Plonsky, L., & Gass, S. (2011). Quantitative Research Methods, Study Quality, and Outcomes: The Case of Interaction Research. Language Learning, 61, 325-366.
https://doi.org/10.1111/j.1467-9922.2011.00640.x
[42] Plonsky, L., & Gonulalb, T. (2015). Methodological Synthesis in Quantitative L2 Research: A review of Reviews and A Case Study of Exploratory Factor Analysis Research. Language Learning, 65, 9-36.
https://doi.org/10.1111/lang.12111
[43] Plonsky, L., & Oswald, F. L. (2010). Meta-Analysis in Second Language Research: Choices and Challenges. Annual Review of Applied Linguistics, 30, 85-110.
https://doi.org/10.1017/S0267190510000115
[44] Plonsky, L., & Oswald, F. L. (2012). How to Do a Meta-Analysis. In A. Mackey, & S. M. Gass (Eds.), Research Methods in Second Language Acquisition: A Practical Guide (pp. 275-295). Blackwell Publishing Ltd.
https://doi.org/10.1002/9781444347340.ch14
[45] Plonsky, L., & Oswald, F. L. (2014). How Big Is “Big”? Interpreting Effect Sizes in L2 Research. Language Learning, 64, 878-912.
https://doi.org/10.1111/lang.12079
[46] Ren, W., Gao, M., & Wang, X. (2020). Research on Language Testing in China and Abroad: A dynamic visual analysis. Chinese Foreign Languages, 17, 77-86.
[47] Small, H. G. (1977). A Co-Citation Model of a Scientific Specialty: A Longitudinal Study of Collagen Research. Social Studies of Science, 7, 139-166.
https://doi.org/10.1177/030631277700700202
[48] Sooryamoorthy, R. (2021). Scientometrics for the Humanities and Social Sciences. Routledge.
https://doi.org/10.4324/9781003110415
[49] Strozzi, F., Colicchia, C., Creazza, A., & Noè, C. (2017). Literature Review on the ‘Smart Factory’ Concept Using Bibliometric Tools. International Journal of Production Research, 55, 6572-6591.
https://doi.org/10.1080/00207543.2017.1326643
[50] Taylor, A. (2013). CALL versus Paper: In Which Context Are L1 Glosses More Effective? CALICO Journal, 30, 63-68.
[51] Teimouri, Y., Goetze, J., & Plonsky, L. (2019). Second Language Anxiety and Achievement: A Meta-Analysis. Studies in Second Language Acquisition, 41, 363-387.
https://doi.org/10.1017/S0272263118000311
[52] Tijssen, R. J. W. & Van Raan, A. F. J. (1989). Mapping Co-Word Structures: A Comparison of Multidimensional Scaling and Leximappe. Scientometrics, 15, 283-295.
https://doi.org/10.1007/BF02017203
[53] Truscott, J. (1996) The Case against Grammar Correction in L2 Writing Classes. Language Learning, 46, 327-369.
https://doi.org/10.1111/j.1467-1770.1996.tb01238.x
[54] Van Beuningen, C. G. V., Jong, N. H. D., & Kuiken, F. (2012). Evidence on the Effectiveness of Comprehensive Error Correction in Second Language Writing. Language Learning, 62, 1-41.
https://doi.org/10.1111/j.1467-9922.2011.00674.x
[55] Varela, O. E. (2017). Learning Outcomes of Study-Abroad Programs: A Meta-Analysis. Academy of Management Learning & Education, 16, 1-73.
https://doi.org/10.5465/amle.2015.0250
[56] Vitta, J. P., & Al-Hoorie, A. H. (2020). The Flipped Classroom in Second Language Learning: A Meta-Analysis. Language Teaching Research, 1-25.
https://doi.org/10.1177/1362168820981403
[57] Wei, F., & Zhang, G. (2020). A Document Co-Citation Analysis Method for Investigating Emerging Trends and New Developments: A Case of Twenty-Four Leading Business Journals. Information Research, 25.
http://informationr.net/ir/25-1/paper842.html
[58] White, H., & McCain, K. (1998). Visualizing a Discipline: An Author Co-Citation Analysis of Information Science, 1972-1995. Journal of the American Society for Information Science, 49, 327-356.
[59] Xu, L., Zhu, X., & Liang, K. (2020). Knowledge Mapping and Trends of Second Language Writing Studies in China and Abroad. TEFLE, No. 3, 88-96.
[60] Yang, L., & Wang, T. (2020). A Visual Analysis of Research on Classroom Assessment in Foreign Language Teaching (2008-2020). Foreign Language Education in China, 5, 34-41.
[61] Yun, J. (2011). The Effects of Hypertext Glosses on L2 Vocabulary Acquisition: A Meta-Analysis. Computer Assisted Language Learning, 24, 39-58.
https://doi.org/10.1080/09588221.2010.523285
[62] Zhou, W., Chen, J., & Huang, Y. (2019). Co-Citation Analysis and Burst Detection on Financial Bubbles with Scientometrics Approach. Economic Research-Ekonomska Istrazivanja, 32, 2310-2328.
https://doi.org/10.1080/1331677X.2019.1645716

Copyright © 2024 by authors and Scientific Research Publishing Inc.

Creative Commons License

This work and the related PDF file are licensed under a Creative Commons Attribution 4.0 International License.