Mapping Knowledge Produced on Problem-Based Learning between 1945 and 2014: A Bibliometric Analysis

Abstract

Problem-based learning (PBL) has been systematized for over forty years. However, there is still a lack of studies that seek to visualize the evolution of scientific literature on the subject. This paper seeks to fill this gap through the development of a bibliometric analysis by carrying out a data search on the Web of Science? database and subsequently transferring the acquired data for treatment to the VantagePoint? software package. The data indicate that research on PBL shows a strong progression since the end of the twentieth century, with a high volume of the most impacting studies being carried out by a group of researchers who are currently linked to or used to be connected to the Maastricht University. As a result of these studies, a geographical (from North America and Europe to Asia) and academic (from research in medical education to other biomedical and engineering areas) spread of scientific production regarding this teaching methodology has arisen.

Share and Cite:

de Pinho, L. , Mota, F. , Conde, M. , Alves, L. and Lopes, R. (2015) Mapping Knowledge Produced on Problem-Based Learning between 1945 and 2014: A Bibliometric Analysis. Creative Education, 6, 576-584. doi: 10.4236/ce.2015.66057.

1. Introduction

Problem-based learning (PBL) was systematized in the medicine course at McMaster University, in Canada, in the late 1960s (Neville & Norman, 2006) . Since then, PBL represents an important change in professional health education, in which higher education institutions from different countries have begun new programs or restructured their existing courses based on PBL (Mamede, Schmidt, & Norman, 2006) . As an example, Brighton University, in England, maintains an Internet directory with information on courses organized as PBL throughout the world (http://feedback.bton.ac.uk/pbl/pbldirectory/index.php). In December 2014, the directory listed 118 courses in 103 institutions, from 35 countries, that apply PBL as a teaching strategy and curriculum-organizing structure.

In a problem-based learning experience, students encounter a problem situation. They ask: What do we know? What do we need to know? And, how can we find out? (Kain, 2003) . PBL instruction is constructed on ill- structured problems. According to Savery (2006) :

“PBL is an instructional (and curricular) learner-centered approach that empowers learners to conduct research, integrate theory and practice, and apply knowledge and skill to develop a viable solution to a defined problem. Critical to the success of the approach is the selection of ill-structured problems (often interdisciplinary) and a tutor who guides the learning process and conducts a thorough debriefing at the conclusion of the learning experience” (Savery, 2006: p. 12) .

In PBL, the problem becomes the vehicle for learning, and students work in small collaborative groups and learn what they need to know in order to solve a problem. The teacher acts as a facilitator to guide student learning through the learning cycle (Hmelo-Silver, 2004) .

“In this cycle, also known as the PBL tutorial process, the students are presented with a problem scenario. They formulate and analyze the problem by identifying the relevant facts from the scenario. This fact-iden- tification step helps students represent the problem. As students understand the problem better, they generate hypotheses about possible solutions. An important part of this cycle is identifying knowledge deficiencies relative to the problem. These knowledge deficiencies become what are known as the learning issues that students research during their self-directed learning (SDL). Following SDL, students apply their new knowledge and evaluate their hypotheses in light of what they have learned. At the completion of each problem, students reflect on the abstract knowledge gained. The teacher helps students learn the cognitive skills needed for problem solving and collaboration. Because students are self-directed, managing their learning goals and strategies to solve PBL’s ill-structured problems (those without a single correct solution), they also acquire the skills needed for lifelong learning” (Hmelo-Silver, 2004: pp. 236-237) .

Studies reveal that there is a positive effect from PBL on student skills. Highlighted among the benefits of applying PBL are the increase in the student’s ability to conduct research, integrate theory and practice, communicate, conduct group work, apply knowledge and skills to develop a viable solution to a problem and develop self-directed studies (Wood, 2003; Dochy et al., 2003; Rikers & Bruim, 2006; Mamede, Schmidt, & Norman, 2006; Hung, 2009; Lopes et al., 2011) .

Despite pronounced progress on the application and research regarding PBL, no bibliometric studies in the literature that examine scientific production on the subject on a broad time scale are available, with the exception of one bibliometric analysis by Xian and Madhavan (2013) regarding the contribution of Howard Barrows to PBL research. Barrows was an American physician who for over forty years has been a leading researcher and propagator on the use of PBL in medical education. In this context, the present study aims to answer some issues regarding PBL in a bibliometric context: i) observe the evolution of scientific production in this regard; ii) identify the contribution and impact of the citations of the most productive authors; iii) investigate the importance of collaborations with the most important authors for scientific production regarding the production of knowledge on PBL; and iv) identify and study the contribution of the journals that have published the most on the subject.

2. Methods

Research on PBL was conducted by a bibliometric analysis, observing the evolution of the scientific production on the subject. To this end, a database was generated with the results acquired by conducting a search on the Web of Science™ database (WoS), on February 12, 2014. The WoS database is a research platform that catalogues multidisciplinary scientific information on books, journals and patents, providing data from 15 other large databases (for more information, visit: http://thomsonreuters.com/thomson-reuters-web-of-science/). The search was conducted by selecting the option for all databases, with the following topic field terms: “problem based learning” (OR) “practice based learning” (OR) “problem based curriculum” (OR) “problem based curricula”, covering the whole time series available in the WoS database, between 1945 and 2014. The results were subsequently refined in the WoS directory itself, to return only the following document types: articles, reviews, editorials or letters. The search results were then imported into the Vantage Point© v. 8.0 program, a text-mining platform in which it is possible to acquire results regarding a certain database (for more information, visit: https://www.thevantagepoint.com/). When required, manual filtering of overlaid data regarding author names and institutions was conducted. Finally, a survey on the final information selected for analysis was conducted, as described below.

The following parameters were selected in the Vantage Point© program: number of publications (NP) by year; NP by type of publication; NP by country; NP by author; NP by journal and the number of times that an author or journal was cited in the acquired publications (C). The following were also observed: journal impact factors (IF-obtained from 2013 Journal Citation Reports); authors or journals according to the number of articles (P), the average number of citations per publication (CPP), proposed by Hsieh and collaborators (2004) , which is achieved by dividing the total number of citations by the total number of publications (C/P), the H index, which is the value in which h author publications have at least h citations (Hirsch, 2005) , and the m value, which is the h value divided by the number of years since the first publication of the author in question (Hirsch, 2005) . To exemplify, the CPP value gives a general idea of the impact of an author, institution, country or journal for a given area, while both the H index and the m value also provide this information, with emphasis, however, on the most cited productions (and possibly the productions with the most impact), with the latter also offering information regarding consistency in the production of papers of impact. When necessary, the value of the coefficient of determination (r²) between certain parameters, obtained with the Bio Estat v. 5.3 software package, was also determined. All figures, except for Figure 3, which was generated in PowerPoint 2010 (with data obtained from the Vantage Point© program), were generated by the Vantage Point© software package.

3. Results and Discussion

The consultation of the WoS platform took place on February 12, 2014 and generated a total of 2990 records. Of this total, 2658 were articles, 161 were editorials, 95 were review articles and 76 were letters. When observing the total number of publications according to the time series applied, it can be observed that the first record in WoS dates from 1981, with a pattern of relatively steady growth during the period (r2 = 0.95), and that 2009 is the year with the highest number records (n = 234, Figure 1), taking into account that 2014 had incomplete data, until the beginning of February, with only eight papers published during this time.

Regarding the countries with the highest number of publications, the United States stands out as the country with the highest expression (n = 952). From a total of 80 countries, the 15 countries with the highest number of publications accounted for 85.3% of the total records (n = 2551, Table 1), in which the United States alone accounted for 31.9% of the production among the listed countries. Figure 2 shows the evolution of scientific pro-

Figure 1. Publication time series between 1981 and 2014 (until the beginning of February). 2990 records were found for this time frame (32 years) during the Web of Science™ search.

Figure 2. Evolution of the scientific production on PBL per most productive countries from 1992 to 2014.

Table 1. Countries with the highest number of publications from 1992 to 2013.

duction of the five countries that showed the highest number of records between 1992 and 2014. A consistent increase in scientific production on PBL can be observed in the United States (r2 = 0.89), especially from the twenty-first century onwards, with the other countries showing a relatively erratic production over time, but still higher than the production observed in the previous period, of the 1990s.

Table 2 lists authors who have published at least ten papers on PBL during the analyzed time period. Authors Henk G. Schmidt and Cees P. M. Van der Vleuten were the most productive, both with 56 publications. However, the former was the most cited, with 2359 citations, and had the highest H index (H = 21). The author who showed the highest CPP value was Cindy E. Hmelo-Silver (CPP = 64.6), while author Jerome I. Rotgans had the highest m value (m = 1.3). The high C, H and m values of the first four authors listed in Table 2 are worth noting, since this indicates that some of their publications have significant impact, even though all authors presented intermediate CPP values. It is necessary to mention that the calculated values regarding the impact of the scientific production of the authors (i.e. CPP, H-index and m value) were obtained using the database acquired by the WoS search, therefore disregarding any publication (and its scientific impact) that was not found in the search, either because they are publications that deal with subjects other than PBL, or because they did not include the terms chosen for the search.

An illustrative map of the co-authorship relationships between the authors listed in Table 2 was generated with the data obtained with VantagePoint© (Figure 3). Although this figure does not illustrate all co-authorship relationships (for example, co-authors not listed in Table 2), along with the subsequent fact that even though some authors showed certain weak or nonexistent interaction networks in Figure 3 this does not necessarily mean that they did not have colleagues working on the articles, this map indicates the existence of a collaboration network among the four most productive authors and five other authors, all of whom are either currently linked to or were at one time connected to the Maastricht University. Authors Henk G. Schmidt and Cees P. M. van der Vleuten show the most interactions with other authors, although the latter has a more intense collaboration with other authors displayed in the network shown in Figure 3. This indicates that, when categorizing by country, despite the USA appearing as the country with the most publications on the subject, the Netherlands, was placed fifth in this regard (Table 1), and is possibly the country with the greatest impact on PBL research. This country has a relatively small group of researchers, but which has contributed both with the amount and quality of their publications,

Table 2. List of authors who published at least 10 papers on PBL, where P = number of publications; C = Total citations; Cr = Author rank in category C; CPP = Average Quotes/Publications; CPPr = Author rank in category CPP; Hr = author rank in category H-index and mr = Author rank in category m value.

Figure 3. Framework of relationships between the authors listed in Table 2.

as measured by the indices relating to paper citations (Table 2 and Figure 3) and answers, in great part, to the relatively constant production on PBL in this country (Figure 2).

Despite this collaborative network shown in Figure 3, other authors have also gained importance in the context of PBL research. Cindy E. Hmelo-Silver (Hmelo Silver, C. E.) for example, does not have papers in collaboration with these researchers, but has been gaining attention regarding debates on the subject. Although she has published relatively few papers, these have high citation values, making her the first in rank regarding the CPP. She also has as well a relatively high H value in relation to the universe shown in Table 2. Thus, it can be inferred that this author already had a relatively high amount of papers and that her peers considered them important. Author Jerome I. Rotgans is also noteworthy for showing the highest m value (Table 2), indicating the great impact of his publications on the academic community in a short period (his first publication dates from 2011). The author is from an institution in Singapore, Nanyang Technological University, a country that recently rose to prominence among the countries that most published on the subject (Table 1). It is also interesting to note his collaborations with Henk G. Schmidt (Figure 3). This relationship denotes the collaborative effort to implement PBL methodology in other countries, and the geographical spread of impacting scientific production in the academic community.

A strong network of academic interactions between researchers associated with the Maastricht University, illustrated in Figure 3, points out that this is a group that works in close collaboration. However, there may be a correlation between the co-authorship of these researchers and the possible occurrence of self-citations or cross quotes, which would increase the citation indices for their papers. However, such questionings are beyond the scope of this study, and, even if this hypothesis were to be confirmed, this is a fact that generates much discussion in bibliometric studies, since their presence may indicate characteristics from a natural growth process of knowledge in the area (for example, in order to increase the validity of an argument) to a tool applied to inflate citation reports (Pichappan & Sarasvady, 2002) .

Regarding the highest impact factor journals, the journals listed in Table 3 are those showing at least 25 publications. A total of 18 journals were listed, with 66.7% belonging to the health education area. Of this total, 41.7% correspond to journals concerning education in the medical field, while the second most significant area of this universe is nursing education (16.7% of the total). The other listed journals are directed to research on teaching in science, biochemistry and engineering. The journal with the most publications on PBL was Medical Education, with 263 publications, which was also the most cited (6248 citations), and which showed the highest H value (38) and IF among the listed journals. The journal with the highest CPP was Academic Medicine (37.5), which also had the highest m value (1.4). The progress of publications of impact in the Journal of Dental Education is worth noting, highlighted due to its 1.1 m value (Table 3). Despite this journal having few publications, they have been published very recently, since 2007, with consistent citations.

When analyzing the time series for the amount of papers published in each journal, it was observed that the journal Medical Education had a peak of publications on PBL between 1995 and 2009 (Figure 4). However, an

Figure 4. Number of publications in the journals listed in Table 3 between 1980 and 2014.

Table 3. Figures relative to publications by journal (P = total number of publications; C = Total number of citations; CPP = Average Citations/Publications; IF = impact factor).

increase in publications on the subject is also observed in Medical Teacher (from 2004 onwards) and the Journal of Dental Education (from 2007 onwards), respectively, the second ranked journal that published the most on the subject throughout the period and the journal with the highest m value (Table 3). The journal with the highest CPP value, Academic Medicine, published relatively consistently on the subject between 1993 and 2003 (Figure 4). The other journals shown in Figure 4 have either been steadily publishing studies on PBL over the years (e.g. Nurse Education Today) or have had major publication peaks on the subject followed by periods in which very little or nothing on PBL is published (e.g. Biochemical Education).

From the information contained in Table 3 and Figure 4, it can be inferred that PBL, which initially appeared only with regard to the teaching of Medicine, took off on its own area over the years. This methodology has now spread into other health areas, especially nursing and dentistry teaching, biology, particularly biochemistry teaching, and engineering teaching. It is also important to note the presence of Asian journals among the most prolific (Table 3), which also indicates a geographical spread of PBL research.

4. Conclusion

Based on the data from the present study, results indicate that research on PBL has steadily increased from the end of the twentieth century. Many of the most impacting studies were conducted by a group of researchers who are currently or have at one time been connected to the Maastricht University, a Netherlands institution that can be considered an important center for the propagation of the methodology. The increase in scientific productions regarding the subject has had consequences that relate to a geographic and academic widespread use and research in the area. It is also worth noting that the use of several bibliometric measurements aids in highlight characteristics regarding production by country (Figure 2), the authors of those countries (Table 2 and Figure 3) and journal publications on PBL (Table 3 and Figure 4). These results suggest the validity of this practice, since information that would have been lost if only one parameter were used was obtained. Since this is, to the best of our knowledge, the first bibliometric analysis related to this topic, this paper seeks to provide an initial basis on which future research can, for example, be applied in monitoring developments in scientific production, and the inclusion of further discussions related to bibliometric science is desirable.

Acknowledgements

The authors would like to thank the Brazilian funding agencies FAPERJ and CNPq for their financial support.

NOTES

*Corresponding author.

Conflicts of Interest

The authors declare no conflicts of interest.

References

[1] Dochy, F., Segers, M., Bossche, P. V., & Gijbels, D. (2003). Effects of PBL: A Meta-Analysis. Learning and Instruction, 13, 533-568.
http://dx.doi.org/10.1016/S0959-4752(02)00025-7
[2] Hirsch, J. E. (2005). An Index to Quantify an Individual’s Scientific Research Output. Proceedings of the National Academy of Sciences, 102, 16569-16572.
http://www.pnas.org/content/102/46/16569
http://dx.doi.org/
10.1073/pnas.0507655102
[3] Hsieh, W. H., Chiu, W. T., Lee, Y. S., & Ho, Y. S. (2004). Bibliometric Analysis of Patent Ductus Arteriosus Treatments. Scientometrics, 60, 205-215.
http://dx.doi.org/10.1023/B:SCIE.0000027793.12866.58
[4] Hmelo-Silver, C. E. (2004). Problem-Based Learning: What and How Do Students Learn? Educational Psychology Review, 16, 3, 235-265.
http://dx.doi.org/10.1023/B:EDPR.0000034022.16470.f3
[5] Hung, W. (2009). The 9-Step Problem Design Process for Problem-Based Learning: Application of the 3C3R Model. Educational Research Review, 4, 118-141.
http://dx.doi.org/10.1016/j.edurev.2008.12.001
[6] Kain, D. L. (2003). Problem-Based Learning for Teachers, Grades 6 - 12. USA: Pearson Education
[7] Lopes, R. M., Filho, M. V. S., Marsden, M., & Alves, N. G. (2011). Aprendizagem baseada em problemas: Uma experiência no ensino da química toxicológica. Química Nova, 34, 1275-1280.
http://dx.doi.org/10.1590/S0100-40422011000700029
[8] Mamede, S., Schmidt, H. G., & Normam, G. R. (2006). Innovations in Problem-Based Learning: What Can We Learn from Recent Studies. Advances in Health Sciences Education, 11, 403-422.
http://dx.doi.org/10.1007/s10459-006-9018-2
[9] Neville, A. J., & Norman, G. R. (2007). PBL in the Undergraduate MD Program at McMaster University: Three Iterations in Three Decades. Academic Medicine, 8, 370-374.
http://dx.doi.org/10.1097/ACM.0b013e318033385d
[10] Pichappan, P., & Sarasvady, S. (2002). The Other Side of the Coin: The Intricacies of Author Self-Citations. Scientometrics, 54, 285-290.
http://dx.doi.org/10.1023/A:1016070029935
[11] Rikers, R. M. J. P., & Bruim, A. B. H. (2006). Introduction to the Special Issue on Innovations in Problem-Based Learning. Advances in Health Science Education, 11, 315-319.
http://dx.doi.org/10.1007/s10459-006-9011-9
[12] Savery, J. R. (2006). Overview of Problem-Based Learning: Definitions and Distinctions. Interdisciplinary Journal of Problem-Based Learning, 1, 9-20.
http://dx.doi.org/10.7771/1541-5015.1002
[13] Wood, D. F. (2003). ABC of Learning and Teaching in Medicine: Problem Based Learning. British Medical Journal, 326, 328-330.
http://dx.doi.org/10.1136/bmj.326.7384.328
[14] Xian, H., & Madhavan, K. (2013). Building on and Honoring Forty Years of PBL Scholarship from Howard Barrows: A Scientometric, Large-Scale Data, and Visualization-Based Analysis. Interdisciplinary Journal of Problem-Based Learning, 7, 132-156.
http://dx.doi.org/10.7771/1541-5015.1325

Copyright © 2024 by authors and Scientific Research Publishing Inc.

Creative Commons License

This work and the related PDF file are licensed under a Creative Commons Attribution 4.0 International License.