Abstract

Climate services (CS) are crucial for mitigating and managing the impacts and risks associated with climate-induced disasters. While evidence over the past decade underscores their effectiveness across various domains, particularly agriculture, to maximize their potential, it is crucial to identify emerging priority areas and existing research gaps for future research agendas. As a contribution to this effort, this paper employs the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) methodology to review the state-of-the-art in the field of climate services for disaster risk management. A comprehensive search across five literature databases combined with a snowball search method using ResearchRabbit was conducted and yielded 242 peer-reviewed articles, book sections, and reports over 2013-2023 after the screening process. The analysis revealed flood, drought, and food insecurity as major climate-related disasters addressed in the reviewed literature. Major climate services addressed included early warning systems, (sub)seasonal forecasts and impact-based warnings. Grounded in the policy processes’ theoretical perspective, the main focus identified and discussed three prevailing policy-oriented priority areas: 1) development of climate services, 2) use-adoption-uptake, and 3) evaluation of climate services. In response to the limitations of the prevalent supply-driven and top-down approach to climate services promotion, co-production emerges as a cross-cutting critical aspect of the identified priority areas. Despite the extensive research in the field, more attention is needed, particularly pronounced in the science-policy interface perspective, which in practice bridges scientific knowledge and policy decisions for effective policy processes. This perspective offers a valuable analytical lens as an entry point for further investigation. Hence, future research agendas would generate insightful evidence by scrutinizing this critical aspect given its importance to institutions and climate services capacity, to better understand intricate facets of the development and the integration of climate services into disaster risk management.

Keywords

Climate Services, Disaster Risk Management, Policy Process, Science-Policy Interface, Institutional Analysis

Share and Cite:

1. Introduction

Amidst the widespread escalating impacts of climate change and the need to make communities climate-risk-resilient are at the heart of many countries’ development and adaptation agendas. In this vein, researchers and experts continue to call for urgent and increased efforts to strive for the development of sound climate services systems to mitigate and manage climate-induced disaster risk worldwide. Tremendous scientific endeavors and investments have been made to develop such services and the concept of CS has been placed at the heart of the international adaptation agenda following the World Climate Conference in 2009. Despite abounding research and investments, climate change impacts on human and natural systems continued to rise, and additional efforts towards more effective climate services for disaster risk management are increasingly important. There is an increasing need to develop a better understanding of the production, dissemination, and implementation of CS, especially in collaboration between scientists and political decision-makers can be enhanced. This systematic literature review summarizes the key findings of scientific literature on the production and dissemination of CS for disaster risk reduction published from 2013 to 2023. This scientific endeavor aims to map emerging priority areas in this field and identify potential research gaps, as an entry point for future research agendas. In doing so, we adopted a systematic literature review grounded in the policy processes theoretical perspectives. In the following section, the international development of initiatives for the promotion of CS is discussed to establish the background of the review.

1.1. The Global Framework for Climate Services (GFCS)

The establishment of the GFCS after the Third World Climate Conference in 2009 (Hewitt et al., 2012; Medri et al., 2012; Vaughan & Dessai, 2014), has emphasized the importance of climate services, especially in the priority sectors, including water, agriculture and food security, health, and disaster risk reduction (Lugen et al., 2019; Vaughan & Dessai, 2014). The GFCS enacts a global framework meant to enable societies to manage the risk associated with climate-related hazards in the context of climate variability through the development of science-based climate information and prediction and its incorporation into planning, policy, and practice (WMO, 2014). The GFCS set up five goals to ensure its effective implementation, namely 1) Reducing the vulnerability of society to climate-related hazards through better provision of climate information; 2) Advancing the key global development goals through better provision of climate information; 3) Mainstreaming the use of climate information in decision-making; 4) Strengthening the engagement of providers and users of climate services; 5) Maximizing the utility of existing climate service infrastructure (WMO, 2014: p. 6). In this line, the GFCS is meant to bridge the interfaces between science, policy, and society for a resilient system to climate risk and defines CS as:

“Providing climate information in a way that assists decision-making by individuals and organizations. A service requires appropriate engagement, along with an effective access mechanism, and must respond to user needs.” (WMO, 2014: p. 11).

As a global initiative, the framework relies on a partnership strategy that entails cooperation among multiple parties such as meteorological and hydrological services, climate researchers, policy-makers, and user communities. Five key components shaped the functioning of GFCS namely 1) Observation and monitoring: to improve the collection and dissemination of climate data and information, including historical data, current observations, and future projections. 2) Climate services: to develop and deliver tailored climate services that are relevant and useful for decision-makers and help them anticipate and manage climate risks and opportunities. 3) Research and modelling: to improve the understanding of climate processes and their impacts, and to develop improved climate models and forecasting tools. 4) Capacity building: to build the capacity of individuals and institutions to use and apply climate information, through training and education programs. 5) Governance and coordination: to provide a platform for coordination and collaboration among various stakeholders involved in climate services, and to ensure effective governance and management of the initiative. Overall, organized around these key components, the GFCS aims to strengthen the provision of climate services at the global, regional, and national levels, and to support the integration of climate information into decision-making processes across various sectors to ensure sustainable development and climate-resilient human and natural systems. While global North governments and scientific institutions have a long history and strong capability to develop CS for disaster risk management, the GFCS has therefore focused on the implementation of various initiatives in the Global South, particularly in Africa. One example is the Global Framework Adaptation Program which has been implemented to build resilience and enhance disaster risk reduction, food security, and health through the development of climate services in Tanzania and Malawi from 2014 to 2017 (Pathak & Lúcio, 2018). The program created an enabling environment ensuring the collaboration of local, and international institutions, as well as policy-makers and scientists, ensuring that CS is effectively integrated into these countries’ planning processes (Pathak & Lúcio, 2018). Thus, under the aegis of the World Meteorological Organization (WMO) and GFCS, a National Framework for Climate Services in both countries was implemented. An ongoing initiative is the six-year Intra-ACP (African, Caribbean, Pacific Group of States) Climate Services and Related Applications Programme (ClimSA), funded by the EU to strengthen the production, availability, delivery, and application of science-based information and build the capacity of decision-makers, which is implemented by the Organization of African, Caribbean, and Pacific States (OACPS) (www.climsa.org).

1.2. Climate Services and Global Climate Change Agenda

Climate services are widely recognized as a critical cog of the wheel in climate change adaptation and particularly disaster risk management (WMO, 2014). As a result, various initiatives have been dedicated to shaping and accelerating the development of an enabling environment for an effective and strong climate services system for the benefit of societies. The Conference of Parties (COP 21) in 2015 has also highlighted the need to develop effective CS. Indeed, article (7) of the Paris Agreement emphasizes the importance of improving the availability and use of climate information, including through the development and dissemination of climate services. It calls for parties to strengthen scientific knowledge on climate in a manner that informs climate services and supports decision-making (UNFCCC, 2015). In light of the high vulnerability of many countries in the Global South to climate change-induced disasters, the need to support them in the development of adequate CS has been recognized. The latest IPCC Sixth Assessment Report (AR6) (IPCC, 2022) also discusses climate services at length. This report highlights the growing demand for climate services, which are essential for supporting adaptation and mitigation efforts in response to climate change. This strongly emphasizes the need for accessible, user-friendly, and tailor-made climate services that can provide timely and accurate information to decision-makers and stakeholders at all levels. Overall, the AR6 recognizes the critical role that climate services play in supporting climate action and the need for continued investment in their development and improvement. More importantly, the report emphasizes the dire need for the evaluation of existing CS. The report suggests that better monitoring and evaluation of existing CS for disaster management could help to address several challenges and harness opportunities, which could assist in developing CS to their full potential (IPCC, 2022).

1.3. Evolution of Climate Services and Global Disaster Risk Management Agenda

Disaster risk reduction (DRR) is earmarked by systematic efforts to manage the causal factors of disasters (UNDRR, 2016). However, it has been argued that risk management should also be included in the agenda (Eslamian & Eslamian, 2021). Considered a component of climate change adaptation (CCA), and in light of increasing climate change uncertainties, disaster risk management is defined as the application of disaster risk reduction policies and strategies to prevent new disaster risks, reduce existing disaster risk, and manage residual risk, contributing to the strengthening of resilience and reduction of disaster losses (UNDRR, 2022). This definition aligns with the perspective of the Post-Disaster Management Action, introduced in 2005, which involved a paradigm shift from a response and relief-centric approach to a proactive and holistic process covering all key aspects, including prevention, mitigation, preparedness, rehabilitation, reconstruction, and recovery, which in turn covers at the same time the aspects of DRR (Babu, 2021). Since climate-related disasters have become a worldwide problem, under the aegis of the United Nations, governments, civil society organizations, and NGOs have engaged in constant efforts to address DRM. These efforts have resulted in international agreements, such as the Yokohama Strategy and Plan of Action for a Safer World in 1994-2005 and the Hyogo Framework for Action 2005-2015 (Figure 1). Building on lessons learned from previous agendas and strategies, the post-2015 agenda, the Sendai Framework for Disaster Risk Reduction 2015-2030 has been updated to include the latest scientific information regarding climate-related disasters. It provides a comprehensive roadmap that reflects the paradigm shift from managing disasters

Figure 1. Climate services and evolution of international agenda on disaster risk.

to managing current and future risks, with a strong strategic focus on building resilience (Eslamian & Eslamian, 2021).

Over the years, the development of international agendas on disaster risk management has been accelerated by the increase in the challenges posed by climate-related extreme events. Thus, most of these agendas stress the need for the development of effective climate services as highly critical for disaster risk management.

1.4. Climate Service for Disaster Risk Management: Early Warning Systems

The literature highlights a wide spectrum of CS, with the Early Warning System (EWS) being the most important in the domain of disaster risk management. An EWS is supposed to be a complete system of climate-related information provided to decision-makers (individuals or institutions) for effective climate-informed preventive and protective actions (Trtanj & Houston, 2021). The early warning includes basic monitoring, forecasts, and predictions that provide advanced warnings to decision-makers to allow preventive action in managing and reducing the impacts of hazardous climate events (Aguirre-Ayerbe et al., 2020; Apergi et al., 2020; Lee & Lam, 2010). The Yokohama Action Plan promoted EWS and the effective dissemination of information through mass media as key factors for successful disaster prevention and preparedness. Most developed countries pledged to provide financial assistance to establish and strengthen early warning systems in disaster-prone developing countries, particularly those in the least developed, landlocked, and small island developing States that led to the establishment of the Trust Fund for the Decade to finance such initiatives. Nevertheless, over the years, important capacity gaps have been identified in the establishment and development of effective EWS, especially within the least developed countries (LCDs) (WMO, 2020). To address this problem, the Climate Risk and Early Warning Systems (CREWS) initiative, pioneered by WMO members has invested US$ 40 million in projects in LCDs to enable the establishment of EWS (WMO, 2023a). Recently, to support those most affected countries by climate-induced disasters, the UNFCCC Warsaw International Mechanism for Loss and Damage, which was launched at COP 28 in 2023, also promoted EWSs as a key measure for averting loss and damage associated with adverse effects of climate change (WMO, 2023b).

While the urgent need for EWS has been established above, there was limited understanding of which components are important to make EWS effective. Thus, back in 2005, the World Conference on Disaster Reduction, adopted the Hyogo Framework for Action 2005-2015, which highlighted the dire need for: “early warning systems that are people-centered, in particular systems whose warnings are timely and understandable to those at risk (...) including guidance on how to act upon warnings (...)” (Para. 17, ii.d.9). Out of the need to provide countries with clear guidance for more effective EWS, the third International Conference on Early Warning (EWC III) in Bonn in 2006, experts and scientists from various backgrounds developed a Checklist of Core Elements that need to be met by an effective and people-centered EWS (ISDR, 2006). These elements include Risk Knowledge, Monitoring and Warning Service, Dissemination and Communication, and Response Capability (Figure 2). To function well, effective governance and institutional arrangements, an approach to early warning involving the perspective and needs of local communities and the consideration of gender perspective and cultural diversity is highly important (WMO, 2018).

In light of the increasing occurrence of climate-related disasters and the increasing need for a more robust EWS, at the end of the implementation of the Hyogo Framework, the Sendai Framework for Action made a significant shift from a single-hazard EWS to a Multi-Hazard Early Warning System (MHEWS) (Golding, 2022). In its design, MHEWS goes beyond the elements of the EWS by implementing a proactive and comprehensive approach to the development and dissemination of the MHEWS (Luther et al., 2017) (Figure 3). A MHEWS is designed to address the individual and combined risks of hazardous events (Amaratunga et al., 2021).

Thus, after the 7^th Session of the Global Platform for Disaster Risk Reduction, which recalled strengthening EWS for DRR, the Third Multi-Hazard Early Warning Conference (MHEWC-III) held in Bali in 2022 also noted the critical need for countries and international organizations to join efforts for the effective development of MHEWS. Thus, in 2023, the United Nations initiative labelled ‘Early Warning for All’ has come to crown the previous efforts. Indeed, given the increasing recognition of the importance of EWS and the ambition to make them for All, the executive action plan 2023-2027 planned an important investment of USD 3.1 Billion (WMO, 2023b).

Figure 2. Key elements of effective early warning system (adapted from UN/ISDR).

Figure 3. Schematic of a multi-hazard early warning system elements (WMO, 2018).

2. Theoretical Framework: Policy Processes and CS for DRM

The theoretical framework that informs this review is rooted in the linear model of the policy process and the interactive model of policy implementation to explore the landscape of climate services for disaster risk management. The rationale for adopting this framework stems from the assumption that climate services, as part of disaster risk management and climate change adaptation, are broadly enshrined in climate change policy. Hence, theoretically and practically, climate services for disaster risk management are reflected in these policy processes. Grounded in the policy process perspective, the framework provides a lens (Larsen & Adu, 2021; Grant & Osanloo, 2014) to capture policy-oriented priority areas in the field of climate services for disaster risk management. It also provides a perspective from which to identify the actors and stakeholders involved, as well as the dynamics related to the field.

Understanding Policy Processes

Policies emerge as a result of social needs or problems (Kulaç & Özgür, 2017). In an attempt to provide a comprehensive definition of policy, Birkland (2020) synthesized a set of attributes of policy such as problem-solving oriented, made in the public interest, interpreted, and implemented by public and private actors; goal-oriented or oriented state, ultimately made by government or via the interplay of government and nongovernmental actors. These elements are considered substantially instrumental to policymaking, which is widely shared by many scholars in the domain.

The linear model, widely known as a staged, sequenced model, represents one of the most traditional theories for understanding policy formulation and implementation (Sutton, 1999), and remains a useful starting point for understanding policy-making processes (Kulaç & Özgür, 2017). According to this model, the policy process is allegedly characterized by a well-ordered arrangement of steps, translating a rational trajectory from problem identification to solution implementation, including agenda setting, which refers to the identification and framing of potential issues to be addressed by decision-makers, decision phase defining the formulation of the policy, and implementation phase in which policy is put in action and evaluation in which the impacts and/or performance of the implemented policy is assessed to determine its success or failure (Grindle, 1991; Gültekin, 2014). This lingering model provides an objective analysis of options and a clear separation between policy and implementation (Thomas & Grindle, 1990; Sutton, 1999). In practice, a proposed reform gets on the agenda for government action, a decision is made on the proposal, and a new policy or institutional arrangement is implemented, either successfully or unsuccessfully (Grindle, 1991: p. 122). Very often, there is a strong focus on policy formulation and policy decision-making, while the implementation phase receives less attention. However, since the outcomes of policy initiatives are often determined by a lack of political will, poor management, or shortage of resources (Sutton, 1999), but not necessarily depending on the policy or decision itself, a proper analysis of a policy process needs to include policy implementation and feedback loops between implementation and policy making.

Although linear policy models have helped to move attention from policy formulation and political decision-making to implementation, they are, according to Sutton (1999), oversimplifying and overlooking the complexity of the policy processes, which are not necessarily linear, and steps within the process may overlap or occur simultaneously. Furthermore, linear models tend to overlook the influence of power dynamics and the diversity of actors influencing agenda-setting, decision-making, and implementation by focusing on the logic and rationality of steps.

In contrast to the traditional linear model, in the interactive models of policy processes, several instrumental elements influence policy outcomes from agenda-setting to implementation. In a seminal work, Sutton screened the contributions of several disciplines, including political science, sociology, anthropology, international relations, and management studies, to provide an overview of interactive models of policy processes. Against the conventional understanding of policy as a linear process, they argued that policy is not simply about analyzing a problem, appraising possible responses, selecting a response rationally, implementing the chosen action, and evaluating the outcome (Clay & Schaffer, 1984). Instead, they suggested that the policy process as more complex and chaotic, shaped by a multitude of purposes and accidents. This perspective challenges the traditional ‘practitioner perspective’ which views policies as ‘objective entities’ that are the result of decisions made by some rational authority and which reorganize bureaucratic action to solve particular ‘problems’ and produce a ‘known’ (or desired) outcome (Clay & Schaffer, 1984). In essence, this perspective emphasizes the complexity and unpredictability of policy-making, highlighting that it is not merely a matter of rational implementation of decisions through selected strategies. Such processes can be understood by using concepts such as policy narratives, policy communities, and change management developed by different scientific disciplines. Policy processes should be both dynamic and processual. Different steps in a given policy process are likely to be reshaped or affected by the interests and reactions of various actors, thus affecting the outcomes (Thomas & Grindle, 1990; Grindle, 1991). Considering implementation as the most crucial component of the process, the interactive model places it at the center of the policy process analysis.

The science-policy interface in the policy processes:

The process of formulating policies is frequently intricate; actors involved in policy-making navigate through an interplay of a multitude of stakeholders and their interests, which often clash (Burton et al., 2019). In this context, it becomes crucial for actors, especially researchers and policymakers, to navigate and learn how to integrate their perspectives into the policy process better. This has led to the emergence of the concept of the science-policy interface (SPIs) (Burton et al., 2019), which are defined as social processes that encompass relations between scientists and other actors in the policy process, and which allow for exchanges, co-evolution, and joint construction of knowledge intending to enrich decision-making (Van den Hove, 2007: p. 1) and represent space where scientific knowledge converges with decision-making processes (Gluckman, Bardsley, & Kaiser, 2021) in the quest to create a common ground to collaborate to resolve practical problems in society (Afrin & Rahman, 2022). Thus, the emergence of studies on science-policy interface issues and the renewed interest among scholars and decision-makers stems from long-lasting philosophical, sociological, and political questions concerning the roles of science in society (Van den Hove, 2007), where the cleavage between scientists and policy-makers is pointed out as the triggering factor of failure in science-informed decision-making. In this context, SPI represents a central element of the policy process, which has various implications for different stages and knowledge production. Thus, at the science-policy interface, Gluckman (2018) identified four categories of roles: knowledge generation, knowledge synthesis, and knowledge brokerage. Knowledge production or generation has a lot of interest among scholars and practices out of the need to develop socially relevant knowledge. Based on Gibbons’s (1994) comprehensive work, two main approaches are widely acknowledged. Mode 1 knowledge production is the traditional and well-established model for generating knowledge within academic institutions, which is characterized by a linear, disciplinary-driven, and academically oriented approach to knowledge creation. Mode 2 knowledge production, on the other hand, is a more collaborative, context-driven, and transdisciplinary approach to knowledge generation. Widely considered as a mechanism for transferring research evidence into policy and practice (Ward, House, & Hamer, 2009), at its simplest level, knowledge brokering simply facilitates the flow of information from one side to another (Shaxson & Gwyn, 2010), as well as transforming it (Meyer, 2010; Meyer & Brun, 2023). Thus, the actors playing these roles are mainly scientists, policymakers, boundary organizations, and other stakeholders (e.g., beneficiaries of a project) (Burton et al., 2019). Thus, in light of the complexity of knowledge production and decision-making, Neßhöver et al. (2013) identified three propositions that are instrumental to the effective science-policy interface, including framing research and policy jointly, promoting inter and transdisciplinary research, and multi-domain working groups that include both scientists and policymakers from various fields and sectors, put in place structures, and incentive schemes that support interactive dialogue in the long-term. Whereas these elements seem to address practical issues arising at the interfaces of science and policy, Van den Hove (2007), identified a range of points of intersections between science and policy, which induce some theoretical considerations that need to be addressed, which are termed science-policy interfaces defined earlier by the author. In this regard, Van den Hove (2007) identified 15 normative requirements guiding the design, implementation, and evaluation of science-policy interfaces (Table 1).

Theoretical Problems at the Science-Policy Intersection:

Integration of Knowledge: The challenge of integrating scientific knowledge with policy-making, considering the different languages and methodologies used in each domain. Complexity and Uncertainty: Policies often deal with complex systems where scientific knowledge is uncertain, incomplete, or contested. Diverse Values and Interests: The intersection of science and policy involves multiple stakeholders with varying values, interests, and power dynamics.

Normative Requirements for Science-Policy Interfaces:

Credibility: SPIs must ensure the scientific credibility of the information provided to policymakers. Salience: The information must be relevant and timely to the policy issues at hand. Legitimacy: SPIs should be perceived as legitimate by all stakeholders; which involves fairness and ethical considerations.

Van den Hove emphasizes that SPIs are not just about facilitating the flow of information from science to policy, but also about co-evolution and joint knowledge construction. This requires a dynamic ecosystem of processes, actors, and organizational arrangements designed to effectively address complex policy problems. The author’s work suggests that addressing these theoretical challenges and adhering to the normative requirements can enhance the design,

Table 1. Theoretical problems at the science-policy intersection and normative requirements for the interfaces.

implementation, and evaluation of SPIs, leading to more informed and effective governance and policy processes. Consequently, as these later answer how science-policy interfaces operate, it is arguably suggested that failing to address them would cost the effectiveness of policy processes and governance.

3. Materials and Methods

Originally used in psychology and health science (Durach et al., 2017; Higgins & Green, 2020; Moher et al., 2014; Tranfield et al., 2003), systematic literature reviews are now recognized as a standard methodology for summarizing and identifying key scientific contributions in many scientific fields (Tranfield et al., 2003; Sharif et al., 2019; Schmid, 2020), including climate change adaptation (Berrang-Ford et al., 2015; Shaffril et al., 2021) and disaster risk management. According to Snyder (2019), traditional literature reviews are often rather opaque regarding the selection and evaluation of literature and tend to lack a rigorous methodology that could result in a significant bias in the reviews. SLR is particularly helpful in research fields that deal with complex research questions and involves a wide range of conceptual and epistemological approaches, as well as diverse information sources (Shaffril et al., 2021). Unlike traditional reviews, SLR, according to Higgins & Green (2020), provides a comprehensive and organized way of reviewing the existing literature, in which the methodology is transparent and replicable. This review follows the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement (Moher et al., 2014) (Figure 4). The process is organized into seven steps drawn from the PRISMA statement checklist (Page et al., 2021), including data collection, data analysis, and data synthesis. How these were performed is described in the following sections.

3.1. Data Collection: Review Concepts and Search Strings

Given the cross-cutting aspect of the subject of inquiry and out of the need to ensure a rich and comprehensive review, the data collection was performed by

Figure 4. Steps of systematic literature review adapted from (Xiao & Watson, 2019).

relying on multidisciplinary sources as recommended in the Cochrane Handbook (Bramer et al., 2017), including Research4life, Tylor and Francis online, Google Scholar, ScienceDirect, and JSTOR using a search string (Figure 5) that refined search results to retrieve articles that were most relevant to the review topic. Records were directly saved into Zotero through the web importer extension in Google Chrome.

To collect records not captured via databases, a snowball search strategy was performed in ResearchRabbit to identify records related to the initial one imported by searching for literature citing the initial hit and relevant papers cited in these texts.

Based on the inclusion and exclusion criteria (Table 2), a three-stage screening was performed to identify the final sample from 2168 records. 1) duplicates were removed and titles were scanned against eligibility criteria. 2) abstracts were screened, and 3) the full content was evaluated. Records collected from databases (1460) and records collected through ResearchRabbit (998) resulted in an initial sample of 2168. Within this sample of 2168 records, by processing the first screening, a substantial number of papers were eliminated because of a minor focus on climate services for disaster risk management. Within the sample, some of the papers combined literature, interviews, and questionnaires; these papers were then considered as satisfying inclusion criteria because they did not rely solely on a literature review. In the final step of the screening, 46 papers (32 from the set of five databases and 14 from ResearchRabbit) were removed out of 288, which resulted in a sample record of 242 papers included in the review.

Figure 5. Search strings.

Table 2. Inclusion and exclusion criteria.

3.2. Data Extraction: Mining, Coding and Analysis

To proceed with data processing (Figure 6), the records were consolidated into a single database using Zotero 6.0.26. Duplicates were removed and initial filtering was performed. The data were then exported to Microsoft Excel for further sorting and organization, facilitating the continued screening and generation of graphs and tables. Data analysis was conducted in two stages: descriptive analysis and thematic/content analysis (Tranfield et al., 2003). The former provides a comprehensive account of the sample included in the review, whereas the latter offers a qualitative thematic/content analysis (Creswell & Creswell, 2018), drawing on key conceptual patterns emerging from the reviewed literature. Thematic/content analysis was used to generate conceptual patterns (Braun et al., 2019; Braun & Clarke, 2013; Flick, 2014), and eligible records were imported into the ATLAS.ti 23 software for analysis. This process facilitated the identification of key themes and patterns in the data. After organizing the data, two coding processes were employed including open coding and axial coding (Adu, 2019; Auerbach & Silverstein, 2003; Saldaña, 2016). At the first stage of open coding, relevant data were condensed to generate initial units of analysis (codes). The axial coding completed this first stage, which consisted of grouping and connecting codes and placing them in new categories. This allowed to come up with sub-priority areas.

As part of the process of data collection, screening, and extraction, a specific checklist (Table 3) was used. This checklist contains meta-data, based on which relevant information was identified and captured for the analysis. By following this procedure, each article was reviewed according to this meta-data checklist.

3.3. Quality Appraisal

The ROBIS (McGuinness & Higgins, 2020) (https://www.riskofbias.info/) tool

Figure 6. Data processing from collection to Analyse.

Table 3. Checklist of meta-data for the review.

*CS: Climate Services, *WCS: Weather and Climate Services, *CIS: Climate Information Services, *MS: Meteorological Services, *DRM: Disaster Risk Management, *DRR: Disaster Risk Reduction.

was applied in this review to assess its quality. This included 1) Relevance Assessment, which evaluated the relevance of the studies with regard to the objective of the review. 2) Identification of concerns with the review process, which identified potential issues or concerns with the process of the review itself, particularly regarding the PRISMA methodology. 3) Bias Risk Judgement, the final step that involved making a judgment about the overall risk of bias and providing an overall assessment of the reliability of the review (Whiting et al., 2016). Within these three stages, four key domains are examined including (i) Study Eligibility Criteria, which assessed the criteria used to select studies for inclusion (Table 2) and ensured that the selection process is unbiased and consistent. (ii) Identification and Selection of Studies, evaluating the methods used to identify and select studies for inclusion in the review. It ensured that all relevant studies had been considered and that the selection process was transparent and replicable. (iii) Data Collection and Study Appraisal, which examined the methods used for data collection and the appraisal of studies by ensuring that the data collected is reliable and that the quality of the studies included in the review has been appropriately evaluated. (iv) Synthesis and Findings, the final domain that evaluated the way findings have been synthesized to ensure that the conclusions drawn are backed by the data and are presented clearly and understandably. The assessment was conducted in five batches, with the risk of bias for each batch detailed in Appendixes 1-5. Thus, after the assessment, most of the studies included in the review were found to have a low to moderate risk of bias, indicating a high level of reliability of the review.

4. Results and Discussion

In the following sections, the results of the literature review are discussed quantitatively and qualitatively. First, the development of the number of publications over time, their geographical reference, and the type of CS intervention they are discussing are listed. After that, a qualitative assessment and qualitative prioritization in line with the institutional and policy-process-oriented objective of this article is discussed.

4.1. Summary of the Record Included in the Review

From an initial sample record of 2458, 2216 documents (including duplicates, and irrelevant documents) were removed through the screening process, resulting in a final sample of 242 based on eligibility criteria (Figure 7). The reviewed record of 242 studies included 201 (83%), journal articles, 33 reports (13 %), and 9 (4%) book sections, all related to climate services and disaster risk management (Figure 8).

4.2. Literature Production on Climate Services and Disaster Risk Management

Over the ten years (2013-2023) concerned by this review, literature production

Figure 7. PRISMA Flowchart.

Figure 8. Distribution of final record included in the SLR.

Figure 9. Temporal evolution of literature on climate services (2013-2023).

has substantially increased. Temporal evolution (Figure 9) portrays that the highest production is made in 2020, followed by 2021, 2019, and 2017, while the lowest scientific literature production is identified in 2013 (taking into account the fact that this work concerned the early publication of 2023).

Among the literature surveyed, Climate Services has the highest number of publications (n = 48) followed by the International Journal of Disaster Risk Reduction (n = 14) and Climate Risk Management (n = 11) summarized in Appendix 6. Concerning reports produced by institutions dealing with climate services and disaster risk management, the CGIAR Research Program on Climate Change, Agriculture and Food Security (CCAFS) has the highest record (n = 7), followed by the World Meteorological Organization (WMO) and the World Bank Appendix 7. Appendix 8 summarizes publishers of books sections included in the review where Springer totalizes the highest (n = 6).

Looking at the geographical zones, the highest percentage of publications is addressing Africa at 36.4%, followed by Europe at 24.8%, Asia at 16.1%, and Oceania at 1.2%. Some studies covered more than one geographical zone as highlighted in (Figure 10). The last category concerns studies that have focused on a global scale with 10.7%.

Figure 10. Distribution of literature production per continent.

4.3. Qualitative Analysis of the Literature Production on Climate Services and Disaster Risk Management

The large array of scientific literature on climate services for disaster risk management and the wide coverage across fields including agriculture, water resources management, and urban development demonstrates the increasing interest in this field. Over the past ten years, scientific production has been abounding with increasing interest from scholars of various backgrounds and multiple framing. By systematically reviewing literature in the timespan of 2013-2023, this work mapped scholarly research priorities in the field of climate services for disaster risk management. This section starts by presenting the climate-induced disasters in the literature and the typology of climate services. The second section delves into the analysis of identified policy-process-related priority areas in the literature.

4.3.1. Climate-Induced Disasters Addressed in the Literature Reviewed

In the current review, flood appears prominently as one of the most frequent climate-induced disasters. In parallel, drought emerges as another prevalent disaster (Figure 11). These findings concur with confident evidence indicating these hydro-climatic extreme events have increased in magnitude at the global scale due to climate change (IPCC, 2022). In Africa, over the last twenty years (2002-2021), the Centre for Research on the Epidemiology of Disasters (CRED) indicated the occurrence of 793 floods and 137 droughts (CRED, 2022) which impacted more than 52 million people in 2022 Global Center on Adaptation (GCA, 2022). Furthermore, food insecurity appears at the third position in the explored literature as a consequential outcome of the interplay of the two first

Figure 11. Frequency of Climate-induced disaster in the literature.

(CRED, 2022). In Asia, recent evidence indicates the occurrence of 81 climate-related disasters in 2022, of which over 83% were floods and storms (WMO, 2022). As a widespread situation witnessed globally, particularly in the south, drought occurrence is worsening, leading to more impacts and damages (Aadhar & Mishra, 2021; Ha, Uereyen, & Kuenzer, 2023) and in the Monsoon regions (Kim et al., 2020). Other climate-related, events found in the literature include cyclones and storms (Figure 11). In Europe, more frequent and hotter heat weaves are expected, while in some parts drought will increase and flooding in other zones (Guerreiro et al., 2018). The occurrence and impacts of drought and flood are well documented as appeared in Lehner et al. (2006) in which these events have been projected to increase.

4.3.2. Types of Climate Services for Disaster Risk Management Addressed in the Literature

Climate services have rapidly and progressively expanded as a field of interest from various actors depending on their logic and research interests (Vaughan & Dessai, 2014). The urgent need to mitigate and face climate-induced disasters has sparked attention to the role climate services play in the DRR cycle in managing this societal risk (Hewitt & Stone, 2021). This has led to the need and development of various typologies of climate services (Bessembinder et al., 2019) for the sake of decision-making for a wide range of end-users. In the quest to meet the needs of both supply and demand sides a wide variety of climate services are developed (Visscher et al., 2020). Throughout the reviewed literature, the typology of climate services for disaster risk reduction encompasses a range of essential components that aim to enhance preparedness, response, and resilience in the face of climate-induced disasters. These services are broadly categorized into three key areas: climate information, impact assessment, and decision support. Climate services provide reliable and timely data on weather patterns, climate variability, and long-term climate projections. This includes meteorological data, such as temperature and rainfall patterns, as well as climate modelling and forecasting. Impact assessment services analyze the potential consequences of climate-related hazards on various sectors, including water, infrastructure, agriculture, health, and ecosystems. These assessments help in understanding vulnerability and risk exposure, facilitating the development of targeted disaster reduction strategies. Decision support services utilize climate information and impact assessments to inform policy and decision-making processes. They involve the development of user-friendly tools, such as risk mapping, and early warning systems, which enable stakeholders to make informed choices and take proactive measures in disaster risk reduction. By combining these different types of climate services at different scales, policymakers, communities, and individuals can better anticipate, prepare for, and respond to climate-induced disasters, ultimately minimizing their impacts and building resilience in a changing climate. In summary, climate services for disaster risk management identified within the literature include meteorological services, weather information, warnings, forecasts (seasonal climate forecast, sub-seasonal to seasonal forecast…), early warning systems (flood early warning, cyclone early warning), and Impact Based Warning System (Figure 12).

4.3.3. Identified Policy-Process-Related Priority Areas in the Literature

The content analysis of the reviewed literature identified three major and interconnected policy-oriented priority areas including (i) development of climate services, (ii) use-adoption-uptake of climate services, and (iii) evaluation of climate services (Figure 13). These priorities are part of disaster risk management and climate change adaptation, which both fall into the realm of climate change policy as part of encouraging science-based and climate-informed policy development (Larosa & Mysiak, 2020). Broadly taken within climate change policy, adaptation is considered not a single activity, but rather a continuous process and ongoing policy cycle to understand risk, develop adaptation options, and assess them (Leal Filho & Jaco, 2020). In the same vein, authors argued for the concept of adaptation services, which complement climate services in the context of risk management. This standpoint on adaptation services concurs with the definition of climate services touted as providing climate information in a way that assists decision-making (WMO, 2014). Similarly, climate services support all phases of disaster risk reduction (Street et al., 2019).

Priority 1: Development of Climate Services

The development of climate services offers reliable decision-support in efforts towards climate change adaptation (Leal Filho & Jaco, 2020). In parallel, as part of their role in decision-making, climate services have been proven to support a better understanding and anticipation of disaster risk. In the explored literature, the Development of Climate services revolves around various aspects namely (1 production: co-production; users-providers engagement), (2 dissemination:

Figure 12. Frequency of different types of climate services for disaster risk management in the literature.

communication; community-based approach; roles of non-state institutions (NGOs and Private sector)), (3 barriers to development: institutional; technical; capacity etc.). To foster effective dissemination, in many projects, a community-based approach is used going beyond a simple top-down approach by involving the community at the heart of the dissemination process. This has brought non-state actors on board since it falls in the realm of their developmental interventions. Non-governmental organizations have made significant efforts in this regard (Kirbyshire & Wilkinson, 2018; Jones, Harvey, & Godfrey-Wood, 2016) in the implementation of climate services programs, bringing a new breath of life to the field of climate services for disaster risk management. By recognizing the widespread impacts of climate change and variability, which became an existential threat to humanity as well as the humanitarian perspective to addressing societal needs to face its impacts, they recognized the need to build a strong and effective climate services system (Coughlan de Perez & Mason, 2014), there is a renewed interest in funding projects to support countries in their effort to build a sustainable climate services system from NGOs and non-state actors, especially in sub-Saharan Africa

Figure 13. Identified policy-oriented priority areas in the literature.

(Harvey et al., 2019). In this context, they are considered key actors, given the recognition of a “missing middle” of two-way brokering, translation, and validation where NGOs and other non-state actors can support innovation and experimentation, particularly in two-way forms of engagement with local actors and knowledge holders (Knudson & Guido, 2019).

Priority 2: Use-Adoption-Uptake of Climate Services

The second priority area extensively documented in the explored literature represents the utilization of climate services. This multifaceted aspect encompasses the use, adoption, and uptake of climate services. Thus, beyond methodological considerations, this conceptual variation focuses mainly on understanding two key elements: enablers and barriers (Antwi-Agyei, Dougill, & Abaidoo, 2021). Interestingly, these elements can sometimes be found within a single factor, which can be both an enabler and a barrier. Climate services are produced to support users’ decision-making processes (Vaughan & Dessai, 2014), which suggests that tailoring these services to meet users’ needs significantly shapes their adoption (Buckland & Campbell, 2021). This emphasizes the positive influence of co-design in climate services on their use and uptake (Buckland & Campbell, 2021). Several factors influence the adoption of climate services. These include gender (Carr & Onzere, 2017), awareness and perception of climate services (Zongo et al., 2015), institutional membership (West, Daly, & Yanda, 2018), institutional design (Vedeld et al., 2021), and institutional coordination and communication (West, Daly, & Yanda, 2018). These factors are closely linked to the concepts of salience, legitimacy, and credibility (West, Daly & Yanda, 2018) that influence the usefulness of climate services to end users (Cash et al., 2002; Clifford, Travis, & Nordgren, 2020). To achieve salience, legitimacy, and credibility, it is crucial to managing boundaries between disciplines, scales, and forms of knowledge (Cash et al., 2002). Furthermore, several factors are related to the roles of non-state institutions or actors in providing and creating favorable conditions to enhance the usability of climate services. One such aspect is the agency of non-state actors, particularly the emergence of NGOs. These organizations act as brokers in the dissemination of climate services, creating an enabling environment that enhances their utilization (Harvey et al., 2019). Consequently, as it is discussed in the literature, the enablers and barriers are arguably attributed to the types of approaches adopted in the process of climate services implementation. In most cases, barriers are considered to reflect the supply-driven approach in which users do not have significant inputs in the process. In contrast, the adoption of a bottom-up, participatory, or co-production approach is reflected in the enablers, since the users’ needs are at the heart of the approach in addition to the continuous interactions over the process.

Priority 3: Evaluation of Climate Services

The emphasis on the evaluation of climate services is widely explored in the dedicated literature (Vaughan et al., 2019). Indeed, the focus of the literature is on the assessment of climate services as an integrated element of the climate services value chain. Scholars identified this priority out of the need to make the processes more effective by evaluating various dimensions of climate services specifically 1 Assessment of use-adoption-uptake (Soares & Dessai, 2015; Amwata, Omondi, & Kituyi, 2018), 2 Assessment of impact, socioeconomic assessment; willingness to pay for climate services (Ouédraogo et al., 2018; Paparrizos et al., 2021), 3 technical performance assessment, 4 Assessment of needs (Vincent et al., 2017), 5 developing assessment methodology (Vogel, Letson, & Herrick, 2017; Tall et al., 2018), 6 Measuring co-production (Visman et al., 2022), 7 Business model of climate services for fostering services’ adoption and boosting the market for climate innovation (Larosa & Mysiak, 2020). Developing assessment methodologies is becoming a critical element, given the need to have robust metrics and indicators for effective evaluation of climate services and ensuring evaluation constitutes a cornerstone of the value chain (Dinku, 2018). The importance of this aspect is emerging towards the co-evaluation approach as it appeared in priorities as a shift for a more comprehensive approach that involved the entire value chain, the actors involved, and their logic in the field (Lugen, 2020).

The Emergence of Co-production Approach

Throughout the reviewed literature, co-production appears as an emerging theme in the climate services processes, which translates the response to the call for a shift in the focus of climate services (Daniels et al., 2020). The traditional approach to the provision of climate services has been dominated by a supply-driven mechanism, which involves essentially a one-way direction in the process (ICPAC, 2021). Thus, despite numerous calls for useful climate science information to support policy, most endeavors were consistent with this linear model (McNie, 2013). The supply-driven or top-down approach is mostly characterized by the availability of data technology, and production, rather than by specific user needs (Hewitt, Mason, & Walland, 2012; Taylor, Dessai, & de Bruin, 2015). However, the approach has been criticized for not adequately considering the needs and capacities of end-users (Sultan et al., 2020). It can lead to the production of climate services that are not relevant or usable for decision-makers at the local level. This is particularly problematic in developing countries, where local capacities and needs may be very different from those assumed by global models. In contrast, a demand-driven, bottom-up approach starts with the needs of the users and builds climate services to meet those needs. This approach can lead to more relevant and usable climate services, though it requires a deep understanding of user needs and the capacity to meet those needs, which can be challenging to achieve in practice (Lourenço et al., 2016), which does not slow down its use. Thus, this approach has been continuously improved and over the last decades, the co-production (which focuses on demand-driven) approach has emerged and widely adopted in implementing climate services programs (Jacobs & Street, 2020) as both the antithesis and the solution to the linear model (Tangney, 2022). Rooted in the theory and practice of Knowledge co-production involves the joint creation of knowledge by experts and non-experts, ensuring that diverse perspectives and experiences contribute to the understanding and resolution of complex issues (Jasanoff, 2004). The process emphasizes the collaborative and interactive process of producing knowledge between scientists, policymakers, stakeholders, and the public toward addressing complex societal issues (Cash et al., 2003; Jasanoff, 2004). Applied to the climate services process, it brings together different knowledge sources and experiences to jointly develop new, combined, and relevant knowledge and systems to enable its intended use in specific decision-making contexts (ICPAC, 2021: p. 21). The process of co-producing knowledge and building the necessary skills and capacity of different user groups, both to guide the production and tailoring of climate information (to meet context-specific needs) and to be able to apply that information (Leal Filho, 2015: p. 15). Indeed, to be more efficient and effective (Vaughan & Dessai, 2014), the climate services field needs to address the approach to production (Weichselgartner & Arheimer, 2019), which involves the shift to a co-production paradigm (Bremer et al., 2019; Katharine Vincent et al., 2018). Co-production can be highly useful for enabling dialogue and collaborating across the value chain, helping create services based on credible, salient, and legitimate knowledge (Hewitt & Stone, 2021). This revealed its critical importance as an impetus towards the effective development of climate services for disaster risk management. Furthermore, co-production emerged as a crosscutting aspect over the three main priority areas identified in the current work namely, (Development; Use-adoption-uptake and Evaluation of climate services). Indeed, the process lies in the effective users’ engagement (Daly & Dessai, 2018; Golding et al., 2017), and that of providers, which may be considered as the two faces of the same coin. Since climate services should be decision-driven, reflecting end-users’ needs, their engagement in the process of production remains critical (Baulenas et al., 2023). However, Bremer et al. (2019) are against the narrowed acceptance of the process, but rather frame it as multi-faceted. Most importantly, co-production has implications for the evaluation of climate services. Indeed, since the users and providers are interactively engaged in the production, so does the evaluation, with the emerging concept and approach of co-evaluation of climate services (Delpiazzo et al., 2022), recognized as important for co-development (Tarchiani et al., 2021) framing (Bremer et al., 2021). An example of a practical and proven application of the co-production approach is described in the Tandem Framework (Daniels et al., 2019, 2020). In this approach, to ensure the effectiveness and long-term sustainability of climate services, it cross-pollinates with the main aspects of the value chain, which are also found in the three major priority areas in the present review. Applied throughout the different aspects the approach involves for instance co-design (Santos & Swartling, 2020), and when it comes to evaluation, considered in terms of co-evaluation (Street et al., 2019; Delpiazzo et al., 2022). In this regard, co-production brings a new perspective to the (co)-evaluation of climate services providing a methodological approach to assess the impacts of the co-production of climate services (Englund, André, & Swartling, 2022) with a focus on the users’ feedbacks.

The supply-driven and co-production approaches vs. linear and Interactive Policy process? A reflection on the implications of the theoretical framework

The interactive model of policy implementation and the linear model of the policy process are combined in the theoretical framework used in this study. This theoretical lens offers a prism through which the three main areas of specified priority are analyzed. Thus, as revealed in the analysis, the field of climate services has historically been dominated by the supply-driven approach, which is typified by a top-down method of production. This method appears to be consistent with the linear model of the policy process, in which policies or services are produced by experts and then transferred down for execution. However, this approach has drawbacks, notably in terms of climate services. The top-down character of the supply-driven strategy might result in misalignment and a certain level of discrepancies between the services offered and the real needs of end-users. This imbalance presents a high likelihood of leading to underutilization of climate services or slowing down the uptake process, which limits their potential to face and manage disaster risks. As a response to these limitations, the co-production approach has evolved and is gaining momentum in this field. Co-production entails active collaboration between service suppliers and end users throughout the entire process, from development to assessment.

This strategy remains therefore in line with the interactive model of policy implementation, which emphasizes the value of iterative learning and feedback loops. Thus, by customizing climate services to end users’ unique needs and settings, co-production ensures that these services are more relevant and widely adopted. However, co-production is not without challenges. Indeed, it requires a significant commitment of time and resources as well as a change in the balance of power such that end users are more actively involved in the process.

Thus, this emerging approach provides at first glance, a theoretical pathway to understanding the interplay of different actors involved in the process and institutional layering, which have major implications for the development of services and their integration into disaster risk management. From a practical perspective, co-production provides a concrete enabling environment for actors in the process, to better engage and overcome barriers and discrepancies.

5. Conclusion

Climate services for disaster risk management are increasingly under scrutiny. Thus, literature production on climate services for disaster risk management has expanded rapidly over the last decade. This paper systematically reviewed 242 records including peer-reviewed papers, book sections, and reports from 2013-2023. Upon initial examination, the comprehensive analysis highlighted major climate-induced disasters including floods, drought, and food insecurity, as top research priorities. Concerning the typology of climate services, as it is addressed in this work, several studies provide empirical evidence in which EWS, SFs, and IBWs are identified as the top research priority.

Informed by a theoretical framework that combines a linear model of the policy process and an interactive model of policy implementation, this paper identified three major policy-oriented priority areas: 1) development of climate services which encompasses various sub-priorities namely production; co-production users-providers ‘engagement; dissemination; communication; community-based approach; roles of non-state institutions; Barriers to development; institutional; technical; capacity. 2) Use-Adoption-Uptake of Climate Services with much more focus on Enablers and Barriers. As a sub-priority, the Roles of non-state institutions acting as providers and users of climate services are also highlighted. 3) Evaluation of climate services emerged as a high-priority area within which assessment of use-adoption-uptake; assessment of impact, socioeconomic assessment; willingness to pay; technical performance assessment; assessment of needs; developing assessment methodology; measuring co-production; business model of climate services emerged as sub-priorities. Reflecting on the theoretical framework, the co-production approach is emerging as a cross-cutting theme across these major priorities in response to the dominant supply-driven or top-down approach in the production, implementation, and dissemination of climate services.

A rigorous and effective review of prior, relevant literature is an essential feature of any future academic project, which creates a firm foundation for advancing knowledge. It facilitates theory development, closes areas where a plethora of research exists, and uncovers areas where research is needed (Webster & Watson, 2002: p.1). As a fundamental goal of literature reviewing (Müller-Bloch & Kranz, 2015), the current work, in addition to mapping priority areas, identifies potential research gaps in the field of climate services for disaster risk management.

The present systematic review of the extant literature on climate services for disaster risk management reveals a substantial need for further investigation from the science-policy interface perspective. This aspect is pivotal to the formulation and implementation of effective policy regarding climate services for disaster risk management and particularly for the interplay of institutions, and climate services capacity. Studying and understanding these aspects from the science-policy interface would allow to improve the understanding of the intricacy of the mechanisms through which climate services can inform and influence policy decisions. Practically, addressing this gap is imperative to enhance the efficacy of disaster risk management strategies and to foster a synergistic relationship between scientific inquiry and policy-making processes.

As in many other sectors like biodiversity governance, in this field, scientists, experts, and decision-makers often operate in different spheres with distinct languages, priorities, and timelines. The science-policy interface acts as a bridge, translating scientific findings into understandable language for policymakers and facilitating dialogue to ensure climate services are effectively integrated into disaster risk decision-making. Science–policy interfaces are social processes that encompass relations between scientists and other actors in the policy process, and which allow for exchanges, co-evolution, and joint construction of knowledge intending to enrich decision-making (Van Den Hove, 2007). In that, they involve the two-way communication between producers and users of scientific knowledge, recognizing that science and policy are two separate but interconnected worlds (Pielke, 2007) and characterized by dynamic interactions between different types of expertise, values, and interests, shaping the use of knowledge in decision-making (Stirling, 2008). Given the commonalities shared by these definitions of SPIs, they appear to be an integral part of the policy process. Concerning the climate services under scrutiny in this work, SPIs entail many implications for institutions and climate services capacity; the process of their development, and integration into disaster risk management. SPIs’ perspective should play crucial roles in the understanding of institutional logic that underpins the development and integration of climate services and disaster risk decision-making process. Consequently, this endeavor can inform theoretically, research practice to unpack more insights concerning the development of climate services and their integration into DRM. It can also inform policy practice in the quest for a better understanding of intricate facets influencing the effectiveness of policy processes in the field.

Acknowledgements

This work is carried out as part of a PhD Fellowship funded by the German Federal Ministry of Education and Research (BMBF) at the West African Science Service Centre on Climate Change and Adapted Land Use (WASCAL). The authors gratefully acknowledge the financial support and resources provided by these institutions, which have made this research possible. The views expressed in this paper are solely those of the authors and do not necessarily reflect the views of the funding body.

Appendix: Supplementary Data

Supplementary material related to this article: Figures and Tables.

Appendix 1. Batch 1 Study Risk of Bias

Appendix 2. Batch 2 Study Risk of Bias

Appendix 3. Batch 3 Study Risk of Bias

Appendix 4. Batch 4 Study Risk of Bias

Appendix 5. Batch 5 Study Risk of Bias

Appendix 6. Journal and Publication

Appendix 7. Institutions and Publication (Report)

Appendix 8. Publisher of Book Chapter

Supplementary Data: Risks of Biases Dimensions

B1: Batch 1 Risks of Bias Matrix.

B2: Batch 2 Risks of Bias Matrix.

B3: Batch 3 Risks of Bias Matrix.

B4: Batch 4 Risks of Bias Matrix.

B5: Batch 5 Risks of Bias Matrix.

Conflicts of Interest

The authors declare no conflicts of interest regarding the publication of this paper.

References