Assessing the Skills of Rossby Centre Regional Climate Model in Simulating Observed Rainfall over Rwanda ()
1. Introduction
Understanding the current and future rainfall trends, and its variability over Rwanda is paramount. The footprints of climate change and variability are evidenced over the region due to increased occurrence of extreme climate events [1] [2] [3]. Rainfed agriculture is the core of the economy hence rainfall variability has detrimental impacts on the socioeconomic of the country [4]. Unfortunately, the main crop growing seasonal rainfall has been decreasing over the country [5] [6] [7] [8] [9], hence threatening food security and livelihood.
Food insecurity and clashes over natural resources have been witnessed over East Africa region due to limited resources. Increased frequency and intensity of occurrences of droughts and floods [10] [11] have exacerbated food insecurity in the already vulnerable zone due to weak adaptive capacity. Prevalence of pests and diseases due to climate change like the recent invasion of desert locust over the East Africa region has threatened many people’s livelihood and the economy at large. The numerical model has become an effective tool to understand the climate change and predict the climate at different time scales. Therefore, gauging the skills of the current RCA4 historical models is crucial in selecting the best performing model that can be used for future climate projections and impacts studies.
Around the globe, the General Circulation Models (GCMs) have been widely used to understand the climate issues [12] [13] [14] [15]. In some regions, the models are skillful like the subtropics [16] [17] while in other regions especially over the tropics, the models show relatively poor skills and huge models’ divergence [13] [15] [18]. This discrepancy in the performance of GCMs has been partly linked to the imperfect parametrization schemes, such as the convective schemes in the GCMS [19]. For one thing, the parameterization schemes may be sensitive to regions, that is, an appropriate scheme for a region may not fit for another region. As the model performance varies for different regions, one of the purposes of this study is to examine the performance in simulating the regional climate in East Africa.
Under the umbrella of the World Climate Research Program (WCRP), CMIP modeling groups have been developing models for the last 20 years [20]. These models have been on improvement in both physical parametrization and horizontal resolution, e.g., most of the CMIP3 atmospheric model has a horizontal resolution of 300 km while most of CMIP5 models have a resolution of around 200 km [21]. Encouragingly, improved performance across CMIP models’ generations have been observed by [22] in simulating observed rainfall over East Africa (EA) when comparing the CMIP3 with CMIP5. The study documented that, CMIP3 was less skillful in simulating the short-term rainfall peak in April but such bias was rectified in CMIP5 that depicted satisfactory skills in capturing the seasonal peak.
The course (low-medium) resolution of GCMs has hindered them from capturing mesoscale features that control rainfall patterns in many parts of the globe [23] [24]. Models with high resolution are anticipated to be capable of simulating skillfully convective mesospheric features and multiple-scale interaction than GCMs with coarse resolution [25] [26]. Due to the limitation of the computing resource and the difficulty in model designing, high-resolution simulation and prediction for a global region is still in its infancy. This promoted the scientists around the globe to widely deploy both dynamical and statistical downscaling techniques to improve on the model’s simulation ability. Despite the advantages brought by downscaling methods, the physics uncertainty derived from the driving GCMs can lead to more uncertainty in the regional climate model (RCMs), which yields the biased results [27] [28].
Remarkably, previous studies that evaluated the skills of CORDEX (Coordinated Regional Downscaling Experiment) historical models consensually agree that RCMs still exhibits inadequate skills to simulate rationally rainfall characteristics over different regions [29] [24] [18]. This can be linked to the fact that the majority of these studies evaluated the first phase of CORDEX models that were driven by ERA-Interim reanalysis data [30] [31]. The systematic errors in simulating the large-scale circulation features will greatly influence the skills of the RCMs in replicating the climate characteristics. This calls for the need to evaluate the skills of the second phase of CORDEX models that are driven by CMIP5 [21], as few attentions has been paid for this aspect.
The main objective of this current study is to evaluate the performance of 10 CORDEX historical Climate models against observed rainfall. The findings of this study are vital for model developers, government policy makers and contribute significantly to the body of growing literature review on the performance of models in simulating observed climate. The remaining part of this paper will address the characteristics of the study domain, the types of datasets and methodology deployed in Section 2. Section 3 will present the findings and discussion at length. Conclusion and recommendations will be drawn in the last Section 4.
2. Study Domain, Data, and Methodology
2.1. Characteristics of the Study Area
Rwanda is a tropical country located on Latitude 1.4˚S to 2.51˚S and Longitude 28.53˚E to 30.53˚E with an area of approximately 26,338 km2 (Figure 1). The
![]()
Figure 1. The insert shows a map of East Africa on the left (a) while the map of Rwanda is on the right (b). The background color shows the elevation (meters).
country experienced dipole rainfall regimes, i.e., March-May (MAM) and September to December (SOND); and two dry seasons, i.e., June-August (JJA) and January-February (JF) on its annual cycle [5] [7] [32] [33]. These rainy seasons are known locally as ‘long-term rainy seasons” that refers to the MAM season and “short-term rainy season” that refers to the SOND season. Such termination is based on the amount of rainfall during the rainy season. Throughout the remainder of this study, MAM and SOND will be used to refer to these two seasons.
Rainfall is highly erratic and unevenly distributed within Rwanda during both rainy seasons despite the country’s location is along the equatorial zone. The country exhibits complex mountainous topography ranging from 900 to 4507 m [5]. High altitude regions like the Virunga volcanic chains and Congo-Nile ridges are located on the northwestern part of the country with a height of 1800 - 4500 m. Those regions receive the largest amount of rainfall with an annual amount ranging from 1300 mm to 1550 mm [32]. The low-lying land lies on the eastern parts of the country. This complex topographic feature highly influences the climate characteristics of the country.
The seasonality of the rainy seasons is modulated by the equatorial rain belt over the region [34] [35]. MAM rainfall season is controlled by mesospheric features like topography and Lakes while SOND rains are controlled by large-scale features like El Niño Southern Oscillation (ENSO), Indian Ocean Dipole (IOD), subtropical Anticyclones [36] - [42]. Remarkably, the convergence of anticyclones, i.e., Mascarene High at Madagascar coast and feeds to the Mozambique coast and St. Helena high conveyed from the South Atlantic Ocean to Congo basin then to Lake Kivu, leads to an anomalous amount of rainfall during MAM season [32]. The dry seasons are caused by cold and dry air masses from the Arabian Sea that forms the Siberian anticyclone [5] [37].
2.2. Data
Several datasets were deployed in this study from 1951-2005. Monthly precipitation data obtained in form of “Precipitation flux" (kg/m2/s) for 10 RCA4 models in Coordinated Regional Climate Downscaling Experiment (CORDEX) was be used. These models are an advancement of the High-Resolution Limited Area Model (HIRLAM) [43], in which the improved dynamical and physical parametrization were applied [44]. This dataset has been developed by the Swedish Meteorological and Hydrological Institute [45] and archived at https://www.smhi.se/en/research/research-departments/climate-research-rossby-centre2-552. The information for the RCA4 models used are illustrated in Table 1. Based on the location of the study domain, we used the RCA4_AFRICA (AFR-44) domain with a resolution of 50 km (approx. 0.44˚). The Digital Elevation Model data was obtained from National Oceanic and Atmospheric Administration archived on https://www.ngdc.noaa.gov/mgg/topo/topo.html.
As the densely populated synoptic stations remain a major challenge over Africa [39], the precipitation data derived from the Global Precipitation Climatology Centre (GPCC) version 8 was used in this study. The available station data were used to validate GPCC dataset which was used in the subsequent analysis
![]()
Table 1. The description of the CMIP5 climate models that were downscaled by the RCA4 group to RCA4, v1.
as the reference dataset. For consistency purposes and fair comparison, both the observational data and the RCA4 model datasets were regridded to a common resolution of 0.44˚ * 0.44˚ using bilinear interpolation method.
2.3. Methodology
This study deployed various scalar statistical metrics to discern the skills of RCA4 models in simulating the observed rainfall characteristics over Rwanda. To obtain the multi-model ensemble mean, equal weight was assigned to each model. This method of building the model ensemble mean was adopted from [46]. Other statistical methods include: Standard deviation, Mean, Pearson Correlation coefficient, Relative percentage bias, Mann-Kendall trend test, and Sen’s slope estimator. These methods can be elucidated mathematically as;
(1)
After obtaining the means of the two datasets, it was essential to get the mean difference between the two using Root mean square error (RMSE). The value of the RMSE is taken as an absolute value. Remarkably, for a fair comparison of the model performance, is to scale the value of RMSE either by the observed mean or the standard deviation. In this study observed mean (
) was used (Equation (3))
(2)
(3)
(4)
To understand the temporal agreement between the simulated rainfall and observation, the Pearson correlation method was used. The coefficient value is obtained by dividing the covariance between the observed and simulated model with the product of their standard deviation as expressed in (Equation (5)). The significance of the correlation value was tested using a t-test at a 95% confidence level.
(5)
Understanding the rainfall trend is important for agriculture purposes and water management. Non-parametric trend detection Modified Mann-Kendall test [47] [48] was used to assess the direction of the trend. The magnitude of the trend was assessed using a non-parametric Theil Sen slope estimator [49]. Non-parametric methods are more robust than their coequal parametric methods since they are not influenced by occurrences of extremes and outliers in the datasets [50]. More details of the methods can be found in [51] [52] [53] [54].
To understand the discrepancy between the observed and simulated mean values, relative bias is used. The relative bias of different models is scaled against the mean observation to unify them for a fair comparison of individual models. The Percentage Relative Bias (P.R.B) can be elucidated mathematically as;
(6)
Taylor diagram [55] was used to assess spatially the performance of the models when statistical measures like standard deviation, correlation, and centered-root mean square error (RMSE) are combined. If the model is near to the reference data (Observation), the model is considered to exhibit reasonable skills in simulating the observation. Additionally, if the models are spread apart from each other, it depicts divergence of the models/lack of consensus in their prediction ability and the opposite is true.
Lastly, we combined these statistical methods using skill scores [56]. The skills of the models at both seasonal and annual scales were subjected to an absolute error to capture their magnitude of deviation from the observation or the ideal values i.e. 1 in the case of the correlation coefficient. Then, we added up the Skill score values (SSV) at all the time scale considered. The smaller the SSV, the better the performance of the models and vice versa is true. The following scalar metrics were used to build the SSV; Normalized Root means square error (nRMSE), Correlation coefficient (CC), absolute relative errors for; relative bias and, Theil-slope estimator (TSA). The SSV can be illustrated mathematically as;
(7)
3. Results and Discussion
3.1. Mean Annual Cycle Characteristics
As described in Sect. 2, the mean rainfall in Rwanda exhibits bimodal rainfall patterns, MAM rainy season and SOND rainy season. Virtually, the skills of the model to simulate the seasonality of rainfall over Rwanda lies primarily in the model’s ability to capture the north-south movement of ITCZ, which is an essential baseline of model’s basic performance. Therefore, we firstly investigate whether the RCA4 models have the ability of duplicating the seasonality of the mean rainfall.
As clearly seen in Figure 2, the mean rainfall simulated by all the models exhibit a semi-annual feature, indicating that all the models can capture the annual cycle of rainfall over Rwanda. Here the GPCC dataset was also evaluated, in order to be used as the reference dataset in the subsequent analysis. In general, the annual cycle of rainfall derived from the GPCC (thick red curve in Figure 2) matches well with the observation (thick blue curve in Figure 2), in terms of both the temporal evolution feature and the magnitude. Specifically, the correlation
![]()
Figure 2. The skills of RCA4 models in simulating the mean annual rainfall cycle (mm/month) over Rwanda from 1951-2005.
coefficient between them reaches more than 0.9, indicating that the GPCC captures the seasonality feature excellently. Therefore, the GPCC data is used as a potential reference in the subsequent analysis.
To assess the ability of RCA4 models in simulating the annual cycle of rainfall in detail, we further examine the dynamical aspects of the annual cycle of rainfall, including the onset, peak, and cessation of rainfall. In general, the RCA4 models can simulate the MAM seasonal rainfall peak, despite showing some biases in the magnitude (Figure 2). Specifically, the MAM seasonal rainfall is underestimated by three models (HadGEM2, NorESM1 and GFDL-ESM2M) while the rest of the models overestimated the seasonal rainfall. In contrast, all the RCA4 models exhibit wet bias when simulating the SOND seasonal rainfall, compared to the observation data and GPCC (Figure 2). The higher values of precipitations given by the models may indicate that the parametrization concern an accumulation effect.
We further assess the RCA4 model performance in simulating the observed mean annual cycle, in terms of three aspects (i.e., correlation, root mean square, and standard deviation). As summarized in Figure 3, the models show reasonable skills and consistency in simulating the mean annual cycle with low spatial variability and normalized RMSE of less than one. In this case, the best performing model is CSIRO while the least performing its MIROC5. These findings are consistent with previous studies that also focused on this region and identified CSIRO has the best performance in simulating the mean annual cycle [57] [15].
3.2. Spatial Distribution of Seasonal Rainfall over Rwanda
Observed spatial patterns depicts that most parts of the country are wet during the MAM rainy season, as shown in Figure 4(a). The regions surrounding Lake
![]()
Figure 3. Summarizing multiple facets of RCA4 climate models in simulating observed mean annual cycle. The alphabetical letters denote; A-CanESM2, B-IPSL, C-CNRM, D-CSIRO, E-EC-Earth, F-Ensemble model, G-GFDL-ESM2M, H-HadGEM2, I-MIROC5, J-MPI, K-NorESM1.
![]()
Figure 4. Spatial distribution of observed MAM seasonal rainfall (a), RCA4 models (b), and bias (c) in mm/year.
Kivu and northwestern part of Rwanda receive the highest amount of rainfall during this season as compared to the other parts. Notably, the eastern parts are found to be drier than the rest of the country (Figure 4(a)). The MME made by the RCA4 models show reasonable skills in simulating this spatial distribution (Figure 4(b)). However, the models underestimate the rainfall amount in most parts of the country, especially for the southeastern region where the underestimation reaches around 100 mm/year. In the western parts, the models slightly overestimated the observed seasonal rainfall by 30 mm/year (Figure 4(c)).
During SOND rainy season, the observed data shows that the country receives more rainfall than that during MAM. The spatial pattern remains relatively the same during the two seasons with the western regions wetter than the eastern side (Figure 5(a)). The MME result made by RCA4 models show relatively low skills in simulating the spatial pattern by documenting a high amount of rainfall on the northern parts of Lake Kivu as compared to the observation (Figure 5(b)). Succinctly, the RCA4 climate models show wet bias in simulating SOND seasonal rainfall over the northern parts of Rwanda while dry bias in the southern region (Figure 5(c)).
![]()
Figure 5. Spatial distribution of observed SOND seasonal rainfall (a), RCA4 models (b), and bias (c) in mm/year.
The biases of the mean precipitation distribution may be linked to the orography. In Rwanda, the precipitation distribution exhibits more rainfall in the western parts, which is closely associated with the fact that the anomalous influx of moisture from Lake Kivu and the Congo airmasses is usually the source of the precipitation in Rwanda. As shown in Figure 1(b), the altitude sharply increases in the western edge of Rwanda and can reach around 4000 meters. Such a sophisticated orography may be beyond the simulation skill in the models and induce the bias in simulating the mean precipitation in Rwanda. It is worth mentioning that the orographic effects have been noted to significantly influence the skills of RCA4 models in simulating observed rainfall over Africa [18] [24]. Conclusively, RCA4 models have the ability in simulating the mean precipitation in MAM and SOND rainy seasons, but still show some biases in their spatial distribution.
3.3 Seasonal MAM Rains Temporal Characteristics
Table 2 shows the ability of the RCA4 climate models in simulating observed MAM seasonal rainfall statistics. Most of the models exhibit dry bias except CSIRO, EC-Earth, IPSL, and MPI that have a wet bias. The ability of the climate models to capture the temporal variability was low as portrayed by the majority of the models exhibiting insignificant negative correlation values. Few models that depicted positive relationships were CanESM2, CNRM, and NorESM1. Notably, none of the models exhibited an insignificant relationship with the observed MAM seasonal rainfall at a 95% significant level.
In the observation, the rainfall exhibits a decreasing trend at a rate of 0.10 mm/year for the MAM rainy season. Unfortunately, only two out of ten models (CNRM and IPSL) were able to capture such decreasing trend. This conforms to
![]()
Table 2. Skills of RCA4 models in simulating MAM seasonal rainfall (mm/year) mean statistics.
the “East Africa Climate paradox” where the observation shows decreasing rainfall trends while the GCMs show positive trends [58].
To summarize pictorially different aspects of the RCA4 models in simulating the observed MAM seasonal rainfall, the Taylor diagram was used (Figure 6). All the RCA4 models have some biases in simulating the mean rainfall in MAM season to various extent, and the MME results outperforms the individual model in simulating the observed MAM rain characteristics. This has been observed in the previous studies over the great horn of Africa that gauged the skills of RCA4 models against observations [18] [59].
3.4. SOND Rainfall Characteristics
The ability of the RCA4 models to simulate the observed SOND rains was also gauged. As shown in Table 3, the results show that all the models as well as the MME have shown wet bias in terms of the mean SOND rainfall, showing an overestimation of mean SOND rainfall to various extent. In terms of the interannual variation, the models exhibit high standard deviation than the observation (Table 3). In terms of long-term trend of SOND mean rainfall, the observation shows an insignificantly increasing trend at a rate of 0.03 mm/year during this period. Interestingly, only three RCA4 models (HadGEM2, IPSL, and MIROC5) were able to capture this increasing trend.
Diagrammatically, the three aspects of the RCA4 models’ skills in simulating the observed SOND rainfall are shown in Figure 7. All the RCA4 models have some biases in simulating the mean rainfall in MAM season to various extent, but the MME results outperforms the individual model in terms of simulating
![]()
Figure 6. Taylor diagram depicting the performance of RCA4 model in simulating MAM seasonal rainfall mean statistics. The alphabetical letters denote; A-CanESM2, B-IPSL, C-CNRM, D-CSIRO, E-EC-Earth, F-Ensemble model, G-GFDL-ESM2M, H-HadGEM2, I-MIROC5, J-MPI, K-NorESM1.
![]()
Table 3. RCA4 Skills in simulating mean SOND rainfall (mm/year) over Rwanda.
![]()
Figure 7. Summarized skills of RCA4 models in simulating observed SOND rains over Rwanda. The alphabetical letters denote; A-CanESM2, B-IPSL, C-CNRM, D-CSIRO, E-EC-Earth, F-Ensemble model, G-HadGEM2, H-GFDL-ESM2M, I-MIROC5, J-MPI, K-NorESM1.
the observed SOND rain characteristics.
3.5. Annual Scale Characteristics
At the annual mean time scale, the ability of the RCA4 climate models to simulate the mean annual statistics are illustrated in the Table 4. The results show that, all the models with exception of EC-Earth exhibit dry bias. The RMSE results show that the models exhibit bias in terms of the spatial distribution of mean rainfall to various extent. The observed annual rainfall over Rwanda shows
![]()
Table 4. The skills of RCA4 models in simulating total annual rainfall (mm/year) over Rwanda.
a decreasing trend at a rate of -0.02mm/year. The three RCA4 models (CNRM, CSIRO, and EC-Earth) are able to capture this decreasing trend, as shown in Table 4).
Notably, the inadequacy of RCA4 models to capture the correct sign of trend is consistent with the finding that was documented by previous studies which also focused on this region [18] [59] [60]. Such bias may be traced back to the driving GCMs models that have been dynamically downscaled. It is worth mentioning that the RCA4 models that are driven by ERA-Interim dataset skillfully simulate rainfall trends compared to those driven by CMIP data [61].
The summary of the skills of RCA4 climate models in simulating the observed annual rainfall is shown using a Taylor diagram in Figure 8. The results show that there is little agreement between the models in capturing the statistical aspects of the annual rainfall over Rwanda.
The RCA4 models exhibits some biases in simulating the annual mean rainfall to various extent. Clearly, the multi-model ensemble result performs better than the individual models in this category.
3.6. Summarizing the Skills of RCA4 Models Using Skill Score
The models were gauged on their capability to simulated seasonal and annual rainfall over Rwanda. The models portrayed mixed performance in simulating rainfall at all scales (Figure 9). Notably, individual models perform differently at a different scale. The RCA4 models show reasonable skills in simulating SOND seasonal rainfall than both MAM and annual rainfall. These findings are coherent with previous studies over the region that deployed different GCMs models [13] [14] [15] [18]. Remarkably, the ensemble model outperforms the skills of the individual model in simulating the observed rainfall features over Rwanda.
![]()
Figure 8. Pictorial depiction of RCA4 models in simulating total annual rainfall. The alphabetical letters denote; A-CanESM2, B-IPSL, C-CNRM, D-CSIRO, E-EC-Earth, F-Ensemble model, G-GFDL-ESM2M, H-HadGEM2, I-MIROC5, J-MPI, K-NorESM1.
![]()
Figure 9. Skill score of the RCA4 historical models in simulating seasonal and annual rainfall over Rwanda. The alphabetical letters denote; A-CanESM2, B-IPSL, C-CNRM, D-CSIRO, E-EC-Earth, F-Ensemble model, G-GFDL-ESM2M, H-HadGEM2, I-MIROC5, J-MPI, K-NorESM1.
The best model in simulating observed MAM seasonal rainfall is CNRM while the least performing model is MPI. During SOND rainy season, MIROC5 outperforms all other models including the ensemble model which is number six out of eleven. The least skilled model in this category is GFDL. At the annual scale, EC-Earth shows more reasonable skills in simulating observed annual rainfall characteristics while MIROC5 was the least skilled. This mixed performance portrays that, no individual model performs skillfully at all time scale. Therefore, model selection should be based on the intended use of the model. This calls for improvement of model parametrization to capture the large-scale factors (ENSO and IOD) and mesospheric factors (Complex topography and significant water bodies) that modulate rainfall over the study domain and the greater East Africa.
Ranking the overall performance of the models is crucial as it acts as a basis for model selection to be used for future climate projection. Therefore, the models can be ranked from the best performing model to the least as; CSIRO, CanESM2, CNRM, GFDL, MIROC5, ENS, EC-Earth, HadGEM2, IPSL, MPI, and NorESM1.
4. Conclusion and Recommendations.
The present study aims at ranking the performance of ten RCA4 historical models that were driven by CMIP5 climate models against the observed gridded datasets as obtained from GPCC from 1951-2005 over Rwanda. The study deployed both statistical metrics and geospatial methods to understand both spatial and temporal skills of the RCA4 models in simulating the observed seasonal and annual rainfall characteristics. For consistency purposes and fair comparison, all the models and the GPCC dataset were regridded to a common resolution of 0.5˚ × 0.5˚ using bilinear interpolation. The results show that Rwanda receives a bimodal rainfall regime; March-May (MAM) and September-December (SOND). The historical RCA4 climate models are inconsistent in simulating this annual cycle with most of the models showing wet bias and three of them exhibiting dry bias during MAM rainy season. However, the historical RCA4 models are coherent in simulating the SOND seasonal rainfall although they exhibit a wet bias.
The RCA4 historical models are skillfully in simulating the mean annual cycle than capturing the temporal pattern at both seasonal and annual scale. Most of the models exhibited inadequate skills in simulating rainfall trends and magnitude. This signifies the inadequacy of the GCMs models even after downscaling to simulate the mechanisms that modulated interannual variability of rainfall over the region. The performance of the RCA4 historical models differs from one-time scale to another with huge bias. The historical models can be ranked from the best performing model to the least as; CSIRO, CanESM2, CNRM, GFDL, MIROC5, ENS, EC-Earth, HadGEM2, IPSL, MPI, and NorESM1. The ensemble model outperforms some individual models in capturing the mean statistics of rainfall over Rwanda.
The findings of this study are meaningful to both policymakers and climate modelers. The modeler can use the results to improve on the models’ parametrization schemes and physics in order to reduce the chronic bias documented. We highly recommend deployment of bias correction techniques on RCA4 future climate models before using them for future rainfall projections and impact studies. Additionally, the selection of the model should be based on the purpose of the study due to mixed performance of the models at different time scale.
Acknowledgements
This work was jointly supported by the National Key Research and Development Program of China [grant number 2019YFC1510004], NSFC Grants (No. 42005020) and the National Key Research and Development Program on Monitoring, Early Warning and Prevention of Major Natural Disaster (2018YFC1506002). Additionally, special thanks to RCA4 group for downscaling the CMIP5 data and ESG for openly availing the data. We are also grateful to Rwanda Meteorology Agency for the provision of the observation data. The authors are grateful to Nanjing University of Information Science and Technology (NUIST), China for creating an enabling environment to carry out research globally. We are highly indebted to the editor for contacting three anonymous reviewers to complete the review in time and we thank the experts in the field who helped improve the work to the current state.