Evaluation of the Climate Prediction System CMA-CPSv3 on Performances of Precipitation and Temperature Prediction over the Tibetan Plateau

Maji Ni, Dunzhu Danzeng, Yangzong Ciren^*

Tibet Climate Center, Tibet Meteorological Bureau, Lhasa, China.

DOI: 10.4236/jdaip.2024.124026                

Abstract

In order to evaluate the prediction performances of the climate prediction system developed by the China Meteorological Administration (CMA-CPSv3) over the Tibetan Plateau region, the precipitation and temperature predicted by CMA-CPSv3 at different lead time was evaluated for the period of 2001-2023, by comparing with observations—the monthly precipitation of the CMAP and 2 m temperature data of the NCEP/DOE reanalysis data. The main conclusions were as follows: 1) the forecast skill of the model is very sensitive to the initial conditions and value, and the model forecast capability decreases rapidly as the lead time is extended. 2) CPSv3 performed well in capturing the spatial distribution of precipitation over Tibet in January, August, October, November and December at 0-month lead, and the forecast skill is relatively poor in March and April. CPSv3 has better performance in the east-central part of the land in January, in the central and western part of the region in March, in the whole region in April, in the northern part of Nagchu, northern part of Chamdo and central part of Shigatse in July, in the eastern part of Chamdo, northern part of Nagchu and northern part of Ali in August, in the whole region in September, and in the cities of Shigatse and Shannan in November. 3) At each lead time, the forecast skill for 2-m temperatures was higher than that for precipitation. For 2-m temperature, CPSv3 performed best in January, May, July, August, September, October, and December, while performing relatively poor in March, April, and November. Specifically, the prediction skill is higher in January and December for most of the regions, for Shigatse and Ali in May, for southern Shannan in July, and for northern Nagchu and southern Shannan in August, and there is some prediction skill in March, April and October, while CPSv3 performed poorly in November.

Keywords

CMA-CPSv3, Prediction Assessment, Precipitation, Temperature, Tibetan Plateau

1. Introduction

The Tibetan Plateau has been called the “third pole” of the Earth due to its special geographic location and complex topographic features [1], and the dynamics and thermodynamics of its special large topography have a profound impact on the global climate and environment [2], such as playing an important role in the formation and maintenance of the Asian monsoon circulation [3]. The Tibetan Plateau is located in the hinterland of the Qinghai-Tibetan Plateau, with an area of more than 1 million square kilometers, an average elevation of 4000 m, and a complex topography, which makes it a sensitive and vulnerable area for climate change [4]. The scarcity of ground-based meteorological observation stations and the lack of observation data on the Tibetan Plateau constrain a comprehensive understanding and knowledge of the overall and regional climate change on the Tibetan Plateau, and thus climate modeling data become an important tool for climate change research on the Tibetan Plateau [5].

The China Meteorological Administration (CMA), the Ministry of Science and Technology (MOST), and the Chinese Academy of Sciences (CAS) jointly released the “China Meteorological Science and Technology Development Plan (2021-2035)” in February 2021, which identifies 46 priority directions for meteorological science and technology work from 2021 to 2035 in nine key areas. The new pattern of meteorological business will be constructed under the framework of new technology system with big data, artificial intelligence, Internet+, cloud computing and so on. Important elements of the development of meteorological science and technology in the new period include research on the fusion and analysis of multi-source, multi-multiple and disordered data, data assimilation methods and the systematic theory and numerical computation scheme of the Earth system model, and the full application of emerging technologies, such as artificial intelligence, in order to improve and perfect the numerical model system. There is an urgent need to vigorously develop China’s autonomous and controllable Earth system numerical prediction model [6]-[8]. Under this background, on September 30, 2021, the Earth System Numerical Forecasting Center of China Meteorological Administration (CMA) was established. In the “14th Five-Year Plan” of China Meteorological Administration (CMA), the development plan of numerical prediction operations has made clear deployment of 28 tasks in six major fields, including short-term climate prediction operations. China Meteorological Administration National Climate Center (BCC) Climate System Prediction Model BCC second-generation model has better performance than the first-generation model in forecasting global temperature, precipitation, and circulation. The seasonal prediction of China’s summer precipitation based on the second-generation short-term climate prediction modeling system of the BCC is evaluated to assess the seasonal prediction of China’s summer precipitation, especially for the areas from the southwestern part of the country to the south of the middle and lower reaches of the Yangtze River, the western part of the Yellow River and Huaihe Plain, the northern part of the northeast China, and the northern Tibet Plateau. The seasonal prediction skill is high, and at the same time, the model has an overall better effect on precipitation distance forecasting [9]. By evaluating the performance of climate prediction models (BCC_CSM, ECMWF_System5, NCEP_CFSv2) on the prediction of pre-winter precipitation and 500-hPa height field over the Tibetan Plateau, it is shown that the BCC_CSM model has the best forecasting skill for historical returns of precipitation, and has the most outstanding performance for climate prediction over the tropical region [10]. The CMA-CPS climate prediction system developed by the China Meteorological Administration established on the basis of the atmosphere-land surface-ocean-sea ice coupled multi-layer climate system model includes multilayer interaction processes, human activities, aerosols, atmospheric chemistry, carbon cycle and dynamic vegetation processes, etc. The coupled multilayer assimilation scheme provides a coordinated and reliable initial analysis field for the climate model, which can be used for sub-seasonal, seasonal, and inter-annual climate prediction, as well as inter-decadal or longer-term climate prediction. The CMA-CPSv3 system consists of a sub-seasonal-seasonal (S2S) prediction subsystem and a seasonal prediction subsystem, and the CMA-CPSv3 seasonal prediction subsystem employs a combination of physical parameterization and first-value perturbation to design 21 ensemble sample members, which provide monthly and seasonal climate predictions for the next one to seven months.

The CMA-CPSv3 system has been put into operational application in March 2021, providing real-time multi-timescale and multi-circle climate prediction products to the whole country.

Currently, the assessment of the CMA-CPSv3 operational prediction system-based prediction products on global and regional scales has been carried out successively. In order to further understand the applicability of the CMA-CPSv3 climate prediction system products in the Tibetan Plateau region, it is necessary to evaluate its prediction performances, and at the same time to make preparations for further improving the predicted effects of the numerical models in the Tibetan Plateau region by applying methods such as the dynamical downscaling.

2. Data and Methods

2.1. Datasets

A set of retrospective seasonal forecasts for past dates have been conducted based on the prediction system. And the CPSv3 hindcasts includes 21 members, the ensemble forecast initiates monthly from January 2001 to present which output prediction products for the next 7 months. In this study we used ensemble means of 21 members results from January 2001 to December 2023, and the climatology is the time average of 2001-2023. And the model data are interpolated to a 2.5˚ × 2.5˚ grid according to the WMO standard. The downloaded historical hindcast data for February and June are missing, as well as forecasted data initiated from these two months. The forecast data in September is missing except for three years from 2021 to 2023.

The assessment of deterministic forecasting methods conducted on two elements, precipitation (PREC), and 2-m air temperature (t2m), with a resolution of 2.5˚ × 2.5˚. The unit of observed precipitation is mm/day, while the unit of model-forecasted precipitation is m/s. For the purpose of a more robust and correct comparison, before the evaluation of CPSv3 hindcasts, the units of model-forecasted and observed precipitation data were harmonized. Results of assessment are presented in flood season (May, July, August and September) and dry season (months excluded flood season: January, March, April, October, November and December).

The following observational datasets are used in this study: 1) monthly precipitation data provided by CMAP (http://www.esrl.noaa.gov/psd/data/gridded/data.gpcp.html.); 2) 2-mtemperaturedatafromNCEP/DOE: (http://www.esrl.noaa.gov/psd/data/gridded/data.ncep.reanalysis2.pressure.html.).

2.2. Methods

Anomaly Correlation Coefficient (ACC) and Temporal Correlation Coefficient (TCC) are used for deterministic forecast testing and evaluation of CPSv3 prediction products.

Let $X_{i,j}$ represent the observed value and $f_{i,j}$ represent the predicted value, where $i=1,2,3,\cdots ,M$ represents the number of grid points or stations in the evaluation area, $j=1,2,3,\cdots ,N$ represents the time series, which needs to be multiplied by the coefficient $W_i$ , $W_i\text{=}1$ when using station information, and $W_i\text{=cos}\left({\varphi }_i\right)$ when using grid point information,, where ${\varphi }_i$ is the latitude at which the grid points are located when performing regional averaging.

2.2.1. Anomaly Correlation Coefficient (ACC)

${\text{ACC}}_j=\frac{{\displaystyle \sum _{i=1}^M\left(X_{i,j}-\overline{X_j}\right)\times {\left(f_{i,j}-\overline{f_j}\right)}^2}}{\sqrt{{\displaystyle \sum _{i=1}^M{\left(X_{i,j}-\overline{X_j}\right)}^2}}\times \sqrt{{\displaystyle \sum _{i=1}^M{\left(f_{i,j}-\overline{f_j}\right)}^2}}}$

where $\overline{X_j}=\frac{1}{M}{\displaystyle \sum _{i=1}^M{X}_{i,j}}$ represents the spatial regional average of observations at a certain time. $\overline{f_j}=\frac{1}{M}{\displaystyle \sum _{i=1}^M{f}_{i,j}}$ represents the spatial average of forecast values at a

given moment in time. The spatial anomaly correlation coefficient (ACC), which is calculated one value per forecast and tests continuous variables, is relatively objective but susceptible to modulation by large-valued regions.

2.2.2. Temporal Correlation Coefficient (TCC)

Evaluation method for the overall forecasting skill of multiple forecasting periods with different forecast lead times for a certain element at each grid point, which pertains to deterministic forecasting of continuous variables. The formula can be written as:

${\text{TCC}}_i=\frac{{\displaystyle \sum _{j=1}^N\left({\chi }_{i,j}-\overline{\chi }\right)\left(f_{i,j}-\overline{f_i}\right)}}{\sqrt{{\displaystyle \sum _{j=1}^N\left({\chi }_{i,j}-\overline{{\chi }_j}\right)}}\sqrt{{\displaystyle \sum _{j=1}^N{\left(f_{i,j}-\overline{f_i}\right)}^2}}}$

The range of TCC_i is (−1, 1). Higher values represent higher prediction skill; the significance is tested using bilateral t-test. The regional mean TCC formula is calculated as:

$\text{TCC}=\frac{{\displaystyle \sum _{i=1}^M{w}_i{\text{TCC}}_i}}{{\displaystyle \sum _{i=1}^M{w}_i}}$

3. Analysis of Results

3.1. Precipitation Evaluation

In order to explore the overall performance of CPSv3 in precipitation prediction, its ability to capture the spatial distributions of precipitation over the Tibetan Plateau as well as its sensitivity to the initial value are discussed. Based on the calculation of the Anomaly Correlation Coefficient (ACC) of the model predictions and observations at different lead times, the inter-annual variations of the ACC in the Tibetan region at different lead times for each month of the flood and dry seasons were analyzed (Figure 1 and Figure 2), and the multi-year average of ACC and the number of years with positive values of ACC during the study period for different lead times in each month were counted (Table 1 and Table 2). This shows that nearly half of the ACC scores of each lead time forecast in most months were negative, and the ACC is sensitive to the initial value of the model. In general, the ACC scores at 0-month lead are higher than those of at other lead times (Table 1), with the multi-year average ACC exceeding 0.3 at 0-month lead in January, August, October, November, and December, which have better forecasting skill than the other months, and followed by those in May and September, while March and April have worst forecasting skills. The number of years with positive ACC for forecasts is also highest at 0-month lead, especially during the flood season (May, July, August, and September). However, the overall precipitation prediction is relatively poor.

Figure 3 and Figure 4 show the average of Temporal Correlation Coefficients (TCC) of precipitation predicted by CPSV3 at different lead times over the Tibetan Plateau during flood season (May, July, August, and September) and dry season (January, March, April, October November, and December), respectively. The

Figure 1. ACC of the predicted precipitation by CPSv3 at different lead times with observation during the flood seasons (May, July, August, and September).

Figure 2. ACC of the predicted precipitation by CPSv3 with observation at different lead times during the dry seasons (January, March, April, October, November and December).

Table 1. Multi-year average of ACC scores of monthly precipitation over the Tibetan Plateau predicted by CPSV3 with observation at different lead times.

Month	Lead-0	Lead-1	Lead-2	Lead-3	Lead-4	Lead-5	Lead-6
January	0.37	0.07	0.04	0.07	0.02	0.05	0.1
March	0.03		−0.02	−0.02	−0.04	0.04	0.01
April	0.08	0.01		0.1	0.02	−0.03	0.06
May	0.25	0.0	0.03		0.06	0.01	−0.02
July	0.13		−0.03	−0.01	0.01		0.03
August	0.31	0.1		0.08	0.01	0.03
September	0.27	−0.04	0.06		0.03	0.11	0.03
October	0.35	−0.02	0.01	0.01		0.15	0.09
November	0.37	0.11	0.0	0.02	−0.06		−0.1
December	0.31	0.05	−0.06	0.01	0.02	0.04

Table 2. Number of years with positive ACC of the predicted precipitation by CPSv3 with observation at different lead times.

Month	Lead-0	Lead-1	Lead-2	Lead-3	Lead-4	Lead-5	Lead-6
January	23	11	14	14	12	13	14
March	12		10	10	9	13	13
April	13	10		15	12	12	12
May	18	10	12		12	12	10
July	14		9	9	12		13
August	20	13		14	12	10
September	21	8	14		14	13	2
October	20	10	9	13		15	14
November	20	14	9	11	9		7
December	19	14	12	10	13	13

Figure 3. Mean TCC of precipitation in flood seasons (May, July, August and September) over the Tibetan Plateau predicted by CPSV3 at different lead times.

Figure 4. Mean TCC of precipitation in the dry seasons (January, March, April, October, November and December) over the Tibetan Plateau predicted by CPSV3 at different lead times.

effect was similar to the ACC scores, also the highest prediction skill was found at 0-month lead for each month, with a significant decrease in skill as the lead time became longer. Therefore, the analysis was focused on lead time 0, which has the highest prediction skill. The spatial distribution of TCC scores at 0-month lead during the flood and dry seasons is given in Figure 5 and Figure 6, with the dotted area passing the test of significance at 90%.

Figure 5. Spatial distribution of precipitation TCC Skills predicted by CPSV3 in Flood Seasons (May, July, August and September) over the Tibetan Plateau at 0-month lead.

Figure 6. Spatial distribution of precipitation TCC Skills predicted by CPSV3 in dry Seasons (January, March, April, October, November and December) over the Tibetan Plateau at 0-month lead.

During the periods of flood seasons, the prediction effect is poor in May, and there is a negative correlation area in the west, the central part of Naqu city, Lhasa city and the east-central part of Rikaze city have better prediction effect in July, CPSv3 also performed better the northern part of Naqu and the northern part of Ali in August, and in most of the region in September, except for Naqu. In the dry seasons, the situation is that the central and eastern parts of the region have better prediction skill in January, the central and western parts of the region in March, Naqu, Shigatse and Ali region in April, and Shigatse, Lhasa and Shannan in November. The correlation is not significant in most of the region in October and December.

3.2. 2-m Temperature Evaluation

From the inter-annual variations of the ACC of 2-m temperature at different lead times for each month in flood and dry seasons (Figure 7 and Figure 8), multi-year averages ACC (Table 3) and the number of years with positive ACC (Table 4) during the study period at different lead times for each month, it can be seen that, similar to the case of precipitation, the ACC scores of the air temperature also decreased with lead time extended. However, the ACC scores at all lead time for 2-m temperature are higher in quantities of positive ACC years during the study period than in the case of precipitation, The mean ACC scores at 0-month lead exceeded 0.3 in January, May, October and December; exceeded 0.23 in July, August and September, while less effective in March, April, and November.

In terms of TCC, the regional mean TCC of 2-m temperature in flood seasons (May, July, August, and September) and the dry seasons (January, March, April,

Figure 7. ACC of the predicted 2-m temperature by CPSv3 at different lead times with observation during the flood seasons (May, July, August, and September).

Figure 8. ACC of the predicted 2-m temperature by CPSv3 with observation at different lead times during the dry seasons (January, March, April, October, November and December).

October November, and December) over the Tibetan Plateau at different lead times predicted by the CPSV3 are still the highest at 0-month lead (Figure 9 and Figure 10). The mean TCC scores at all other lead times were higher than the corresponding precipitation. The mean TCC scores at 0-month lead during flood seasons (May, July, August and September) were 0.32, 0.23, 0.25, 0.24, respectively, and during the dry season months (January, March, April, October, November and December) amounted to 0.41, 0.12, 0.07, 0.35, 0.2, 0.32, respectively.

Figure 9. Mean TCC of 2-m temperature in flood seasons (May, July, August and September) over the Tibetan Plateau predicted by CPSV3 at different lead times.

Figure 10. Mean TCC of 2-m temperature in the dry seasons (January, March, April, October, November and December) over the Tibetan Plateau predicted by CPSV3 at different lead times.

Table 3. Multi-year average of ACC scores of monthly 2-m temperature over the Tibetan Plateau predicted by CPSV3 with observation at different lead times.

Month	Lead-0	Lead-1	Lead-2	Lead-3	Lead-4	Lead-5	Lead-6
January	0.41	0.0	0.04	0.02	0.04	0.06	0.04
March	0.1		−0.01	0.1	0.06	0.01	0.07
April	0.07	0.08		0.02	−0.01	0.02	−0.02
May	0.32	−0.06	0.04		0.05	0.05	0.01
July	0.23		−0.01	−0.07	0.03		0.04
August	0.25	0.11		0.0	0.0	−0.0
September	0.24	0.08	0.05		0.0	0.06	−0.19
October	0.35	0.16	0.15	0.11		0.15	0.03
November	0.2	0.11	0.09	0.07	−0.02		0.07
December	0.32	0.16	0.11	0.16	0.08	0.19

Table 4. Number of years with positive ACC of the predicted 2-m temperature by CPSv3 with observation at different lead times.

Month	Lead-0	Lead-1	Lead-2	Lead-3	Lead-4	Lead-5	Lead-6
January	22	12	11	12	11	14	12
March	15		12	14	15	11	17
April	14	12		13	7	11	10
May	20	7	11		12	9	9
July	20		11	10	11		13
August	19	12		10	10	9
September	19	14	13		11	14	1
October	20	16	15	14		17	12
November	19	15	14	13	11		15
December	22	16	14	16	14	16

From the spatial distribution of TCC scores predicted by CPSv3 at 0-month lead (Figure 11 and Figure 12), it can be seen that the flood seasons has a better prediction effect in May, with Rikaze and Ali passing the 90% confidence test, and higher prediction skill in the southern part of Shannan in July and the northern part of Nagchu and the southern part of Shannan in August. The situation in the dry seasons is that high prediction skills exists in most of the region in January and December, followed by March, April and October, and lowest prediction skill is in November.

Figure 11. Spatial distribution of 2-m temperature TCC Skills predicted by CPSV3 in Flood Seasons (May, July, August and September) over the Tibetan Plateau at 0-month lead.

Figure 12. Spatial distribution of 2-m temperature TCC Skills predicted by CPSV3 in dry Seasons (January, March, April, October, November and December) over the Tibetan Plateau at 0-month lead.

4. Conclusions

Based on the CMA-CPSv3 system hindcasts data from 2001 to 2023, we evaluate the performances of precipitation and temperature prediction at different lead times. The main conclusions are summarized as below.

The model forecast skill is sensitive to the initial value, and the model forecast performance decreases rapidly as the lead time is extended.

When the initial forecasting time is the same month, the ability to grasp the spatial distributions of precipitation over the Tibetan Plateau is better in January, August, October, November, and December, with some prediction skill in May and September, and relatively poor prediction skill in March and April. Spatially, CPSv3 has better performance in the east-central part of the land in January, in the central and western part of the region in March, in the whole region in April, in the northern part of Nagchu, northern part of Chamdo and central part of Shigatse in July, in the eastern part of Chamdo, northern part of Nagchu and northern part of Ali in August, in the whole region in September, and in the cities of Shigatse and Shannan in November.

The prediction skill for different lead time to 2 m temperatures is higher than in the case of precipitation, with relatively better prediction performance in January, May, July, August, September, October, and December, and performed relatively poorer in March, April, and November. Specifically, the prediction skill is higher in most of the region in January and December, for Shigatse and Ali in May, for southern Shannan in July, and for northern Nagchu and southern Shannan in August, and there is some prediction skill in March, April and October, while CPSv3 performed poorly in November.

Acknowledgements

This study was supported by the Department of Forecasting and Networking, China Meteorological Administration (Grant No. FPZJ2023-124, FPZJ2023-125).

Conflicts of Interest

The authors declare no conflicts of interest regarding the publication of this paper.

References

[1]	Yao, T.D., Chen, F.H., Cui, P., et al. (2017) From the Qinghai-Tibet Plateau to the Third Pole and Pan-Third Pole. Journal of the Chinese Academy of Sciences, 32, 924-931.
[2]	Xu, X.D., Zhao, T.L., Lu, C.G., et al. (2014) Characteristics of Atmospheric Water Cycle over the Qinghai-Tibet Plateau. Acta Meteorologica Sinica, 72, 1079-1095.
[3]	Wu, G., Duan, A., Liu, Y., Mao, J., Ren, R., Bao, Q., et al. (2014) Tibetan Plateau Climate Dynamics: Recent Research Progress and Outlook. National Science Review, 2, 100-116.[CrossRef]
[4]	Qin, D.H. and Ding, Y.J. (2009) Cryospheric Change and Its Impacts: Status, Trends and Key Issues. Advances in Climate Change Research, 5, 187-195.
[5]	Xiao, L.H., Gao, Y.H., FEIC, et al. (2016) Dynamical Downscaling Simulation of Extreme Air Temperature over the Tibetan Plateau. Plateau Meteorology, 35, 574-589.
[6]	Zhou, T.J., Chen, Z.M., Zhou, L.W., et al. (2020) Simulations and Projections of the Western Pacifle Subtropical High in CMIP5 Models. Acta Meteorologica Sinica, 78, 332-350.
[7]	Zhang, W.L., Zhang, J.Y. and Fan, G.Z. (2015) Evaluation and Projection of Dry-and Wet-Season Precipitation in Southwestern China Using CMIP5 Models. Chinese Journal of Atmospheric Sciences, 39, 559-570.
[8]	Zhou, T.J., Zou, L.W., Wu, B., et al. (2014) Development of Earth/climate System Models in China: A Review from the Coupled Model Intercomparison Project Perspective. Journal of Meteorological Research, 28, 762-779.[CrossRef]
[9]	Zhang, D.Q., Sun, F.H. and Zhang, Y.C. (2018) Evaluation of Seasonal Prediction for Summer Rainfall in China Based on BCC Second Generation Short-Range Climate Forecast System. Plateau Meteorology, 38, 1229-1240.
[10]	Shen, H.Y., Wen, T.T., Zhao, X.R., et al. (2023) Evaluation of Multi-Model Precipitation Simulation over the Tibetan Plateau in Early Winter. Arid Zone Research, 40, 1-14.