Calibration and Quantitative Forecast of Extreme Daily Precipitation Using the Extreme Forecast Index ( EFI )

Based on the extreme forecast index (EFI) of ECMWF, the “observed” EFI (OEFI) of observation is defined and the EFI is calibrated. Then the EFI equivalent percentile (EFIEP) and EFI equivalent quantile (EFIEQ) are designed to forecast the daily extreme precipitation quantitatively. The formulation indicates that the EFIEP is correlated not only to the EFI but also to the proportion of no precipitation. This characteristic is prominent as two areas with nearly same EFIs but different proportions of no precipitation. Cases study shows that the EFIEP can forecast reliable percentile of daily precipitation and 100% percentiles are forecasted for over max extreme events. The EFIEQ is a considerable tool for quantitative precipitation forecast (QPF). Compared to the probabilistic forecast of ensemble prediction system (EPS), it is quantitative and synthesizes the advantage of extreme precipitation location forecast of EPS. Using the observations of 2311 stations of China in 2016 to verify the EFIEP and EFIEQ, the results show that the forecast biases are around 1. The threat scores (TS) for 20 years return period events are about 0.21 and 0.07 for 36 and 180 hours lead times respectively. The equivalent threat scores (ETS) are all larger than 0 and nearly equal to the TS. The TS for heavy rainfall are 0.23 and 0.07 for 36 and 180 lead times respectively. The scores are better than those of high resolution deterministic model (HRDet) and show significant forecast skills for quantitative forecast of extreme daily precipitation.


Introduction
Extreme precipitations always accompany disasters in China [1].To forecast them is the primary goal globally.Recently, as the development of ensemble prediction system (EPS), many tools have been developed to forecast them and the forecast skills arise steadily [2].
One of the very popular tools is the Extreme Forecast Index (EFI) [3] of European Centre of Medium Weather Forecast (ECMWF) which is the weighted integral of the differences between the cumulative probability distribution functions (CPDF) of ECMWF EPS and the "model" climate established from the re-forecast dataset.The EFI is based on the hypothesis that if the forecast "weather" is extreme and relative to the "model" climate, the real weather would be an extreme event compared to the real climate [4].Because the EFI uses the same model re-forecast dataset, it is free of model bias.And it is a good tool for the area where observations are scarce.Many researches show that the EFI is an effective indication for extreme events, including extreme wind [5], windstorms [6] and heavy rainfall [7].
However, the disadvantage of EFI is obvious too, because its meaning is implicit [8].Typical question is: what does precipitation EFI of 0.7 mean.Is it the probability or intensity of some special event-50 mm/day precipitation for example?Forecasters cannot directly use it to forecast the probability or intensity of the extreme precipitation events.
To address these criticisms, many researches have been done.There are mainly two kinds of them.One is EFI thresholds estimation through verification for the need of application.Boisserie et al. [6] used the EFI and "Shift of Tails" (SOT) of 30 years to calibrate the forecast for extreme windstorms in France and got the thresholds and measures for different ranks events through verification.[5] verified the gust EFI for three airports' daily wind extremes and the EFI thresholds are estimated.Dong et al. [9] verified the EFI and SOT for extremely high and low temperature and extreme precipitation in China and estimated the thresholds for different percentiles.

Petroliagis and Pinson
The other kind of research is producing new products whose meanings are explicit for extreme weather forecast.Prates and Buizza [8] introduced a product of probability of return level (PRET) from the observation and re-forecast datasets.Using the Generalized Extreme Value (GEV) distribution, the return levels for different return periods of observation and model are estimated and the PRET is calculated for different return levels.It not only can provide similar information to the EFI but also is more explicitly understood.Standardized Anomaly Forecasting (SAF) is another product for extreme weather forecast which is the normalized difference between the forecast and expected value of climate distribution [10].SAF is normalized by the climatological standard deviation, so it can be immediately understood.Guan and Zhu [11] verified and compared the SAF and EFI, and found that the SAF and EFI are highly correlated and the Q.Dong performances of them for extreme cold events in America are nearly similar.
The EFI has its own advantage.By using the same model re-forecast as "model" climate, the EFI is considered independent of model bias which is common for numeric weather prediction models and difficult to be estimated and corrected [12] [13] [14].On the other hand, through revised formulation of EFI and continuous improvement of re-forecast suite [4], the forecast skill of EFI for extreme events has been improving continuously.
The EFI is the integral of the differences between the CPDFs of EPS and "model" climate.It concentrates the information.Meanwhile there is no observation index corresponding to the EFI explicitly.It made the EFI difficult to be understood directly.To make the meaning of EFI significantly, the observation index-the "observed" EFI (OEFI) is established firstly from the daily precipitation amount observation and real climate of meteorological stations in this work.
Then, the OEFI and EFI are compared with each other and their mapping relationship is estimated to calibrate the EFI.The EFI equivalent percentile (EFIEP) and equivalent quantile (EFIEQ) are designed based on the calibrated EFI.The EFIEP is the percentile forecast and the EFIEQ is a quantitative forecast of extreme event.The daily precipitation amount observation in 2016 of China is used to verify the EFIEP and EFIEQ.This paper studies the two forecast modes of OEFI and EFI in order to improve the capability of calibration and quantitative forecast of extreme daily precipitation.
After this introduction, the formulations of EFI, calibration approach, EFIEP, EFIEQ and the datasets used are described in Section 2. In Section 3, two extreme rainfall cases and one extreme snowfall case of China are presented and the verification results are shown in Section 4. The conclusions and discussions are drawn in Section 5.

EFI and Definition of "Observed" EFI (OEFI)
To measure the differences between the CPDFs of EPS and "model" climate, the EFI is a weighted integral of the differences.The weights make the EFI more sensitive to the tails of probability density function [15].For the precipitation, of which there are many samples of no precipitation, the EFI formulation is: where p is the "model" climate probability and  is sensitive to the tails of probability distribution [15].
The EFI cannot be understood directly and there is no "observations" related to the EFI directly.Then, define the "observed" EFI (OEFI) to be the integral of differences between CPDFs of real climate and real weather.The cumulative probability of real weather takes the value 1 or 0 according to whether the climatic quantile is larger than the observation or not, same to the approach of Brier score for probability forecast verification [16].If the percentile of observation corresponding to the real climate is p 1 , the OEFI can be written as: Integrating this equation can get the result: ( ) Journal of Geoscience and Environment Protection in which 1 1 arcsin p θ = . The OEFI is a function of p 1 and p 0 and is an increasing function of p 1 and decreasing one of p 0 (Figure 2).It is obvious that the percentile of precipitation relative to the climate must be larger than the proportion of no precipitation (Figure 2).The OEFI is equal to −1, only when the p 0 is 0 and p 1 is 0 too (Figure 2).Because θ 1 is between the interval of [0, π/2] and A(θ) is a decreasing function of θ between the range of [π/2, 0], OEFI is between the interval of [−1, 1] which is same as the EFI.When the precipitation is equal to or larger than the max of the real climate, the p 1 is equal to 1 and the OEFI is equal to 1 consistently.If the observation is the median relative to the climate-p 1 is 0.5, and p 0 is 0, OEFI is 0. It means that OEFI is negative if the observation is below the climate median, and positive if the observation is above the climate median.This characteristic is similar to the EFI [3].When the p 1 is equal to p 0 the OEFI is not constant and different from p 0 .For extreme events of 95% percentile relative to the climate, the OEFIs are about 0.71 for p 0 equal to 0.1.When the p 0 is equal to 0.6, which is a typical value for Yangtze and Huaihe River basins of China in summer, the OEFI is about 0.62, which is close to the suggested thresholds by ECMWF.correlated to the OEFI significantly.As the EFI tends to 1, the OEFI leans towards 1 too.It means that when the forecast "weather" is an extreme event relative to the "model" climate the real weather is likely to be an extreme event according to the real climate.The hypothesis of EFI is acceptable.

Relationship between the EFI and OEFI
The scatter points are located at the diagonal mostly (Figure 3) which means the OEFI value is nearly same to the EFI qualitatively and the definition of OEFI is reasonably.In west of China and north of China, there is always no precipitation for a month-p 0 is 0 and p 1 is 0 too, so the OEFIs are 0 frequently.On the other hand, the EPS would forecast precipitation probability at these areas always, and the EFI would not be 0. Therefore, there are more 0 of OEFI compared to the EFI (Figure 3).
Because the precipitation, precipitation EFI and OEFI are not normally distributed, the standard linear regression with least squares methods cannot be used to estimate the relationship [17].So, the quantile mapping method is used to map the EFI to the OEFI [18] [19].The method is simple: since the samples of OEFI are the same to that of EFI, the EFI quantiles are directly mapped onto the OEFI quantiles, in which the quantiles are replaced by the ranks.The red line in Figure 3 indicates the quantile-quantile (Q-Q) plots of the 36-h EFI forecast Q. Dong against the corresponding OEFI for the 2311 stations of China in 2015.The quantile mapping method is applied to the EFI to calibrate it and would make the expectation of forecast bias to be 1.
With the calibrated EFI (CEFI), the expected percentile of precipitation event-defined as EFI equivalent percentile (EFIEP), could be calculated with the inverse function of (3): Figure 4 shows the 36-h forecasts of EFI against the observed percentile of the 2311 stations of China in 2016 with p 0 of samples between 0.59 and 0.61 and the theoretical relationship between the EFI and the EFIEP from Equation ( 4) with p 0 equal to 0.6 for different lead times.This scatterplot is common and wide used in the EFI verification and application and the distribution pattern is similar to the past works [5] [6] [9].It shows significant correlation between the EFI and the percentiles of events and can be used to verify the EFI and determine the thresholds for exact percentile events.Compared to linear regression, the non-linear correlation of Equation ( 4), using calibrated EFI with quantile mapping, is more reasonable.
The black line in Figure 4 indicates the relationship of Equation ( 4) with no EFI calibration.It is close to the observations, which indicates that the hypothesis of EFI is reasonably.For the same EFIEP, the EFI thresholds decrease with increasing forecast times, which is consistent with the observational verifications [20].This characteristics cannot be shown by the no EFI calibration.As the EFI From this formulation of Equation ( 4), it is indicated that the EFIEP is an increasing function of no precipitation proportion p 0 and EFI, because A(θ) is a decreasing function and CEFI is an increasing function of EFI.When the EFI is 1 which means that all the members exceed the "model" climate maximization, the EFIEP is 1 too.It means that the quantitative forecast of extreme precipitation is equal to or exceeds the real climate maximization.
Making the p 0 equal to 0.6, which is a characteristic value at the Yangtze River basin of China in summer, the EFI is 0.73 and 0.83 for 97.5% and 99% EFIEP respectively.These values are close to the optimal EFI threshold for forecasts of 97.5% and 99% percentile precipitation events under the criteria of TS maximization [9].So, the EFIEP is an effective index for quantitative forecast of extreme precipitation and its meaning is understandable directly.With the EFIEP and real climate, the EFI equivalent quantile (EFIEQ) is defined and estimated as a quantitative forecast of extreme daily precipitation.

Real Climate and Verification Dataset
The primary factor of the EFIEP is the establishment of real climate.In the for- For the calculation of the OEFI of 2015 in Figure 3 and Figure 4 The OEFIs in 2015 are used to establish the calibration relationship between the EFI and CEFI.Then the approach is applied to the EFI forecast in 2016 and the verification is estimated against the observations.The return period is used instead of the event percentile related to the real climate.Because 31 days of moving window is applied, the return period is calculated by the formulation: ( ) The unit of RP is year.When the climate max is exceeded and the p 1 is equal to 1, the RP is defaulted to 36 years, because the real climate is sampled from 35 years dataset.

Extreme Precipitation in North China
On 19 and 20 July 2016, extreme precipitation happened at North China around Beijing (Figure 1) and some stations exceeded the maximization of historical records and some exceeded the maximization of July in history [26] [27] [28].The cyclone effected north of North China including Beijing at 20 July 2016 and more precipitation happened in Beijing [26].Here, just the precipitation happened at 19 July is researched.
The EFI forecast from 16 July onward, which corresponds to lead time of 3 -4 days, captured the extreme precipitation event well [28].As Figure 6(c) shows, the 36-h EFI forecast is larger than 0.7 at North China.It reaches 0.9 at middle of Shanxi and southwest of Hebei provinces, which means high probability of extreme precipitation at this area.But how much probability and what is the intensity of the extreme precipitation will happen cannot be told by EFI or SOT.
Other information, probability forecast from ensemble prediction system for example, should be considered to answer these questions [4].
With the definition of OEFI and calibration of the EFI, the EFI forecast was converted to EFIEP and EFIEQ.For the 36-h forecast, the EFIEQ are larger than 100 mm at southwest Hebei and northeast Henan provinces with return periods of about 30 years.The EFIEQ and EFIEP are smaller than the observations.But, The high resolution deterministic (HRDet) forecast always underestimates the precipitation.The 48-h quantitative precipitation forecast (QPF) of HRDet at Beijing is about 22 mm.However the EFIEQ is 49.5 mm which is closer to the observation of 50.2 mm (Figure 1).The EFI cannot be understood directly and cannot be solely used to forecast the intensity or probability of extreme precipitation.For extreme events, the HRDet always underestimate the intensity or rar-Q.Dong Journal of Geoscience and Environment Protection ity.The EFIEQ is a reasonable tool to forecast the extreme precipitation quantitatively.
For 84-h forecast, the EFIEP forecast the percentiles of 100% at southwest of Hebei and northeast of Henan provinces and the EFIEQ reaching the max of climate (Figure 7).So the EFIEQ cannot forecast the quantitative precipitation over the max of climate.It is the limitation of EFI and the reason of design of SOT.The extreme precipitation at middle of Shanxi province is missed by the EFIEQ and EFIEP which is resulted from the missing forecast of EFI.
The EFI of 84-h forecast is smaller than that of 36-h, but the EFIEP and EFIEQ are not smaller than that of 36-h.It indicated the advantage of lead time dependent calibration approach of EFI.
There are some stations' daily precipitation exceeds the max of real climate [26].These characteristics are indicated by the EFIEP forecast of 100% percentile.However the amount of daily precipitation was underestimated by the EFIEQ (Figure 7).This is the drawback of quantile estimation from real climate.
The EFIEQ cannot exceed the max of real climate from the statistical estimation.
In summary, Figure 6 and Figure 7 is not the same as the main reason is the forecast time is not the same. .So it is the effect of p 0 in the Equation (4).The EFIEP is not dependent on p 1 only.In the former research [5] [6] [9], the EFI is only correlated to the event's percentile or rank relative to the climate, and the performances for different EFI thresholds for different extreme events are estimated.In this work, the EFI is correlated not only to the percentile but also to the proportion of no precipitation (Figure 2).The p 0 are about 58% and 65% at east of Zhejiang and west of Fujian provinces respectively at 16 September.So for the same EFI, the EFIEP are different for the two areas.It is more realistic to the observation.

Typhoon Meranti Heavy Rainfall in East China
The return period forecast of 120-h forecast is more than 35 years at east of Zhejiang province, indicating extreme precipitation.The EFIEQ shows nearly 300 mm daily precipitation at those areas, although the exact position of extreme precipitation is biased.For the medium range forecast, the EFI misses the extreme precipitation at south of Jiangsu province and the EFIEP misses too.

Heavy Snowfall in Northwest China
In 19 and 20 February 2017, heavy snow happened at Xinjiang province, China.

Verification Result
The EFIEP shows significant skills for percentile forecast.Figure 10 shows the performances of EFIEP for different percentiles of Chinese 2311 stations in 2016.
The TS and ETS decrease with the elongated lead times and increasing return periods.The TS is about 0.4 and 0.3 for the 36-h and 180-h forecast respectively for 1 year return period event.It is about 0.21 and 0.07 for 20 years return period event.The ETS is all larger than 0 and nearly equal to the TS.The forecast bias is around 1, except the 48-h, 60-h and 72-h forecast which are nearly 1.5.
Heavy rainfall which is defined as more than 50 mm precipitation per day in China always results to flood, urban waterlogging, mountain torrent and other related disasters, especially heavy rainfall for continuous days [1].QPF is important for disaster prevention.The HRDet QPF underestimates the precipitation commonly and the EPS does not give QPF directly.
The EFIEQ underestimates the precipitation too (Figure 11).It is resulted the EPS probability of the exact quantile of climate probability p (Figure1).p 0 is the proportion of no precipitation days in the climate and

Figure 1 .
Figure 1.Real climate, observation and EPS 48-h forecasts of 24 hours precipitation amount from 00 UTC 19 to 00 UTC 20 July 2016 of Beijing with corresponding EFI of 0.651.The forecast of ECMWF high resolution deterministic model (HRDet) is also shown.The EFIEP is 0.977 and EFIEQ is 50.2 mm that is better than the EPS mean and HRDet.Because of lack of re-forecast dataset, the "model" climate is shown schematically.

Figure 3
Figure 3 shows the scatter plot between the 36-h EFI forecast and the corresponding OEFI for the 2311 meteorological stations of China in 2015.The EFI is

Figure 3 .
Figure 3. Relationship between EFI and "observed" EFI (OEFI) using the observations of 2311 stations of China in 2015 and the corresponding EFI forecast of 36 hours lead time.The red line indicates the quantile mapping relationship between EFI and OEFI.

Figure 4 .
Figure 4. Scatterplot between the EFI of 36 hours lead time and the corresponding observation percentiles for the 2311 stations of China in 2016 with samples p 0 between 0.59 and 0.61 and the theoretical relationship between the EFI of different lead times and the observation percentiles with p 0 equal to 0.6 (solid lines).
of EFI, the "model" climate is based on 5 weeks of re-forecasts run every Monday and Thursday with 10 perturbed and 1 unperturbed member of the last 20 years.There are totaling 11*9*20 = 1980 re-forecast samples of the "model" climate for each grid point.This approach allows seasonal variations and model drift to be taken into account[4].The "model" climate is within a sampling moving window about 29 days and represents the last 20 years variation.The real climate should maintain these features of "model" climate.The sampling moving window of 31 days is applied.The sample number of real climate should be close to that of "model" climate.The sampling years should be as long as possible to collect enough samples.But, it could not be too long to avoid new interdecadal variations.The precipitation observations have been recorded from 1951 for about 600 stations.Until 1980s, the stations increase to about 2300.However, the climate and precipitation exhibit decadal variability[21].East Asian climate has experienced an interdecadal scale transition since the late 1970s[22] [23][24], including precipitation[25].The extreme precipitation exhibits changes in the past 60 years in China[1].This shifting signal in late 1970s does not exist in the "model" climate, because only the last 20 years re-forecast dataset is used.The real climate should exclude this signal too.Therefore, the observations from 1981 to the last year are used to establish the real climate.
The event was connected to a synoptic scale cyclone.A westerly trough moved from central Asia.As it reached west of North China, south wind advected warm and wet air from South China to ahead of the trough and intensified it significantly.A low pressure system developed rapidly.As the cyclone moved to North China, deepening cyclone and action with the subtropical high of Northwest Pacific (SHNP) at Southeast of the cyclone, a southerly low level jet (SLLJ) was established.This SLLJ characterized by over 20 m/s speed at the core and expanded from South China Sea to North China (Figure5(a)).It advected wet and warm air to North China and formed a vapor belt with specific humidity around 20 g/kg expanded from South China Sea to North China.The warm and wet air fueled the cyclone and resulted to the extreme precipitation.At the high level of troposphere, there was a trough at North China and the accompanied high level jet (HLL) was featured (Figure 5(b)).The PV anomaly injection at high level and the secondary vertical circulation accompanied with the HLL, triggered or intensified the convection and precipitation.The 24 hours precipitation amount from 0000 UTC 19 to 0000 UTC 20 July

Figure 5 .
Figure 5. (a) The 500 hPa height (black line every 40 m), 850 hPa wind, special humidity (shaded; unit: g/kg) and wind speed (red line every 4 m/s from 12 m/s) and (b) 200 hPa height (black line every 40 m), wind and wind speed (shaded; unit: m/s) at 1200 UTC 19 July 2016.The blue rectangular indicates the area of Figure 6.

Figure 6 .
Figure 6.The observation of 24 hours precipitation amount from 0000 UTC 19 to 0000 UTC 20 July 2016 (a) and the return periods relative to the real climate (b) and the corresponding 36-h forecast of EFI (contours), EFIEQ (c), and the return periods (d).

Figure 10 .
Figure 10.The forecast performances of EFIEP for different percentiles represented by return periods for the 2311 stations of China in 2016.

Figure 11 .
Figure 11.The scatterplot between daily precipitation observations and the corresponding EFIEQ 36-h forecasts for the 2311 stations of China in 2016.The red line indicates the linear fit by least square method.