Assessment of Different WRF Configurations Performance for a Rain Event over Panama

A set of 15 combinations of microphysics and cumulus parameterizations for the WRF numerical model were tested in the forecast of a rain event on January 16 2018 over Panama. GPM satellite, upper air soundings and ground weather stations were used to evaluate the performance of each configuration. Different metrics were used to obtain the configurations that produced the best forecasts. The analysis showed a strong fluctuation of the diurnal precipitation cycle from one region to another. From the set of tests carried, BMJ with Morrison performed slightly above the other combinations in representing well the diurnal cycle for some domain areas, followed by Thom-Gr, WSM6-BMJ and WSM6-KF. While the experiments had a better performance forecasting over sea, they were not able to match the highest values of total precipitation. Regarding spatial rain distribution, all BMJ cumulus combinations showed the best skills mainly over the area comprising the Panama Canal while the others tended to overestimate the rain. Finally, most BMJ experiments showed the highest correlation factor and performed quite well representing the vertical profiles of relative humidity, temperature and wind.


Introduction
Rainfall events (RE) are among the phenomena that cause significant negative impacts on the economy and society. This kind of events is quite frequent in Panama, where national and regional important activities such as the operations at both the Panama Canal and the international flight hub, have often been affected. Therefore, the implementation of a numerical weather forecast system becomes a high priority contribution to the improvement of early warning mechanisms and the development of new climate services oriented to energy, agriculture and transport sectors.
Panama, a relatively small country located in the extreme eastern part of Central America is frequently affected by RE of different levels of magnitude. In a general sense, high amounts of precipitation occur throughout the whole year but they are more frequent in summer. Large-scale weather systems such as disturbances associated with the Intertropical Convergence Zone (ITCZ) and cold fronts penetrating from northern high latitude appear as two of the major circulation systems that trigger convective activity [1] [2]. The narrow and elongated position of the country between the Pacific Ocean and the Caribbean Sea, as well as the moisture that is transported from both bodies of water, is also a key factor [3]. Finally, the influence of orography over the low-level convergence tends to enhance the convection activity and favors the occurrence of heavy rainfall episodes. As has been pointed by [4] [5] a significant convective activity takes place over the Panama Bight with strong afternoon rainfall over the narrow isthmus of Panama.
Precipitation episodes and particularly RE are very difficult to predict over short timescales in a complex region like Panama, where several different mechanisms interact. Despite Global Numerical Weather Prediction Models have continuously improved during the last years, their low resolution prevents them from producing a reasonably good forecast in such complex regions where convection is a critical factor producing RE. The Regional Models, which represent explicitly many of these complexities like the convection and the interactions with large-scale weather phenomena appear as a good option for implementing a forecast precipitation system [6] [7] that can produce useful and relevant information for a wider user community and to create the base for future development of end-user services [8] [9] [10].
This work presents an assessment of the ability of the advanced research core of the Weather Research and Forecast model (hereafter referred to as WRF) to forecast precipitation over Panama by exploring different model configurations based on various combinations of cumulus and microphysics parameterizations. The model outputs are thoroughly evaluated using observations from different available sources to find physically-based answers to the produced forecasts.
WRF is a very popular regional model widely used for research [6] [7] [11] and real-time operational forecasting [12] [13] [14] [15] of a variety of meteorological events, such as extreme rainfall. Despite the large amount of published work that can be found on the use of this model around the world, there are just a few studies in Central America [16] [17] and in particular over Panama [2].
The paper has been organized as follows. Section 2 offers the details of the WRF version employed in this study as well as the different model configura-

Model Configurations, Data and Methods
WRF-ARW V3.8.1 was used to carry out the simulation of the precipitation events. The initialization and boundary conditions every 3 hours, were provided by NOAA's Global Forecast System (GFS) with 0.5˚ resolution. Figure 1 shows the three simulation domains defined, with resolutions of 27, 9, and 3 km. The verification was focused in the innermost domain.
The main details of the physical configuration used for the model are pre-   Short radiation Dudhia scheme: Simple downward integration allowing efficiently for clouds and clear-sky absorption and scattering [33] Surface layer Eta similarity: Used in Eta model [34] Surface physics Noah Land Surface Model: Unified NCEP/NCAR/AFWA scheme with soil temperature and moisture in four layers, fractional snow cover and frozen soil physics [35] Planetary boundary layer Quasi-Normal Scale Elimination PBL [36] The BMP's include three single-moment schemes (Lin [18], WSM6 [19], and Thompson [20] [21]) and two double-moment schemes (WDM6 [22] and Morrison [23]). In the case of CU, the schemes selected were the convective adjustment scheme of Betts-Miller-Janjic (BMJ) [24] and the two mass flux schemes Grell-Freitas (Gr) [25] and Kain-Frisch (KF) [26].
Single-moment BMP's prognoses the third-order moment of the size distributions (mass mixing-ratio variables) of different water categories. Lin scheme simulates five classes of hydrometeors (cloud water, cloud ice, rain, snow, and hail). The exponential size distribution and terminal speed formulation are used to represent only the precipitation particles: rain, snow, and hail, including ice sedimentation. WSM6 is based on WSM5 [27], adding processes related to graupel. The terminal velocity for graupel is calculated using a mass-weighted expression following [18] [28]. Thompson has implemented a generalized gamma distribution shape for each hydrometeor, a variable gamma distribution shape parameter for cloud water droplets based on observations and the calculation of the intercept variable of rain and graupel depending on the respective mixing ratios. Double-moment BMPs were developed to improve the explicit representation of clouds and precipitation in mesoscale atmospheric models. The scheme predicts the evolution of mass, as well as number densities of the five hydrometeor types, cloud droplets, raindrops, cloud ice, snow and graupel [22] [23]. Since the number concentrations of all these hydrometeors are calculated explicitly, the relevant homogeneous and heterogeneous nucleation processes have been parameterized including the activation of cloud condensation nuclei, which is not predicted in most state-of-the-art cloud-resolving models.
The BMJ scheme of CU introduces the parameter "cloud efficiency" to calculate the relaxation time and the deep convection using specific reference states in the Betts-Miller parameterization [29] [30]. The Kain-Fritsch parameterization scheme uses a one-dimensional entraining/detraining plume model assuming that any mixture that becomes negatively/positively buoyant detrains/entrains from/into the cloud. On the other hand, the Grell scheme implemented in the WRF model follows the method developed by [31], to represent an automatic smooth transition as the resolution increases. Table 2 summarizes the relevant characteristics of the different sources of available data that were used in this work. ERA-Interim reanalysis [37] was employed for the description of the synoptic conditions during the selected day. Precipitation data within the CVA comes from a group of 38 weather stations (WS) operated by the Meteorological Service of Panama and the Panama Canal Authority (PCA). Also precipitation data from the Global Precipitation Mission (GPM) [38] were used, considering their high spatial and temporal resolution as well as their availability for the time period of this study, this allowed these data to be very conveniently compared to the model's precipitation output fields. A set of other  meteorological data such as upper air soundings provided by the PCA was also used to assess the WRF skill in relation to precipitation characteristics and other processes associated with its formation. Figure 2 shows the location of the WS.

Methodology
The methodological approach used involved the following steps. First, a comparison of 24 hours accumulated precipitation from WS and their corresponding underlying grid-point cells of GPM was made. The aim of the comparison was to assess the accuracy of GPM to represent the rainfall behavior over Panama. WRF experiments were then bilinearly regridded to the GPM grid and statistically compared, adopting different widely used metrics. These metrics included Pearson's correlation coefficient (r) and a set of categorical indexes [39] [40], that were calculated to explore the rainfall detection performance of WRF experiments. The indexes were: the probability of detection (POD), false alarm ratio (FAR), and critical success index (CSI). POD is used to describe the extent of rainfall events that are correctly forecasted; FAR reflects the extent of rainfall events forecasted that are false alarms, and CSI illustrates the overall proportion of rainfall events that are correctly detected in the forecast. Table 3 presents the mathematical expressions for computing the categorical indexes.

Synoptic Situation Analysis
The surface chart (      medium levels a layer with a relative humidity of 50% and 10 to 15 knots east winds is observed, while high levels show southwestern and western winds with a relative humidity of 50% to 60%. The wind vertical profile shows little shear and the trigger temperature is 29.6 degrees.

Correspondence between GPM Estimated Precipitation and Surface Stations Reports
As it was mentioned in the previous section, in order to assess the performance of the experiments with a better spatial and temporal resolution, the precipitation estimated by the GPM product was used. However, as it is not a direct measurement, a characterization of the product is needed. Figure 5 presents the 24 hour accumulated rainfall registered by the WS (Figure 5(a)) and the same variable estimated by GPM ( Figure 5(b)  Although there are quantitative differences in the precipitation obtained from GPM for the case studied in this investigation, this data was useful in the spatial sense, for the verification of the experiments developed. Also it was important for covering the regions where no weather stations are present, for example near the border with Colombia. Hence, these data were used to illustrate rainfall at sub-daily values in the absence of both stations and hourly stations data.

Precipitation Verification
Diurnal cycle graphics of precipitation for the 15 combinations of BMP and CU on a nine boxes mosaic that divides the WRF inner domain over Panama is presented in Figure 6. Each graph is located over its corresponding geographic region and a shaded map of the 24 hours GPM accumulated precipitation is shown in the background. Wherever significant total rainfall occurred, most of the combinations properly represented the corresponding peaks, although in general with slight displacements in time (boxes 1, 2, 5, and 7). For these boxes there stand out, with better results the combinations Mor-BMJ, Thom-Gr, WSM6-BMJ and WSM6-KF.
The rainfall estimated by GPM for regions 6 and 9, that cover the Caribbean Sea was not significant and it was over these areas where more than half of the combinations approached best the accumulates from GPM. A similar behavior occurred at box 1 which is also located over the Pacific Ocean. This indicates that most of the combinations presented greater ability to forecast precipitation  over sea for the case studied. This is an expected performance, considering that over sea, the surface friction is lower leading to a more uniform wind field and less pronounced temperature changes, allowing a better representation of convective processes. Lin-KF, Mor-KF and Thom-KF presented the mayor overestimation in these areas. At box 4 however, despite having a large part of the area occupied by sea, the model had difficulties representing the rain in the earliest and latest hours of the day for all configurations, with a huge overestimation. Something similar occurred with box 3 but less pronounced and with greater dispersion in forecasts. Over zone 8, which presents irregular coastal areas, the model presented difficulties, overestimating precipitation. It also occurred at box 7 where overestimation occurred in the afternoon. Once more combinations with the KF cumulus parameterization produced the higher values of accumulated rainfall been the ones with mayor differences from GPM.
Regions 2 and 5, also enclosed water-land interface zones and the mountainous system (Central-American Sierra Madre). In spite of it, most of the configurations were able to forecast the precipitation with low errors, representing well, though earlier, the peaks of precipitation. Mor-BMJ achieved better representation of the diurnal cycle over the box number 5, where weather stations recorded the most significant accumulated precipitation.
In general, the ability of each combination of BMP-CU varies significantly from one region to another, so it can't be easily identified, which ones achieved better accuracy. As a common feature, all experiments overestimate the hourly total area of precipitation in each box, presenting higher precision at regions where sea is predominant. It was found that experiments that use KF as CU strongly overestimate the rainfall values while the ones with BMJ fit better the daily behavior of the precipitation.  overestimation encountered over the CVA in the diurnal cycle analysis. For this particular case, just like with POD, the best combination of parameterizations was BMJ with Lin, while the experiments that have KF as CU showed the worst skill.
Pearson correlation was computed for both GPM (Figure 9(a)) and WS (Figure 9(b)) 24 hours accumulated precipitation. This correlation is spatial in the sense that was calculated for each combination, for the entire CVA and all weather stations. All experiments here showed a poor skill, especially when compared with GPM. Values lower than 0.4 indicate that the combinations of BMP-CU failed, possibly because areas of precipitation are shifted in space, misshapen or not forecasted. WDM6-BMJ, stands out slightly in the verification with GPM with 0.3 as correlation and WSM6-BMJ highlights as the best with values around 0.8 in the comparison with WS. Focusing on the experiments assessments with weather stations, another two combinations that used BMJ as CU, reached high correlations values. These are Mor-BMJ and Thom-BMJ, both over 0.5. Again BMJ stands out as the better CU selection in three of the combinations studied. On the contrary, those that use Gr cumulus parameterization showed no correlations or negative correlations with real values.
As a verification of the displacement and/or deformation of the areas of precipitation, Figure 10(a) presents the one hour accumulated rainfall from GPM for Jan 17 2018 0000 UTC. Figure 10 Figure 10(e) show the categorical maps resulting from the comparisons with GPM. It can be observed that the forecasted areas for both combinations are quite smaller than those shown on the GPM chart. It can be seen as well that there are zones of precipitation registered by GPM that the experiments did forecast, but spatially shifted (see for instance the precipitation area over Veraguas). Other precipitation events shown in GPM were missing in the experiments, for example the one southwest of Panama City. All of this could explain the presence of large areas of missing and false alarm categories, and therefore the low areal correlations between GPM and BMP-CU combinations. KF overestimation of precipitation is also made evident in the numerous areas of false alarm that appear in Figure 10

Verification with an Upper Air Sounding
In order to achieve a better understanding of the results, an evaluation with the sounding at Jan 17 2018 0000 UTC, was carried out. The numerical vertical profiles for wind velocity, relative humidity and temperature were extracted from each experiment following the real sounding trajectory. Figure 11 shows  All experiments were able to represent the vertical humidity profile close to the sounding measurement ( Figure 11(b)). However, between 700 and 600 hPa, they showed difficulties in the estimation of the variable, this could be related to the amount of vertical levels defined in the model grid. It is appreciable that BMJ and KF schemes combined with all BMP schemes are the ones that represented best the vertical profile of relative humidity. Even though BMJ does not account for any changes that occur below the cloud base [24], it showed the most appropriate behavior of humidity at lower levels (except when combined with WSM6).
In the BMJ scheme, moisture profiles are a determinant factor in the identification of the different states of convective equilibrium. The scheme also succeeded to represent well the profiles of humidity and temperature (not shown), which are used to calculate the entropy and "cloud efficiency", that measures the ability of the convective column to transport the enthalpy in the vertical producing the least possible precipitation. This could explain the ability presented by BMJ in well representing the total amount of precipitation as well as the behavior of the diurnal cycle, for this case studied. The rain in the sounding area was mainly caused by diurnal heating. The temperature and humidity profiles reflected the adjustments made by the scheme that forced them to climatological reference profiles. Those schemes managed to adequately represent the vertical profile of both variables. The BMJ parameterization, which is supported by the concept that convection is a purely thermodynamic process, was therefore the most capable of reproducing precipitation in the area.
On the other hand, Gr presented less skill predicting the variables, overestimating the precipitation in most combinations. A study conducted in the tropical zone describing the African monsoon presented similar difficulties with this parameterization [41]. All KF combinations also overestimated the amount of rainfall, which may be related to the fact that this scheme tends to leave an extremely deep saturated layer after convection [42] in order to better represent

Conclusions
The main purpose of this work was to assess the ability of the WRF model to forecast rainfall events in Panama. In this regard, the performance of WRF was  The categorical evaluation of the occurrence of precipitation demonstrates the best ability of those schemes that combine BMJ as cumulus parameterization. This happens mainly over the area comprising the Panama Canal where Lin-BMJ presented a clear improvement over all other cumulus schemes. The overall WRF performance of rainfall/no-rainfall events forecast was fairly poor and it explains the overestimation found in the diurnal cycle within the validation area. The worst combinations were those that include the KF parameterization. For all combinations, there were low spatial correlation values of predicted rainfall with respect to GPM estimates, suggesting a model deficiency to represent the spatial distribution of precipitation for the selected study case. When spatial correlations were computed versus the station data, higher values emerged. Here also BMJ combinations were better. The experiments that use the Gr scheme stood out as the worst, showing no correlations or negative correlations with observations from stations.
All experiments represented the vertical humidity profile close to the sounding. It is appreciable that BMJ and KF schemes combined with all BMP schemes yielded the best humidity profiles. BMJ showed the most appropriate behavior of humidity at lower levels. While most BMJ experiments underestimated the wind profiles at lower levels, the rest of the BMP-CU combinations and also WSM6-BMJ exceeded the intensity values. For medium levels BMJ represented better the changes in wind velocity with the exception also of WSM6-BMJ. In general, most of the combinations followed the profile peak above 200 hPa level reported by the sounding.
A general feature that stands out from all the tests performed is that changes in cumulus parameterizations have a higher impact on the model's performance statistics than those in microphysics, with BMJ scheme yielding the best results.