Validating the California Puff ( CALPUFF ) Modelling System Using an Industrial Area in Accra , Ghana as a Case Study

The performance of the California Puff (CALPUFF) modelling system has been evaluated using a case study in Ghana. The performance evaluation consisted of a quantitative comparison of dispersion simulation results of SO2 and NO2 with measurements at the Tema Oil Refinery, and meteorological simulation results with observations from the Tema Meteorological Station, both in the Greater Accra region of Ghana. Four statistical indicators—Index of Agreement (IOA), Fractional Bias (FB), Normalized Mean Square Error (NMSE) and the Pearson correlation coefficient(R) employed in the assessment indicate sufficient reliability of both CALPUFF and its meteorological simulator, CALMET. IOA values of 0.73 and 0.67 and FB values of 1.65 and 1.42 were obtained for SO2 and NO2 respectively. IOA between measured and modelled emissions were 0.72 and 0.69 for SO2 and NO2 respectively. The correlations between the simulated and observed emission were 0.66 and 0.08 for SO2 and NO2 respectively. An IOA value of 0.66 was obtained for both wind speed and wind direction with correlations of 0.29 and 0.58 in comparison with observations from the meteorological station.

order for model results to be acceptable, the model must be validated to demonstrate that it can produce reliable results for a given modelling scenario.
The kind of data needed for verifying model output depends on the model itself and the users' needs.For models with meteorological pre-processors or coupled meteorological/chemical models, atmospheric variables observation in some points of the domain are required in order to validate results.Observations can be made at ground level or with a vertical profile, in the case of three dimensional fields.In the case of chemical species concentration, monitoring stations could supply data needed to check model results [1].[1]- [3] propose some statistical performance measures namely: Correlation Coefficient (R), Mean Bias (MB), Fractional Bias (FB), Normalised Mean Square Error (NMSE), Geometric Mean (GM), Geometric Variance (GV) and Index of Agreement (IOA).
The coefficient of correlation is the measurement of the relationship between observed and predicted values.It indicates the tendency of the predicted values to change with a change in the observed values.A value of R close to unity implies good model performance.The NMSE measures the random spread of the values around the mean and characterizes the amount of deviation between predictions and observations.A good model will have an NMSE value of 0. The IOA reflects the degree to which the observed variable is accurately predicted.The IOA varies from 0 (the theoretical minimum for an inadequate prediction) to 1 (perfect accuracy between the predicted and observed values).The FB is a measure of the systematic bias of the model.It indicates the tendency and the sign of the deviation.A negative FB value indicates model over-prediction and a positive value, an under-prediction.
In most cases, a model is considered acceptable if most of its predictions are within a factor of 2 of the observations [4] [5].On the other hand, studies by [6]- [8] report that a model can be deemed acceptable if: NMSE ≤ 0.5, −0.5 ≤ FB ≤ +0.5, and IOA > 0.5.
The objective of this study is to assess the performance of the CALPUFF modelling system using measured SO 2 and NO 2 data at the Tema Oil Refinery and meteorological data from the Tema Meteorological Station.

Study Area
The study area, capturing Ghana's only refinery in Tema, the emission source for this study and spanning a domain size of 60 km × 60 km, is located in the Greater Accra Region of Ghana stretching along the Ghanaian Atlantic coast and extending a bit north into Ghana's interior as shown by Figure 1.The Universal Transverse Mercator (UTM) coordinate of the southwestern corner of the domain in zone 30 are 795 km easting and 600 km northing.
The study area covers the Tema municipal area, parts of the Accra Metropolitan area and extends to parts of the Eastern region.The southern boundary of the metropolis of Accra which coincides with that of the study area is the Gulf of Guinea.Tema is the most industrialised city in Ghana and is locally nicknamed the "Harbour Town" because of its status as Ghana's largest seaport.It is a major trading center, home to Ghana's only oil refinery, the Tema Oil Refinery, and other numerous factories, and is linked to Accra by a highway and railway.

Model Description and Simulation Set-Up
The 3-component CALPUFF modeling system consists of CALMET (a 3-D meteorological model), CALPUFF (an air quality dispersion model) and CALPOST (a post-processing package).CALPUFF is a non-steady-state Lagrangian puff dispersion model, which simulates the effects of time and space-varying meteorological conditions on pollution transport, transformation, and removal.It is capable of modeling instantaneous or continuous releases over distances ranging from tens of meters to hundreds of kilometers.CALMET is a diagnostic meteorological field generator.It combines data from surface stations, upper air stations, over water stations, precipitation stations, with gridded three-dimensional meso-scale data and geophysical data like land use, terrain elevations, to develop fine-scale winds and other meteorological fields consistent with the terrain and land-use resolved on the fine scale grid.The CALMET model can be derived with data either generated by prognostic meteorological models or station observations or combination of both.The three-dimensional meteorological fields generated by CALMET are utilized by CALPUFF to calculate the dispersion of emissions over a specified area.CALPOST is a post-processor designed to average and report concentration results [9] [10].In this study, CALMET model was initialised with Weather Research and Forecasting (WRF) data to develop the meteorological field for CALPUFF due to the unavailability of a complete (surface and upper air) meteorological data from local meteorological stations.The WRF model is a fully compressible, non-hydrostatic mesoscale model with a hydrostatic option which uses a terrain-following hybrid sigma-pressure vertical coordinate in its meteorology simulation [11] [12].Two one-way nested computational domains were set with horizontal grid spacing of 24 km and 12 km representing horizontal grid dimensions of outer and inner domains respectively.The inner domain covered the whole of Ghana within which the study area is located.The model was run with Global Forecast System (GFS) 1 degree reanalysis as initial and boundary conditions with full physics processes included in order to reproduce real meteorological phenomena.Other options include WRF Single Momentum (WSM) 6 Class Graupel for Microphysics Scheme, Rapid Radiative Transfer Model (RRTM) scheme for long wave radiation, Dudhia scheme for short wave radiation, Monin-Obukhov scheme for surface layer, and NOAH model (5 soil layers) for land-surface interactions, while the dynamic options were left as default to produce hourly averages [13].WRF data for a period of 1 -31 January, 2008 was generated.CALMET then interpolated and adjusted the coarse resolution of WRF data by accounting for the kinematic effects of higher resolution terrain and land-use.CALMET utilized a divergence-minimization scheme to solve for horizontal components of wind field and similarity theory to extrapolate a vertical profile of wind velocity.The meteorological domain is a 60 km x 60 km with a grid resolution of 1 km.
CALPUFF was then executed with estimated emission rates of SO 2 and NO 2 from the refinery.The CALPUFF model predicts concentrations at specific points or receptors, which are established by the user within the modeling domain.Of the 38 receptor locations specified in the study domain, only one (Tema Oil refinery) also served as a monitoring site [14].Therefore, evaluation of the dispersion model was performed with measured results from this station.

Model Evaluation
To validate CALPUFF, measured data of SO 2 and NO 2 emissions made from the 10 -21 January, 2008 were used.The Differential Optical Absorption System (DOAS) used for the measurements was installed on the premises of the Tema Oil Refinery to monitor and measure the concentration of gases emitted into the atmosphere due to its activities and also due to the operations of other industries within its environs.Details of the set-up and results can be obtained in [15].
To determine the reliability of the simulation data, verification of simulated values using the WRF and CALMET models was conducted for surface wind speed and direction.Datasets for January 2008 from the Tema Meteorological Station (TMS) were used.Due to the large number of missing data of the other surface wind parameters as well as data format constraints, only wind speed and direction were validated.Figure 2 shows the locations of the TOR and the TMS in the terrain map of the study area.
The statistical verification of model performance in this study was performed using four statistical indicators namely the Index of Agreement (IOA), Fractional Bias (FB), Normalized Mean Square Error (NMSE) and Pearson Correlation Coefficient (R) proposed by [2] [3] which are in agreement with the USEPA guidelines pertaining to model evaluation protocol.The formulae used to derive these indicators are given by: ( ) ( ) where C o is the observed quantity, C p the predicted quantity, C oi and C pi are the observed and predicted quantities respectively for N cases.

Validation of CALPUFF
Plots of measured and modelled SO 2 and NO 2 are seen in Figure 3 and Figure 4 respectively.
The performance assessment of the model, based on direct quantitative comparisons of observed and predicted mean concentrations as seen in Figure 3 and Figure 4, reveals that CALPUFF predictions for SO 2 were better than NO 2 concentrations.From the 10 -12, 17 -19 and 21 January, predicted SO 2 concentrations closely approached measurements.Quantitative agreement between predicted and modelled NO 2 concentrations is excellent on 13 January and good for 19 -21.However, from 13 -16 January, CALPUFF significantly under-predicted the measured values.Despite these differences between predicted and measured values on some days, the trends in the measurements are accurately predicted especially for SO 2 .The simulation results did not take into account SO 2 and NO 2 background concentrations because this data was not available.This may explain the reason for this under-prediction.The measured concentrations are likely to include emissions from other sources which are not considered in the simulations.Another important reason could be the assumption made that emission rates from the refinery are constant.This is unlikely to be representative of actual operating conditions at the refinery.It was, however, necessary to make this assumption because information on the hourly, daily and seasonal variations of emission rates was not available.Additionally, possible errors associated with the measurements of SO 2 and NO 2 at the measurement location cannot be overlooked.Results of model performance evaluation after applying the USEPA guidelines pertaining to model evaluation protocol are presented in Table 1.
The index of agreement between predicted concentrations and measurements is better for SO 2 and NO 2 .The correlation coefficient of SO 2 with observations is satisfactory but that for NO 2 is negatively weak as shown in Table 1.The NO 2 emissions from automobiles make up a large fraction of ambient NO 2 levels.It is therefore probable that a large fraction of the measurements consists of contributions from the vehicular emission, revealed in the weak correlation and the poor index of agreement.CALPUFF under-predicts NO 2 by a smaller factor than SO 2 as revealed by their FB values.The NMSE values for SO 2 are acceptable whiles that of NO 2 is not.Therefore, based on these indices, the performance of CALPUFF can be described as acceptable.

Validation of CALMET and WRF Models
Plots of CALMET and WRF outputs with available surface observations (wind speed and direction) for January 2008 from the Tema Meteorological Station (TMS) are shown in Figure 5 and Figure 6.Wind roses generated from the models are also presented in Figure 7 and Figure 8.
It is apparent from Figure 5 that WRF and CALMET models generally under-predicted wind speeds from 9 -14 January and over-predicted speeds on 16 -24 of January.However, on the 11, 16, 21 and 23 days, CALMET predictions closely approach observations whiles WRF predicts similar speed on January 15. Figure 6 reveals the over-prediction of wind direction by both models for most part of the period considered.From the other statistical performance measures given in Table 2, it can be observed that both CALMET and WRF share similar indices rating their performances on the same scale with CALMET indicating slightly higher correlation coefficients for both wind direction and speed than WRF.Under-prediction by CALMET is greater than by WRF as seen in the FB values.
The wind rose generated from CALMET as seen in Figure 7, indicated the predominant prevailing wind is south south-westerly (26%) whiles southerly and southwesterly represent about 26%.The remaining winds are spread across a wide spectrum ranging from westerly to north north-easterly.WRF wind rose as presented by Figure 8, predicted an increased percentage of south south-westerly winds (31%) than CALMET.It also indicates 12% of southerly winds and 12% of south-westerly winds which is lower than CALMET.It is clear that the percentages of the other winds are also lower in the WRF wind rose than the CALMET.
The predominant dominant wind speed class predicted by both models is 3.3 -5.4 m/s with CALMET predicting a higher percentage (75%) than WRF (64%).30% of the total winds have speeds between 5.4 -8.5 m/s according to WRF whiles CALMET predicts only 3% of that class.Generally, average wind speeds is higher for     WRF than CALMET.Calm winds in both modelled wind (>0.5 m/s) account for only 0.28% of the data.Based on the statistics, it appears that CALMET predictions better approximate observations as it is able to capture small scale effects which cannot be accounted for by WRF.Numerical meteorological models tend to have spatial and temporal phase errors in simulating surface wind.This is because the lowest-level WRF winds must be extrapolated down to ground-level to initialize CALMET.Thus unless the vertical resolution is quite fine, it would be expected that the near-surface CALMET output winds will be biased towards the trends seen in the lowest level of the WRF data, which has a higher elevation in the region compared to the surface stations [16].

Conclusion
The CALPUFF modelling system has been validated using measured concentrations of SO 2 and NO 2 at the Tema Oil Refinery and meteorological data from the Tema Meteorological Station in the Greater Accra Region of Ghana.The statistical measures between observations and simulation results are indicative of the acceptable performance of the dispersion model, CALPUFF, its meteorological component, CALMET and the prognostic WRF model.

Figure 1 .
Figure 1.Map of study area.

Figure 2 .
Figure 2. Terrain Map of the study area showing the refinery and the Tema Meteorological Station.

Figure 3 .
Figure 3. Plots of measured and modelled SO 2 concentrations.

Figure 4 .
Figure 4. Plots of measured and modelled NO 2 concentrations.

Figure 5 .
Figure 5. Plots of observed and modelled wind speeds.

Figure 6 .
Figure 6.Plots of observed and modelled wind direction.

Table 1 .
Statistical performance indices of the CALPUFF model.

Table 2 .
Statistical performance indices of CALMET and WRF models.