Evaluating and Improving the Effectiveness of Vehicle Inspection and Maintenance Programs: A Cost-Benefit Analysis Framework

Motor vehicle inspection and maintenance (I/M) programs are designed to identify high-emitting vehicles and mitigate their impacts on air quality and climate. I/M programs have been traditionally ranked superior among various vehicle emission control measures by the results of cost-benefit analysis, based on the assumption that these programs will achieve the targeted emission reduction outcomes. However, the actual effects of I/M programs may be greatly uncertain and when this uncertainty is taken into account, these programs may become suboptimal. This study develops a new a cost-benefit analysis framework that links various program design consideration, such as program participation rate, identification rate and effective repair rate, to the public health benefits as well as costs of the programs. This framework helps decision makers to investigate minimum implementation requirements that at least ensure the benefits are greater than the costs of implementing the programs in order to improve the overall effectiveness of the I/M programs. To illustrate the applications of the framework, it was applied to a particulate matter oriented I/M program targeting all diesel-fueled vehicles in the city of Bangkok, Thailand, a large metropolitan area that has been suffering from severe ambient PM pollution mainly attributable to its wide use of diesel-fueled vehicles and motorcycles. It was found that the health benefits achieved from the program are sensitive to several key program design elements, including participation rate and problem vehicle identification rate, fraction of effective repairs and illegal operation rate. Other variables, such as the testing cut-points and vehicle population growth rate, only have modest effects on the overall emission reduction and consequent health benefits. Overall, the performance of multiple variables associated with I/M program design needs to be improved simultaneous in order to achieve the targeted benefits of the program. How to cite this paper: Li, Y. (2017) Evaluating and Improving the Effectiveness of Vehicle Inspection and Maintenance Programs: A Cost-Benefit Analysis Framework. Journal of Environmental Protection, 8, 1541-1566. https://doi.org/10.4236/jep.2017.812095 Received: September 29, 2017 Accepted: November 26, 2017 Published: November 29, 2017 Copyright © 2017 by author and Scientific Research Publishing Inc. This work is licensed under the Creative Commons Attribution International License (CC BY 4.0). http://creativecommons.org/licenses/by/4.0/


Introduction
Motor vehicle inspection and maintenance (I/M) programs are designed to identify high-emitting vehicles and mitigate their impacts on air quality and climate [1]. In western developed countries such as the United States (US), these programs have been considered to be cost-effective and are required by the Clean Air Act Amendments of 1990 in regions with the most challenging air pollution problems [2] [3]. The main purpose of I/M programs is to encourage better maintenance for in-use vehicles and to assure the vehicle emission control systems are functioning properly through periodic inspections. The rationale for an I/M program is that the emission distribution among a vehicle population is highly skewed: A small portion of vehicles (estimated at 5% -10%), sometimes called gross polluter vehicles, is responsible for a substantial fraction (variously estimated at 50% to 80%) of total vehicle emissions [4] [5]. Moreover, not only old vehicles can be gross polluters, but also vehicles of all model years may include some proportion of gross polluters [5], due to the factor that vehicle emission levels are heavily dependent on maintenance. This problem can be even more pronounced in developing countries, where vehicles have a long lifetime and are often poorly maintained [6]. In this case, upgrading maintenance practices and replacing the worst engines should be considered first before moving on to better technology [7]. In addition, despite technological and regulatory advances, new vehicle standards are not sufficient to achieve pollution abatement goals if vehicles deteriorate rapidly, resulting in increasing emission rates [6]. Therefore, to control rapidly growing vehicle emissions, governments must not only affect the behavior of vehicle manufacturers and fuel suppliers, but also the actions of drivers in terms of how well they maintain their vehicles regardless of their vehicle ages [8].
However, although simple in concept, the detailed design and implementation of I/M programs is challenging. For example, when emission control equipment malfunctions, vehicle performance may be unaffected, hence the driver has no private incentive to seek repairs, and demanding private expenditures of money and time by vehicle owners will create the usual tensions that lead many actors to try to evade the regulation in numerous ways [8]. The practices of I/M programs in the US have shown various barriers that may cause failure of these programs to generate the emission reduction originally anticipated by policy makers. For instance, motorists may have many opportunities to evade required repairs, such as testing vehicles numerous times until they happen to pass. By among the drivers of a region [8].
Due to these challenges, the actual cost-effectiveness of I/M programs often remain uncertain. In theory, vehicular emission reductions available from I/M programs are mainly determined by the failure thresholds, or cut points, used to identify problem vehicles, but are also quite sensitive to a variety of factors, such as the actual percentage of problem vehicles identified, the percentage of problem vehicles waived from repairs or operating illegally, the emission reduction achieved by repairs, the durability of repairs, and so on [2]. In practice, these factors are often overlooked in designing and evaluating a regional I/M program.
This paper aims at developing a cost-benefit analysis framework for evaluating the effectiveness of vehicle inspection and maintenance programs, on the basis of the emission reduction assessment tool called "I/M Design" developed by [2] (referred to as EISINGER2005 hereafter), and the health benefit analysis tool developed in our previous study [9]. This combined analysis framework incorporates various factors that affect the level of emissions achieved by an I/M program, and links emission reduction with health benefits (avoided mortality and illnesses). The new framework is then applied to a hypothetical particulate matter (PM) oriented I/M program targeting all diesel-fueled vehicles in the city of Bangkok, Thailand, a megacity that has been suffering from severe adverse health effects attributable to PM for a few decades, to illustrate how the framework may help to evaluate the cost-effectiveness of I/M programs.

Theoretical Framework
I/M programs are one of the essential policy tools to control emissions from in-service vehicles in severely polluted urban areas [2]. An I/M program has the potential to reduce emissions in a number of ways, such as better maintenance of vehicles by motorists as a result of the program, repairs made in anticipation of an I/M inspection (referred to as pre-test repairs) or as a result of failing the test, and early scrapping of vehicles that are not worth repairing [3].   • Vehicles scrapped: emission reductions will be generated from scrapping gross polluting vehicles that would otherwise continue to be used, and replacing them by low emission vehicles. This variable also includes high-pol- Equation (6)  cific contexts and issues of concern with respect to the interested area and programs 1 .
The present study used this spreadsheet tool as a cost-benefits framework that links I/M design considerations with health benefits associated with the programs in order to understand the impacts of some key issues regarding I/M design, such as compliance rates, testing cut-points and effectiveness of repairs, on the potential health benefits of the programs.
Our previous study estimated the potential health benefits associated with the proposed PM-oriented I/M programs targeting all diesel-fueled vehicles and motorcycles in the city of Bangkok, Thailand, a megacity in Asia that has been suffering from severe adverse health effects attributable to ambient PM for decades [9]. In that study, the health benefits as a function of different levels of PM 10 (all particulates with an aerodynamic diameter of less than or equal to 10 μm) emission reductions were analyzed and compared with the social costs of the I/M programs (these different levels of reductions were considered due to the significant uncertainty involved in the actual emission reduction benefits of I/M programs). It was found that a minimum of about 4% reduction of the total PM 10 emissions from motor vehicles is required in order for the total benefits to 1 Based on a conversation with the author of the paper, Dr. Douglas S. Eisinger.   Table 1.

A Framework to Estimate the Effectiveness of I/M Programs
In addition to the input variables listed in Table 1, the spreadsheet also needs the following inputs related to the characteristics of the vehicle population studied: 1) Problem vehicles as percent of total vehicles: The values used in the spreadsheet were consistent with the assumptions made in our previous study-10% of buses and heavy trucks, 17.5% of light trucks and 25% of motorcycles in the BMR are problem vehicles [9]. In the uncertainty analysis, the upper and lower bounds of this parameter were assumed to be 1.5 times and half of the mean estimate, respectively. In lack of empirical evidence to support the form of PDF of this parameter, the uniform distribution was selected based on the authors own judgment; 2) Problem vehicles as percent of total PM emissions: Studies usually suggest that the gross-polluting vehicle pool is responsible for a substantial fraction-ariously estimated at 50% to 80%-of total vehicle emissions [4] [5]. Based on this, this study assumes that 50% and 80% are the lower and upper limits, respectively, and the mean value of them, 65%, is the best estimate of total vehicular emissions are generated by problem vehicles. Also, the uniform  It has been proposed that the new PM-related I/M programs should be linked to vehicle registrations and managed by a central database in order to significantly improve the levels of participation in the programs. It is expected that, with the government's strong will and efforts to curb severe air pollution in the BMR, the participation rate of the programs can be high. A mean estimate of 90% participation rate is assumed in this study. However, sensitivity analysis will test the role of this variable on the overall emission reductions by I/M. For simplicity, it was assumed that the fraction of problem vehicles is the same in the participation group as in the "non-participation group", although in reality, problem vehicles are more likely to escape from the inspection process. Diesel vehicles may deteriorate rapidly without proper maintenance.
On the contrary, a well-maintained diesel vehicle will generally retain a good emissions performance throughout its operating life [11]. It is expected that the updated I/M programs in the BMR should be able to motivate vehicle owners to better maintain their vehicles in anticipation of the effective inspection process.
In lack of more available information, the US values were used in this study. 1 The input values used in EISINGER2005 were based on light-duty vehicle hydrocarbon (HC) inspection data from an enhanced I/M program in southern California's South Coast Air Basin. 2 The original study examines the emission reduction benefits by an I/M program in a previous year using data on the actual number of vehicles inspected. It assumed that all vehicles subject to inspection participated in the program. 3 Triangular distribution was selected for all the variables in Table 1 except for VehWaive. There is little empirical evidence to support the PDFs of the variables in Table 1. Therefore, the PDFs were selected based on the author's own judgment. When there is more confidence in values near the central value than in values far away on either side, the triangular distribution was selected. In the case of the variable VehWaive, there is no reason to believe that some values between the lower and upper limits are larger than the others, and therefore, the uniform distribution was selected. Furthermore, the range was obtained based on the mean and the known theoretical limit of the variable, namely, 0% for the lower limit or 100% for the upper limit. Journal of Environmental Protection identify the worst polluters and minimize false failures [2]. In reality, vehicle emission rates usually span a wide spectrum. Conceptually, if an I/M targets a 25% reduction of the total emissions from motor vehicles, a cut-point equal to 75% of the current fleet-average emission rate will reduce the emission rates of all vehicles to 75% of the current level or lower, and thus ensure that the 25% emission reduction target is reached with confidence. However, given that the emission rates of the large portion of "good" vehicles (e.g. 90% of total vehicles) are usually much lower than the small portion of problems vehicles (e.g. 10% of total vehicles), it is not necessary, or probably not feasible either, to cut the emission rates of all vehicles to 75% of the current average level or lower in order to achieve the 25% reduction goal 2 . More stringent cut-points may be able to fail more vehicles, in particular those with emission rates close to the failure cut-points. However, more stringent cut-points are also likely to increase the social costs of I/M programs, and to suffer from problems such as technological infeasibility and motorist acceptance of the programs. While how to select and modify testing cut-points in I/M design to optimize the program effectiveness is beyond the scope of this study, this study uses the "ideal" cut-points discussed above, i.e. cut-points equal to 75% of the baseline fleet-average emission rates for each vehicle type in the BMR for the "best estimate" case, followed by an examination of the impacts of alternative cut-points on overall emission reduction levels. Just for comparison, the enhanced I/M in southern California's South Coast Air Basin studied in EISINGER2005 used a rate of 86% of the baseline the fleet-average emission rate as the failure cut-point (the baseline rate was 1.25 g/mi and the cut-point was 1.08 g/mi). Table 2 summaries the fleet-average  1 The baseline emission rates were derived from data published in [9] using PM 10 emission rates in the year 2000 and the assumption that a 5% annual PM emission factor decrease rate for all types of vehicles in the BMR. 2 The emission rates under the I/M scenario were calculated as baseline rate × 75%, assuming cut-points equal to 75% of the baseline fleet-average emission rates for each vehicle type. 2 A hypothetical example is provided here: Assuming that a vehicle fleet has an average emission rate of 1.0 g/km. 10% of all vehicles are gross polluting and they are responsible for 50% of the total emissions. Based on this information, it can be derived that the average emission rates for good and problem vehicles are 0.56 g/km and 5.0 g/km, respectively. As long as the average emission rate for problem vehicles goes down to 2.46 g/km, the fleet-average rate will decrease to 0.75 g/km. Therefore, if all problem vehicles can be properly identified and fixed, a cut-point of 2.46 g/km will ensure the 25% emission reduction goal accomplished. EISINGER2005 considered that a small fraction (in the range of 0% -7.5%) of the initial problem vehicles seek repairs in anticipation of I/M tests, and they were assumed to be "good" vehicles in inspection and pass the I/M test. Although the emission reductions resulting from this kind of "pre-test" repairs were taken into account in EISINGER2005, these reductions are only responsible for a small fraction of total emission reduction benefits achieved by I/M programs, approximately ranging from 0% -2%. It is expected that the fraction of problem vehicles seeking emission repairs before I/M will be even smaller in a developing country than that in the US, given that people are generally less wealthy and less able to afford the costs of maintenance and repairs. For simplicity, this study did not consider the emission reductions resulting from pre-test repairs.

Estimating Emission Reduction Effectiveness of the PM-Related I/M Programs in the BMR
Using the "best estimate" values of the variables listed in Table 1 and the estimated 2008 vehicle population, emission rates and VKT [9], the "I/M Design" spreadsheet was run for the year 2008. The results show that the PM-oriented I/M programs are expected to reduce total PM 10 emissions from motor vehicles in the BMR by 10.6%. Table 3 summarizes the findings. Therefore, in the "best estimate" case, the proposed PM-oriented I/M programs in the BMR are expected to yield health benefits that exceed the social costs of the programs (the "threshold" for achieving this goal is a 4% overall PM emission reduction achieved by the programs, as found in [9].

Examining the Roles of Key Design Elements on the Emission Reduction Benefits of I/M Programs
A series of sensitivity analyses were conducted to examine the sensitivity of the effectiveness of the I/M programs in terms of the percent of overall PM 10 emission reduction to the key design elements.

The Effects of Testing Cut-Points on Overall Emissions Benefits
The "best-estimate" in Section 3.1 is based on the assumption that failure cut-points are 75% of the baseline emission rates for each vehicle type. Since the cut-points determine the size of the initial problem vehicle pool (a more stringent testing cut-point is likely to result in more vehicles with "excess" emissions and, hence, subject to repair or replacement), changes to the cut-points will re-  was assumed that slight changes in cut-points do not change the values of the two variables (this may be true in the case that the majority of good vehicles have emission rates much lower than the cut-points, and the majority of problem vehicles have emission rates much higher than the cut-points), so in this case failure cut-points only affect the new emission rates of problem vehicles after repairs and retests. Based on this assumption, a cut-points sensitivity analysis was conducted. Figure 2 shows the results.  Figure 2 indicates that the when cut-points decrease from 100% to 60% of the baseline emission rates, the percent of overall emission reduction increases from 9.8% to 11.1%. Therefore, cut-points modifications within a certain range (e.g. from 60% -100% of the  baseline emission rates in this case) only have modest effects on the overall emission reduction benefits (the percent of overall PM 10 emission reductions changes from approximately 10% to 11% as the result of changing the failure cut-points from 60% -100% of the baseline emission rates), because most vehicles are considered to emit at levels well outside the range (either higher or lower the cut-points). This study considers that other I/M design elements may have more significant effects on the emission reduction benefits achieved by I/M programs. In general, important variables in I/M design that policy makers need to address include program participation rate, problem vehicle identification rate, effective emission repair rate and problem vehicle illegal operation rate. In the following section, the impacts of these variables on emission reduction were analyzed.

Key Variables Affecting I/M Effectiveness 1)
Participation rate and problem vehicle identification rate associated with I/M programs. Participation rate (PartiRate) and problem vehicle identification rate (IdenRate) are two key elements to address in designing I/M programs. Participation rate represents the levels of program enforcement. A successful I/M program minimizes vehicle violations (vehicles required by an I/M program do not participate in the program). The "best estimate" case in Section 3.1 assumes the majority (90%) of vehicles in the BMR required by I/M will participate in the programs, i.e. they will undertake appropriate emission inspection (reasons discussed in Table 1). However, if a large fraction of vehicles subject to I/M tests escape from the inspection process, the emission reduction benefits of I/M are expected to decrease considerably. Problem vehicle identification rate represents the ability of I/M programs to identify vehicles that exceed the emission standards and thus need emission repairs.
Sensitivity tests of each of the two variables were conducted by changing the  Figure 3 shows the results. The square-marked and the triangle-marked lines represent the percent of overall PM emission reductions achieved by I/M programs as a function of program participation rate (PartiRate) and problem vehicle identification rate (IdenRate), respectively (as noted in the figure). Each of the two lines was generated by incrementing the value of an individual variable (PartiRate or IdenRate) by 10% at a time (starting from 0% and ending at 100%), while setting all other inputs to their best estimates. And the red solid line represents the minimum percentage of emission reduction required in order for the benefits of the programs to outweigh the costs (the value was 4% as found in [9]). Figure 3 indicates that both participation rate and problem vehicle identification rate are important determinants of overall PM emission reduction benefits achieved by I/M programs. For participation rate, when the value of this variable increases from 0% (lower bound) to 100% (upper bound), the percent of overall emission reductions from vehicles increases from 0.6% to 11.7%; for problem vehicle identification rate, the percent of overall emission reductions from vehicles increases from 0.6% to 20.6% when the variable's value changes from the lowest to the highest. Comparing the effects of the two variables in Figure 3 shows that problem vehicle identification rate has a greater impacts on the overall emission reduction benefits than program participation rate, since for the same increment (e.g. 10%) in the two variables, the incremental emission reduction benefits resulting from the change in the problem vehicle identification rate In order to achieve the goal of 4% PM emission reduction from motor vehicles, the participation rate is required to be greater than 30%, if all other inputs remain the same values as in the "best estimate" case. And the requirement for problem vehicle identification rate is 17% when setting the other variables in the spreadsheet to their "best estimate" values.
2   determinants of the effectiveness of emission repairs. Comparing the three marked lines in Figure 4 shows that while the same increment in any one of the three variables results in approximately similar incremental emission reduction benefits, the impact of initially effective repair rate (GoodRep) is slightly greater than the impacts of the other two variables. The 4% emission reduction target requires a minimum of 22% of repair work initially effective (GoodRep), or 25% of excess emissions from identified problem vehicles reduced by repairs (ExEm), or 26% of repairs that properly pass an I/M remain durable until the next I/M inspection (DurRep).
3) The impacts of illegal operation by problem vehicles. Illegal operation here refers specifically to failed vehicles that continue to run on roads without appropriate repairs or certificates of waiver (the variable IllegalVeh in Table 1). There are other types of illegal operation in I/M program implementation. For example, vehicles may run on roads without taking the inspection required by the programs. This latter type of illegal operation is considered in the program participation rate variable, so it is not taken into account here. Illegal operation by failed vehicles may considerably damage the performance of I/M programs, since these vehicles are identified as gross emitters. Using the same sensitivity test approach as in Figures 3-5 was generated, which shows the effects of failed vehicle illegal operation rate on the levels of overall emission reduction achieved by I/M programs.
Therefore, the increases in illegal operation rate by failed problem vehicles can substantially reduce the emission reduction benefits achieved by I/M programs. In order to achieve the goal of 4% PM 10 emission reduction from motor vehicles, the rate of failed problem vehicle illegal operation should not go over 75%, presuming that the performance of the other variables is at the level of the "best

Improving the Emission Reduction Effectiveness of I/M Programs
For the PM-oriented I/M programs in this study, a 25% PM 10 Table 1, the improvement in the performance of any individual element is not sufficient to achieve the upper bound target of 25% overall emission reduction initially expected in proposing the programs to be adopted in the BMR.
For example, when one of the key variables discussed above reaches the upper bound, i.e. 100% (for IllegalVeh, the upper bound is 0%), while holding the other variables the same as in the "best estimate" case, the percent of overall PM 10 emission reduction is summarized in Table 3.
The results in Table 4 show in the case that only one variable in the spreadsheet increases while the values of the others remain the same as assumed in Table 1, even if the variables achieve complete success, the maximum level of PM emission reduction benefits is 20.6% (when IndenRate reaches 100%). Therefore, the performance of more variables needs to be improved simultaneously.  replaced variable values used and the new emission reduction estimates.

Emission Reduction Benefits Due to the Change in Vehicle Population Growth
The "best estimate" in Table 3 was based on the assumption that the implementation of the new PM-related I/M programs causes 10% decrease in average annual vehicle growth rate in the BMR [9]. This assumption is associated with the variable BenefitsGrow (the percent of emission reductions achieved due to the changes in vehicle population growth as a result of I/M enforcement). Sensitivity test was conducted by changing the percentage decrease in annual vehicle growth rate from 10% to 0% (no change in the annual growth rate), 20% or 30%, while holding all other input unchanged. Figure 6 summarizes the results.
Past experience in rapidly developing metropolitan areas in Asia shows that the introduction of vehicle I/M programs may slightly slow down the fast growth of motor vehicles in these areas and it is expected that the percent decrease in average annual vehicle growth rate falls into the range of 0% -30%. Figure 6 indicates that the change in the assumption about the percent decrease in annual vehicle growth rate has modest impact on the overall emission reduction benefits, when the change falls into the range of 0% -30%.

Uncertainty Analysis Results
Contribution to variance is a measure of the fraction of the total uncertainty (variance) in the risk estimate that comes from the uncertainty in a particular parameter, when all parameters are allowed to vary simultaneously [12]. Research to reduce uncertainty then should focus limited resources on narrowing the uncertainty in the premise showing the greatest contribution to variance [12]. Ta Table 1, and were performed in the Oracle Crystal Ball software. The sample size was set as 5000. Table 6 indicates that the problem vehicle identification rate of light-duty trucks showed the greatest contribution to variance. The contribution to variance of this premise is large probably both because the overall PM 10 emission reductions are sensitive to this variable (as found in Figure 3), and because the Figure 6. Impact of the change in vehicle growth rate on the overall PM 10 emission reductions by the I/M programs.  Table   1). Moreover, the problem vehicle identification rates of several other vehicle types (buses, city trucks and motorcycles) are also among the top 10 premises that contribute the greatest to the total uncertainty. Therefore, policy design considerations need to focus on increasing problem vehicle identification rate in order to narrow its uncertainty and improve its effectiveness. Over all, the sensitivity analysis performed here indicates that, in order to increase the problem vehicle identification rate, a key point is to improve testing procedure to maximize the ability of the programs to detect vehicles that need emission repairs. Second, studies have suggested that using more stringent testing cut-points can increase the percent of problem vehicles that are discovered by the inspection [2]. Further research on how to maximize the problem vehicle identification rates associated with I/M programs is warranted.

Conclusions
Since air pollution control usually imposes substantial costs on a society, an un-