A Comparison of National Immunization Programme Target Population Estimates with Data from an Independent Source and Differences in Computed Coverage Levels for the Third Dose of DTP Containing Vaccine

Background: Comparison of target populations for immunization used by national immunization programmes with independent sources can be useful for identifying irregular patterns. Similarly, understanding differences in computed coverage levels that result from changes in target population estimates can be important. Methods: Using data reported annually by national immunization programmes to WHO and UNICEF, we compared the national number of births and surviving infants with estimates reported by the United Nations Population Division (UNPD). We also re-computed and compared coverage levels for the third dose of DTP containing vaccine (DTP3) using the nationally reported number of children vaccinated with DTP3 (the numerator) and the nationally reported number of children in the target population (the denominator) and compared this value with DTP3 coverage computed using the nationally reported number of children vaccinated and the UNPD estimate of the national number of surviving infants as an independent denominator. Results: We observed differences in the number of births and surviving infants reported by national immunization programmes compared with those estimated by the UNPD. Year-to-year changes in the number of births and surviving infants reported by national immunization programmes often exceeded those estimated by the UNPD. The re-computed administrative coverage levels for DTP3 using a nationally reported target population tended to be higher on average than those re-computed using the UNPD target population estimates. Conclusion: Target population estimates are a challenge for immunization programmes, and comparison to independent sources can be useful. There is increasing need to trace and better understand the processes and conditions affecting the enumeration and recording of the number of children in the target population for immunization services and the number of children vaccinated while recognizing that the challenge to do so is greater in some locations than others.


Introduction
We have previously described differences in the implied infant mortality rate (IMR) computed using data provided by national immunization programmes and esti-mates of IMR from the United Nations Interagency Group on Child Mortality Estimation (IGME) and the United Nations Population Division (UNPD) [1]. The results highlighted the inconsistencies present between estimates of IMR used by national immunization programmes and those produced by external sources. The current report updates analyses comparing the number of births and surviving infants reported by national immunization programmes with those estimated by the UNPD and extends prior work by reporting observed differences in computed coverage levels that result from changes in target population estimates.

Methods
Since 1998, WHO and UNICEF have jointly collected national-level data on the incidence of selected vaccinepreventable diseases, immunization coverage, recommended immunization schedules, vaccine supply and other information on the structure, policies and performance of national immunization systems through the WHO/ UNI-CEF Joint Reporting Form on Immunization (JRF) [2]. (Since 2000, more than 95% of WHO Member States have reported annually.) As part of this annual reporting exercise, in addition to immunization coverage national authorities also report the estimated number of children vaccinated according to administrative data (i.e., reports from health service providers) and the number of children in the national target population for each antigen in the national immunization schedule.
The appropriate target population for vaccines administered at birth (eg., Bacille Calmette-Guérin or BCG) is the estimated number of live births in the country. For other antigens, the target population most often used for computing coverage is the number of surviving infants. (NB. Some countries use live births as the official denominator for computing coverage levels for DTP containing vaccine, polio vaccine, hepatitis B and Haemophilus influenzae type b containing vaccines, and yellow fever vaccine. Historically, countries with high infant mortality have chosen not to use the number of live births as a target population for vaccines administered after birth since an immunization programme's performance might not be accurately reflected in the presence of high infant mortality even if the programme was vaccinating each and every child that survived.) Using databases maintained by the WHO and UNICEF (data as of July 2013), we abstracted the national number of 1) live births and 2) surviving infants reported by national immunization programmes in the JRF for the period 2000-2012 as well as the nationally reported 3) number of children vaccinated with three doses of DTP containing vaccine (DTP 3 ) from routine administrative reporting systems. Administrative data are based on aggregated reports from service providers to local health authorities of the number of vaccinations administered during a given period. We compared the national number of births and surviving infants with estimates reported by the UNPD World Population Prospects, 2012 edition [3]. We re-computed DTP 3 coverage using the nationally reported number of children vaccinated with DTP 3 (the numerator) and the nationally reported number of children in the target population (the denominator) and compared this value with DTP 3 coverage computed using the nationally reported number of children vaccinated and the UNPD estimate of the national number of surviving infants as an independent denominator.

National Number of Births and Surviving Infants
The results of comparing national numbers of births and surviving infants for 2012 estimated by the UNPD with those reported by national immunization programmes are displayed in Figures 1 and 2. The map shows the effects on numbers of births and surviving infants, in relative percent terms, of changing from a target population dataset using nationally reported totals to one using UNPD totals. The differences that can result from such changes in target population are evident when considering the extreme case of Myanmar, where the UNPD estimates a total number of births for 2012 of 921,941, while the national immunization programme estimates 1,602,657, a difference of +74%; or of Eritrea, where the UNPD estimates a total number of births for 2012 of 229,517, while the national immunization programme estimates 111,469, a difference of −51%. Elsewhere, differences are smaller, but a large number of countries show absolute differences of greater than 5% for number of births (n = 80 countries) and for number of surviving infants (n = 77). Median values of absolute differences in the number of births and surviving infants from the two sources varied by greater amounts in low-income countries than in the high-income countries ( Table 1). For countries defined as "least developed" [4], the median absolute difference in the number of births was 9.4% (IQR, 10.9) while the median absolute difference was 5.9% (IQR, 10.3%) for the remaining countries. Similar differences were observed for surviving infants. We examined 580 reporting events during a five-year reporting period (2008-2012) from 116 countries which reported national numbers of births and surviving infants. For 21% (n = 124) of these reporting events, the year-toyear relative percent change in the number of births reported by national immunization programmes exceeded ±5%. In contrast, year-to-year percent change in the UNPD estimated national number of births for the same countries across the same period exceeded ±5% in only one instance, for Sudan from 2010 to 2011. The 124 report-   ing records with extreme (>±5%) year-to-year changes in the nationally reported number of births were from 23 countries (6 low-income 1 , 15 middle-income 2 , 2 highincome 3 countries), and a percent difference between the national number of births for 2012 estimated by UNPD and reported by national immunization programmes in excess of 5% points was observed in 103 of the 124 records. In China, the year-to-year change in reported national number of births was +11% between 2007 and 2008 and was +7% between 2011 and 2012 while that for the UNPD estimated national number of births ranged from <1% to 2%. In Rwanda, the national number of births reported by the immunization programme declined 20% between 2011 and 2012 following 3% to 4% increases from 2008 to 2011 in contrast with 1% to 2% increases per UNPD estimates over the same period.
In the 456 reporting events (out of 580) for 2008-2012 with a year-to-year relative percent change in the nationally reported number of births within ±5%, the median year-to-year percent change (absolute value) was 2.4% based on national data and 1.0% based on UNPD estimates. The median year-to-year percent change (absolute value) in the national number of births during 2008-2012 was 2.6% based on nationally reported data compared to 1.7% per UNPD estimates in 26 low-income countries represented; 2.1% based on nationally reported data compared to 0.8% per UNPD estimates in 57 middle-income countries; and 2.7% based on nationally reported data compared to 0.7% per UNPD estimates in 10 high-in-come countries. Among these 2032 events where administrative coverage could be re-computed using a reported number of children vaccinated and reported target population, coverage levels were compared to coverage levels re-computed using a number of surviving infants estimated by the UNPD (i.e., an "independent" denominator). The re-computed administrative coverage levels for the third dose of DTP containing vaccine using a nationally reported target population tended to be higher on average than those re-computed using the UNPD target population estimates (average percent difference, −5.1%; median percent difference, −1.1%; IQR, 14.5).  Differences in the re-computed DTP 3 coverage levels using national reported target population and UNPD estimates changed over time. During 2000-2004, the average percent difference between re-computed coverage levels using national reported target population data and that using UNPD estimates was −8.1% (n = 748 reporting events). During 2005-2009, the average percent difference was −4.3% (n = 800 reporting events) while that for 2010-2012 was −1.8% (n = 4 84 reporting events). As a sensitivity analysis, we excluded reporting events for which the percent difference between re-computed coverage levels exceed 100% (n = 16). In doing so, we observed an average percent difference between re-computed coverage levels using national reported target population data and those using UNPD estimates of

Comment
We observed differences in the number of births and surviving infants reported by national immunization programmes compared with those estimated by the UNPD.
Year-to-year changes in the number of births and surviving infants reported by national immunization programmes often exceeded those estimated by the UNPD and in some instances were potentially inconsistent with general rules-of-thumb for population growth or decline. In some instances, differences may arise from changes in target populations with a tendency to not update or revise timeseries of target population data. For example, in the case of Rwanda, the apparent abrupt change between 2011 and 2012 in the nationally reported number of births is the result of a programme change to use the number of children vaccinated with BCG vaccine as the target population estimate in 2012 without a retrospective update of the time-series.
Immunization target population size estimates are a recognized challenge for programme managers [1], and the World Health Organization has recently developed an approach to assist national immunization programmes to assess estimates of target population size at the national level [5]. For example, immunization programme managers might consider referencing the UNPD estimates, or other independent data sources, for their country to establish plausibility boundaries for year-to-year changes in the number of births and/or surviving infants. When such boundaries are exceeded, further enquiry of the national data may be triggered. Conducting a time series review of programme data and examination of implied infant mortality rate values [1] may also be useful.
Using data reported by national immunization programmes and an independent data source, we observed differences in re-computed coverage levels for DTP 3 coverage from those computed using nationally reported data. Re-computed administrative coverage levels for DTP 3 based on nationally reported target population data tended to be higher than those based on UNPD target population estimates. While examining re-computed coverage levels, DTP 3 coverage levels exceeded 100% in some (108; 5%) reporting events with typographical errors, transposed digits and dropped digits in the reported data among the identifiable problems. For example, in one instance a typographical error was likely present as the national immunization programme reported 144,900 children vaccinated among a reported target population of 14,456 surviving infants resulting in a computed coverage level of 1002%; the national immunization programme reported 98% administrative coverage. In another country, the number of children vaccinated was reported as 5,938,998 among a target population of 719,617 resulting in a computed coverage level of 825%. If one deletes one of the "9" values from the number of children vaccinated (e.g., 593,898), the computed coverage level equals 83%, which corresponds to the administrative coverage reported by the country. In other instances, the underlying source of problems leading to implausible re-computed coverage levels is not readily apparent. For example, in one country, the reported number of children vaccinated was 101, 185 among a reported target population of 71,002 resulting in a computed coverage level of 143%, which is equal to the administrative coverage level reported by the national immunization programme. In this case, the reported numbers may have been transposed (i.e., number of vaccination children, 71,002; number of children in target population, 101,185) or a digit may have been left off (i.e., reported target population could be 171,002 rather than 71,002). The WHO and UNICEF continue to work with authorities in national immunization programmes to improve reporting practices and highlight discrepancies when they are identified. Potential solutions include the incorporation of embedded data checks in the reporting tools used by WHO and UNICEF to collect immunization system performance data. Unfortunately, such steps likely fall short of what is really needed which is a culture change around the central role of data in improved programme management and investment beyond data collection in data analysis, synthesis and dissemination.
In summary, with increasing attention toward immunization coverage levels in the international development community and concerns about the quality of the underlying empirical data, it is important to further understand patterns in immunization coverage and the processes used to produce the underlying empirical data. Recent annual reports on the number of children vaccinated with three doses of DTP containing vaccine by national immunization programmes to the WHO and UNICEF is quite complete though falls short of 100% (93% of reporting events during 2000-2012 where administrative coverage was included, which constituted 87% of all reporting events, also reported the number of children vaccinated). Although differences in the reported number of births and surviving infants by national immunization programmes compared with those estimated by independent sources may not be surprising, the observed patterns in year-toyear relative changes reported by national immunization programmes often differed greatly between sources and in some instances were potentially inconsistent with general rules-of-thumb for population growth or decline. While independent sources of target population estimates are no more likely to be accurate than national estimates, observed inconsistencies between sources might trigger further investigation. Population data are a central input in immunization coverage and therefore deserving of greater attention. Moving forward, additional investment and political will needs to be directed towards developing robust civil registration systems [6] as well as ensuring high levels of awareness of the civil registration process and its importance among parents or caregivers, their ability to access civil registrar services, and caregivers' willingness to interface with appropriate authorities [7]. In addition, there is increasing need to trace and better understand the processes and conditions affecting the enumeration and recording of the number of children in the target population for immunization services and the number of children vaccinated while recognizing that the challenge to do so is greater in some locations than others.