Measuring Effectiveness of Health Program Intervention in the Field

Om Prakash Singh, Santosh Kumar

Indian Institute of Health Management Research, Jaipur, India.
Suresh Gyan Vihar University, Jaipur, India.

DOI: 10.4236/iim.2012.45029                

Abstract

Improving and sustaining successful public health interventions relies increasingly on the ability to identify the key components of an intervention that are effective, to identify for whom the intervention is effective, and to identify under what conditions the intervention is effective. Bayesian probability an “advanced” experimental design framework of methodology is used in the study to develop a systematic tool that can assist health care managers and field workers in measuring effectiveness of health program intervention and systematically assess the components of programs to be applied to design program improvements and to advocate for resources. The study focuses on essential management elements of the health system that must be in place to ensure the effectiveness of IMNCI intervention. Early experiences with IMNCI implemented led to greater awareness of the need to improve drug delivery, support for effective planning and management at all levels and address issues related to the organization of work at health facilities. The efficacy of IMNCI program from the experience of experts and specialists working in the state is 0.67 and probability of effective- ness of all management components in the study is 58%. Overall the standard assessment tool used predicts success of around 39% for the IMNCI intervention implemented in current situation in Rajasthan. Training management component carried the highest weight-age of 21% with 73% probability of being effective in the state. Human resource management has weight-age of 13% with 53% probability of being effective in current scenario. Monitoring and evaluation carried a weight-age of 11% with only 33% probability of being effective. Operational planning carried a weight-age of 9% with 100% probability of being effectively managed. Supply management carried a weight-age of 8% with zero probability of being effective in the current field scenario. In the study, each question that received low score identifies it as a likely obstacle to the success of the health program. The health program should improve all sub-components with low scores to increase the likelihood of meeting its objectives. Public health interventions tend to be complex, programmatic and context dependent. The evaluation of evidence must distinguish between the fidelity of the evaluation process in detecting the success or failure of the intervention, and relative success or failure of the intervention itself. We advocate management attributes incorporation into criteria for appraising evidence on public health interventions. This can strengthen the value of evidence and their potential contributions to the process of public health management and social development.

Keywords

Effectiveness; Efficacy; Performance; Evaluation; Measuring; Capacity Building of Health Interventions

Conflicts of Interest

The authors declare no conflicts of interest.

References

[1]	WHO, “Improving Performance,” The World Health Report, World Health Organization, Geneva, 2000, p. 151.
[2]	A. B. Zwi, “How Should the Health Community Respond to Violent Political Conflict?” PLoS Medicine, Vol. 1, No. 1, 2004, p. e14. doi:10.1371/journal.pmed.0010014
[3]	P. Bolton, J. Bass, L. Murray, et al., “Expanding the Scope of Humanitarian Program Evaluation,” Prehospital & Disaster Medicine, Vol. 22, No. 5, 2007, pp. 390-395.
[4]	B. A. Weisbrod, “Costs and Benefits of Medical Research: A Case Study of Poliomyelitis,” Journal of Political Economy, Vol. 79, No. 3, 1971, pp. 527-544.
[5]	D. S. Shepard, L. Sanoh and E. Coffi, “Cost-Effectiveness of the Expanded Programme on Immunization in the Ivory Coast: A Preliminary Assessment,” Social Science & Medicine, Vol. 22, No. 3, 1986, pp. 369-377. doi:10.1016/0277-9536(86)90136-X
[6]	R. E. Glasgow, T. M. Vogt and S. M. Boles, “Evaluating the Public Health Impact of Health Promotion Interventions: The RE-AIM Framework,” American Journal of Public Health, Vol. 89, No. 9, 1999, pp. 1322-1327 doi:10.2105/AJPH.89.9.1322
[7]	L. Gilson, “Discussion in Defense and Pursuit of Equity,” Social Science & Medicine, Vol. 47, No. 12, 1996, pp. 1891-1896.
[8]	E. A. Suchman, “Evaluative Research: Principles and Practice in Public Service and Social Action Programs,” Russell Sage Foundation, New York, 1967.
[9]	E. I. Struening and M. Guttentag, “Handbook of Evaluation Research,” Sage Publishing, Thousand Oaks, 1975.
[10]	L. W. Green, M. W. Kreuter, S. G. Deeds and K. B. Partridge, “Health Education Planning: A Diagnostic Approach,” Mayfield, Mountain View, 1980.
[11]	R. A. Windsor, T. Baranowski, N. Clark and G. Cutter, “Evaluation of Health Promotion and Education Programs,” Mayfield, Mountain View, 1984.
[12]	S. H. Starr, “C4ISR Assessment: Past, Present, and Future,” 8th ICCRTS, Washington DC, 17-19 June 2003.
[13]	N. Sproles, “Establishing Measures of Effectiveness for Command and Control: A Systems Engineering Perspective,” DSTO report, DSTO.GD-0278, Fairbairn, 2001.
[14]	J. T. Dockery, “Why Not Fuzzy Measures of Effectiveness?” 11th ICCRTS Coalition Command and Control in the Networked Era, Cambridge, 3-4 May 1986.
[15]	N. Smith and T. Clark, “An Exploration of C2 Effectiveness: A Holistic Approach,” 2004 Command and Control Research and Technology Symposium, San Diego, 15-17 June 2004.
[16]	N. Sproles, “Formulating Measures of Effectiveness”, System Engineering, Vol. 5, No. 4, 2002, pp. 253-263. doi:10.1002/sys.10028
[17]	B. Berg, “Qualitative Research Methods for the Social Sciences,” 4th Edition, Allyn and Bacon, Boston, 2001.
[18]	Experiment-Resources.com., 3 May 2012. http://www.experiment-resources.com/experimental-research.html
[19]	A. Maynard, “Evidenced-Based Medicine: An Incomplete Method for Information Treatment Choices,” Lancet, Vol. 349, No. 9045, 1997, pp. 126-128. doi:10.1016/S0140-6736(96)05153-7
[20]	R. H. Moos, “Understanding Program Environments. New Direction in Program Evaluation,” 40th Jossey Bass Higher Education and Social and Behavioral Sciences Series, San Francisco, 4 March 2008, pp. 7-24.
[21]	C. Potter and R. Brough, “Systemic Capacity Building: A Hierarchy of Needs,” Health Policy and Planning, Vol. 19, No. 5, 2004, pp. 336-345. doi:10.1093/heapol/czh038 PMid:15310668
[22]	S. Simmonds and N. Bennett-Jones, “Human Resource Development: The Management, Planning, and Training of Health Personnel,” Health Policy and Planning, Vol. 4, No. 3, 2005, pp. 187-196. doi:10.1093/heapol/4.3.187
[23]	M. A. Schellenberg Jr., de Savigny D. B. J., T. Lambrechts, C. Mbuya, L. Mgalula and K. Wilczynska, “The Effect of Integrated Management of Childhood Illness on Observed Quality of Care of Under-Fives in Rural Tanzania,” Health Policy and Planning, Vol. 19, No. 1, 2004, pp. 1-10.
[24]	G. W. Pariyo, E. Gouws, J. Bryce and G. Burnham, “Improving Facility-Based Care for Sick Children in Uganda: Training Is Not Enough,” Health Policy and Planning, Vol. 20, No. 1, 2005, pp. i58-i68. doi:10.1093/heapol/czi051 PMid:16306071
[25]	S. Arifeen, “With Benefit of Hindsight: Policy and Program Lessons from the MCE,” Key Findings from the Multi-Country Evaluation of IMCI Effectiveness, Cost and Impact (MCE) ICCDDR, B, Centre for Health and Population Research, Mexico city, 2004.
[26]	J. Amaral, A. J. Leite, A. J. Cunha and C. G. Victora, “Impact of IMCI Health Worker Training on Routinely Collected Child Health Indicators in Northeast Brazil,” Health Policy and Planning, Vol. 20, No. 1, 2005, pp. i42-i48. doi:10.1093/heapol/czi058 PMid:16306068
[27]	L. Huicho, M. Davila, M. Campos, C. Drasbek, J. Bryce, and C. G. Victora, “Caling up Integrated Management of Childhood Illness to the National Level: Achievements and Challenges in Peru,” Health Policy and Planning, Vol. 20, No. 1, 2005, pp. 14-24.
[28]	A. Rowe, F. Onikpo, M. Lama, F. Cokou and M. S. Deming, “Management of Childhood Illness at Health Facilities in Benin: Problems and Their Causes,” American Journal of Public Health, Vol. 91, No. 10, 2001, pp. 16251635. doi:10.2105/AJPH.91.10.1625
[29]	J. Nsungwa-Sabiiti, G. Burnham and G. Pariyo, “Implementation of a National Integrated Management of Childhood Illness (IMCI) Program in Uganda,” Journal of Health and Population in Developing Countries, 2004, pp. 5-15.
[30]	K. Buse, N. Mays and G. Walt, “Making Health Policy,” Open University Press, Maidenhead, 2005.
[31]	C. G. Victora, L. Huicho, J. J. Amaral, J. Schellenberg, F. Manzi, E. Mason and R. Scherpbier, “Are Health Interventions Implemented Where They Are Most Needed? District Uptake of the Integrated Management of Childhood Illness Strategy in Brazil, Peru and the United Republic of Tanzania,” Bulletin of the World Health Organization, Vol. 84, 2006, pp. 792-801. doi:10.2471/BLT.06.030502