Teaching Biomedical Statistics to Nurse-Practitioners in Sub-Saharan Africa—The Example of “Intention to Treat” Shows Our Challenges and Dilemmas

Abstract

Introduction: African nurse practitioners experience specific challenges, when faced with complex clinical trials implemented in their country. Method: Teaching challenges for African nurse practitioners were extracted from courses we conducted in Malawi. Participants attending the courses were nurse practitioners at various stages of their education ranging from nurses and medical assistants to the clinical officer with a BSc degree. Results: We identified four dilemmas for our participants: the “Taliban dilemma”, the “significance dilemma”, the “drop-out dilemma” and the “reality dilemma”. These dilemmas lead to five teaching challenges in the African context. Challenges in the context are theoretical complexity, imposed opinions, hierarchical implications, African enthusiasm and mysticism. Conclusions: The nurse practitioners, working in remote district hospitals need specific support to be able to scrutinize research papers which are meant to be implemented in their hospitals in order to secure their cooperation and dedication.

Share and Cite:

Pollach, G. , Graf, U. and Place, D. (2014) Teaching Biomedical Statistics to Nurse-Practitioners in Sub-Saharan Africa—The Example of “Intention to Treat” Shows Our Challenges and Dilemmas. International Journal of Clinical Medicine, 5, 1221-1227. doi: 10.4236/ijcm.2014.519156.

1. Introduction

Working and teaching in the largest hospital of Malawi, the Queen Elizabeth Central Hospital in Blantyre most of our staff in Malawian anesthesia and intensive care are non-physicians, doing the job of trained doctors. Nevertheless they have had only a limited exposure to formal training in mathematics and statistics. These colleagues are called “anesthetic clinical officers” and are similar to what would be called nurse practitioners in the US.

It is important for us to teach our non-physician staff the basic statistics. This does not relate so much to pure theory, but to the evaluation of large studies, affecting our daily work.

The discussion of trials, which are near to real life conditions is not about deep statistical teaching and not so much about teaching itself, but to find a way to make changes in daily clinical work for our frontline staff really happening. Our nurse anaesthetists are the backbone of clinical anaesthesia in the whole country and so their ability to read medical trials will directly influence their clinical work.

So we try to incorporate some practical statistical teaching in lectures and seminars for our anaesthetic clinical officers and other nurse practitioners in surgery or obstetrics.

The discussion of research papers in the training of clinically working colleagues might not make good statisticians out of them but would raise their motivation, ability, conceptual thinking, self esteem, give them confidence to read these papers, as it was recognized by Holmes and Dodd (2012) [1] .

We usually do not discuss means, standard deviations and other descriptive statistics, leaving that to the statistics departments, but we do discuss important statistical findings in papers relevant to African anaesthesia, emergency medicine and critical care.

2. Method

During seminars for different cadres of nurse practitioners and other non-physician health professionals we discussed a published large randomized trial comparing fluid administration in a common type of emergency situation in African children related to a severe infection either by bolus or without a bolus [2] . We selected this well known trial (the FEAST trial with 3141 African children) because it will probably change the way thousands of African nurses and clinicians will care for millions of sick children during the next few years. It showed quite a low overall mortality, suggesting that the clinical part was well managed and it was credited of being especially statistically robust and well managed.

We used our usual one to three day long courses with around 600 Malawian front-line staff to detect on how we can raise their commitment to introduce new clinical knowledge, which successful research trials might bring to the medical fraternity. These courses were very much clinically orientated and supplied direct hands-on knowledge on HDU, ICU treatment, obstetric emergencies trauma procedures, oxymetrie and pediatric anaesthesia. The training and the discussions with the clinicians provided us with the qualitative knowledge and its consequences, we are presenting here in this qualitative study.

For randomized controlled clinical trials in biomedical research like the FEAST trial the “intention-to-treat” principle constitutes the gold standard when dealing with drop outs, withdrawals and missing data. Due to its practical prominence and theoretical importance we often had to discuss this principle and use it here as an example.

Randomized clinical trials (RCTs) with ITT analyses dominate with approximately 50% of RCTs in the major journals [3] - [6] . Typically ITT is used for two challenges. Either to deal with non-adherence and drop outs or to treat missing data in combination with “Last Observation Carried Forward (LOCF)” or other suggested imputations. Even proponents of ITT acknowledge that “no consensus exists about how missing responses should be treated in intention to treat analyses...” [5] and so we concentrated on the first point with our non-physicians.

Intention-to-treat (ITT) analyses means that a possible bias through dropouts and non-adherent subjects should not occur at the end of a research study. This is done by the inclusion of everybody who ever was randomized, regardless of adherence, change in regimes or any other reason for non adherence. The standard procedure is to “analyze as randomized” [7] . Very often ITT is not only considered as the gold standard, but the language, used by its proponents to defend it may be interpreted as being inappropriately aggressive by Africans. Some of our colleagues tend to “shut-down” their attention and interest than. This type of sentences can be found quite dominating in papers on ITT: “... investigators must apply the intention-to-treat principle” [3] ; “once randomized always analyzed” [8] , “... ignores noncompliance, protocol deviations, withdrawal...” [8] , “Well- designed RCTs require strict adherence to the intention-to-treat principle...” [6] .

Having been through these exercises with different cadres we spotted several recurrent challenges to the understanding of the concept of “intention-to-treat” (ITT) for our participants. Frustrating for us and strangely enough for the attentive students who were asking intelligent questions, was the fact that we were not able to easily retrieve literature discussing some of these points—and most textbooks were silent on them so that the above linguistic impression was pronounced and we were left a little bit in despair.

We tried to establish what were the major dilemmas for our anaesthetic clinical officers and cooperating nurse-practitioners from the other departments.

3. Results

3.1. Our Dilemmata

3.1.1. The “Taliban Dilemma”

Living in times when health care workers are killed for offering polio-vaccination the “Taliban dilemma” emerged. We came up with a theoretical randomized trial comparing two different treatment options for diarrhea. Patients belonging to each of the two treatment arms are placed in two different houses of the hospital. The Taliban (or any of the multiple Asian or African terrorist groups) throw a bomb and all participants in one house are dead before the start of the therapy in this house. Due to ITT the mortality in this treatment group would be 100% and therapy in the second house would be superior for any therapy ever tested against diarrhea.

It was clear to all participants that this is only a theoretical reflection, but it was apparent that when you can construct a situation in which a concept is nonsense, than the concept needs an amendment by telling us when it is no longer applicable. Meaning: Which percentage of deaths in one group is acceptable before it is a danger to the ITT concept? In a concept which is embedded so deeply in the biostatistical thinking a rule of thumb, as advocated [9] [10] , cannot be the solution and no practical solution to address this problem could be found in the literature—leaving our participants unsatisfied.

3.1.2. The “Significance Dilemma”

In the above mentioned FEAST trial we saw that significance for the primary end point (48-hour mortality) could be attributed mainly to the fact that death before the initiation of treatment (8 children died before treatment) did only occur in the bolus groups.

With per-protocol analysis, and without ITT analyses, a significant result for the trial (p < 0.05) would have been reached only narrowly. This would have reduced its importance for the future therapy in millions of children in the tropics greatly.

Independent of the trial in question the groups felt that there should be a mechanism to deal with the fact, that ITT was not, as it is usually assumed, “watering down the results”, but did to the contrary and was responsible for the then claimed “robustness” of the result.

How far can a statistical parameter like ITT be allowed to be decisive for the outcome of a trial with severe ethical implications in the treatment of African children? Where is the threshold to render a statistically sound and significant trial to clinical invalidity? We have to admit that we found the literature silent on this topic and we were not in a position to offer a sound explanation to our colleagues.

3.1.3. The “Drop-Out” Dilemma

ITT is considered to be the gold standard solution on how to treat drop-outs in a trial. In order to do so it treats all drop-outs the same way: “once randomized—always analyzed”.

Frequently participants felt that not all drop-outs are the same and that at least drop-outs due to death before the initiation of treatment need a closer scrutiny.

Usually we do not consider death as a special form of drop-out because you never can be sure that it is not related to the therapy even when therapy did not start. The famous example of Paul Meier is than cited in which a patient, participating in a study on ischemic heart disease dies on a boat trip after he was seen to enter the boat with two six-packs of beer. Meier makes the point, that even when it appears that the patient might have fallen into the water because he was drunk and many researchers would not like this death in their respective treatment group—there is still the chance that during the search for the body the police finds all beer bottles closed on the ground of the lake, what means that he should be counted as dead in his treatment group. When he would have been really drunk and the bottles never would have been recovered than the consequence of ITT would have been only a watering down of the positive effect of one of the treatment arms; so the consequence is: “Unless the possibility of bias cannot be confidently rejected... [11] ” ITT should be done.

This famous argument neglects the point that there might be in some cases a definite proof that death has nothing at all to do with the research question. Our friend might have died through a variety of strange reasons independent of his ischemic heart disease and discovered by the post mortem of his old colleague who was with him on the boat trip (or the children in an African trial might have been killed by their desperate mothers because they did not any longer believe that their child was salvageable and had to make sure that the soul of the child did not leave the body with its last breath—so they kept the mouth and nose close to prevent the last breath and so deliberately suffocated the child, as it was happening in Somalia [12] ).

It was thought that it was particularly important to ascertain the cause of death in patients participating in RCT, when data are analysed by ITT and leading to marginal results concerning statistical significance. This should lead to the consequence to conduct thorough death analyses, whenever ITT asks to count an early death for one of the randomized groups. The death analyses need to be the more thorough, the more important the death is for the significance in the respective treatment arm. Even when a real proof might be rare it should not be neglected when available.

Moreover “proof” might not be necessary, but a probability of the non-relation of death which is higher than the significance level the trial used for its results.

When proof is available, than there is no reason at all to incorporate this death via ITT in one of the treatment groups. Statistical thinking, which deals with relativities should not be more important than absolute truth for the evaluation of a trial.

The opinion in the seminars was, that more work is needed to follow up the drop-outs, not more statistical sophistication.

3.1.4. The “Reality” Dilemma

ITT pretends to be a better reflection of reality than per-protocol analysis because its results should reflect reality more. In reality patients may default, they may die, patients withdraw, protocols may be changed etc.

In Malawi public health (not clinical medicine) is nowadays the dominating force in medicine and for all public health implications the above notion is certainly true—when we consider whether a type of therapy works better for thousands of people, when we look at the pharmacoeconomic implications of a therapy and when we look at the justification of funding for a new therapy. The same is true when we consider whether surgery or a pill is better—incorporating long delays for surgery due to a lack of surgeons, anaesthesiologists and intensive care beds into our considerations. Let ITT rule all trials concerned with these problems.

Despite being stated often [3] the need to look at the treatment of the single patient is not often realized. What we do is to look at a group of patients; and so we come to the question of a clinical officer working lonely in a rural hospital in the highlands of Malawi:

“What are the chances for the patient considering the pill or the surgery when treatment is given now and by myself?” Our colleague is able to do it now and here, he is not interested to incorporate waiting lists in study results or any reason for drop out because his patient needs a decision now and he will not drop out and cannot withdraw. The question is not what results are produced by surgery for a group of patients with several possibilities of a falsely low mortality-rate of surgery [11] (some have to wait, some want to have their last party first etc.), but he wants to know which “pure” therapy is the best.

This is the real clinical question, which renders all discussion about withdrawal, drop out, death on the waiting list, lacking measurements or change of protocol (which all are so much in favor of ITT) to insignificance [13] [14] .

We agree with Dallal, that ITT changes the research question as such, because ITT measures the effect of assignment. Despite it suggests moreover that this answer is the answer to the measurement of adherence too [7] . Per protocol analyses answers the question of adherence without the suggestion to answer a second additional question. Excluding non-adherent patients will destroy the unbiased comparison in the randomization because “empirical evidence suggests that participants who adhere tend to do better” [3] [15] —but that’s what we want for our nurse practitioners in the districts of rural Malawi—that their patients do better.

No analysis is perfect and we have to match the correct one with the problem in question. We cannot pretend that by using ITT we will produce always a more robust answer to the question which therapy is better particularly in the context of a clinician directly delivering the treatment in a defined setting of a peripheral hospital. ITT is not a golden bullet.

It’s the basic question of medicine: What is helping this particular patient at this moment when the non- physician clinician has to choose between two viable options.

We suggest to let ITT answer public health questions by incorporating all problems watering down possible therapeutic effects and let per protocol analyses answer the real clinical question, posed by our frontline staff— the nurse practitioner.

4. Discussion

Anaesthetic clinical officers in Malawi were not happy with the fact, that it is not often admitted that the concept of ITT still contains problems longing for discussion and fixing. For the sake of credibility of statistical teaching in our African environment this is an issue which needs to be addressed. Only few authors, as Gupta [8] accept that there are “many arguments” valid in questioning ITT and that “... exclusion of some randomized subjects in a justified way” might be possible. But even these authors usually only deal with the softer challenges (procedural difficulties, the problem of patients crossing over, watering down the results, how to replace lacking data [8] ) and do not explicitly address the above mentioned dilemmas and they see per-protocol analyses only as an (inferior) subset of ITT analyses.

There are ways to try to cope with this challenge. Accepting some change in the pure ITT analyses, so then it is called “modified intention-to-treat (mITT)” [6] , but the definition of mITT remains unclear [16] . Moreover mITT is not only not defined, but it is associated with negative connotations, because it was introduced into the edge of research financed by pharmaceutical industries or other for-profit agencies. This happened despite the fact, that RCTs, analysed by mITT in well renowned journals (JAMA; NEJM, Lancet), were scrutinized and no major differences in their methodological quality were found [6] ; and despite the fact, that it is not clear that ITT is better in non-inferiority trials [17] . Again, it appears that dissent against the predominant narrative of pure ITT has its challenges.

Furthermore the following challenges appeared for our trainees in Malawi:

1) Statistics can threaten fearful individuals [18] not only in developed countries—even when modern methods as discussed in the journal “Teaching Statistics” are used to debug the teaching on randomization of its mysticism e.g. with cockroaches [19] . This is much more the case in countries were the basic understanding of science still remains challenging for the nurse practitioner just graduating from secondary school.

2) Random assignment and selection seem to be difficult to grasp in any case and students even distrust it [20] [21] . The scientific and teaching communities need to develop methods that will capture the enthusiasm of our non-physician staff or we will lose the interest of the very people who have to implement our research studies for the African good.

3) Our experience shows that African university students do not cherish strong, quasi officially imposed opinions and remarks (even when correct), because that seems to exclude a discussion on how to handle a problem e.g. in a journal article concerned with African affairs and topics related to their daily work experience. Not far away from brushing away criticism with sentences like “this holds because the stats appear to be robust” is than the suggestion of a neocolonial attitude in sciences.

4) On the other hand our non-physician students are sometimes all too eager to rely on authorities and to take strong statements—as a law which should not be questioned. This is detrimental to any scientific thinking and the development of a critical mind.

5) We have to be careful not to disengage our nurse practitioners by robbing them of their enthusiasm and spirit through an arrogant behavior which leads to the impression statisticians and clinicians are not able to communicate. When they are not felt to be taken seriously and when their intellectual approach is not acknowledged, they may fall prey to the lures of non-evidence based medicine. They sometimes have the same problems like the villagers who can’t distinguish research and clinical work [22] . This would be detrimental to their work in daily life. Taboo, witchcraft, intellectual bypass and magic bullets are still lingering around in not really consolidated health and education systems.

We do not have a lot of time to discuss statistics in our clinical teaching anyhow, so we would prefer not to have to deal with these additional challenges in our teaching.

As clinical teachers we should make a strong case for a curriculum sensitive to the cultural attitude of our students when confronted with strong opinions and well established concepts in statistics. We, but not all of our students, know that all concepts are challenged from time to time as stated by Lu and Henning 2012 in the cases of “probability” or the “statistical population” [23] . The general opinion [7] that the only question is “... how wrong the model is” [23] still needs to be nourished in Africa.

The main goal should be to help our nurse practitioners to understand basic principles and concepts of statistics to enable them to form their own well funded opinion on the value of research papers and to be an informed consumer of statistical teaching [24] .

5. Conclusions

We do think that we have raised some valid points through our nurse practitioner teaching, which are not adequately discussed in the biomedical literature—partly because the intention to treat principle is so dominant and the opinion is widespread that it does not need further discussion.

Due to the predominance of public health and epidemiology this is of special interest for resource poor countries. The growing focus on mathematical exactness leading to “robust statistics that holds” leaves the clinical teacher, faced with intelligent non-physician staff and their critical questions in despair.

Conflicts of Interest

The authors declare no conflicts of interest.

References

[1] Holmes, K.Y. and Dodd, B.A. (2012) Teaching Statistics Using Classic Psychology Research: An Activities-Based Approach. Teaching Statistics, 34, 13-17.
http://dx.doi.org/10.1111/j.1467-9639.2011.00499.x
[2] Maitland, K., Kiguli, S., Opoka, R.O., et al. (2011) Mortality after Fluid Bolus in African Children with Severe Infection. New England Journal of Medicine, 364, 2483-2495.
http://dx.doi.org/10.1056/NEJMoa1101549
[3] Montori, V.M. and Guyatt, G.H. (2001) Intention-to-Treat-Principle. CMAJ, 165, 1339-1341.
[4] Ruiz-Canela, M., Martinez-Gonzalez, M.A. and de Irala-Estevez, J. (2000) Intention to Treat Analysis Is Related to Methodological Quality. BMJ, 320, 1007-1008.
http://dx.doi.org/10.1136/bmj.320.7240.1007
[5] Hollis, S. and Campbell, F. (1999) What Is Meant by Intention to Treat Analysis? Survey of Published Randomized Controlled Trials. BMJ, 319, 670-674.
http://dx.doi.org/10.1136/bmj.319.7211.670
[6] Montedori, A., Bonacini, M.I., Casazza, G., Luchetta, M.L., Duca, P., Cozzolino, F. and Abraha, I. (2011) Modified versus Standard Intention-to-Treat Reporting: Are There Differences in Methodological Quality, Sponsorship, and Findings in Randomized Trials? A Cross-Sectional Study. Trials, 12, 58.
http://dx.doi.org/10.1186/1745-6215-12-58
[7] Dallal, G.E. (1998) The Little Handbook of Statistical Practice.
www.jerrydallal.com/LHSP/LHSP.HTM
[8] Gupta, S. (2011) Intention-to-Treat Concept: A Review. Perspectives in Clinical Research, 2, 109-112.
http://dx.doi.org/10.4103/2229-3485.83221
[9] Schulz, K.F., Grimes, D.A., Altman, D.G. and Hayes, R.J. (1996) Blinding and Exclusions after Allocation in Randomised Controlled Trials: Survey of Published Parallel Group Trials in Obstetrics and Gynaecology. BMJ, 312, 742-744.
http://dx.doi.org/10.1136/bmj.312.7033.742
[10] Schulz, K.F. and Grimes, D.A. (2002) Sample Size Slippages in Randomised Trials: Exclusions and the Lost and Wayward. Lancet, 359, 781-785.
http://dx.doi.org/10.1016/S0140-6736(02)07882-0
[11] European Coronary Surgery Study Group (1979) Coronary-Artery Bypass Surgery in Stable Angina Pectoris. Survival at Two Years. Lancet, 313, 889-893.
http://dx.doi.org/10.1016/S0140-6736(79)91372-2
[12] Pollach, G. (1994) Personal Experiences in Kismayo General Hospital. Kismayo, Somalia.
[13] Tamura, H. (1990) Modelling of Statistical Investigation. Teaching Statistics, 12, 84-85.
[14] Schiffner, R., Schiffner-Rohe, J., Gerstenhauer, M., Hofstädter, F., Landthaler, M. and Stolz, W. (2001) Differences in Efficacy between Intention-to-Treat and Per-Protocol Analyses for Patients with Psoriasis Vulgaris and Atopic Dermatitis: Clinical and Pharmacoeconomic Implications. British Journal of Dermatology, 144, 1154-1160.
http://dx.doi.org/10.1046/j.1365-2133.2001.04234.x
[15] Horwitz, R., Viscoli, C., Donaldson, R.M., Murray, C.J., Ransohoff, D.F., Berkman, L., et al. (1990) Treatment Adherence and Risk of Death after Myocardial Infarction. The Lancet, 336, 542-545.
http://dx.doi.org/10.1016/0140-6736(90)92095-y
[16] Deng, C.Q. (2014) Intention-to-Treat and Modified Intention-to-Treat Analysis in Clinical Trials. PPD Development Research Triangle Park, NC 27560.
http://webspace.webring.com/people/eu/um3826/ITT JSM2004.ppt
[17] Piaggio, G., Elbourne, D.R., Altmann, D.G., Pocock, S.J. and Evans, S.J., CONSORT Group (2006) Reporting of Non-Inferiority and Equivalence Randomized Trials: An Extension of the CONSORT Statement. JAMA, 295, 1152-1160.
http://dx.doi.org/10.1001/jama.295.10.1152
[18] Williams, A.S. (2012) Statistics Anxiety and Instructor Immediacy. Journal of Statistics Education, 18, 1-18.
http://www.amstat.org/publications/jse/v18n2/williams.pdf
[19] Wagler, A. and Wagler, R. (2013) Randomizing Roaches: Exploring the “Bugs” of Randomization in Experimental Design. Teaching Statistics, 36, 13-20.
[20] Derry, S., Levin, J.R., Osana, H.P., Jones, M.S. and Peterson, M. (2000) Fostering Students Statistical and Scientific Thinking: Lessons Learned from an Innovative College Course. American Educational Research Journal, 37, 747-773.
http://dx.doi.org/10.3102/00028312037003747
[21] Sawilowsky, S.S. (2004) Teaching Random Assignment: Do You Believe It Works? Journal of Modern Applied Statistical Methods, 3, 221-226.
[22] Molyneux, C.S., Wassenaar, D.R., Peshu, N. and Marsh, K. (2005) “Even If They Ask You to Stand by a Tree All Day, You Will Have to Do It (Laughter)…!”: Community Voices on the Notion and Practice of Informed Consent for Biomedical Research in Developing Countries. Social Science & Medicine, 61, 443-454.
http://dx.doi.org/10.1016/j.socscimed.2004.12.003
[23] Lu, Y. and Henning, K.S.S. (2012) Are Statisticians Cold-Blooded Bosses? A New Perspective on the “Old” Concept of Statistical Population. Teaching Statistics, 35, 66-71.
http://dx.doi.org/10.1111/j.1467-9639.2012.00524.x
[24] Chance, B.L. (2002) Components of Statistical Thinking and Implications for Instruction and Assessment. Journal of Statistics Education, 10, 1-18.

Copyright © 2024 by authors and Scientific Research Publishing Inc.

Creative Commons License

This work and the related PDF file are licensed under a Creative Commons Attribution 4.0 International License.