Bias-Variation Dilemma Challenges Clinical Trials: Inherent Limitations of Randomized Controlled Trials and Meta-Analyses Comparing Hernia Therapies


Purpose:Evaluation of hernia therapies according to the current rules of Evidence Based Medicine is widely reduced to results of RCTs or meta-analyses. RCTs have been accepted as a most important tool to confirm a superior effect of an intervention. Unfortunately, in hernia surgery, comparisons of RCTs and correspondingly their use in meta-analyses, are not, surprisingly often, able to confirm any significant impact of a specific procedure due to intrinsic restrictions in a multi-causal setting with its web of influences. Methods:Based on our own experiences of clinical studies in surgery, the present article outlines several situations, with their respective reasons, which argue the severe limitations of RCTs and meta-analysis to define an optimum treatment. Results:Meta-analyses accumulate the variations of each trial, which then may cover any clear causal relationship. RCTs usually are dealing with subgroups of standard patients thus excluding the majority of our patients. Low statistical power of current cohort sizes restricts the analysis of subgroups or of effects with low incidences. Simple comparisons of means frequently are hampered by nonlinear relationships to outcome. The relevance of a specific variable is difficult to separate from other influences. The limited surveillance period of studies ignores a delayed change in outcome. Randomization cannot guarantee a standardized patient’s condition. All the arguments have to be considered as a crucial and fundamental consequence of the bias-variance dilemma or principle of uncertainty in medicine, and underline the many limitations of RCTs to evaluate any specific impact of hernia therapies on e.g. infection, pain or recurrence. Conclusions: Many surgical issues cannot be and should not be investigated by RCTs, in particular, if a marked patients’ heterogeneity has to be considered or the low incidences of the outcome readout cannot be addressed with sufficient statistical power without getting lost in the variation mire. Registries with their non-restricted data-acquisition should be regarded as reliable alternatives for postoperative outcome quality surveillance studies.

Share and Cite:

Klinge, U. , Koch, A. , Weyhe, D. , Nicolo, E. , Bendavid, R. and Fiebeler, A. (2014) Bias-Variation Dilemma Challenges Clinical Trials: Inherent Limitations of Randomized Controlled Trials and Meta-Analyses Comparing Hernia Therapies. International Journal of Clinical Medicine, 5, 778-789. doi: 10.4236/ijcm.2014.513105.

Conflicts of Interest

The authors declare no conflicts of interest.


[1] Gabriel, S.E. and Normand, S.L. (2012) Getting the Methods Right—The Foundation of Patient-Centered Outcomes Research. New England Journal of Medicine, 367, 787-790.
[2] Phillips, B., Ball, C., Sackett, D., et al. (2009) Oxford Centre for Evidence-Based Medicine—Levels of Evidence (March).
[3] Miserez, M. and Fitzgibbons Jr., R.J. (2012) Schumpelick V: Meta-Analyses of Lightweight versus Conventional (Heavy-Weight) Mesh in Inguinal Hernia Surgery. Hernia, 16, 503.
[4] Sanson-Fisher, R.W., Bonevski, B., Green, L.W., et al. (2007) Limitations of the Randomized Controlled Trial in Evaluating Population-Based Health Interventions. American Journal of Preventive Medicine, 33, 155-161.
[5] Black, N. (1996) Why We Need Observational Studies to Evaluate the Effectiveness of Health Care. BMJ, 312, 1215-1218.
[6] Memon, M.A., Khan, S. and Osland, E. (2012) Meta-Analyses of Lightweight versus Conventional (Heavy Weight) Mesh in Inguinal Hernia Surgery. Hernia, 16, 497-502.
[7] Rothwell, P.M. (2005) External Validity of Randomised Controlled Trials: “To Whom Do the Results of This Trial Apply?”. Lancet, 365, 82-93.
[8] Juurlink, D.N., Mamdani, M.M., Lee, D.S., et al. (2004) Rates of Hyperkalemia after Publication of the Randomized Aldactone Evaluation Study. New England Journal of Medicine, 351, 543-551.
[9] Brody, B.A., Ashton, C.M., Liu, D., et al. (2013) Are Surgical Trials with Negative Results Being Interpreted Correctly? Journal of the American College of Surgeons, 216, 158-166.
[10] Schmedt, C.G., Sauerland, S. and Bittner, R. (2005) Comparison of Endoscopic Procedures vs Lichtenstein and Other Open Mesh Techniques for Inguinal Hernia Repair: A Meta-Analysis of Randomized Controlled Trials. Surgical Endoscopy, 19, 188-199.
[11] Forbes, S.S., Eskicioglu, C., McLeod, R.S., et al. (2009) Meta-Analysis of Randomized Controlled Trials Comparing Open and Laparoscopic Ventral and Incisional Hernia Repair with Mesh. British Journal of Surgery, 96, 851-858.
[12] Goodney, P.P., Birkmeyer, C.M. and Birkmeyer, J.D. (2002) Short-Term Outcomes of Laparoscopic and Open Ventral Hernia Repair: A Meta-Analysis. Archives of Surgery, 137, 1161-1165.
[13] Yin, Y., Song, T., Liao, B., et al. (2012) Antibiotic Prophylaxis in Patients Undergoing Open Mesh Repair of Inguinal Hernia: A Meta-Analysis. American Surgeon, 78, 359-365.
[14] Sanabria, A., Dominguez, L.C., Valdivieso, E., et al. (2007) Prophylactic Antibiotics for Mesh Inguinal Hernioplasty: A Meta-Analysis. Annals of Surgery, 245, 392-396.
[15] Sanchez-Manuel, F.J., Lozano-Garcia, J. and Seco-Gil, J.L. (2012) Antibiotic Prophylaxis for Hernia Repair. Cochrane Database of Systematic Reviews, 2, CD003769.
[16] Shankar, V.G., Srinivasan, K., Sistla, S.C. and Jagdish, S. (2010) Prophylactic Antibiotics in Open Mesh Repair of Inguinal Hernia—A Randomized Controlled Trial. International Journal of Surgery, 8, 444-447.
[17] Billroth, T. (1869) Chirurgische Erfahrungen, Zürich 1860-1867. Langenbecks Archiv fü Chirurgie, 10, 1-113
[18] Rothwell, P.M. (2006) Factors That Can Affect the External Validity of Randomised Controlled Trials. PLoS Clinical Trials, 1, e9.
[19] Hoer, J., Lawong, G., Klinge, U. and Schumpelick, V. (2002) Factors Influencing the Development of Incisional Hernia. A Retrospective Study of 2983 Laparotomy Patients over a Period of 10 Years. Der Chirurg, 73, 474-480.
[20] Rosemar, A., Angerås, U., Rosengren, A. and Nordin, P. (2010) Effect of Body Mass Index on Groin Hernia Surgery. Annals of Surgery, 252, 397-401.
[21] Klosterhalfen, B. and Klinge, U. (2013) Retrieval Study at 623 Human Mesh Explants Made of Polypropylene—Impact of Mesh Class and Indication for Mesh Removal on Tissue Reaction. Journal of Biomedical Materials Research Part B: Applied Biomaterials, 101, 1393-1399.
[22] Gastinger, I., Koch, A., Marusch, F., Schmidt, U., Köckerling, F. and Lippert, H. (2002) Significance of Prospective Multicenter Observational Studies for Gaining Knowledge in Surgery. Der Chirurg, 73, 161-166.
[23] Grenander, U. (1951) On Empirical Spectral Analysis of Stochastic Processes. Arkiv för Matematik, 1, 503-531.
[24] Geman, S., Bienenstock, E. and Doursat, R. (1992) Neural Networks and the Bias/Variance Dilemma. Neural Computation, 4, 1-58.
[25] Agarwal, R. (2013) What Can We Learn from Null Randomized Controlled Trials? Journal of the American Society of Nephrology, 24, 691-693.
[26] Janes, A., Cengiz, Y. and Israelsson, L.A. (2004) Preventing Parastomal Hernia with a Prosthetic Mesh. Archives of Surgery, 139, 1356-1358.
[27] Hannink, G., Gooszen, H.G. and Rovers, M.M. (2013) Comparison of Registered and Published Primary Outcomes in Randomized Clinical Trials of Surgical Interventions. Annals of Surgery, 257, 818-823.
[28] Muysoms, F., Campanelli, G., Champault, G.G., DeBeaux, A.C., Dietz, U.A., Jeekel, J., et al. (2012) EuraHS: The Development of an International Online Platform for Registration and Outcome Measurement of Ventral Abdominal Wall Hernia Repair. Hernia, 16, 239-250.
[29] Ayer, G. (2010) Die Zeit Billroths in Zürich 1860-1867. In: Peiper, H.J. and Hartel, W., Eds., Das Theodor-Billroth-Geburtshaus in Bergen auf Rügen, Wallstein-Verlag, Göttingen, 151.

Copyright © 2022 by authors and Scientific Research Publishing Inc.

Creative Commons License

This work and the related PDF file are licensed under a Creative Commons Attribution 4.0 International License.