Quality Appraisal of Clinical Guidelines in Obstetrics and Gynecology in India ()
1. Background and Introduction
Clinical practice guidelines offer recommendations that are informed by a systematic review of evidence. They offer an assessment of benefits and harms of management strategies for optimal patient care [1]. Contextual guidelines are easier to adopt and easier to implement thereby positively impacting health care [2] [3]. Some of the Low Middle Income countries have adopted the existing western guidelines to formulate health policy. However, this has its problems of adaptation and implementation due to the lack of resources and funds.
However, multiple guidelines on the same health condition often result in recommendations that are inconsistent [4] [5] and confusing. For the guidelines to be reliable, their methodological rigour and transparency need to be ensured [1] [6].
With the establishment of National Rural Health Mission (NRHM) in 2005, guideline development gathered pace in India. Recent focus on Universal Health Care (UHC), health insurance [7] and health financing [8] are compelling reasons for India to have its own well-formulated and validated clinical practice contextual guidelines.
This paper reports the outcomes of systematic assessment of the published Indian guidelines related to Obstetrics and Gynaecology using the validated WHO Appraisal of Guidelines for Research & Evaluation II (AGREEII) tool [9].
The objective of the study is to present the quality of Indian clinical practice guidelines published in the specialty of obstetrics and gynecology.
2. Methods
All reported guidelines in obstetrics and gynecology published by Indian authors from 1st Oct 2013 to 1ST April 2018 were accessed after a detailed search of web portals of Governmental and non-government professional bodies. Systematic search for relevant publications was conducted in the following websites.
1) National Health Mission, NRHM Guidelines. Available from: http://www.nrhm.gov.in/nhm/nrhm/guidelines/nrhm-guidelines.html. [Last accessed on 1st April 2018].
2) http://www.fogsi.org/category/archives-resources/clinical-guidelines.
3) http://nhm.gov.in/nrhm-components/rmnch-a/maternal-health/guidelines.html.
4) https://www.gfmer.ch/Guidelines/Obstetrics_gynecology/Obstetrics_gynecology.htm, India.
5) https://www.researchgate.net/.
6) https://www.nhp.gov.in.
7) http://healthfacilityguidelines.com/Guidelines/ViewPDF/HFG-India/part_b_obstetrics_unit.
8) https://www.icogonline.org.
9) http://clinicalestablishments.gov.in.
10) http://aogd.org/.
11) http://www.narchi.org/.
12) http://indmed.nic.in/.
13) https://www.ncbi.nlm.nih.gov/pubmed/.
14) https://scholar.google.co.in/.
Article type limits were ‘‘Guideline’’, ‘‘Practice Guideline’’, and ‘‘Clinical Practice Guideline’’. Documents in the form of circular, memos, checklists, government orders but labelled as “guidelines” were excluded. The screening of titles and abstracts and subsequently of full texts was performed by the two authors independently.
Each guideline was subjected to inclusion using 3-point assessment criteria (relevance, clarity of intervention/outcome, and appropriate use of healthcare resources).
3. Definitions
• Relevance: Guideline addresses an important health condition of high/moderate prevalence.
• Clarity of intervention/outcome: The intervention/outcome needs to be clearly defined e.g. drug dosages and formulations according to local availability.
• Appropriate use of health resources: Clarity regarding referrals, patient monitoring, surgical interventions.
The guidelines were rated as “Low”, “Moderate” or “high” for each field by each author independently. Guidelines that received the combination of “Moderate” and or “High” were included for appraisal. Each CPG was then appraised using AGREE II checklist independently by the two authors and scored. Discrepancies in the screening of title, abstracts as well as full texts were resolved by discussion between the authors.
4. Results
A total of 70 CPGs related to Obstetrics and Gynecology published in India were retrieved. After the removal of duplicates, 47 CPGs were considered for screening (Figure 1). Only 8 guidelines met the inclusion criteria. These were assessed and scored using AGREE II checklist by each author independently. Individual domain and overall scores were calculated using the AGREEII Score Calculator. The number of authors for CPG varied from 5 - 24. Four guidelines focused on treatment, one on screening and treatment and two addressed prevention, screening and treatment (Table 1). While seven of the 8 guidelines scored over 75% for scope and purpose, the rigor for development, applicability and editorial independence were found to be poor (Figure 2). None of the guidelines referenced the target user. The overall assessment scores ranged from 8% to 22% with a median score of 15%. None of the guidelines were recommended as “Yes”
![]()
Table 1. Characteristics of included guidelines in the study.
by either of the reviewers. Only 1 review had identified cost as one of the focus areas as part of the guideline. There were no provisions for peer review, audit of practice or updating later in any of the reviews. Disclosures, conflict of interest or funding information was not available in any of the CPGs (Figure 2).
5. Discussion
Several guidelines, manuals, and reference materials on maternal health have been developed in India by Government and professional bodies to enhance the quality of gynecological services, antenatal care, safe delivery and perinatal care but the quality of these guidelines has been found to be modest to low as was also documented in our study. In most cases, the methods used fell short of basic standards and were not based on research evidence. The low scores for rigor of development and editorial independence observed in our study were also documented by Sonawane et al in the publication on Appraisal of Maternity Management and Family Planning Guidelines using AGREE II [10].
Guideline production is time-consuming and expensive [11]. It is estimated to require 14 - 16 months to construct each guideline assuming the authors strictly adhere to their tasks and proposed timelines.
![]()
Figure 2. Standardised AGREE II domain scores.
In our study, there were no plans for updating any of the guidelines. It is generally accepted that guidelines require to be re-evaluated at least every 3 - 4 years one of the key components under the domain of rigor of development. However, the recommendation to review all guidelines and to replace those more than 5 years old may not be feasible due to cost and time constraints. If guidelines are not updated as per current evidence, it should be clearly noted. Older guidelines should be better retired.
We accept the suggestion that it might be better to have a modular approach to guideline development [12] : instead of commissioning a guideline on broad topic e.g. “management of endometriosis” focus on more specific clinical questions might be cost-effective and practical. Hence the title, “Dienogest in management of pain due to mild to moderate endometriosis” is more specific. Since management of a disease will be covered over several guidelines instead of a single publication, updates may be provided to individual modules of a disease management process instead of having to reconstruct an entire guideline, many parts of which may not change over time.
Guideline development process requires rigour and finally and open discussion with stakeholders including patient representatives. All the CPGs in our study were published from medical colleges with no participation of the stakeholders down the stream. The poor scores in the “stakeholder involvement” domain are due to non-involvement of healthcare professionals other than doctors. Involvement of patient representatives, non-academic clinicians and clinicians from non-elite institutions is expected to improve translation and implementation of the guideline. The heavy academic focus in guideline development also leads to inadequate consideration of putting recommendations into practice resulting in poor scores in the “applicability” domain.
Academic elitism in authorship of the CPGs was an important observation made by Bhaumik et al. [13]. Both at selection of panel and consultative processes for formulating recommendations, work and suggestions of members from elite and tertiary teaching college institutions carry more importance compared to those from smaller institutions. The lack of formal methods for formulating recommendations is particularly problematic where the guideline panel composition and the consultative process are heavily biased in favour of the academia in elite institutions making it difficult for all voices to be heard or and ensure fair weight age to everyone’s argument. The need to utilise whatever capacity that is available in-country other than attitudinal barriers wherein guideline development is seen as an academic activity could be possible reasons for this observation. This also emphasizes the need for capacity building and training of policy makers as well as professionals at all levels for the formulation of clinical practice guidelines.
In their mixed method study by Bhaumik et al. [13] interview of the authors helped to understand the reasons for low to moderate scores of several parameters. The main impediments included poor governance structures and inadequate in-country capacity for evidence search, syntheses and guideline methodology. A previous study on the growth of Cochrane in India found that systematic review authors (indicative of capacity) are limited to elite institutes only [14]. The INDEX TB guideline which received high scores in the study by Bhaumik et al., had engaged Cochrane consultants (from India and United Kingdom) for evidence search, appraisal and methodological advice [13].
In the case of National Institutes of Health, the process of evidence presentation is followed by a period of discussion to which any interested party, including the public, may contribute. The resulting consensus statement is approved after revision based on these comments and subsequently published in a peer reviewed journal [12]. This ensures the acceptance and implementation of the guideline. The NICE process involves setting up advisory groups that bring together technical expertise and relevant “lay” or service user experience. Such activity brings strengths in terms of plurality of experiences, perspectives, and backgrounds to inform evidence-based recommendations [15].
Another important step is for societies to adopt the GRADE framework. GRADE (Grading of Recommendations, Assessment, Development and Evaluations) is a transparent framework for developing and presenting summaries of evidence and provides a systematic approach for making clinical practice recommendations [16] [17].
It is the most widely adopted tool for grading the quality of evidence and for making recommendations with over 100 organizations worldwide officially endorsing GRADE. GRADE “evidence profiles” are created to summarize abstracted data across multiple studies and the quality of evidence is assessed as high, moderate, low, and very low. The advantage of GRADE is that the strength of recommendations is not based solely on the quality of evidence; 3 additional factors are considered when attaching a strength of recommendation—1) the balance between harms, and benefits between the intervention and the comparator; 2) the clarity of patient preferences for various outcomes of the intervention and comparator; and 3) whether the intervention represents an appropriate use of healthcare resources [12].
The second domain with the lowest scores, editorial independence, includes recording conflict of interest of guideline development group members and influence of a funding body. Conflict of interest, association with industry, and details of funding body (if any) need to be mandatorily recorded at release of guideline. These details were not forthcoming in any of the guidelines in our study resulting in possibility of conflict of interest/bias. Conflict of interest (COI) is an important potential source of bias in the development of guidelines [18] [19]. Practices for declaring COI, association with the industry, and the funding body need be promoted in guideline development.
In their study on publications by Hauffman et al. reporting guideline appraisals with AGREE II, the impact of the 6 domain scores on the overall assessment of guideline quality was examined using a multiple linear regression model. The results of the regression analyses showed a statistically significant influence of all domains on overall guideline quality, with Domain 3 (rigour of development) having the strongest influence. For the recommendation for use, the results showed a significant influence of Domains 3 and 5 (applicability). Furthermore, all analyses showed a statistically significant influence of Domain 5 on over all assessments also [20]. The strong influence of Domain 3 on the overall assessments is supported by earlier research which suggests that this domain is a stronger indicator of guideline quality than the other domains [21] [22]. This requires urgent attention in the context of Obstetrics and Gynecology clinical practice guidelines published in India. The existing challenges include lack of capacity in developing guidelines that are methodologically robust compounded by deficiency of good quality primary research.
Cost of management is an important area that needs to be addressed specially in the setting of low resource countries. Sensitisation and training are also required in this area.
Limitations of the study: The lack of a dedicated guideline database or electronic repository may have limited the access to Indian guidelines, raising a possibility that some important guidelines have been missed. Interview of the members involved in guideline development would have added to the strength of the study. There is also the lack of data regarding the uptake of existing guidelines.
6. Conclusions
This attempt at appraising guidelines has thrown light on various aspects of guideline development and the prerequisites for their implementation in the Indian context. There should be a robust process for developing guidelines with a thrust on documentation in line with international standards with a focus on openness and inclusiveness. Progress toward better quality guidelines, which are developed in a transparent, evidence-based and a systematic manner in India would require governance, planning and dedicated funding supported by changes in the medical curriculum and capacity building efforts. Issuing agencies need to adopt policies to make panels more representative, search and appraise evidence appropriately, have formal process for formulating recommendations and disclose conflict of interests.
Establishing or nominating an independent national agency for guideline development, implementation and monitoring housed within the Directorate of Health Services could be an option. Involving stakeholders from the group of care givers, nurses, and users in the development process will improve the quality and inclusiveness of guidelines. The development of clinical guidelines should be a tool in medical and paramedical education to inculcate the value of evidence-based practice.
The costs for producing each guideline are quite high, even with the mostly volunteer efforts of guideline authors. There remains a great challenge to motivate the participation of busy clinicians to the extent necessary to achieve the quality we all desire. Finally, we require methodologists who are not only experts in the conduct of systematic reviews and meta-analyses but have undergone additional training in the GRADE process. The combination of these factors limits the number of guidelines that can be produced using the rigorous process that is necessary to ensure validity.
There is a need for sensitization and capacity building of clinicians and public health professionals on the development of CPG related to obstetrics and gynecology in India.