Blog Viewer

Guideline Development Within the Neurocritical Care Society

By Currents Editor posted 04-05-2018 13:51

Kumar-Mona.jpgMcNett-Molly.jpgBy Mona Kumar, MD, FNCS (left), and Molly McNett, PhD, RN, CNRN

The development of evidence-based clinical practice guidelines(CPGs) is one of the most valuable and visible functions of the Neurocritical Care Society (NCS). As such, development of scientifically sound CPGs is central to the mission of the society. Development of quality CPGs follow structured frameworks, which include a formalized needs assessment, identification of relevant topics, balanced writing groups, rigorous evaluation of the scientific literature, utilization of a systematic methodology to generate practice recommendations, as well as an exhaustive peer review process and methods to seek external endorsement, publication and dissemination of content to relevant stakeholders. Thus, the timeline for CPG development typically extends over a two-year period. In this article, we briefly discuss the history of CPGs in medicine, the NCS process of de novo CPG development and the procedure for external guideline endorsement. 

A History of Guidelines

CPGs are relatively new in the practice of medicine. The Institute of Medicine’s (IOM) Omnibus Budget Reconciliation Act of 1989 created the Agency for Health Care Policy and Research (AHCPR), now known as the Agency for Healthcare Research and Quality (AHRQ), to focus on outcomes and effectiveness research. AHRQ was designed to address escalating healthcare costs, variations in medical practice, inefficient health services, and to determine whether various incentives could improve utilization. As part of that charge, AHRQ partnered with the IOM to create CPGs and jointly they published a report entitled “Clinical Practice Guidelines: Directions for a New Program.” The IOM identified four major problems with guideline efforts up to that time:1

  1. The guideline development process was inefficient and not coordinated between societies.
  2. The guideline development process often lacked methodological quality controls.
  3. Most guidelines failed to meet standards of quality assurance, cost control, comparison with alternative practices and medical liability reduction.
  4. Due to methodological limitations, the impact of clinical practice guidelines on improving quality of care was indeterminate.
In 2008, the U.S. Congress instructed the Secretary of Health and Human Services to contract with the IOM to develop evidence based, methodological standards for systematic reviews and CPGs in order to ensure that “organizations conducting such reviews have information on methods that are objective, scientifically valid, and consistent.”2

In 2011, The IOM updated its definition of CPGs, describing them as “statements that include recommendations intended to optimize patient care that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options.”3 CPGs are comprised of two main parts: 1) a systematic review of the evidence bearing on a clinical question, focused on the strength of the evidence on which clinical decision making for that condition is based; and 2) a set of recommendations, involving both the evidence and value judgments regarding benefits and harms of alternative care options, addressing how patients with that condition should be managed, with everything else being equal. The 2011 definition stipulates that for CPGs to be trustworthy, they should be: 1) based on systematic review of the existing evidence; 2) developed by knowledgeable, multidisciplinary panels of experts and representatives from affected groups; 3) cognizant of important patient subgroups and preferences; 4) based on an explicit and transparent process that minimizes biases and conflicts of interest; 5) provide ratings of both the quality of evidence and the strength of recommendations; and 6) reconsidered and revised as appropriate.3 This definition underscores the importance of the systematic review and stipulates that both benefit and harm assessments are essential characteristics of CPGs. According to the IOM, this definition separates CPGs from other statements, such as consensus statements, practice parameters, white papers and expert advice.

Differentiating Between CPGs and Other Documents

NCS employs the aforementioned IOM definition of CPGs in our guideline development processes. Consensus statements are similar to CPGs in that they are recommendations developed using available evidence; however, consensus statements also utilize expert opinion (consensus) in areas where high quality clinical data is limited or does not exist. Development of consensus statements follow a structured process that includes formation of a multidisciplinary team of experts, as well as formalized vetting and adjudication of evidence and expert opinion to generate recommendations. In contrast, a position paper is a document that sets forth the opinion or stance of the society on a particular topic. The document includes background information and discussion on a topic to provide a thorough understanding of issues and rationale behind the position4-5 but does not necessarily employ a systematic review and critical evaluation of the literature to generate recommendations. Similarly, a white paper is an authoritative report that provides facts about a particular issue and includes frameworks or strategies for potential solutions,4-5 but not recommendations based on evidence. Thus, white papers are used primarily for educational purposes. The distinctions between these types of documents are important to evaluate the quality and use of scientific evidence, and also to determine resource allocation, review process and endorsement procedures.

GRADE Methodology

The NCS process for guideline development is largely derived from the Institute of Medicine and further guided by the Grading of Recommendations Assessment, Development and Evaluation (GRADE) process.6 The GRADE working group began in the year 2000 as an informal collaboration of people with an interest in addressing the shortcomings of grading systems for scientific literature in healthcare. The working group has developed a common, sensible and transparent approach to grading quality (or certainty) of evidence and strength of recommendations. GRADEprovides a systematic and transparent framework for clarifying questions, determining the outcomes of interest, summarizing the evidence that addresses a question, and moving from the evidence to a recommendation or decision. The system is designed for reviews and guidelines that examine alternative management strategies or interventions, which may include no intervention or current best management. GRADE may be applied to a broad scope of clinical questions, including diagnosis, screening, prevention and therapy.

The Guidelines Committee (GC) of NCS chose GRADE methodology over alternate systems because of its simplicity, flexibility and adaptability (Figure 1). In GRADE, recommendations have two components: a two-level representation of the strength of recommendation (strong or weak) and a four-level representation of the quality of the evidence (high, moderate, low and very low)7. A recommendation is a strong recommendation to do (or not do) something, where the benefits clearly outweigh the risks (or vice versa) for nearly all patients. The quality of evidence in GRADE reflects confidence in the estimates of benefits, harms and burdens. Prior grading systems had many disadvantages, including the lack of separation between the quality of evidence and strength of recommendations, the lack of transparency about judgments, and the lack of explicit acknowledgment of values and preferences underlying the recommendations. In addition, the existence of many, often scientifically outdated, grading systems has created confusion among guideline developers and end users. The GRADE approach was developed to overcome the shortcomings of previous grading systems. Advantages of GRADE over other grading systems include:7

  • Developed by a widely representative group of international guideline developers
  • Clear separation between judging confidence in the effect estimates and strength of recommendations
  • Explicit evaluation of the importance of outcomes of alternative management strategies
  • Explicit, comprehensive criteria for downgrading and upgrading quality of evidence ratings
  • Transparent process of moving from evidence to recommendations
  • Explicit acknowledgment of values and preferences
  • Clear, pragmatic interpretation of strong versus weak recommendations for clinicians, patients, and policy makers
  • Useful for systematic reviews and health technology assessments, as well as guidelines.
More than 100 organizations from 19 countries around the world have endorsed or are currently using the GRADE process.

The Guideline Development Process at NCS

Development of CPGs by the NCS takes approximately two years from identification of the topic to publication and dissemination. Our process is outlined in Table 1 and detailed below:

1. Perform a needs assessment: The critical first step in CPG development is to determine an existing deficiency in clinical practice. NCS performs a needs assessment by querying its membership annually. At the beginning of each calendar year, NCS sends out a member survey to identify possible guideline topics. Additionally, other medical societies often invite us to collaborate on CPG development on topics to meet the needs of the broader medical community. An optimal CPG topic addresses a disease that is common or a population that is large, where practice changes would be most impactful. However, the topic must be feasible. The IOM has identified the following prioritysetting criteria for guideline development:

  • Disease burden – extent of morbidity and mortality imposed by a condition
  • Controversy – degree of controversy or uncertainty about a particular topic
  • Cost – economic cost associated with a particular condition
  • New evidence – new data exists with the potential to change prior conclusions
  • Potential impact – potential to improve decision-making for patient or provider
  • Public or provider interest – stakeholders want to inform decision-making
  • Variations in care – reduce variations in prevention, diagnosis, or treatment
2. Determine suitable topics: The results of the member surveys and external society requests are collated and tallied. The Guidelines Committee (GC) discusses the possible options, briefly reviews the extant literature and votes to determine the three to five best topics. These topics are ranked and presented to the Executive Committee, which makes the ultimate decision about the next guideline topic at the annual meeting in March.

3. Formation of a balanced writing group: After determination of the topic, the GC chooses topic co-chairs. Co-chairs are chosen largely for their expertise and knowledge of the subject matter. Prior guidelines experience is preferred for at least one chair. With guidance from the GC, the topic co-chairs assemble a multidisciplinary and well-balanced team of experts in the relevant clinical topic. The group should be balanced with regard to profession (e.g., MD, RN, PharmD), seniority, sex/gender, race, guidelines experience, nationality and membership status. Determination of whether patient groups or advocates should be involved in the process is made at this stage. If partnership with external societies is enviable, the GC will reach out to establish a memorandum of understanding and to determine whether a formal designee should be included on the writing committee.

4. Train in the GRADE process: After the writing committee has been assembled, the next step is to train members in GRADE methodology. Every writing group is assigned a GRADE Coach, or methodologist, who has attended national GRADE training conferences. The GRADE Coach educates the writing committee on GRADE methodology and formulation of appropriate questions. These questions are framed using the PICO (P-patient, I-intervention, C-comparators, O-outcome[s]) criteria. Questions are limited to approximately 10-15.

5. Consult a librarian: Once the PICO questions are formulated, a librarian is consulted to assist in an unbiased and systematic review of the literature. It has become more common for librarians to assist with systematic reviews and CPGs as ‘‘best practice’’ standards evolve for guideline development.8

6. Evaluate the extant literature: A systematic review and metaanalysis of relevant literature is then performed for each specific PICO question. The evidence is then rated using dimensions defined by GRADE: risk of bias, inconsistency, imprecision, indirectness, publication bias, large effect or dose effect of the intervention, and plausible residual confounding. We currently employ a software program designed to standardize and facilitate this review process. The web-based application uploadsreferences, enhances screening, assigns reviews and tracks progress. Importantly, it also creates evidence profiles or summary of findings tables, which display the overall quality of evidence rating for each outcome as well as across outcomes. These tables are critical for proof of transparency in the process and are required for posting on the AHRQ repository for CPGs, the National Guideline Clearinghouse. Recommendations are then formulated as strong or weak/conditional for, or against, a management strategy. There are specific considerations for the quality of evidence and also the balance of benefit/harm, patients’ values and preferences, as well as resource utilization.

7. Vote on recommendations: The writing group convenes in person at the annual meeting to vote on the recommendations. Each evidentiary table is reviewed and determination of recommendations is made according to the GRADE process.

8. Draft the manuscript: After recommendations are determined, the writing group prepares a manuscript for publication. A medical writer is then contracted to assist in the final composition and editing of the manuscript.

9. Perform review/seek endorsements: Once the manuscript of the CPG is complete, NCS calls upon the membership to review the document. Members have one to two weeks to perform an exhaustive review. The results are collated and summarized by the GC co-chairs elect, who present the comprehensive assessment to the GC for discussion. The feedback is then provided to the writing committee for incorporation into the final draft. Once the final draft is complete, it is sent to external societies for endorsement. Endorsement or affirmation may be subject to methodology employed, and may include addition revisions if warranted.

10. Publish and publicize the guideline: Once the review and endorsement process is complete, the CPG is presented at the annual meeting with concomitant publication in Neurocritical Care, with all relevant endorsements. The CPG is also posted on the NCS website to facilitate access. Once posted on the NCS website, the CPG is also submitted for posting on the National Guideline Clearinghouse website. Required for posting on that website is a comprehensive review of the methodology, a timeline of the process, the evidentiary tables and completion of a formal checklist. The GC updates the advocacy and publicity committees to promote visibility of the CPG.

How can you get involved?

If you are interested in participating in the guidelines process, there are many opportunities for involvement. Society members are encouraged to submit topic ideas via our annual survey to the membership, which is a critical component in identifying potential topics. Alternatively, members can volunteer to review drafts of internal guidelines, or to review CPGs from other societies to determine level of NCS endorsement. Members interested in joining the GC should respond to the member and leadership surveys conducted every spring, as these are the most direct routes to committee membership. Members involved in CPG development with other societies, including (but not limited to) the American Academy of Neurology, the American Stroke Association and the Society for Critical Care Medicine are encouraged to inform the GC co-chairs of current projects. Knowledge of NCS member participation on external guidelines and projects facilitates coordination and collaboration with partnering societies, and can streamline review and endorsement processes.


1. Institute of Medicine (IOM) Committee to Advise the Public Health Service on Clinical Practice Guidelines. Clinical Practice Guidelines: Directions for a New Program. Washington DC: National Academies Press, 1990.

2. United States Congress. Medicare Improvements for Patients and Providers Act of 2008, Public Law 110-275. Retrieved February 2018: PLAW-110publ275.pdf.

3. Institute of Medicine. Clinical Practice Guidelines We Can Trust. Washington DC: National Academies Press, 2011.

4. American Academy of Family Physicians. AAFP Definitions for Policy Statement, Position Paper, and Discussion Paper. Retrieved February, 2018: html

5. Ingravallo F, Dietrich CF, Gilja OH, Piscaglia F. Guidelines, clinical practice recommendations, position papers, and consensus statements: Definition, preparation, role, and application. Ultraschall in Med 2014: 35(05): 395-399.

6. Guyatt GH, Oxman AD, Schunemann HJ, Tugwell P, Knottnerus A. GRADE guidelines: A new series of articles in the Journal of Clinical Epidemiology. J Clin Epidemiol 2011; 64(4): 380-2.

7. Schünemann H, Brożek J, Guyatt G, Oxman A, editors. GRADE handbook for grading quality of evidence and strength of recommendations. Updated October 2013. The GRADE Working Group, 2013. Retrieved February 2018: guidelinedevelopment. org/handbook.

8. Cooper, ID, Crum JA. New activities and changing roles of health sciences librarians: A systematic review, 1990-2012. Journal of Medical Library Association 2013; 101(4): 268-277.


NCS_201803_Guideline_Developmen_1.jpg NCS_201803_Guideline_Developmen_2.jpg

By Christa O’Hana S. Nobleza, MD, MSCI, (left) and Diana Greene-Chandos, MD, FNCS (right) The Women in Neurocritical Care (WINCC) recently announced the creation of the WINCC Mentorship Subsection headed by Christa O’Hana S. Nobleza, MD, MSCI. This Subsection is composed by a multidisciplinary ...
By Currents Editor   Fostering research that advances the field of neurocritical care is one of NCS’ top strategic priorities. The NCS Research Committee reviews survey requests and grants approval for surveys to be sent to NCS members. Below, you will find the latest approved surveys. Please ...
By Sasha Yakhkind (left) and Christa O’Hana S. Nobleza, MD, MSCI (right) Do you have questions regarding your professional life? Is there a conundrum at work that requires an outside perspective? The Trainee Section and Women in Neurocritical Care (WINCC) Section have teamed up to publish a new ...