In 2006, a process was initiated to develop evidence-based paediatric guidelines directed towards physicians for reduction of cardiovascular risk. In contrast to earlier consensus-based guidelines from the National Heart, Lung, and Blood Institute, this process was to be evidenced based.^{1, 2} The function of the committee of experts appointed to execute this process was thus somewhat different from that in the past, where the role of the members of the committee was to contribute scientific expertise and identify literature that supported the opinions of the members of the committee with regard to the best strategies to promote reduction of cardiovascular risk. The aim of this manuscript is to briefly describe the process undertaken to write the evidence-based guidelines from the National Heart, Lung, and Blood Institute, beginning with the search for evidence, then the process of review of the evidence, and finally the writing of the final document. This manuscript will then conclude with some thoughts on how this process might be adapted in developing guidelines for caring for patients with congenital cardiac disease.

The search for evidence

The first step in the development of a guideline is to identify the questions to be answered by the guideline. For the guidelines for reduction of cardiovascular risk for children by the National Heart, Lung, and Blood Institute, the charge was to develop recommendations for physicians integrated across all the major risk factors identifiable in children and also integrated developmentally. This charge is extraordinarily broad based compared with other processes of development of guidelines, which often focus on a specific disease for which defined treatments are available. The complexity of the task is better understood when one considers that the endpoints of cardiovascular disease – including myocardial infarction, stroke, congestive cardiac failure, and aortic aneurysms – do not become prevalent until the fifth decade of life and are associated with a broad range of exposures to risk, including:

• • genetic disorders such as familial hypercholesterolaemia;

• • specific diseases such as Type I diabetes mellitus;

• • behavioural choices and addiction such as use of tobacco; and

• • common exposures seen in the population such as sedentary lifestyle and poor nutritional choices.

Applicable studies would almost exclusively address a portion of the risk over the course of life and atherosclerosis rather than specifically relate risk to hard endpoints of cardiovascular disease.

The questions developed took two forms. The first was a series of questions that provided the background for considering the need for strategies of reducing risk in children. That is, what was the evidence that adult onset cardiovascular disease began in youth? These questions probed the evidence for several domains:

• • the early development of atherosclerosis;

• • the relationship of risk factors for cardiovascular disease assessed in youth to the onset of atherosclerosis and other cardiovascular end-organ injuries; and

• • the relationship of risk factors for cardiovascular disease measured in youth to later events of cardiovascular disease, atherosclerosis, and other measures of injury to target organs measured in adulthood.

Without convincing evidence for the chronic nature of cardiovascular disease, the rationale for intervention would collapse. The second form of questions was specifically related to the outcome of interventions to modify risk factors. Was risk reduction in youth successful and did it have an impact on the future development of atherosclerosis, other forms of injury to target organs, or rates of events of cardiovascular disease in adulthood?

The review of evidence was conducted by an agency independent from the panel of experts, but based on the critical questions developed by the panel. Major databases were searched including PubMed and Cochrane reviews. Owing to the extraordinary number of articles identified, the search strategy was limited to the following types of articles:

• • systematic reviews,

• • meta-analyses,

• • randomised clinical trials, and

• • longitudinal observational and epidemiological studies.

Key elements were abstracted from each article including the population of the research, nature of the intervention if present, and important outcomes. Findings from each article were tabulated by risk factor and presented to the committee of experts for review and grading of the quality of the findings of the research.

Grading the evidence

Methods to grade evidence generally rate the quality of specific findings of the research based on the design of the study and consistency across multiple studies.³ Randomised clinical trials conducted in populations similar to those for which the guideline is intended are considered most important, followed by observational studies, then smaller clinical studies, and in the lowest tier is expert opinion supported by experimental data or limited clinical observations. Meta-analyses and systematic reviews are important to this process as they can provide vital information on consistency of findings across studies and quality of research.

For this guideline from the National Heart, Lung, and Blood Institute, the methodology of the American Academy of Pediatrics was chosen for several reasons:

• • this guideline from the National Heart, Lung, and Blood Institute was specifically developed for children and

• • this guideline from the National Heart, Lung, and Blood Institute was meant to interface with existing guidelines from the American Academy of Pediatrics to avoid redundancy and conflicting information for practitioners, such as counselling against use of tobacco by parents should be the same for prevention of cardiovascular disease as it might be for prevention of other diseases.³

The committee of experts felt that the grading of evidence must be performed in a way that was relevant to the task at hand, which had several unique aspects described above including:

• • the chronic nature of the disease under study,

• • the long period of time between the presence of antecedents of disease and events, and

• • the diversity of aetiologies that contributed to risk.

The committee reflected on those types of studies that established for them as individuals that prevention of cardiovascular disease beginning in youth was important. This process led to the affirmation as Level II quality evidence – below randomised clinical trials – of two types of studies:

• • high-quality observational studies with consistent evidence across studies conducted in diverse populations and

• • genetic natural history studies as models for chronic exposure either to risk or to its absence.

The latter has been called Mendelian randomisation: the likelihood of a given child receiving a specific gene in a family is random, and thus outcomes in affected and unaffected family members can be compared.Reference Davey Smith⁴ Examples of observational studies given special emphasis included Pathobiological Determinants of Atherosclerosis in Youth, Bogalusa, Muscatine, and Young Finns.Reference Gidding⁵ Examples of genetic defects informing the process included familial hypercholesterolaemia and change of function mutations in PCSK9.

Subgroups of the committee of experts then took the results of the independent search of the literature, further abstracted the relevant literature for scientific content, and then graded the evidence using the above schema. At least two members of the committee reviewed each article identified by the independent review.

Writing the guideline

It was not until the review of evidence was complete that the panel of experts seriously considered the exact recommendations to be made. Although the panel recognised that certain types of recommendations would be needed – such as screening of levels of cholesterol, treatment with drugs, and dietary counselling – discussion of the specifics of these recommendations was not begun until after evidence had been accumulated to inform the actual process. The background providing the rationale for intervention was reviewed. Then, for each risk factor for cardiovascular disease, a series of recommendations was developed; these recommendations received a grade of evidence and a strength of recommendation. The strength was based on three factors:

• • the quality of the evidence,

• • the vote of the committee – the more unanimous the vote, the higher the strength, and

• • the recognised impact of the recommendation.

Although the process of development of the guideline appears objective, at many points judgement by experts was critical. This need for judgement by experts was particularly true because of the almost complete absence of two types of evidence:

• • The first was data about cost: if a particular strategy was recommended, was there information on the ratio of cost to benefit of this recommendation to society?

• • The second was the generalisability of the recommended intervention to physicians in practice. For example, a clinical trial could demonstrate that a particular dietary intervention would lower cholesterol, but no evidence existed to support that the generalist could counsel the dietary intervention effectively. This dilemma contrasts the known efficacy of the intervention against the effectiveness of the intervention in actual practice.

An incongruous example of how these considerations might influence a recommendation relates to counselling against the use of tobacco. Excellent evidence from randomised trials documents the efficacy of counselling for cessation of smoking; however, in clinical practice, the intervention consumes time. In the context of a busy general paediatric practice where visits for maintenance of health last 15 minutes, is this intervention useful when there are many other competing health-related claims for that vital 15 minutes and, although proven, when the rate of success of intervention is low? In general, the final recommendations reflect the known efficacy of an intervention rather than effectiveness, as studies demonstrating effectiveness of successful interventions in practice simply did not exist.

A final process in writing the first draft of the guideline was directed towards integrating the recommendations in two ways: developmentally and across strata of risk. For the former, specific ages at which to consider assessment of risk and treatment were identified with a goal of being consistent with the Bright Futures Program of the American Academy of Pediatrics. A simple example is to begin screening of blood pressure at age 3 years. For the latter, consideration was given to specific scenarios of multiple risks of cardiovascular disease, for example the patient with diabetes and high levels of low-density lipoprotein cholesterol.

Perhaps one of the most profound lessons from the development of the guideline is that the word “final” has very little meaning. Throughout the entire process, numerous feedback loops exist. These loops of feedback include revising the search of the literature to ensure that all crucial papers related to the topic have been retrieved, and revision of drafts based on identified inconsistencies or deliberation of the committee. In addition, critical to contemporary development of guidelines is external review by key stakeholders including various scientific groups and governmental agencies where recommendations might influence policy. After this external review, the guideline is again revised. Publication can then occur – hopefully later in 2011 for this guideline; however, for a guideline to have lasting value, it should be associated with a process for implementation and a process for revision of recommendations based on new research.

Implications for guidelines related to congenital cardiac disease

Determining best practices for the care of patients with congenital cardiac disease is extraordinarily difficult because of many factors:

• • chief among these factors is the rarity of many of the diseases,

• • the “craft” nature of many of the treatments, that is, dependent on individual skill and experience,

• • the heterogeneity of the clinical spectrum of disease, and

• • the role of new technology in advancing newer treatments faster than older treatments may be evaluated.

However, as the field has matured and concerns about late outcomes of prior interventions have assumed prominence, the need for evidence-based processes to guide care can no longer be ignored.

In developing the guidelines directed towards physicians for reduction of cardiovascular risk, an important moment for the committee of experts came when the group was forced to confront the question, as described above, as to what evidence convinced the group that early prevention of cardiovascular disease was important. For reduction of cardiovascular risk, the types of studies that were uniformly cited were

• • studies linking risk factors for cardiovascular disease measured in youth to atherosclerosis,

• • observational studies linking risk in youth to either risk as adults or outcomes in adults, and

• • genetic studies as a proof of concept.

This scenario led to modifications of the paradigm of evidence based on scientific consensus.

With regard to congenital cardiac disease, an analogous critical process of thought can occur. The question can be asked, what are the best types of studies that can be conducted to demonstrate the usefulness of what we do on a daily basis?Reference Gidding⁶ If the problem is deciding whether a new technology is worthwhile, then we need to study key outcomes related to substituting the new technology for the old technology, including:

• • cost,

• • impact of treatment on outcomes,

• • ratio of risk to benefit to patients, and

• • comparative effectiveness of competing technologies.

If the problem is one of “craft”, that is, a high variability of outcomes across many centres for a specific technique, then randomised trials will not be the answer. Instead, we will need to get away from the model of idiosyncratic approaches by centre and use collaborative learning techniques across centres to demonstrate that variability in practice can be reduced. And if the problem is one of clinical equipoise among various treatment options, then randomised clinical trials may establish a firm evidence base for interventions.