Appendix 1: grading the recommendations

Grading Methodology

To accurately assess the strength of evidence available, the NHMRC methodology (FORM) was used in this clinical practice guideline to formulate and grade recommendations. The aim of this approach by NHMRC is to assist clinical practice guideline developers with a structured process for evaluating the evidence base corresponding to a particular key clinical question, in the context of the setting in which it is to be applied.⁹

The grading methodology allows for both the quality of the evidence and the strength of recommendations to be determined. Where insufficient evidence exists to formulate a grade, a practice point may be assigned instead. The NHMRC grading framework allows for these practice points to be included when developers consider it is important to provide non-evidence-based guidance.⁹

The NHMRC Evidence Statement Form sets out the basis for rating five key components of the ‘body of evidence’ for each recommendation. These components are:

1. The evidence base, in terms of the number of studies, level of evidence and quality of studies (risk of bias).
2. The consistency of the study results
3. The potential clinical impact of the proposed recommendation
4. The generalisability of the body of evidence to the target population for the guideline
5. The applicability of the body of evidence to the Australian healthcare context⁸.

The first two components describe the internal validity of the study data in support of efficacy (for an intervention), accuracy (for a diagnostic test), or strength of association (for a prognosis or aetiological question). As suggested, the third component gives the likely clinical impact of the proposed recommendation. The final two components assess external factors that may influence the effectiveness of the proposed recommendation in practice, in terms of generalisability of study results to the intended target population for the Guideline and setting of the proposed recommendation, and applicability to the Australian (or other local) health care system.⁸

These described components should be rated according to the body of evidence matrix (refer to Table 5). The matrix system is used to summarise the rating of the five key components which allows each recommendation to be assigned an overall NHMRC Grade of Recommendation (A-D).⁹

Table 5: NHMRC Body of evidence matrix⁸

^#Level of evidence determined from the NHMRC evidence hierarchy

*If there is only one study, rank this component as ‘not applicable’

^{^}For example, results in adults that are clinically sensible to apply to children OR psychosocial outcomes for one cancer that may be applicable to patients with another cancer

There is also capacity to note any other relevant factors that were considered by the guideline developers and the respective Working Group when judging the body of evidence and developing the wording of the recommendation.

The NHMRC grades given (A-D) are intended to indicate the strength of the body of evidence underpinning the recommendation (refer to Table 6).  Grade A or B recommendations are generally based on a body of evidence that can be trusted to guide clinical practice, whereas Grades C or D recommendations must be applied cautiously to individual clinical and organisational circumstances and should be interpreted with care. A recommendation cannot be graded A or B unless evidence base and consistency of the evidence are both rated A and B respectively.⁹

Table 6: Definition of NHMRC grades of recommendations^8,9

Note: This table is replicated in Clinical practice recommendations and practice points

By referring to the statements of evidence in combination with the NHMRC body of evidence matrix, a grade for each recommendation was derived from the respective grades allocated to the five key components. Grading the components of consistency, clinical impact, generalisability and applicability, was undertaken by the Working Group members, who discussed each section, and based on consensus achieved across the Working Group, arrived at these ratings.

The use of the NHMRC evidence hierarchy Table, categorises the respective study level according to the study design (refer to Table 3). This is used to determine the respective grades for evidence base and consistency of the recommendation.

Implementing the NHMRC Evidence Hierarchy, each included study in a systematic review should be assessed according to the following three dimensions of evidence:

1. Strength of evidence (level of evidence, quality of evidence (risk of bias) and statistical precision.
2. Size of effect (assessing the clinical importance of the findings of each study and hence addressing the clinical impact component of the body of evidence matrix.
3. Relevance of evidence (translation of research evidence into clinical practice and is potentially the most subjective of the evidence assessments).

Table 7: NHMRC Evidence Hierarchy: designations of ‘levels of evidence’ according to type of research question⁸