Clinical Indications and Quality Assurance

Accuracy

The accuracy of a numeric measurement, such as wall thickness, aortic jet velocity, or aortic diameter, is expressed as the agreement between the echocardiographic measurement and a reference standard. These measurements reflect continuous variables; a continuous range of values is recognized from the smallest to largest seen in clinical practice. For example, aortic jet velocity ranges from <1 m/s to as high as 6 m/s. The numeric reference standard may be an anatomic measurement at surgery or autopsy, direct measurements in an experimental model, or comparison of echocardiography with other imaging techniques or hemodynamic recordings. Published data on the accuracy of echocardiography are shown in tables in each chapter of this book. More recent studies use an approach called Bland-Altman analysis, which compares the deviation of each measurement (echocardiography and the reference standard) with the mean of both measurements. Older validation studies typically report correlation coefficients and regression equations with standard errors for each measurement.

For echocardiographic diagnoses that are either present or absent (called categorical variables), accuracy reflects the certainty with which a specific diagnosis can be confirmed or excluded based on the test results ( Fig. 5.1 ). An example is echocardiography for the diagnosis of endocarditis: the patient either has or does not have endocarditis; no range of values exists. Accuracy for this type of test is described in terms of sensitivity and specificity. The sensitivity of a test is the degree to which it identifies all patients with the disease; specificity is the degree to which a test identifies all patients without the disease.

• ▪

  Sensitivity = “True-positive” test results / All patients with the disease = TP / (TP + FN)

• ▪

  Specificity = “True-negative” test results / All patients without the disease = TN / (TN + FP)

where TP is true positive, FN is false negative, TN is true negative, and FP is false positive.

Accuracy indicates the percentage of patients in whom the test results are correct in identifying the presence or absence of disease.

• ▪

  Accuracy = True positives + True negatives / Total number of tests = (TP + TN) / All tests

Using a diagnostic test to determine whether a disease is present or absent depends on the cutoff value or breakpoint used to define the test as abnormal. Sensitivity and specificity are related inversely to each other; in general, the higher the sensitivity, the lower is the specificity and vice versa. Whether a higher sensitivity is preferable to a higher specificity depends on the clinical question. If the goal of the test is identification of all patients with the disease, a high sensitivity is preferable. If the goal is confirmation of the diagnosis in an individual patient, a high specificity is preferable.

The relationship between sensitivity and specificity can be evaluated quantitatively for any given diagnostic test by graphing the sensitivity ( y -axis) versus 1 − specificity ( x -axis), with each point on the curve representing a different breakpoint defining the test as abnormal. The area under the curve reflects the clinical value of the test, with a larger area indicating a more reliable diagnostic test. The point on the receiver-operator curve where sensitivity and specificity are maximized indicates an appropriate breakpoint ( Fig. 5.2 ).

Precision

The reproducibility of echocardiographic imaging and Doppler data is affected by variability in:

• ▪

  Recording

• ▪

  Measurement

• ▪

  Interpretation

In addition, variability can occur both when the same person repeats the data acquisition or measurement at a different time (intraobserver variability) and when data acquisition or measurement is performed by different people (interobserver variability). These sources of imprecision are major limitations of echocardiography in clinical practice. Several approaches to improving the precision, and thus reliability, of echocardiographic data are used. Appropriate training and experience help ensure correct acquisition of data, including correctly aligned image planes and Doppler recordings, optimization of instrument parameters, and standardized study protocols. Measurement precision is improved with adherence to published standards, quality control in each laboratory, and comparison with reference standards when possible. Interpretation variability is minimized by using standard terminology and diagnostic criteria, developing a consensus approach to reporting in each laboratory, and comparing images and Doppler data with previous recordings in that patient whenever possible; that is, the report should specify whether a change from previous studies has occurred based on direct comparison of the recorded data, with side-by-side measurements as needed. Measurement variability is reported in each chapter when this information is available.

Expertise

The quality of an echocardiographic examination is highly dependent on the expertise of the sonographer performing the study, the physician interpreting the data, and the expertise of the laboratory. Optimal acquisition of image and Doppler data requires experience, in addition to education and training. A physician's interpretation is affected both by the data acquired (e.g., if images of a ventricular thrombus are not recorded, the physician will not see it) and by the education, training, and experience of that physician. Laboratory expertise affects data quality in terms of study protocols, time allocation and efficiency, instrumentation, and the group expertise of the sonographers and physicians. Thus, echocardiographic studies performed in different laboratories are not always comparable, and published studies on the accuracy of echocardiographic diagnosis may not apply to all diagnostic examinations.

Predictive Value

A major limitation of applying sensitivity and specificity data to an individual patient is the problem of whether a particular patient has a “true” or a “false” test result. Predictive values indicate the percentage of patients with a positive test result who have the suspected disease and the percentage with a negative test result who do not have the suspected disease:

• ▪

  Positive predictive value = true positives divided by all positives

• ▪

  Negative predictive value = true negatives divided by all negatives

However, predictive values are determined by the prevalence of disease in the population studied and also by the sensitivity and specificity of the test. Intuitively, this is obvious, comparing the use of echocardiography to “screen” healthy young subjects for endocarditis (many false-positive results because of ultrasound imaging artifacts) versus the same test in patients who have a new murmur, fever, and positive blood culture results, with a high prevalence of disease. The finding of a valvular vegetation on echocardiography in the latter group has a much higher predictive value for a diagnosis of endocarditis than in the healthy subjects, even though the sensitivity and specificity of echocardiography for diagnosing endocarditis are the same in both groups. Thus, the positive or negative predictive value of a test reflects disease prevalence as well as test accuracy.

Likelihood Ratio

The likelihood ratio indicates the relative likelihood of disease in an individual patient, based on a positive or negative test result. The likelihood ratio for a positive test result is calculated as:

or

A positive likelihood ratio >10 indicates an excellent test, and a ratio of 5 to 10 indicates a good test.

The likelihood ratio for a negative test result is calculated as:

or

A negative likelihood ratio <0.1 indicates an excellent test, and a ratio of 0.1 to 0.2 indicates a reasonably good test.

For example, diagnosis of left ventricular (LV) thrombus by echocardiography, assuming a sensitivity of 95% and a specificity of 88%, has a positive likelihood of 7.9 (a good diagnostic test) and a negative likelihood ratio of 0.06 (an excellent diagnostic test). The positive likelihood is not excellent because ultrasound artifacts may be mistaken for a ventricular thrombus. The excellent negative likelihood depends on a high-quality echocardiographic study and the expertise of the sonographer to ensure that an apical thrombus is not missed by echocardiographic imaging.

Pre-test and Post-test Probability

Another approach to the use of sensitivity and specificity data in patient management is to consider relevant clinical data along with the test result ( Fig. 5.3 ). The value of a diagnostic test increases when the pre-test likelihood of disease is integrated with the test results to derive a post-test likelihood of disease. This approach is known as Bayesian analysis. For example, the pre-test likelihood of severe aortic stenosis in an asymptomatic 30-year-old woman without a systolic murmur is very low. An echocardiogram purporting to show severe aortic stenosis most likely is an erroneous interpretation (a false-positive test result). In this setting, the result does not increase the post-test likelihood of disease very much. In contrast, in an elderly man with a 4/6 aortic stenosis murmur and symptoms of angina, syncope, and heart failure, the diagnosis of severe valvular aortic stenosis can be made with a high level of certainty even before any test is performed. The echocardiogram serves only to confirm the diagnosis and define the severity of obstruction. In general, diagnostic tests are most helpful when the pre-test likelihood of disease is intermediate so that the test result will substantially change the post-test likelihood of disease.

The most comprehensive approach to the evaluation of a diagnostic test is clinical decision analysis. Clinical decision analysis incorporates several rigorous approaches to the problem of clinical prediction, with the method most applicable to a diagnostic test (e.g., echocardiography) being the threshold approach. The basic tenet of clinical decision analysis as applied to a diagnostic test is that the test results should have an impact on patient care by either:

• ▪

  Prompting a change in therapy or

• ▪

  Leading to a change in the subsequent diagnostic strategy in that patient

This basic assumption is formalized in the threshold model of decision analysis. In this approach, two disease probability thresholds are defined for the diagnostic test:

• ▪

  A lower threshold below which the risk of the test is greater than the risk of not treating the patient and

• ▪

  An upper threshold above which treating the patient is a lower risk than performing the test

The intermediate range—in which the risk of treating or not treating the patient is greater than the risk of the diagnostic test—is known as the testing zone ( Fig. 5.4 ). For any specific indication, the testing zone for echocardiography generally is wide because of the low risk and high accuracy of this technique. However, both an upper threshold and a lower threshold still are definable for echocardiography. The upper threshold is reached in situations in which the diagnosis is clear, and echocardiographic examination would only delay appropriate treatment. For example, a patient with a classic presentation of an ascending aortic dissection (chest pain, wide mediastinum, peripheral pulse loss) requires prompt surgery. Any delay caused by unnecessary diagnostic testing could result in additional morbidity or mortality.

It is tempting to assume that no lower end to the test zone for echocardiography exists, given the absence of known adverse biologic effects of this procedure. However, the risk of the test also includes the risks of additional diagnostic tests or even erroneous treatment choices resulting from a false-positive or false-negative echocardiographic findings. For example, an echocardiogram is not indicated to evaluate for aortic dissection in a young patient with atypical chest pain and a normal physical examination, electrocardiogram, and chest radiograph . If a false-positive echocardiographic diagnosis leads to further evaluation with cardiac catheterization, any complications from the invasive procedure ultimately can be considered a consequence of the echocardiographic results. Thus, a lower limit to the test zone does exist for echocardiography and can be defined for each specific diagnostic indication by applying decision analysis techniques. Other clinical decision analysis approaches have been applied to specific clinical problems that use echocardiographic data as a branch point in the decision analysis tree.