Meaningful Retrospective Analysis


Summary of Key Points

  • Confounders in clinical investigation are systematic errors that cause a tendency toward erroneous results. The two major forms of bias in retrospective studies are selection bias and missing data.

  • Case-control studies are used to identify factors that might lead to a particular outcome by looking retrospectively and comparing cases with a particular outcome to controls without that outcome. These types of studies are ideal when the incidence of a particular outcome is rare (e.g., infection).

  • Large administrative databases containing vast amounts of patient data have increased our opportunity to generate meaningful evidence using retrospective analysis.

  • The state inpatient databases (SIDs) are part of the Healthcare Cost and Utilization Project (HCUP) sponsored by the Agency for Healthcare Research and Quality. A SID contains demographic data, admission date and discharge date, procedure, and diagnosis codes. Some of the SIDs contain patient identifiers and permit longitudinal analysis. For example, SIDs have been used to study reoperation rates following spine surgery.

  • The nationwide inpatient sample is part of the HCUP sponsored by the Agency for Healthcare Research and Quality and represents a weighted sample from the SIDs that is readily generalized to the entire U.S. nonfederal hospital universe. The NIS has been used extensively to study complication rates following surgery and has also been used to assess comparative hospital charges for various spinal procedures.

  • The NIS contains no patient identifiers and can only be used to study single hospital admissions. The NIS contains no patient-reported outcomes data.

  • Clinical registries contain vast amounts of patient data collected for clinical purposes using epidemiological methods to systemically gather data for specific patient populations. They are able to generate powerful retrospective analyses to evaluate healthcare quality and value and clinical outcomes.

Today, there is a growing need to supplement our education with a systematic understanding of the principles of evidence-based practice. This chapter explores the value of the retrospective approach and explains how to interpret clinical evidence when the clinical data are not randomized, or even prospective. New approaches using modern statistical methods with administrative databases have improved our ability to extract meaningful conclusions from retrospective data. By understanding both the strengths and weaknesses of the retrospective study, we can learn to harness the power of the approach without falling prey to false conclusions. Indeed, there is a role for retrospective studies, and it is likely that the approach will have a lasting presence in understanding outcomes from spine surgery.

Retrospective Clinical Studies

Retrospective studies represent a major portion of the available evidence in neurological surgery. Although these types of studies constitute level III or IV evidence (Oxford Center for Evidence-based Medicine Levels of Clinical Evidence [ Table 177.1 ]), they nonetheless represent the majority of neurosurgery evidence to date. There are three major types of nonrandomized clinical studies:

  • 1.

    Case-control studies (Oxford Center Level III Evidence): Case-control studies are used to identify factors that may be associated with a particular outcome by looking retrospectively and comparing cases with a particular outcome to controls without that outcome. When the endpoint is infrequent, the case-control method is particularly useful, because prospective studies would likely require large numbers of patients to have enough power to detect differences between cohorts if the differences did exist. Misclassification of true case-control studies in the neurosurgical literature occurs often.

  • 2.

    Nested case-control studies: One variation of the case-control study is the nested case-control study, in which cases of a condition are identified from a defined cohort, and, for each case, a selected number of matched controls are chosen. This strategy has the advantage of potentially limiting not only the costs that might be associated with doing a large prospective trial but also the confounding biases typically associated with traditional case-control studies, in which the case and control populations differ substantially in ways that might not be apparent to the investigators evaluating the results.

  • 3.

    Case series (Oxford Center level IV evidence): A case series refers to a study of one group of patients with a discrete outcome regardless of whether they have had specific exposure, or patients with a discrete outcome with a specific exposure. These studies tend to be descriptive and are best used to provide outcomes data for new techniques or for the treatment of rare disorders. These studies do not include control groups.

Table 177.1
Medical Levels of Clinical Evidence
Modified from Oxford Center for Evidence-based Medicine Levels of Evidence and Grades of Recommendation. 2001. Accessed January 14, 2020. Available at: http://www.cebm.net/index.aspx?0=5513 .
Level of Clinical Evidence Description
I Well-executed randomized controlled trial
II Prospective cohort study with controls
III Case-control study
IV Case series (without control group)
V Expert opinion or theory

Limitations of Randomized Clinical Trial Methodologies

Although the randomized controlled trial (RCT) represents the highest level of clinical evidence, significant barriers exist to performing an RCT in spine surgery. The heterogeneity of spine diseases, the requirement for equipoise, and the learning curve associated with novel procedures are often cited as common challenges in performing RCTs. Even when performed, the results of RCTs are difficult to interpret or do not provide a clear answer to the research question. For these and many other reasons, nonrandomized clinical studies, including retrospective studies, remain an important research tool for spine surgeons. In fact, the number of published retrospective clinical studies continues to increase. Prospective registries may represent an alternative to the RCT, although they pose the risk of generating data that can be difficult to interpret without clearly defined entry and exclusion criteria and control groups.

You're Reading a Preview

Become a Clinical Tree membership for Full access and enjoy Unlimited articles

Become membership

If you are a member. Log in here