Epidemiology and Research Methodology

Observational Studies

There are two main types of observational studies: cohort studies and case–control studies. A cohort is a group that has something in common; in epidemiology this is frequently risk of developing a disease of interest. Cohort studies enroll a population at risk and follow them for a period of time. Individuals who develop the disease in that time are then compared with individuals who remain disease-free.

Cohort studies in vascular care

There have been many cohort studies performed in vascular surgery pertaining especially to the utility of endovascular aneurysm repair (EVAR) versus open surgical repair (OSR) in the treatment of abdominal aortic aneurysms. One prominent trial was conducted by the OVER Veterans Affairs Cooperative Study Group, which recruited 881 patients and randomized them to EVAR or OSR. Notably, this study found that overall survival at 14 years of follow-up was similar between patients randomized to EVAR and those randomized to OSR ( Fig. 1.1A ). Other notable cohort studies have been performed in vascular surgery using cohorts of patients described in Medicare claims, as well as cohorts from the Society for Vascular Surgery’s Vascular Quality Initiative (VQI) registry.

For example, Columbo et al. utilized a database that linked the VQI registry to Medicare claims to study 12,911 patients who had undergone EVAR and the long-term effects of the procedure. This group found that a third of EVAR patients were at risk of reintervention and further identified five clinical factors at the time of the initial repair that were associated with a higher risk of reintervention ( Fig. 1.1B ).

Experimental Studies

The other large class among study designs is the experimental study . Unlike observational studies, experimental trials involve introducing participants to an exposure of interest. One benefit of experimental studies is the ability to randomize participants, commonly via a randomized controlled trial (RCT).

The benefit of randomization is the avoidance of bias. Randomization ensures that known factors are evenly distributed between the exposure and control groups. Further, it also ensures the even distribution of unknown factors. Thus, in a well-designed RCT, complex statistical models are not necessary to control for confounding factors, as long as randomization is performed in a well-designed and well-executed fashion.

There are several ways of structuring a randomization to address potential issues including complete randomization of the entire study population, block randomization, and adaptive randomization. For complete randomization , each new patient is randomized without prior influence on previously enrolled patients. The expected outcome at the completion of the trial is an equal distribution of patients within each treatment group, although unequal distribution may occur by chance, especially in small trials.

In a cluster randomization , groups of individuals (i.e. communities, schools, hospital systems, etc.) are randomized to treatment arms. This methodology is useful when complete randomization is difficult to implement, and other factors can confound randomization at the individual level. One example of such a study is the Preferences for Open Versus Endovascular Repair of Abdominal Aortic Aneurysm (PROVE-AAA) trial. This cluster randomized trial aimed to determine the efficacy of a validated decision aid to enable better matching between a patient’s ultimate repair modality and their preoperative preference. This study’s unit of randomization was each participant location. Each location was randomized to receiving a decision aid or not receiving a decision aid. All subsequent patients in each location would then receive or not receive the decision aid on the basis of their treatment location. As treating patients who have used the decision aid may change how a surgeon interacts with those who do not have the decision aid, a cluster randomized trial was used to ensure that a surgeon’s actions are not “contaminated” by the decision aid itself. This methodology further ensures better assessment of the decision aid’s efficacy.

Experimental studies face stricter ethical and patient safety requirements than their observational counterparts. One basic assumption of experimental trials is clinical equipoise , or the existence of more than one generally accepted treatment. This must exist both to create the situation where the research that is being undertaken will lead to clinical relevant information and that the treatment options to which a participant is randomized will not be assuming risk of care that is known to be inferior. Whereas you could not randomize people to observation only for a ruptured aortic aneurysm, for certain populations you could make an argument for endovascular versus open repair. This type of situation often arises when clinical experts professionally disagree on the preferred treatment method. It is worth noting that although the field may have equipoise, individual healthcare providers or patients may have bias for one treatment. In such a case, enrollment in an RCT may be difficult because the patients or their providers are not willing to be subject to randomization. A recent example of a large clinical trial in vascular care where equipoise in treatment options has been compared is the Best Endovascular vs. Best Surgical Therapy in Patients with Critical Limb Ischemia, or BEST-CLI trial ( www.bestcli.com ). This large multicenter, NHLBI-funded trial compares open surgery to endovascular treatments using a pragmatic study design. Results from this landmark trial are expected to be reported in the fall of 2020.

Although RCTs represent the pinnacle in clinical design, there are many situations in which RCTs are impractical or impossible. Clinical equipoise may not exist, or common sense could prevent randomization of well-established practices, such as the use of parachutes during free fall. RCTs can also be costly to conduct and must generate a new control group with each trial. For this reason, some studies are single-arm trials that use historical controls similar to the case–control design. In addition, patient enrollment may also be difficult, particularly if patients or clinicians are uneasy with the randomization of treatment. RCTs can also have methodologic and interpretative limitations. For example, if study patients are analyzed by their assigned randomization grouping ( intent to treat ) studies with asymmetric or high overall dropout and/or crossover rates may not reflect actual treatment effects. Given the cost and time required, RCTs are often conducted in high-volume specialty centers; as a result, enrollment and treatment of study patients may not reflect the general population with the disease or providers in the community. Finally, as with any analysis, inaccurate assumptions made in the initial power calculations may lead to failure to capture a true effect.

Special Techniques: Meta-Analysis

Meta-analysis is a statistical technique that combines the results of several related studies to address a common hypothesis. The first use of meta-analysis in medicine is attributed to Smith and Glass in their review of the efficacy of psychotherapy in 1977. By combining results from several smaller studies, researchers may decrease sampling error and increase statistical power, thus helping to clarify disparate results among different studies.

The related studies must share a common dependent variable. Effect size specific to each study is then weighted to account for the variance in each study. Because studies may differ in patient selection and their associated independent variables, a test for heterogeneity should also be performed. Where no heterogeneity exists ( P > 0.5), a fixed-effects meta-analysis model is used to incorporate the within-study variance for the studies included. A random-effects model is used when concern for between-study variance exists (0.5 > P > 0.05). When heterogeneity among studies is found, the OR should not be pooled and further investigation for the source of heterogeneity may then exclude outlying studies.

The weighted composite dependent variable is visually displayed in a forest plot along with the results from each study included. Each result is displayed as a point estimate, with a horizontal bar representing the 95% confidence interval for the effect. The symbol used to mark the point estimate is usually sized proportional to other studies to reflect the relative weight of the estimate as it contributes to the composite result. For example, Columbo and colleagues examined bleeding risk associated with continuing aspirin during non-cardiac surgery, with an effect size shown in the forest plot shown in Figure 1.2 . Classically, meta-analyses have included only RCTs, but observational studies can also be used. ^, Inclusion of observational studies can result in greater heterogeneity through uncontrolled studies or controlled studies with selection bias.

The strength of a meta-analysis comes from the strength of the studies that make up the composite variable. Furthermore, if available, the results of unpublished studies can also potentially influence the composite variable, because presumably many studies with nonsignificant results are not published. Therefore, an assessment of publication bias should be included with every meta-analysis. Publication bias can be assessed graphically by creating a funnel plot in which the effect size is compared with the sample size or another measure of variance. If no bias is present, the effect sizes should be balanced around the population mean effect size and decrease in variance with increasing sample size. If publication bias exists, part of the funnel plot will be sparse or empty of studies. Begg’s test for publication bias is a statistical test that represents the funnel plot’s graphic test. The variance of the effect estimate is divided by its standard error to give adjusted effect estimates with similar variance. Correlation is then tested between the adjusted effect size and the meta-analysis weight. An alternative method is Egger’s test , in which the study’s effect size divided by its standard error is regressed on 1/standard error. The intercept of this regression should equal zero, and testing for the statistical significance of nonzero intercepts should indicate publication bias.