Improving Quality and the Value of a Vascular Registry to the Practice

Introduction

Registries contain large bodies of information, generally collected in the course of routine clinical practice. The usefulness or value of that information depends on many issues but primarily whether it is used for a purpose consistent with the intent of the registry and whether the registry data are regarded as valid. This chapter summarizes the characteristics of registries, and examines how they might – or might not – add value to the practice of vascular surgery. A comprehensive review of medical registries is also available for those who wish more detail.

The Definition of a Medical Registry

Although most practitioners intuitively understand the concept, it is useful to unambiguously define a medical registry. Some have applied the term to almost any database that collects information about patients or clinical events. However, there are specific characteristics that make a collected data set a true registry. The most useful definition was advanced by Solomon et al. in 1991 and subsequently subscribed by many others: a systematic collection of a clearly defined set of health and demographic data for patients with specific health characteristics, held in a central database for a predefined purpose. ^,

Understanding the predefined purpose of the registry is paramount, as this Inclusion Principle will govern downstream decisions about what data should be collected. In other words, what characteristic(s) of a patient or medical event qualifies it for registry inclusion? The diversity of medical registries arises from the various reasons or predefined purposes for their existence.

One way of operationalizing this definition has been termed the MDR - OK model:

M ergeable data facilitates aggregation of information submitted from various individual sources that can be combined into a larger, more representative, dataset.

D ata set standardization means that the same characteristics are collected on every patient or case included in the registry. The predefined purpose of the registry will inform the process of selecting the characteristics to be collected for later analysis. Adherence to collecting a clearly defined standardized data set should reduce the subsequent problem of missing information during analyses. Completeness of data collection is a key factor in determining the quality or external validity of a registry.

R ules for data collection provide guidelines for translating raw source data into definitions established by the registry. These guidelines are typically contained in a protocol and data dictionary developed during conception of the registry. Observance of these rules will ensure that data in the registry are reproducible and consistent, even if collected by different people at different times in different locations about different patients or events. The degree to which these rules are complied with affects the consistency of a registry, which is a measure of internal validity. Examples of this process are mapping blood pressure information into a registry data element such as “Hypertension” or creatinine values into a data element termed “Renal Insufficiency.”

The aforementioned characteristics are fundamental to a medical data registry. Several additional features may enhance the value of a registry:

O bservation over time means that a registry collects longitudinal patient data, which are then linked to the initial patient record.

K nowledge of outcomes may allow a registry to indicate which characteristics collected as part of the initial registry record are more or less likely to lead to a particular clinical outcome. It is important to recognize that the outcome in question must be as carefully defined and standardized as any of the initial data elements. Outcomes assessment may be determined actively by their specific addition to the registry by its participants or passively by linkage to administrative data. An example of the latter is tracking vital status by using a common identifier such as a social security number to link the initial registry record with the Social Security Death Index. Registries are becoming increasingly sophisticated in developing relationships and linkages with administrative data sources (see Linkage with Administrative Datasets to Monitor Device-Related Outcomes, below).

Why Registries Exist

Like virtually all physicians, vascular surgeons strive to provide the best outcomes for their patients. To do this, information is needed about which procedures or treatments work best for which patients at which times. It is an axiom of quality improvement that “you can’t improve what you don’t measure.” Registries measure activities that occur during routine clinical care, as opposed to information derived from randomized clinical trials, which are in reality experiments designed to answer specific questions or test explicitly stated hypotheses.

Further, in recent years, many payers have restricted the use of new or evolving technology unless the results of these procedures are followed carefully in registries. ^, For example, the FDA has also used medical registries for the purpose of post-approval device surveillance of TEVAR and TAVR devices. ^,

There is a well-recognized hierarchy of medical evidence based primarily on whether the conclusion delivered by a particular type of study is felt to be biased ( Fig. 201.1 ). ^, Systematic meta-analyses of related randomized controlled trials (RCTs) are regarded as the highest quality evidence followed by individual RCTs themselves. However, RCTs have numerous drawbacks. Randomization is unethical in situations where clinical equipoise does not exist, no matter how valuable or useful the answer might be. The highly restrictive inclusion/exclusion criteria of RCTs may raise questions about whether the results apply to the broader population of patients seen in routine clinical practice. RCTs are rarely powered to detect safety signals.

The disadvantages of RCTs create the niche and rationale for medical registries. Randomization is not a prerequisite for enrollment, so there are rarely ethical objections. Maintaining confidentiality of protected health information is usually the major concern with registries. Systematic and broad registration of patients or events according to the predefined inclusion principle should make registry-based results more generalizable to everyday patients and minimize selection bias. The large case volume accumulated by many registries allows detection of relatively low-frequency events that are unlikely to be recognized in an RCT or in a single center’s experience. Broad participation also facilitates benchmarking, which is the comparison of an individual or group’s performance to the average performance (or best performance) of the rest of the group.

Determinants of the Value of a Registry

The quality of data collected and maintained by a registry is the greatest determinant of that registry’s value. Two major attributes of registry data quality are recognized: completeness and accuracy .

Completeness is a measure of the extent to which all data eligible for registry entry have actually been included. Since one of the major advantages of registries is their increased representativeness or broad inclusion of relevant cases, any concern that a registry’s data is not inclusive or representative will lower its value and raise questions of selection bias. Lack of completeness is a threat to a registry’s external validity or generalizability. These issues may be more likely to affect procedure-based as opposed to disease-based registries, as well as those that depend on self-reported data as opposed to data entry by a third party or automatic process. Lack of completeness would severely compromise the value of a registry established for the purpose of public reporting. Another issue that may affect a registry’s external validity is cost of participation. If cost becomes a barrier to participation, data that are entered into the registry may be less representative, again raising the possibility of selection bias.

Accuracy refers to the degree to which registered data conform to reality, or a predetermined gold standard. Flaws in data accuracy will primarily affect a registry’s internal validity or confidence that conclusions based on registered information are free from bias or error.

An appropriately focused auditing strategy or policy should be a feature of all high-quality registries. The nature and scale of the audit should be consistent with the registry’s purpose and will ideally test for both completeness and accuracy. Concerns about validity pertain to all types of registries, whether they exist primarily for research or other clinical purposes. For instance, a registry’s signal about poor clinical outcomes may be dismissed if there is suspicion that the underlying data are suspect.

There is also a tension between registry complexity and both internal and external validity. Typically, the more complex registries will collect and contain more clinical information, theoretically increasing their value since they can analyze the relationship of more clinical characteristics with outcomes than registries that contain simpler data sets. A hierarchy of data for arthroplasty registries has been described that can be extrapolated to other clinical, procedure-based registries.

• Level 1 – Simple demographics, date of procedure, dx, type of procedure, medical device info, surgeon and hospital identifier

• Level II – Patient clinical characteristics, surgical technique, intraoperative events/processes

• Level III – Patient Reported Outcomes (PROs), clinical or functional outcomes, economic data

• Level IV – Radiographic assessments (possibly by a central imaging core).

Obviously a registry collecting Level III or IV information will have much richer data and be able to address a wider variety of clinical questions than a registry that only contains Level I information. At first glance, it may appear to deliver greater value. However, a complex registry that strives to collect detailed information but sacrifices representativeness or has quality issues because of complicated or confusing data entry processes may ultimately not be as useful as a simpler registry. In general, a registry should be no more complex than necessary to meet its stated purpose.

At inception and throughout its life cycle, a registry must address and balance multiple issues that create tension between its value and validity. Although registries are not typically as costly as RCTs, they are frequently not subsidized or externally funded. Thus even a relatively low-cost registry may be very expensive for some centers or groups.

Where Registries Fit as Medical Evidence

As noted above, there is a hierarchy of medical evidence ranging from systematic reviews of randomized controlled trials regarded as the most authoritative to isolated case reports regarded as least authoritative. RCTs are so highly regarded because they are prospectively designed as focused clinical experiments and the randomization allocation minimizes the possibility that the trial outcomes will be affected by confounding influences. However, RCTs cannot address every issue for which we need data to help guide clinical decision making.

While data from registries do not represent the highest level of evidence, they do provide valuable information that cannot be obtained from any other source. There are essentially three types of studies that leverage registry data: cohort analysis, case–control and cross-sectional studies ( Fig. 201.1 ; Table 201.1 ).

Differences between registry data and those collected in the course of a randomized controlled trial are important to recognize. First, subject enrollment into a trial is a strictly controlled event occurring only after stringent assessment of carefully defined, prospectively applied inclusion and exclusion criteria. Enrollment criteria for registry entry are usually not so strict giving registries a “real-world” flavor, but the inclusion of ineligible subjects may corrupt the relationship between a particular characteristic and outcome. Also, treatment allocation in a registry is almost never random, so the factors governing the choice of one type of treatment over another are frequently unknown.

Since the participants in randomized trials are so valuable, follow-up for adverse events and clinical outcomes is typically active and aggressive. This type of information may only find its way into registries passively and can produce problems with missing data in terms of outcomes of interest. Further, outcome ascertainment in a formal trial is frequently blinded to the treatment allocation but not necessarily so in a registry. When an outcome of interest such as mortality is definitive and not open to interpretation, linkage to administrative databases may alleviate this problem. For less definitive outcomes, the possibility of ascertainment bias remains a significant issue in registries.

Registry data are often affected by various confounders (e.g., selection bias, ascertainment bias, inconsistent treatment protocols and others that may not be recognized), so data analysis can be complicated. Numerous statistical methods to account for known confounding have been developed, including adjustment techniques, regression analysis, instrumental variable analysis, and various types of propensity-matching.

For the clinician evaluating a registry-based article, a standardized approach can be useful to help determine the quality of the conclusions as they may impact practice, as outlined in Table 201.2 .

A randomized registry trial is an effort to leverage the existing infrastructure of high-quality registries and still gain the powerful advantages of randomization. A randomized registry trial is a clinical experiment targeting a specific question conducted within the context of a registry, thus avoiding the time-consuming and expensive task of creating the infrastructure necessary to answer the question. During a randomized registry trial, a “randomization module” unique to the question at hand is applied at the time of registry entry. Randomized registry trials still require informed consent and equipoise regarding the risks or benefits to the patient of allocation to one group or the other.

Probably the best example of a randomized registry trial is the TASTE (Thrombus Aspiration in ST Elevation MI) study performed in Scandinavia. ^, This study randomized patients experiencing STEMI to a specific attempt at thrombus aspiration during percutaneous coronary intervention (PCI) versus standard PCI alone. Enrollment was relatively straightforward, since STEMI patients in Scandinavia are typically entered into the Swedish Web System for Enhancement and Development of Evidence-Based Care in Heart Disease Evaluated According to Recommended Therapies (SWEDEHEART), a well-developed registry. Following informed consent, patients were randomly allocated to thrombus aspiration or standard PCI. The outcome was “hard” (mortality at 30 days) and ascertained by querying SWEDEHEART. Most patients agreed to participate, and the incremental cost of enrolling a patient in the TASTE study, as opposed to just the underlying registry, was $50. Thrombus aspiration was found not to provide a survival advantage, compared with standard PCI for STEMI.

Existing Vascular Surgery Registries