Improving Health-Care Quality Through Measurement


“Wherever we see systematic measurement of results in health-care–no matter what the country–we see those results improve.” Michael E. Porter, Harvard Business Review (October 2013)

Extensive evidence exists demonstrating that health-care is plagued with overutilization of inappropriate services, underutilization of appropriate services, and avoidable medical errors. Over the last few decades, the emphasis on “closing the quality gap” in health-care has escalated and intensified. Furthermore, measures of quality have become tools to curb rising and unsustainable health-care costs. To understand the evolving landscape of health-care quality and national payment reform initiatives, it is important to have a fundamental understanding of what quality in health-care is and how it is most appropriately measured and used to, ultimately, improve the care of our patients.

In this chapter, we first present an overview of quality in health-care. Using this background, we then describe how health-care quality can be measured. Last, we describe four ways quality measures are commonly used in health-care: quality improvement, accreditation and verification, public reporting, and value-based care.

What Is Quality in Health-Care?

Quality can be considered the gap between the care delivered and the care that should be delivered. As highlighted in the seminal publication Crossing the Quality Chasm , abundant evidence suggests that health-care falls short in six dimensions: (1) safety, (2) effectiveness, (3) patient-centeredness, (4) timeliness, (5) efficiency, and (6) equity. These six dimensions comprise the overall concept of health-care quality ( Table 45.1 ). Improving these six dimensions would therefore improve health-care quality by ensuring patients receive care that meets their needs and is based on the best scientific knowledge. Recognizing that health-care quality is multidimensional is the first step to being able to understand and, most importantly, to improve it.

Table 45.1
Six dimensions of health-care quality according to the National Academy of Medicine.
Aim Definition Example
Safety Avoiding injuries to patients from the care that is intended to help them. Chipped tooth during intubation
Effectiveness Providing services based on scientific knowledge to all who could benefit and refraining from providing services to those not likely to benefit. Early removal of Foley catheters
Patient-centeredness Providing care that is respective of and responsive to individual patient preferences, needs, and values and ensuring that patient values guide all clinical decisions. Preoperative goals of care discussion
Timely Reducing waits and sometimes harmful delays for both those who receive and those who give care. Available operating room for emergencies
Efficient Avoiding waste, including waste of equipment, supplies, ideas, and energy. Unnecessary preoperative echocardiogram
Equity Providing care that does not vary in quality because of personal characteristics such as sex, gender, ethnicity, geographic location, and socioeconomic status. Care provided to patients in rural settings

In the United States, these six dimensions of quality are most prominent in the National Strategy for Quality Improvement in Health-care, or simply the National Quality Strategy (NQS). Led by the Agency for Health-care Research and Quality (AHRQ), the NQS was developed through a transparent and collaborative process with the goal to align health-care quality improvement efforts across national, federal, state, and private-sector stakeholders. Many agencies of the US Department of Health and Human Services (HHS) have adopted the NQS, including the Centers for Medicare and Medicaid Services (CMS), as a framework to improve health-care quality.

In addition to the inherent multidimensional nature of quality, there are two additional concepts that should be understood. First, improving quality along one of these dimensions does not exclude the ability to improve along any of the other dimensions of quality. That is, tradeoffs are not inherent. A corollary to this is that improvement along one dimension of quality does not necessarily result in improvement along another dimension. In many instances, although the dimensions of quality are all interconnected to some degree, effort must be spent on simultaneously improving all dimensions of quality.

Second, it is important to recognize that although all dimensions of quality are important, certain dimensions might be considered more important depending on which stakeholder is evaluating the health-care system. For instance, patients have different perspectives of what is “high quality” compared with what payers, policymakers, or providers might consider to be of high quality. Patients might value longer, unhurried face-to-face time during clinic visits with their health-care provider, whereas health-care administrators might wish to minimize this time to maximize the number of patients that can be seen in a day.

Why Improve Quality?

It is important to understand the motivators for measuring and improving quality: What drives quality? These motivations depend on the perspectives of the stakeholders evaluating the care provided and the types of rewards involved. Rewards can be categorized into those that are intrinsic or extrinsic. Intrinsic rewards represent motivations driven by philosophy, such as teaching, learning, or healing. Arguably, health-care professionals entered the profession for intrinsic rewards: to heal patients and to cure them of their ailments. Extrinsic rewards, in contrast, are material goals, such as financial gain. The challenge in health-care has been to figure out when each of these motivations is most effective as a lever to improve quality. Indeed, there exists concern that measuring and reporting of health-care quality might result in punitive action by administrative managers and policymakers, resulting in magnified conflicts between intrinsic and extrinsic rewards. Quality problems typically occur not because of a failure of goodwill, knowledge, effort, or resources devoted to health-care, but because of fundamental shortcomings in the ways care is organized, delivered, or both. Ideally, intrinsic (e.g., curing patients) and extrinsic (e.g., financial gain) motivators can work in tandem to improve health-care quality.

Framework for Measuring Quality

Quality measures (performance measures, metrics, indicators, etc.) are tools used to quantify the care provided to patients. When used in health-care, quality measures assist in determining how well care is provided for certain aspects of care, for certain conditions, or for various populations or communities. In 1966, Donabedian presented the most commonly used framework by which to measure quality in health-care today: structure, process, and outcome ( Box 45.1 ).

Box 45.1
Donabedian Framework for Measuring Quality.

  • “Structure” refers to the characteristics of the setting in which care is provided. The validity of structure as a measure of quality is predicated on the idea that appropriate resources are needed to deliver care of high quality. Without certain fundamental structural resources, high-quality care could not be provided. Examples include the number of ICU beds, nurse staffing levels, level of trauma services, or availability of an adverse event reporting mechanism. Structural measures of quality are typically the easiest to measure. They have been the motivation for many accreditation programs, such as those of the Joint Commission, to denote a facility provides high-quality care or, more precisely, has the capacity to provide high-quality care.

  • “Process” reflects how the care was delivered. A process measure seeks to determine whether high-quality clinical care was properly practiced or delivered. The administration of appropriate antimicrobial surgical prophylaxis within 60 minutes of surgical incision is a quintessential process measure in perioperative medicine. Performing this process leads to higher quality care because patients are less likely to develop a postoperative surgical site infection; that is, the higher the compliance with this process measure, the better the outcome. The validity of a process measure, therefore, depends entirely on the existence of a causal relationship between the process and the outcome. According to Donabedian, process also “includes the patient’s activities in seeking care and carrying it out.” Thus, whether a patient discontinued their anticoagulation medication within the appropriate timeframe before their operation is another example of a process measure.

  • “Outcome” refers to the effects of the care delivered on the health status of the patient or population receiving that care. Classically, these are stated in terms of recovery, restoration or improvement of function, and survival. Many stakeholders want to measure outcomes because they are concrete and represent the end-product of the care delivered. However, outcomes are challenging to measure compared with other types of quality measures. An appropriate outcome measure must consider several factors: appropriate time horizon, consistency of data collection methods for tracking outcomes, attribution of members of the surgical team, the need for a large sample size to detect a statically significant outcome, cost for the infrastructure required to measure and collect data, consistent analysis methods, and risk adjustment.

The types of measures used depends on the available scientific knowledge connecting them to the quality gap and on the resources needed to measure them. Measuring the totality of care, which often encompasses all types of quality measures across all six dimensions ( Table 45.1 ), is optimal. Structural measures are only as good and useful as the strength of their link to desired processes and outcomes ( Fig. 45.1 ). Similarly, process and outcome measures must be related to each other (i.e., causal) in measurable and reproducible ways to be truly valid and reliable measures of quality. Because of these concerns, outcomes, which represent the bottom line, are arguably more attractive to measure. However, as discussed, without a high level of rigor, there is the likelihood of misclassifying care, which can have unintended consequences, such as rewarding poor care or penalizing optimal care.

Fig. 45.1, Relationship between structure, process, and outcome measures.

Fig. 45.2, Anatomy of a proportion quality measure.

In addition to “structure,” “process,” and “outcome,” any combination of these measures can be combined to form a composite measure. A composite measure combines the results of two or more component quality measures, each of which individually reflect quality of care, combined into a single quality measure with a single score to provide a more summative picture. Although attractive because they can measure care across a continuum and can be easier to understand by patients, composite measures have their own share of difficulties. For instance, combining a measure in which performance is generally poor with a measure in which performance is generally good results in a composite measure that may appear average.

As discussed earlier, the perspective of which outcome is important to measure often varies by stakeholder. For example, a patient who did not experience any complications may be dissatisfied with the surgery she received because she was not able to return to her usual activities of daily living as she had hoped, or because her postoperative pain did not resolve as she might have expected and compromised her quality of life. In these situations, the patient perspective can be a more meaningful measure of success. Patient-reported outcome performance measures (PRO-PMs) are quality measures based upon aggregated patient-reported outcomes (PROs) data. PROs measure a patient’s health status, quality of life, health behavior, or experience of care using information that comes directly from the patient, family, or caregiver without interpretation by a health-care provider or anyone else. PRO-PMs share the same qualities of traditional outcome measures (i.e., need for risk adjustment) except they are evaluated from the patient perspective.

Development and Endorsement of Quality Measures

Numerous stakeholders are involved in improving health-care quality, including federal (e.g., AHRQ, CMS) and state government, payers (e.g., Blue Cross Blue Shield), purchasers (e.g., Pacific Business Group on Health), professional societies (e.g., American Medical Association), provider organizations (e.g., American Hospital Association), industries (e.g., Pfizer), nonprofit organizations (e.g., National Committee on Quality Assurance), community health agencies (e.g., Wisconsin Collaborative for Health-care Quality), patients and patient advocates (e.g., National Partnership for Women and Families), and foundations (e.g., Robert Wood Johnson Foundation). Accordingly, just as many have developed and continue to develop quality measures, both broadly across different health-care settings and specific to their agendas, specialties, and constituencies’ needs. Although there are many general principles that should be followed to develop measures that are meaningful, valid, and reliable, Donabedian astutely condensed them into four basic questions :

  • 1.

    Who is being assessed?

  • 2.

    What are the activities being assessed?

  • 3.

    How are these activities supposed to be conducted?

  • 4.

    What are they meant to accomplish?

Overall, it is most important to answer the last question and to define the purpose of the quality measure: what is the gap that, when addressed, will improve quality? In general, this involves compiling the evidence base to identify the current practice and what is the “best practice” as supported by current scientific knowledge. The difference between what is done and what should be done is the quality gap. Once these questions are answered, then the technical details of the measure, such as inclusion/exclusion criteria, risk adjustment variables, validity, reliability, can be specified, tested, and implemented ( Box 45.2 ).

Box 45.2
Calculating quality measures.

Quality measures can be expressed in three common ways, depending on how the intended data are to be calculated: (1) proportion, (2) continuous, (3) and ratio.

  • 1.

    A proportion measure is a score derived by dividing the number of cases that meet a criterion for quality (the numerator) by the number of eligible cases within a given time frame (the denominator) where the numerator cases are a subset of the denominator cases (e.g., percentage of eligible women with a mammogram performed in the last year). Fig. 45.2 depicts a proportion measure.

  • 2.

    A continuous variable measure is a measure score in which each individual value for the measure can fall anywhere along a continuous scale and can be aggregated using a variety of methods, such as the calculation of a mean or median (e.g., mean number of minutes between presentation of chest pain to the time of administration of thrombolytics).

  • 3.

    A ratio measure is a score that is derived by dividing a count of one type of data by a count of another type of data (e.g., the number of patients with central lines who develop infection divided by the number of central line days).

Fig. 45.3, Hospitals participating in the ACS NSQIP in 2018.

Endorsement by the National Quality Forum (NQF) is considered by many to be an essential stamp of approval for quality measures. Founded on recommendation of the 1998 President’s Advisory Commission on Consumer Protection and Quality in the Health-care Industry, the NQF is an independent organization that brings together public- and private-sector stakeholders from across the health-care system to determine high-value measures for improving the nation’s health and health-care. The NQF measure endorsement process, also referred to as the Consensus Development Process, provides the nation with a centralized portfolio of quality measures that meet rigorous evaluation criteria and could be implemented in both accountability and quality improvement programs. Examples of NQF-endorsed perioperative measures are shown in Table 45.2 . In addition to endorsement, the NQF aims to accelerate development of needed measures, to identify high-priority measures, to harmonize measures, to drive more effective implementation of priority measures, and to understand better what does and does not work in quality measurement.

Table 45.2
Selected National Quality Forum (NQF)-endorsed perioperative quality measures.
Measure title (NQF measure number) Description Measure type Developer/steward
Preoperative beta blockade (0127) Percentage of patients aged 18 years and older undergoing isolated CABG who received beta blockers within 24 hours preceding surgery. Process The Society of Thoracic Surgeons
Perioperative antiplatelet therapy for patients undergoing carotid endarterectomy (0465) Percentage of patients undergoing CEA who are taking an antiplatelet agent (aspirin or clopidogrel or equivalent such as Aggrenox/Tiglacor, etc.) within 48 hours prior to surgery and are prescribed this medication at hospital discharge following surgery. Process Society for Vascular Surgery
Prevention of CVC-related bloodstream infections (2726) Percentage of patients, regardless of age, who undergo CVC insertion for whom CVC was inserted with all elements of maximal sterile barrier technique, hand hygiene, skin preparation and, if ultrasound is used, sterile ultrasound techniques followed. Process American Society of Anesthesiologists
Postoperative respiratory failure rate (PSI 11) (0533) Postoperative respiratory failure (secondary diagnosis), mechanical ventilation, or reintubation cases per 1000 elective surgical discharges for patients ages 18 years and older. Outcome Agency for Health-care Research and Quality
Perioperative temperature management (2681) Percentage of patients, regardless of age, who undergo surgical or therapeutic procedures under general or neuraxial anesthesia of 60 minutes duration or longer for whom at least one body temperature greater than or equal to 35.5°C (or 95.9°F) was achieved within the 30 minutes immediately before or the 15 minutes immediately after anesthesia end time. Outcome American Society of Anesthesiologists
Risk-adjusted, case mix-adjusted elderly surgery outcomes measure (0697) Hospital-based, risk-adjusted, case-mix adjusted elderly surgery aggregate clinical outcomes measure of adults 65 years of age and older. Outcome American College of Surgeons
CABG, Coronary artery bypass graft; CEA, carotid endarterectomy; CVC, central venous catheter.

Although NQF endorsement is important and a recognizable achievement, quality measure endorsement is not always necessary. The process of NQF endorsement takes considerable time, which must be weighed against the need for improvement. NQF endorsement is often sought for “high stakes” measures used for purposes of national, large-scale accountability programs (i.e., public reporting or pay-for-performance), such as the CMS Quality Payment Program (QPP) or CMS Hospital Value-based Purchasing Program. The need for quality measure endorsement thus depends on how the quality measure will be used.

NQF Measure Applications Partnership

The NQF also convenes the Measure Applications Partnership (MAP), which provides pre-rulemaking guidance to the HHS for the inclusion of certain quality measures in public reporting and performance-based payment programs. The MAP was mandated in the Affordable Care Act (ACA) to make way for significant enhancements to the traditional federal rulemaking process by providing a forum for public and private partnerships to provide feedback on quality measures before they are proposed for use in federal regulations. HHS selected the NQF to provide this pre-rulemaking input guided by the three-part aim of the NQS: better care, better health, and lower cost. The MAP is charged with providing a coordinated look across federal programs. It identifies measure gaps and recommends measures for use in approximately 20 federal quality programs, including the CMS QPP.

Uses of Quality Measures

Quality measures, once developed, specified, and potentially NQF-endorsed, are commonly used in four ways: (1) improving care services and delivery, (2) accreditation and verification, (3) public reporting, and (4) incentive payments or value-based care. All uses share the same goal of improving the quality of health-care delivered but do so with different means depending on the perspective of the stakeholder and the motivation for reward. For instance, clinicians, motivated by intrinsic rewards, use quality measures to assess clinical practices better and to track their implementation. Payers and insurers, motivated by extrinsic rewards, use quality measures to reward success financially and penalize failure. Public reporting of quality measures increases transparency, allowing patients to make more informed choices about providers and facilities.

Quality Improvement

Quality measures have been traditionally used for internal monitoring and quality and improvement (QI/PI) activities. Using quality measures in this way helps providers and institutions track ways to improve care and is related to professional and personal commitments to care as well as institutional expectations. Although quality measures have long been used for internal monitoring and reporting, providers have not been able to compare themselves easily with others because the data elements and collection instruments have not been synchronized. Collecting and reporting the same data using the same specifications helps organizations and providers understand how they perform compared with other organizations. It also enables them to identify opportunities for focused quality improvement efforts ( Box 45.3 ). With the rise of national quality programs, registries, and “big data,” groups of hospitals can network with each other and learn which interventions worked or failed at everyone’s respective institutions.

Box 45.3
Diminishing variation in perioperative outcomes: the American College of Surgeons’ National Surgical Quality Improvement Program (ACS NSQIP).

Gaps in care quality can be considered as unwarranted variation in the care delivered. The presence of variation in medical practice has been described since 1938 when Dr. James Alison Glover published his classic study on the incidence of tonsillectomy in school children in England and Wales, uncovering geographic variation that defied any explanation other than variation in medical opinion on the indications for surgery. Today, there are countless studies in the literature that have revealed similar variation across myriad medical conditions and procedures. Measuring unwarranted variation in care is a fundamental step to improving it. The American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) is one well-established tool that has been shown to improve surgical quality by accurately measuring variation in postoperative outcomes and feeding these data back to hospitals to drive improvement.

In the 1980s, it was insinuated that the operative mortality at Veterans Affairs (VA) hospitals varied in comparison with the national average. In response, the National VA Surgical Risk Study (NVASRS) was formed in 1991, which established a data collection program that systematically abstracted surgical outcomes and comorbidity data from the clinical record and found that risk-adjusted mortality statistics in the VA were not different than those outside the VA. The success of the NVASRS gave rise to the VA National Surgical Quality Improvement Program (NSQIP), which quantified variation in perioperative outcomes among VA hospitals and fed the results back to frontline providers. The program demonstrated a 43% reduction in 30-day postoperative morbidity, 47% reduction in mortality, and a decrease in median length of stay from 9 to 4 days between 1991 and 2006. Encouraged, the VA NSQIP partnered with the ACS to develop the Patient Safety in Surgery (PSS) Study with financial support from the AHRQ to scale and spread these successes into the private sector. Results demonstrated that implementing the NSQIP framework in the private sector was also associated with significant reductions in surgical complications. In October 2004, this seminal study established the ACS NSQIP as it is recognized today, and improvements continue to be seen. For instance, Cohen et al. demonstrated that for hospitals currently in the ACS NSQIP for at least 3 years, 69%, 79%, and 71% showed improvement in mortality, morbidity, and surgical site infections, respectively. They estimated, for every 10,000 surgical procedures, the improving hospital would have avoided seven deaths, converted 150 patients from having one or more complications to having none, and converted 66 patients from having one or more SSI to having none. Placed into the context of the more than 30 million operations performed in the United States annually, these improvements are substantial.

Today, more than 700 hospitals in the United States and internationally participate ( Fig. 45.3 ), accruing more than 1,000,000 operations annually. Built upon a foundation of high-quality clinical data with the help of data abstractors collecting data following standardized definitions directly from the medical record, the ACS NSQIP recognizes that data are necessary but not sufficient to achieve quality improvement. Decreasing practice variability through local standardization has been identified as one approach to improve quality. Often, hospitals are presented with clinical guidelines that might not be wholly implementable at the local level. However, hospitals typically will start with the guidelines as a “straw man,” and then, with certain tweaks or modifications, are better able to implement them locally. As a corollary to this principle of standardization, it is important that hospitals continually perform surveillance of their data to elucidate whether further modifications to their standardized processes are needed—a cycle of continuous quality improvement. Case studies have been developed to help novice hospitals gain insight into how they might improve their care. Successful hospitals of different sizes and types (for example, rural or urban, teaching or nonteaching) were identified to share information, such as which problems were targeted and which strategy was adopted, how a team was assembled and who it included, which barriers were faced and how they were overcome, and which “pearls” could be shared with others working in the same areas.

Quality Improvement Methodologies

There are numerous tools and strategies, mostly process improvement tools adopted from industry, to improve quality in health-care. Each approach relies on collecting data using quality measures to identify areas in need of improvement and to test potential solutions. The most common process improvement tools are (1) Institute for Health-care Improvement’s Model for Improvement, (2) Six Sigma, (3) Lean, and (4) Lean Six Sigma.

The Model for Improvement relies on three key questions and the Plan-Do-Study-Act (PDSA) cycle to structure continual improvement of a given process. The three key questions are:

  • 1.

    What are we trying to accomplish?

  • 2.

    How will we know that change is an improvement?

  • 3.

    What change can we make that will result in an improvement?

After addressing these three questions, the PDSA cycle is used as the framework for an efficient trial and learning methodology—an iterative, step-wise process to achieve improvement. In the “Plan” stage, objectives are set, a plan is defined, and a hypothesized outcome is articulated. In “Do,” the team implements the plan, documents any problems or unexpected findings, and begin their data analysis. In “Study,” data analysis is completed, and the results are compared to the predicted outcomes. In “Act,” the team identifies further changes that need to be made and decides whether another PDSA cycle will be necessary.

In the 1980s, Motorola declared that it would achieve a defect rate of no more than 3.4 parts per million within 5 years, which represented defect rates outside of six standard deviations (sigma) from the mean. Six Sigma thus refers to a process of improving quality by eliminating variation. Six Sigma depends upon the Define-Measure-Analyze-Improve-Control (DMAIC) framework, which has been shown effective in reducing length of stay after hip replacement surgery, decreasing discharge time, increasing efficiency in radiology suites, increasing hand hygiene compliance, and improving operating room recovery room processes.

Lean is an approach that considers any resources that are not allocated to the goal of creating value for the customer wasteful, and thus should be eliminated. Developed in the 1950s by Toyota, waste (muda) in health-care can take many forms, such as redrawing a blood sample that was previously drawn but lost by the laboratory. Lean tools are like those of DMAIC, including process mapping, work and process observation, standardizing processes, use of checklists, and error proofing. Value-stream mapping is a technique that is more specific to Lean management. In these maps, processes are mapped along with information and material flows. By mapping current states, value-added and nonvalue-added steps can be identified and properly addressed. In comparison to Six Sigma, where waste results from variation, Lean aims to identify wasteful steps in a process.

Both Six Sigma and Lean approaches have individual benefits that in practice complement each other. Quality improvement projects can either have a Lean-focused approach, which apply best practices and focus on implementing standard solutions to increase speed and reduce lead and process time, or a Six Sigma-focused approach, which is often used in more complex problems that involve data-based analytic methods and stress control mechanisms. The DMAIC framework can be used in either case.

Accreditation and Verification

Accreditation, or verification, refers to an approval process conducted by independent (nongovernmental) bodies to certify that an institution or individual has met certain standards. Although there are multiple types of accrediting bodies, such as educational (e.g., Accreditation Council for Graduate Medical Education) and professional societies (e.g., ACS Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program), the most well-known accrediting bodies are the Joint Commission for hospitals and the National Committee for Quality Assurance (NCQA) for health plans.

You're Reading a Preview

Become a Clinical Tree membership for Full access and enjoy Unlimited articles

Become membership

If you are a member. Log in here