Generating Knowledge from Information, Data, and Analyses

What It Is About

Cardiac surgical procedures, particularly coronary artery bypass grafting (CABG), are the most quantitatively studied therapies in the history of medicine. These studies reveal a complex, multifactorial, and multidimensional interplay among patient characteristics, variability of the heart disease, effect of the disease on the patient, conduct of the procedure, and response of the patient to treatment. Because cardiac surgeons were “data collectors” from the beginning of the subspecialty, it is understandable that efforts to improve the quality of medical care while containing costs found cardiac surgical results (outcomes) an easy target. The dawn of medical report cards made it evident that multiple factors influencing outcome must be taken into account to make fair comparisons of outcomes (see “ Risk Stratification ” and “ Risk Adjustment ” in Section VI ). This scrutiny of results, often by the media, reveals that variability in performing technical details of operations, coupled with environmental factors often not under direct control of cardiac surgeons, contribute to differences in results.

Propensity toward data collection in cardiac surgery was reinforced in the 1970s and early 1980s by challenges from cardiologists to demonstrate not simply symptomatic improvement from operative procedures, but improved survival and long-term quality of life (appropriateness). This resulted in one of the first large-scale, government-funded registries and an in-depth research database ( Box 6-1 ) of patients with ischemic heart disease, as well as a rather small, narrowly focused randomized trial (Coronary Artery Surgery Study ). It stimulated subsequent establishment by the Society of Thoracic Surgeons (STS) of what is now the largest non-governmental registry of cardiac surgical data.

Thus, it is important for all in the subspecialty of cardiac surgery, not just those engaged in bench, translational, or clinical research, to (1) understand how information generated from observations made during patient care is transformed into data suitable for analysis, (2) appreciate at a high level what constitutes appropriate analyses of those data, (3) effectively evaluate inferences drawn from those analyses, and (4) apply new knowledge to better care for individual patients.

It is our desire that the reader realize these goals and not conclude prematurely that this chapter is simply a treatise on biostatistics, outcomes research, epidemiology, biomathematics, or bioinformatics. ¹

1 In its narrowest definition, bioinformatics is a collection of methods devised to process genomic data, representing the reality that advances in genomics require sophisticated and often new computer algorithms. Some would say bioinformatics is the next frontier for statistics, others for machine learning. This is the narrow view. The National Institutes of Health (NIH) in the United States has provided a broader view. The NIH Biomedical Information Science and Technology Initiative Consortium agreed on the following definition of bioinformatics , “recognizing that no definition could completely eliminate overlap with other activities or preclude variations in interpretation by different individuals and organizations”: Bioinformatics is research, development, or application of computational tools and approaches for expanding the use of biological, medical, behavioral or health data, including those to acquire, store, organize, archive, analyze, or visualize such data ( http://grants1.nih.gov/grants/bistic/CompuBioDef.pdf ). They go on to define computational biology in a broader context than genomics as “the development and application of data-analytical and theoretical methods, mathematical modeling and computational simulation techniques to the study of biological, behavioral, and social systems.” Thus, to lead from information to new knowledge, they envision bringing together quantitative needs in structural biology, biochemistry, molecular biology, and genomics at the microscopic level, and medical, health services, health economics, and even social systems disciplines at the macroscopic level, with analytic tools from computer science, mathematics, statistics, physics, and other quantitative disciplines. This vision transcends current restrictiveness of traditional biostatistics in analysis of clinical information. This is why we emphasize in this chapter that the material is not simply for surgeons, their clinical research team, and consulting and collaborating biostatisticians, but also for a wider audience of professionals in a variety of quantitative disciplines.

Who Should Read It

This chapter should be read in whole or in part by (1) all cardiac surgeons, to improve their comprehension of the medical literature and hone their skills in its critical appraisal; (2) young surgeons interested in becoming clinical investigators, who need instruction on how to pursue successful research (see Technique for Successful Clinical Research later in this section); (3) mature surgeon-investigators and other similar medical professionals and their collaborating statisticians, mathematicians, and computer scientists who will benefit from some of the philosophical ideas included in this section, and particularly from the discussion of emerging analytic methods for generating new knowledge; and (4) data managers of larger clinical research groups who need to fully appreciate their pivotal role in successful research (Appendix 6A), particularly as described in Sections I, II, and III of this chapter.

The potential obstacle for all will be language. For the surgeon, the language of statistics, mathematics, and computer science may pose a daunting obstacle of symbols, numbers, and algorithms. For collaborating statisticians, mathematicians, and computer scientists, the Greek and Latin language of medicine is equally daunting. This chapter attempts to surmount the language barrier by translating ideas, philosophy, and unfamiliar concepts into words while introducing only sufficient statistics, mathematics, and algorithms to be useful for the collaborating scientist.

Because this chapter is intended for a mixed audience, it focuses on the most common points of intersection between cardiac surgery and quantitative science, with the goal of establishing sufficient common ground for effective and efficient collaboration. As such, it is not a substitute for statistical texts or academic courses, nor a substitute for the surgeon-investigator to establish a collaborative relationship with biostatisticians, nor is it intended to equip surgeons with sufficient statistical expertise to conduct highly sophisticated data analyses themselves.

How It Has Evolved

At least three factors have contributed to evolution of Chapter 6 from edition to edition of this book: increasing importance of computers in analyzing clinical data, introduction of new and increasingly appropriate and applicable methods for analyzing those data, and growing importance of nontraditional machine learning methods for mining medical data.

Thus, the title of this chapter in Edition 1 was “Surgical Concepts, Research Methods, and Data Analysis and Use.” Its sections highlighted (1) surgical success and failure, (2) incremental risk factors, (3) research methods, (4) methods of data presentation and analysis and comparison, (5) decision making for individual patients, and (6) improving results of cardiac surgery. All remain important, but progress in each of these areas warrants a fresh approach. In the first edition, formulae were provided for surgeons to implement on a new generation of programmable calculators. These programs provided confidence limits and simple statistical tests that continue to be valuable, particularly in reading and evaluating the literature; however, with the passage of time, increasing sophistication and complexity of programmable calculators have taken implementation of the programs out of the reach of most surgeons. Therefore, we have eliminated those appendices.

In Edition 2, an organizing schema for clinical research was developed, and the name of the chapter was changed to reflect it: “The Generation of Knowledge from Information, Data, and Analyses.” But this schema did not lead to a matching organizational format for the chapter; it paved the way for one. At the time of its writing, there was explosive progress in techniques for analyzing time-related events, so an important portion of the chapter was devoted to this topic. Effective analysis of time-related events was no longer possible with programmable calculators; they demanded powerful computer resources.

In Edition 3, the progression from information to data to analyses to knowledge became the explicit organizing schema for the chapter. Sophisticated methods for longitudinal data analyses and comparative effectiveness assessment were introduced. Whereas the statistician was once the surgeon's primary collaborator in data analysis, that edition introduced computer scientists and mathematicians as partners in collaborative research.

In this edition we introduce other collaborators in the fields of artificial intelligence, ontology, and machine learning. In doing so, we expand themes hinted at in the third edition and hint at new techniques on the horizon. This edition is also strongly influenced by the Institute of Medicine's (IOM) Learning Healthcare System initiative and comparative effectiveness emphases of the IOM and NIH. Undoubtedly, evolution of this chapter will continue in subsequent editions, because new methods are constantly being developed to better answer clinical questions.

How It Is Organized

The organizational basis for this chapter is the Newtonian inductive method of discovery. It begins with information about a microcosm of medicine, proceeds to translation of information into data and analysis of those data, and ends with new knowledge about a small aspect of nature. This organizational basis emphasizes the phrase, “Let the data speak for themselves.” It is that philosophy that dictates, for example, placing “Indications for Operation” after, not before, presentation of surgical results throughout this book.

How to Read This Chapter

Unlike most chapters in this book, whose various parts can be read somewhat randomly and in isolation, Section I of this chapter should be read in its entirety before embarking on other sections. It identifies the mindset of the authors; defends the rationale for emphasizing surgical success and failure; contrasts philosophies, concepts, and ideas that shape both how we think about the results of research and how we do research; lays out a technique for successful clinical research that parallels the surgical technique portions of other chapters; and for collaborating statisticians, mathematicians, and computer scientists engaged in analyzing clinical data, lays the foundation for our recommendations concerning data analysis.

Much of the material in this introductory section is amplified in later portions of the chapter, and we provide cross-references to these to avoid redundancy.

Economics

The economics of health care are driving changes in practice toward what is hoped to be less expensive, more efficient, yet higher quality care. Interesting methods for testing the validity of these claims have become available in the form of cluster randomized trials. In such trials (e.g., a trial introducing a change in physician behavior), patients are not randomized, physicians are (patients form the cluster being cared for by each physician)! This leads to inefficient studies that nevertheless can be effective with proper design and a large enough pool of physicians. It is a study design in which the unit of randomization (physician) is not the unit of analysis (individual patient outcome). Such trials appear to require rethinking of traditional medical ethics.

Innovation

Just when it seems that cardiac surgery has matured, innovation intervenes and drives new knowledge, both from proponents and opponents. Innovation occurs at several levels. It includes new devices; new procedures; existing procedures performed on new groups of patients, such as the elderly and the fetus; simplifying and codifying seemingly disparate anatomy, physiology, or operative techniques; standardizing procedures to make them teachable and reproducible; and introducing new concepts of patient care (the intensive care unit, automated infusion devices, automated care by computer-based protocols). Many of these innovations have had applications beyond the boundary of cardiac surgery.

Yet, innovation is often at odds with cost reduction and is perceived as being at odds with traditional research. In all areas of science, however, injection of innovation is the enthalpy that prevents entropy, stimulating yet more research and development and more innovation. Without it, cardiac surgery would be unable to adapt to changes in managing ischemic heart disease, potential reversal of the atherosclerotic process, percutaneous approaches to valvar and congenital heart disease, and other changes directed toward less invasive therapy.

What is controversial is (1) when and if it is appropriate to subject innovation to formal clinical trial and (2) the ethics of innovation in surgery, for which standardization is difficult.

Reducing the Unknown

New knowledge in cardiac surgery has been driven from its inception by a genuine quest to fill voids of the unknown, whether by clinical research or laboratory research (which we do not emphasize in this chapter, although the principles and recipe for success are the same as for clinical research). This has included research to clarify both normal and abnormal physiology, but also to characterize the abnormal state of the body supported on cardiopulmonary bypass.

Clinical research has historically followed one of two broad designs: randomized clinical trials and nonrandomized studies of cohorts of patients (“clinical practice”), as detailed later in this section under “ Clinical Trials with Randomly Assigned Treatment ” and “ Clinical Studies with Nonrandomly Assigned Treatment ,” respectively. Increasing emphasis, however, is being placed on translational research—that is, bringing basic research findings to the bedside. John Kirklin called this the “excitement at the interface of disciplines.” Part and parcel of the incremental risk factor concept (see “ Incremental Risk Factor Concept ” in Section IV ) is that it is an essential link in a feedback loop that starts with surgical failure, proceeds to identifying risk factors, draws inferences about specific gaps in knowledge that need to be addressed by basic science, generates fundamental knowledge by the basic scientists, and ends by bringing these full circle to the clinical arena, testing and assessing the value of the new knowledge generated for improving medical care.

Surgical Success and Failure

Results of operative intervention in heart disease, particularly surgical failure, drive much of the new knowledge generated by clinical research. In the late 1970s and early 1980s, a useful concept arose about surgical failures. That is, in the absence of natural disaster or sabotage, there are two principal causes of failure of cardiac operations (or other treatments) to provide a desired outcome for an individual patient: (1) lack of scientific progress and (2) human error.

The utility of this concept is that it leads to the programmatic strategies of research on the one hand and development on the other. Thus, lack of scientific progress is gradually reduced by generating new knowledge (research), and human error is reduced in frequency and consequences by implementing available knowledge (development), a process as vital in cardiac surgery as it is in the transportation and manufacturing sectors.

Error

Increased awareness of medical error is driving the generation of new knowledge, just as it is driving increasing regulatory pressure and medicolegal litigation. The UAB group was one of the first to publish information about human error in cardiac surgery and place it into the context of cognitive sciences, human factors, and safety research. This interface of disciplines is essential for facilitating substantial reduction in injury from medical errors.

Human Error

As early as 1912, Richardson recognized the need to eliminate “preventable disaster from surgery.” Human errors as a cause of surgical failure are not difficult to find, particularly if one is careful to include errors of diagnosis, delay in therapy, inappropriate operations, omissions of therapy, and breaches of protocol.

When we initially delved into what was known about human error in the era before Canary Island (1977), Three-Mile Island (1979), Bhopal (1984), Challenger (1986), and Chernobyl (1986), events that contributed enormously to knowledge of the nature of human error, we learned two lessons from the investigation of occupational and mining injuries. First, successful investigation of the role of the human element in injury depends on establishing an environment of non-culpable error . The natural human reaction to investigation of error is to become defensive and provide no information that might prove incriminating. An atmosphere of blame impedes investigating, understanding, and preventing error. How foreign this is from the culture of medicine! We take responsibility for whatever happens to our patients as a philosophical commitment. Yet cardiac operations are performed in a complex and imperfect environment in which every individual performs imperfectly at times. It is too easy when things go wrong to look for someone to blame. Blame by 20/20 hindsight allows many root causes to be overlooked.

Second, we learned that errors of omission exceed errors of commission . This is exactly what we found in ventricular septal defect (VSD) repair ( Table 6-1 ), suggesting that the cardiac surgical environment is not so different from that of a gold mine, and we can learn from that literature.

These two lessons reinforced some surgical practices and stimulated introduction of others that were valuable in the early stages of growth of the UAB cardiac surgery program: using hand signals for passing instruments, minimizing distractions, replying simply to every command, reading aloud the protocol for the operation as it proceeds, standardizing apparently disparate operations or portions thereof, and focusing morbidity conferences candidly on human error and lack of knowledge to prevent the same failure in the future. To amplify, these practices might be enunciated as a “culture of clarity”—in today's terms, a culture of transparency—the end result of which is a reproducible and successful surgical endeavor. In the operating room, each individual on the surgical team is relaxed but alert:

• ▪

  Hand signals serve to inform assistants and the scrub nurse of anticipated needs for a relatively small number of frequently used instruments or maneuvers.

• ▪

  Spoken communication is reserved for those out of the field of sight (i.e., the anesthesiologist and perfusionist). When verbalized, “commands” are acknowledged with a simple reply: “thank you,” “roger,” “yes.” Even those individuals out of the field learn to anticipate these events or commands.

• ▪

  Anticipated deviations from the usual are presented a few minutes to a day or two before the event. (In teaching settings, residents are encouraged to write an operative plan in the preoperative note.) Unanticipated deviations are acknowledged to all concerned as soon as possible.

• ▪

  Successful routines are codified. These include chronology for anticoagulation and its reversal, myocardial management routines (induction of cardioplegia, intervals of cardioplegia reinfusion, controlled myocardial reperfusion before aortic clamp removal), and protocols controlled by the surgeon for commencing and weaning the patient from cardiopulmonary bypass.

• ▪

  Technical intuitive concepts are articulated. For example, some think the VSD in tetralogy of Fallot is a circular hole. Thus, closing such a hole would simply involve running a suture circumferentially to secure a patch. Kirklin and Karp were able to describe the suture line as having four different areas of transition in three dimensions and precisely articulated names for those transitions. Each had a defined anatomic relationship to neighboring structures, so the hole became infinitely more interesting!

• ▪

  Discussion of surgical failure is planned for a time (e.g., Saturday morning) when distractions are minimal. The stated goal is improvement, measurable in terms of reproducibility and surgical success. The philosophy is that events do not simply occur but have antecedent associations, so-called root-cause analysis. An attempt is made to determine if errors can be avoided and if scientific knowledge exists or does not exist to prevent future failure.

A major portion of the remainder of this chapter addresses acquisition and description of this new knowledge.

Lack of Scientific Progress

A practical consequence of categorizing surgical failures into two causes is that they fit the programmatic paradigm of “research and development”: discovery on the one hand and application of knowledge to prevent failures on the other. The quest to reduce injury from medical errors that has just been described is what we might term “development.” The remainder of this chapter focuses mainly on the portion of the paradigm that is research, but also more narrowly on clinical research .

Deduction versus Induction

“Let the data speak for themselves.”

Arguably, Sir Isaac Newton's greatest contribution to science was a novel intellectual tool: a method for investigating the nature of natural phenomena. His contemporaries considered his method not only a powerful scientific investigative tool, but also a new way of philosophizing applicable to all areas of human knowledge. His method had two strictly ordered aspects that for the first time were truly systematically expressed: a first, and extensive, phase of data analysis whereby observations of some small portion of a natural phenomenon are examined and dissected, followed by a second, less emphasized, phase of synthesis whereby possible causes are inferred and a small portion of nature revealed from the observations and analyses. This was the beginning of the inductive method in science: valuing first and foremost the observations made about a phenomenon, then “letting the data speak for themselves” in suggesting possible natural mechanisms.

This represented the antithesis of the deductive method of investigation that had been so successful in the development of mathematics and logic (the basis for ontology-based computer reasoning today). The deductive method begins with what is believed to be the nature of the universe (referred to by Newton as “hypothesis”), from which logical predictions are deduced and tested against observations. If the observations deviate from logic, the data are suspect, not the principles behind the deductions. The data do not speak for themselves.

Newton realized that it was impossible at any time or place to have complete knowledge of the universe. Therefore, a new methodology was necessary to examine just portions of nature, with less emphasis on synthesizing the whole. The idea was heralded as liberating in nearly all fields of science.

As the 18th century unfolded, the new method rapidly divided such diverse fields as religion into those based on deduction (fundamentalism) and those based on induction (liberalism), roughly Calvinism vs. Wesleyan-Arminianism. This philosophical dichotomy continues to shape not just the scientific but the social, economic, and political climate of the 21st century.

Determinism versus Empiricism

Determinism is the philosophy that everything—events, acts, diseases, decisions—is an inevitable consequence of causal antecedents: “Whatever will be will be.” If disease and patients’ response to disease and to disease treatment were clearly deterministic and inferences deductive, there would be no need to analyze clinical data to discover their general patterns. Great strides are being made in linking causal mechanisms to predictable clinical response (see Classification Methods in Section VI ). Yet many areas of cardiovascular medicine remain nondeterministic and incompletely understood. In particular, the relation between a specific patient's response to complex therapy such as a cardiac operation and known mechanisms of disease appears to be predictable only in a probabilistic sense. For these patients, therapy is based on empirical recognition of general patterns of disease progression and observed response to therapy.

Generating new knowledge from clinical experiences consists, then, of inductive inference about the nature of disease and its treatment from analyses of ongoing empirical observations of clinical experience that take into account variability, uncertainty, and relationships among surrogate variables for causal mechanisms. Indeed, human error and its opposite—success—may be thought of as human performance variability.

Collectivism versus Individualism

To better convey how new knowledge is acquired from observing clinical experiences, we look back to the 17th century to encounter the proverbial dichotomy between collectivism and individualism, so-called lumpers and splitters or forests and trees.

In 1603 during one of its worst plague epidemics, the City of London began prospective collection of weekly records of christenings and burials. In modern language, this was an administrative database or registry (see Box 6-1 ). Those “who constantly took in the weekly bills of mortality made little use of them, than to look at the foot, how the burials increased or decreased; and among the casualties, what has happened rare, and extraordinary, in the week current,” complained John Graunt. Unlike those who stopped at counting and relating anecdotal information, Graunt believed the data could be analyzed in a way that would yield useful inferences about the nature and possible control of the plague.

His ultimate success might be attributed in part to his being an investigator at the interface of disciplines. By profession he was a haberdasher, so Graunt translated merchandise inventory dynamics into terms of human population dynamics. He described the rate of goods received (birth rate) and the rate of goods sold (death rate); he then calculated the inventory (those currently alive).

Graunt then made a giant intellectual leap. In modern terms, he assumed that any item on the shelf was interchangeable with any other (collectivism). By assuming, no matter how politically and sociologically incorrect, that people are interchangeable, he achieved an understanding of the general nature of the birth-life-death process in the absence of dealing with specific named individuals (individualism). He attempted to discover, as it were, the general nature of the forest at the expense of the individual trees.

Graunt then identified general factors associated with variability of these rates (risk factors, in modern terminology; see Multivariable Analysis in Section IV ). From the City of London Bills of Mortality, he found that the death rate was higher when ships from foreign ports docked in the more densely populated areas of the city, and in households harboring domestic animals. Based on these observations, he made inferences about the nature of the plague—what it was and what it was not—and formulated recommendations for stopping its spread. They were crude, nonspecific, and empirical: avoid night air brought in from foreign ships (which we now know is not night air but rats), flee to the country, separate people from animal vectors, and quarantine infected individuals. Nevertheless, they were effective in stopping the plague for 200 years until its cause and mechanism of spread were identified.

Lessons based on this therapeutic triumph of clinical investigation conducted more than 300 years ago include the following: (1) empirical identification of patterns of disease can suggest fruitful directions for future research and eliminate some hypothesized causal mechanisms, (2) recommendations based on empirical observations may be effective until causal mechanisms and treatments are discovered, and (3) new knowledge is often generated by overview (synthesis), as well as by study of individual patients.

When generating new knowledge about the nature of heart disease and its treatment, it is important both to examine groups of patients (the forest) and to investigate individual therapeutic failures (the trees). This is similar to Heisenberg's uncertainty principle in chemistry, thermodynamics, and mechanics, in which physical matter and energy can be thought of as discrete particles on the microhierarchical plane (individualism, splitting, trees), and as waves (field theory) on the macrohierarchical plane (collectivism, lumping, forests). Both views give valuable insights into nature, but they cannot be viewed simultaneously. Statistical methods emphasizing optimum discrimination for identifying individual patients at risk tend to apply to the former, whereas those emphasizing probabilities and general inferences tend to apply to the latter.

Continuity versus Discontinuity in Nature

When we turn our focus from named individuals experiencing surgical failure to groups of patients, data analysis becomes mandatory to discover relationships between outcome and items that differ in value from patient to patient (called variables ). A challenge immediately arises: Many of the variables related to outcome are measured either on an ordered clinical scale (ordinal variables), such as New York Heart Association (NYHA) functional class, or on a more or less unlimited scale (continuous variables), such as age. Three hundred years after Graunt, the Framingham Heart Disease Epidemiology Study investigators were faced with this frustrating problem. Many of the variables associated with development of heart disease were continuously distributed ones, such as age, blood pressure, and cholesterol level. To examine the relationship of such variables to development of heart disease, it was then accepted practice to categorize continuous variables coarsely and arbitrarily for cross-tabulation tables. Valuable information was lost this way. Investigators recognized that a 59-year-old's risk of developing heart disease was more closely related to that of a 60-year-old's than to that of the group of patients in the sixth vs. seventh decade of life. They therefore insisted on examining the entire spectrum of continuous variables rather than subclassifying the information.

What they embraced is a key concept in the history of ideas—namely, continuity in nature . The idea has emerged in mathematics, science, philosophy, history, and theology. In our view, the common practice of stratifying age and other more or less continuous variables into a few discrete categories is lamentable because it loses the power of continuity (some statisticians call this “borrowing power”). Focus on small, presumed homogeneous groups of patients also loses the power inherent in a wide spectrum of heterogeneous but related cases. After all, any trend observed over an ever-narrower framework looks more and more like no trend at all! Like the Framingham investigators, we therefore embrace continuity in nature unless it can be demonstrated that doing so is not valid, useful, or beneficial. (Modern methods of machine learning that use classification methods may seem to stumble at this point, but repetition of analyses over thousands of sampled data sets combined with averaging achieves a close approximation to continuity in nature; see Classification Methods in Section VI .)

Single versus Multiple Dimensionality

The second problem the Framingham investigators addressed was the need to consider multiple variables simultaneously. Univariable (one variable at a time) statistics are attractive because they are simple to understand. However, most clinical problems are multifactorial. At the same time, clinical data contain enormous redundancies that somehow need to be taken into account (e.g., height, weight, body surface area, and body mass index are highly correlated and relate to the conceptual variable “body size”).

Cornfield came to the rescue of the Framingham investigators with a new methodology called multivariable logistic regression (see “ Logistic Regression Analysis ” in Section IV ). It permitted multiple factors to be examined simultaneously, took into account redundancy of information among variables (covariance), and identified a parsimonious set of variables for which the investigators coined the term “factors of risk” or risk factors (see “ Parsimony versus Complexity ” later in this section and Multivariable Analysis in Section IV ).

Various forms of multivariable analysis, in addition to logistic regression analysis, have become available to clinical investigators. Their common theme is to identify patterns of relationships between outcome and a number of variables considered simultaneously. These are not cause-effect relations, but associations with underlying causal mechanisms (see discussion of surrogates under Multivariable Analyses in Section IV ). The relationships that are found may well be spurious, fortuitous, hard to interpret, and even confusing because of the degree of correlation among variables. For example, women may be at a higher risk of mortality after certain cardiac procedures, but female gender may not be a “risk factor,” because other factors, such as body mass index, may be the more general variable related to risk, whether in women or men. Even so, it is simultaneously true that (1) being female is not per se a risk factor, but (2) women are at higher risk by virtue of the fact that on average they are smaller than men.

This means that a close collaboration must exist between statistical experts and surgeons, particularly in organizing variables for analysis.

Linearity versus Nonlinearity

Risk factor methodology introduced another complexity besides increased dimensionality. The logistic equation is a symmetric S-shaped curve that expresses the relationship between a scale of risk, called logit units , and a corresponding scale of absolute probability of experiencing an event ( Fig. 6-1 ). Because the relationship is not linear, it is not possible to simply add up scores for individual variables and come up with a probability of an event, a technique that has been attempted in other settings (see Risk Stratification in Section VI ).

The nonlinear relationship between risk factors and probability of outcome makes medical sense. Imagine a risk factor with a logit unit coefficient of 1.0 (representing an odds ratio of 2.7; Box 6-3 and see Fig. 6-1 ). If all other things position a patient far to the left on the logit scale, a 1-logit-unit increase in risk results in a trivial increase in the probability of experiencing an event. But as other factors move a patient closer to the center of the scale (0 logit units, corresponding to a 50% probability of an event), a 1-logit-unit increase in risk makes a huge difference. This is consistent with the medical perception that some patients experiencing the same disease, trauma, or complication respond quite differently. Some are medically robust because they are far to the left (low-risk region) on the logit curve before the event occurred. Others are medically fragile because their age or comorbid conditions place them close to the center of the logit curve. For the latter, a 1-logit-unit increase in risk can be “the straw that breaks the camel's back.” It is this kind of relation that makes it hard to demonstrate, for example, the benefit of bilateral internal thoracic artery grafting in relatively young adults followed for even a couple of decades, but easy in patients who have other risk factors. The same has been demonstrated for risk of operation in patients with aortic regurgitation and low ejection fraction.

This type of sensible, nonlinear medical relation makes us want to deal with absolute risk rather than relative risk or risk ratios (see Box 6-3 ). Relative risk is simply a translation of the scale of risk, without regard to location on that scale. Absolute risk integrates this with the totality of other risk factors.

Raw Data versus Models of Data

Importantly, the Framingham investigators did not stop at risk factor identification. Because logistic regression generates an equation based on raw data, it can be solved for a given set of values for risk factors. The investigators devised a cardboard slide rule for use by lay persons to determine their predicted risk of developing heart disease within the next 5 years.

Whenever possible and appropriate, results of clinical data analyses should be expressed in the form of mathematical models that become equations. These can be solved after “plugging in” values for an individual patient's risk factors to estimate absolute risk and its confidence limits. Equations are compact and portable, so that with the ubiquitous computer, they can be used to advise individual patients (see “ Decision Making for Individual Patients ” in Section V ).

It can be argued that equations do not represent raw data. But in most cases, are we really interested in raw data? Archeologists are interested in the past, but the objective of most clinical investigation is not to predict the past, but to draw inferences based on observations of the past that can be used in treating future patients. Thus, one might argue that equations derived from raw data about the past are more useful than raw, undigested data.

Nihilism versus Predictability

One of the important advantages of generating equations is that they can be used to predict future results for either groups of patients or individual patients. We recognize that when speaking of individual patients, we are referring to a prediction concerning the probability of events for that patient; we generally cannot predict exactly who will experience an event or when an event will occur. Indeed, whenever we apply what we have learned from clinical experience or the laboratory to a new patient, we are predicting. This motivated us to develop statistical tools that yield patient-specific estimates of absolute risk as an integral byproduct. These were intended to be used for formal or informal comparison of predicted risks and benefits among alternative therapeutic strategies.

Of course, the nihilist will say, “You can't predict.” However, in a prospective study of 3720 patients in Leuven, Belgium, we generated evidence that predictions from multivariable equations are generally reliable (see “ Residual Risk ” in Section VI ). We compared observed survival, obtained at subsequent follow-up, with prospectively predicted survival. The correspondence was excellent in 92% of patients. However, it was poor in the rest ( Fig. 6-2 and Table 6-2 ; see also “ Residual Risk ” in Section VI ). A time-related analysis of residual risk identified circumstances leading to poor prediction and revealed the limitations of quantitative predictions: (1) When patients have important rare conditions that have not been considered in the analysis, risk is underestimated. (2) When large data sets rich in clinically relevant variables are the basis for prediction equations, prediction should be suspect in only a small proportion of patients with unaccounted-for conditions (see “ Residual Risk ” in Section VI for details). Except for these limitations, multivariable equations appear capable of adjusting well for different case mixes.

This analysis has important implications for “report card” registries used in institutional comparisons. Surgeons often object to these comparisons on the basis that risk adjustment (see Risk Adjustment in Section VI ) accounts for neither risk stratification (see Risk Stratification in Section VI ) nor rare combinations of risk factors that defy prediction. They may be correct. Unfortunately, in advising patients about operation, we find that these are the very individuals whose risk is difficult to predict on clinical grounds and for whom we wish we had good prediction equations.

The amount of data necessary to generate new knowledge is much larger than that needed to use the knowledge in a predictive way. To generate new knowledge, data should be rich both in relevant variables and in variables eventually found not to be relevant. But for prediction, one needs to collect only those variables used in the equation (see Risk Adjustment in Section VI ) unless one is interested in investigating reasons for lack of prediction (see Residual Risk in Section VI ).

Blunt Instruments versus Fine Dissecting Instruments

A related use of predictive equations is in comparing alternative therapies. Some would argue that the only believable comparisons are those based on randomized trials, and that documented clinical experiences are irrelevant and misleading. However, many randomized trials are homogeneous and focused and are analyzed by blunt instruments, such as an overall effect. On the other hand, real-world clinical experience involves patient selection that is difficult to quantify, may be a single-institution experience with limited generality except to other institutions of the same variety, is not formalized unless there is prospective gathering of clinical information into registries, and is less disciplined. Nevertheless, analyses of clinical experiences can yield a fine dissecting instrument in the form of equations that are useful across the spectrum of heart disease for comparing alternative treatments and therefore for advising patients (see “ Clinical Studies with Nonrandomly Assigned Treatment ” later in this section).

Parsimony versus Complexity

Although clinical data analysis methods and results may seem complex at times, as in the large number of risk factors that must be assessed for comparing treatment strategies in ischemic heart disease, an important philosophy behind such analysis is parsimony (simplicity). We have discussed two reasons for this previously. One is that clinical data contain inherent redundancy, and one purpose of multivariable analysis is to identify that redundancy and thus simplify the dimensionality of the problem. A second reason is that assimilation of new knowledge is incomplete unless one can extract the essence of the information. Thus, clinical inferences are often even more digested and simpler than the multivariable analyses.

We must admit that simplicity is a virtue based on philosophical, not scientific, grounds. The concept was introduced by William of Ocken in the early 14th century as a concept of beauty—beauty of ideas and theories. Nevertheless, it is pervasive in science.

There are dangers associated with parsimony and beauty, however. The human brain appears to assimilate information in the form of models, not actual data (see “ Human Error ” earlier in this section). Thus, new ideas, innovations, breakthroughs, and new interpretations of the same data often hinge on discarding past paradigms (thinking “outside the box”). There are other dangers in striving for simplicity. We may miss important relations because our threshold for detecting them is too high. We may reduce complex clinical questions to simple but inadequate questions that we know how to answer.

For analyses whose primary purpose is comparison, it is important, when sufficient data are available ( Box 6-4 ), to account for “everything known.” In this way the residual variability attributed to the comparison is most likely to be correct.

New Knowledge versus Selling Shoes

The philosophies described so far focus on the challenge of generating new knowledge from clinical experiences. However, other uses are made of clinical data.

Clinical data may be used as a form of advertising. Innovation stems less from purposefulness than from aesthetically motivated curiosity, frustration with the status quo, sheer genius, fortuitous timing, favorable circumstances, and keen intuition. With innovation comes the need to promote. However, promotional records of achievement should not be confused with serious study of safety, clinical effectiveness, and long-range appropriateness.

Of growing importance is the use of clinical information for regulation or to gain institutional competitive advantage. Using clinical outcomes data to rank institutions or individual doctors has become popular in the United States (see Risk Stratification and Risk Adjustment in Section VI ). Many surgeons perceive clinical report cards as a means for punishment or regulation. What is troubling is that their use is based on a questionable quality-control model of outlier identification. Because doctors are people and not machines, this approach generates counterproductive ethical side effects, including defensiveness and hiding the truth. It hinders candid, non-accusatory (non-culpable), serious examination of medical processes for the express purpose of improving patient care (see “ Human Error ” earlier in this section).

Critics of clinical report cards charge that to improve their rankings, some institutions refuse to operate on sicker patients. In several studies of community hospitals by the UAB group, it was shown that they could indeed improve their risk-unadjusted rankings by restricting surgery to low-risk cases. However, their results in such patients were often inferior to those of institutions of excellence operating on similar low-risk patients. That is, their risk-adjusted mortality was high even for low-risk cases.

With the intense focus on institutional performance, another undesirable side effect of data analysis decried years ago has crept back in: undue emphasis on hospital mortality and morbidity. Studies of hospital events have the advantage of readily available data for extraction, but early events may be characterized incompletely. After repair of many congenital and acquired heart diseases, early risk of surgery extends well beyond the hospital stay. This has led to reflection on the effect of time frame on studies of clinical experiences. Use of intermediate-term data is likely to characterize the early events well, but requires cross-sectional patient follow-up. Long-term follow-up is essential to establish appropriateness of therapy, but it is expensive and runs the risk of being criticized as being of historical interest only.

Yet another reason for interest in clinical information is to use it for profit or corporate advantage. At present, the philosophies of scientific investigation and business are irreconcilable. One thrives on open dissemination of information, the other on proprietary information offering a competitive advantage. In an era of dwindling public resources for research and increasing commercial funding, we may be seeing the beginning of the end of open scientific inquiry.

Past versus Future

Is there, then, a future for quantitative analysis of the results of therapy, as there was in the developmental phase of cardiac surgery? Kirklin and Barratt-Boyes wrote in their preface to the second edition of this book:

As already noted in both the first and second editions, the last section of each chapter is on indications for operation. In the future, regulations of policymakers may need to be added to the other variables determining indications.

On the horizon is the promise that medicine will become decreasingly empirical and more deterministic. However, as long as treatment of heart disease requires complex procedures, and as long as most are palliative in the life history of chronic disease, there will be a need to understand more fully the nature of the disease, its treatment, and its optimal management. This will require adoption of approaches to data that are inescapably philosophical.

Clinical Trials with Randomly Assigned Treatment

Controlled trials date back at least to biblical times when casting of lots was used as a fair mechanism for decision making under uncertainty (Numbers 33 : 54). Solomon noted, “The lot causeth disputes to cease, and it decideth between the mighty” (Proverbs 18 : 18). An early clinical trial took place in the Court of Nebuchadnezzar, king of Babylon (modern Iraq). He ordered several gifted Hebrew youths (“well favored, skillful in all wisdom, cunning in knowledge, and understanding science”) to reside at his palace for 3 years as if they were his own children. He proposed to train them in Chaldean knowledge and language. Among them were Daniel and the familiar Shadrach, Meshach, and Abednego. Daniel objected to the Babylonian diet, and so proposed a 10-day clinical trial: The Hebrews would be fed a vegetarian diet with water, while the children of the king would be fed the king's meat and wine. After 10 days, the condition of the Hebrews was determined to be better than that of the king's children, and they received permission to continue to eat their own diet (Daniel 1:1-15). (This is remarkably reminiscent of the contemporary controversy surrounding carbohydrate-rich vs. protein-rich diets.)

The first modern placebo-controlled, double-blinded, randomized clinical trial was carried out in England by Sir Austin Bradford Hill on the effectiveness of streptomycin vs. bed rest alone for treatment of tuberculosis, although 17th- and 18th-century unblinded trials have been cited as historical predecessors.

Clinical trials in which cardiac surgical procedures and medical therapy have been randomly assigned have made major contributions to our knowledge of treatment and outcomes of heart disease. Notable examples are the Veterans Administration (VA) study of CABG, the Coronary Artery Surgery Study (CASS) trial of CABG, the European Coronary Surgery Study trials, and the PARTNER trial of percutaneous aortic valve replacement. Trials of CABG vs. percutaneous coronary intervention have also been important (e.g., the Balloon Angioplasty Revascularization Investigation [BARI]).

Randomization of treatment assignment has three valuable and unique characteristics:

• ▪

  It eliminates selection factors (bias) in treatment assignment (although this can be defeated at least partially by enrollment bias).

• ▪

  It distributes patient characteristics equally between groups, whether they are measured or not, known or unknown (balance), a well-accepted method of risk adjustment.

• ▪

  It meets assumptions of statistical tests used to compare end points.

Randomized clinical trials are also characterized by concurrent treatment, excellent and complete compilation of data gathered according to explicit definitions, and proper follow-up evaluation of patients. These operational byproducts may have contributed nearly as much new knowledge as the random assignment of treatment.

Unfortunately, it has become ritualistic for some to dismiss out of hand all information, inferences, and comparisons relating to outcome events derived from experiences in which treatment was not randomly assigned. If this attitude is valid, then much of the information now used to manage patients with cardiac disease would have to be dismissed and ignored! Investigations concerning differences of outcome among different physicians, different institutions, and different time periods would have to be abandoned. However, moral justification may not be present for a randomized comparison of procedures and protocols that clinical experience strongly suggests have an important difference. (The difficulty of recruitment in BARI reflects this problem.) In fact, when Benson and Hartz investigated differences between randomized trials and observational comparisons over a broad range of medical and surgical interventions, they found “little evidence that estimates of treatment effects in observational studies reported after 1984 are consistently larger than or qualitatively different from those obtained in randomized controlled studies.” (See, however, the rebuttal by Pocock and Elbourne. ) These findings were confirmed by Concato and colleagues. Nevertheless, we acknowledge a hierarchy of clinical research study designs, and the randomized trial generates the most secure information about treatment differences.

Trials in which treatment is randomly assigned are testing a hypothesis, and hypothesis testing in general requires a yes or no answer unperturbed by uncontrollable factors. Thus ideally, the study is of short duration, with all participants blinded and a treatment that can be well standardized. However, in many clinical situations involving patients with congenital or acquired heart disease, the time-relatedness of freedom from an unfavorable outcome event is important and can jeopardize interpretation of the trial. This is because individual patients assign different values to different durations of time-related freedoms, in part because differing severities of disease (and corresponding differences in natural history) affect different time frames and in part because the longer the trial, the more likely there will be crossovers (e.g., from medical to surgical therapy). Also, the greater the number of risk factors associated with the condition for which treatment is being evaluated, the greater the potential heterogeneity (number of subsets) of patients with that condition and the greater the likelihood that a yes/no answer will apply only to some subset of patients. In such situations, a randomized trial may have the disadvantage of including only a limited number of subsets. It may in fact apply to no subset, because the “average patient” for whom the answer is derived may not exist except as a computation. Trials have addressed this problem by basing the randomization on subsets or by later analyzing subsets by stratification (but see concerns raised by Guillemin ) or by multivariable analysis.

These considerations, in addition to ethical concerns, have fueled the debate about whether surgery is an appropriate arena for randomized trials of innovation, devices, and operations. Some argue strongly that randomization should be required at the outset of every introduction of new therapy. In three related articles arising from the Balliol Colloquium held at the University of Oxford between 2007 and 2009, clinicians and anesthesiologists sought to clarify the issues surrounding surgical clinical trials. They recognized important stages in developing a surgical technique, starting with innovation, progressing through development and exploration, to assessment and long-term outcomes. They then explore options for evaluative studies and barriers to each, including sham operations and nonoperative treatment alternatives. They end with an IDEAL model for surgical development (idea, development, exploration, assessment, long-term study) and the role of feasibility-randomized-trials in exploration, and definitive trials in assessment, and registries in long-term surveillance.

Steven Piantadosi of Johns Hopkins University describes a number of important methodological problems with conducting successful surgical trials, however (personal communication; November 2001):

• ▪

  Operations are often not amenable to blinding or use of placebos (sham operations), although there is growing acceptance of this in some cases of surgery, in part because of the huge placebo effect of surgery. This can introduce bias that may be impossible to control; however, thoughtful and creative study designs can often produce substantial blinding, such as of those assessing outcome.

• ▪

  Selection bias is difficult to avoid. He notes that it is insufficient to compare patients undergoing operation with those who do not, no matter how similar the groups appear, unless every patient not undergoing operation is completely eligible for surgical intervention. Judgment is a characteristic of a good surgeon, and the better the surgical judgment, the more likely bias will enter any trial of surgical vs. nonsurgical therapy, even if it is the bias of selecting patients for the trial.

• ▪

  Surgical therapy is skill-based. Therefore, any result obtained from a trial consists of the inextricable confounding of (1) procedure efficacy and (2) surgical skill.

• ▪

  Surgery is largely unregulated. Every operation is different, and particularly in treatment of complex congenital heart diseases, tailoring operations to the specific anomaly is expected and often necessary for patient survival. There is little uniformity from patient to patient to provide a basis for randomizing therapy.

Given these potential obstacles to adequate evaluation of surgical procedures ever occurring, McCulloch has proposed a hybrid strategy that begins with a prospective but nonrandomized surgical study during the dissemination phase of development (phase II) that progresses to a phase III randomized clinical trial. During the phase II study, learning curves are determined, a likely treatment effect is identified for sample-size calculation, consensus is built, and quality measures to confirm delivery of intended operations are drawn up. Failure of these preliminary steps is perhaps what has stirred much of the controversy over the STICH trial of the Dor procedure.

Moses and others present the case for a balance between randomized clinical trials and observational clinical studies. However, observational studies are beset with these same problems of selection bias and skill variance; thus, not to be overlooked are the development and rapid introduction of powerful new methods for drawing causal inferences from nonrandomized trials (see “ Causal Inferences ” later in this section).

Clinical Studies with Nonrandomly Assigned Treatment

Propensity Score

The most widely used balancing score is the propensity score. It provides for each patient an estimate of the propensity toward (probability of) belonging to one group vs. another (group membership) . Here we describe (1) designing the nonrandomized study, (2) constructing a propensity model, (3) calculating a propensity score for each patient using the propensity model, and (4) using the propensity score in various ways for effecting a balanced comparison.

Designing the Nonrandomized Study

The essential approach to a comparison of treatment outcomes in a nonrandomized setting is to design the comparison as if it were a randomized clinical trial and to interpret the resulting analyses as if they emanated from such a trial. This essential approach is emphasized in Rubin's 2007 article, “The Design versus the Analysis of Observational Studies for Causal Effects: Parallels with the Design of Randomized Trials.”

As noted by Rubin, “I mean all contemplating, collecting, organizing, and analyzing data that takes place prior to seeing any outcome data.” He emphasizes by this statement his thesis that a nonrandomized set of observations should be conceptualized as a broken randomized experiment…with a lost rule for “patient allocation, and specifically for the propensity score, which the analysis will attempt to construct.” For example, the investigator should ask, “Could each patient in all comparison groups be treated by all therapies considered? If not, this constitutes specific inclusion and exclusion criteria. If this were a randomized trial, when would randomization take place? One must only use variables to construct a propensity score that would be known at the time randomization would have occurred, not after that; this means that variables chosen in the propensity score analysis are not those that could possibly be affected by the treatment.

Constructing a Propensity Model

For a two-group comparison, typically, multivariable logistic regression is used to identify factors predictive of group membership (see “ Logistic Regression Analysis ” in Section IV ). In most respects, this is what cardiac surgery groups have done for years—find correlates of an event. In this case, it is not risk factors for an outcome event, but rather correlates of membership in one or the other comparison group of interest.

We recommend initially formulating a parsimonious multivariable explanatory model that identifies common denominators of group membership (see Multivariable Analysis in Section IV ). Once this traditional modeling is completed, a further step is taken to generate the propensity model , which augments the traditional model by other factors, even if not statistically significant. Thus, the propensity model is not parsimonious. The goal is to balance patient characteristics by whatever means possible, incorporating “everything” recorded that may relate to either systematic bias or simply bad luck, no matter the statistical significance. (However, this is not to say that the addition of nonsignificant variables is done carelessly; the same rigor in variable preparation described in Multivariable Analysis in Section IV is mandatory.) It is important to use as many continuous variables as possible to represent these patient characteristics, because it produces a fine, as opposed to coarse, set of values when the propensity score is calculated.

When taken to the extreme, forming the propensity model can cause problems because medical data tend to have many variables that measure the same thing. The solution is to pick one variable from among a closely correlated cluster of variables as a representative of the cluster. An example is to select one variable representing body size from among height, weight, body surface area, and body mass index.

Calculating a Propensity Score

Once the propensity modeling is completed, a propensity score is calculated for each patient. A logistic regression analysis, such as is used for the propensity model, produces a coefficient or numeric weight for each variable ( Box 6-5 ). The coefficient maps the units of measurement of the variable into units of risk. Specifically, a given patient's value for a variable is transformed into risk units by multiplying it by the coefficient. If the coefficient is 1.13 and the variable is “male” with a value of 1 (for “yes”), the result will be 1.13 risk units. If the coefficient is 0.023 for the variable “age” and a patient is 61.3 years old, 0.023 times 61.3 is 1.41 risk units.

Box 6-5

Regression

Sir Francis Galton, cousin of Charles Darwin, explored the relation between heights of adult children and average height of both parents (midparent height). He found that children born to tall parents were in general shorter than their parents, and children born to short parents, taller. He called this “regression towards mediocrity.” He even generated a “forecaster” for predicting son and daughter height as a function of father and mother height. It is presented in an interesting way as pendulums of a clock, with chains around two different-sized wheels equivalent to the different weights (regression coefficients) generated by the regression equation!

Today, any empirical relation of an outcome or dependent variable to one or more independent variables (see later Box 6-18 ) is termed a regression analysis . Several of these are described below.

Linear

The form of a linear regression equation for a single dependent variable Y and a single independent variable x is:

$Y=a+bx$

where a is called the intercept (the estimate of Y when x is zero), and b is the slope (the increment in Y for a one-unit change in x ). More generally, when there are a number of x 's:

$Y={\beta }_0+{\beta }_1{x}_1+\cdots +{\beta }_k{x}_k$

where β ₀ is the intercept, x ₁ through x _k are independent variables, and β ₁ through β _k are weights, regression coefficients, or model parameters (see later Box 6-13 ) that are multiplied by each x to produce an incremental change in Y.

It would be surprising if biological systems behaved as a series of additive weighted terms like this. However, this empirical formulation has been valuable under many circumstances in which there has been no basis for constructing a biomathematical model based on biological mechanisms (computational biology).

An important assumption is that Y is distributed in Gaussian fashion (see Box 6-15 ), and this may require the scale of the raw data to be mathematically transformed.

Log-Linear

A log-linear regression equation has the following form:

$\text{Ln(}Y)={\beta }_0+{\beta }_1{x}_1+{\beta }_2{x}_2+\cdots +{\beta }_k{x}_k$

where Ln is the logarithm to base e . Such a format is used, for example, in the Cox proportional hazards regression model (see Section IV ). However, in studies of events (see “ Logistic Regression Analysis ” and Time-Related Events in Section IV ), the estimation procedure does not actually use a Y . Rather, just as in finding the parameter estimates called mean and standard deviation of the Gaussian equation (see Box 6-15 ), parameter estimation procedures use the data directly. Once these parameters are estimated, a predicted Y can be calculated.

Logit-Linear

A logit-linear regression equation, representing a mathematical transformation of the logistic equation (see Fig. 6-1, A and Section IV ), has the following form:

$\text{Ln}[P/(1-P\left)\right]={\beta }_0+{\beta }_1{x}_1+{\beta }_2{x}_2+\cdots +{\beta }_k{x}_k$

where Ln is the logarithm to base e and P is probability. The logit-linear equation is applicable to computing probabilities once the β s are estimated.

Model

A model is a representation of a real system, concept, or data, and particularly the functional relationships within these; it is simpler to work with, yet predicts real system, concept, or data behavior.

Mathematical Model

A mathematical model consists of one or more interrelated equations that represent a real system, concept, or data by mathematical symbols. These equations contain symbols that represent parameters (constants) whose values are estimated from data and an estimating procedure.

A mathematical model may be based on a theory of nature or mechanistic understanding of what the real system, concept, or data represent (biomathematical models or computational biology). It may also be empirical. The latter characterizes most models in statistics, as depicted previously. The Gaussian distribution is an empirical mathematical model of data whose two parameters are called mean and standard deviation (see Box 6-15 ). All mathematical models are more compact than raw data, summarizing them by a small number of parameters in a ratio of 5 to 10 or more to 1.

Linear Equation

When applied to mathematical models, it is an equation that can be solved directly with respect to any of its parameter values by simple mathematical manipulation. A linear regression equation is a linear model.

Nonlinear Equation

When applied to mathematical models, a nonlinear equation is an equation that cannot be solved directly with respect to its parameter values, but rather must be solved by a sequence of guesses following a recipe (algorithm) that converges on the answer (iterative). The logistic equation is a nonlinear model.

One continues through the list of model variables, multiplying the coefficient by the specific value for each variable. When finished, the resulting products are summed. To this sum is added the intercept of the model (see Box 6-5 ), and the result is the propensity score. Note that technically, the intercept of the model, which is constant for all patients, does not have to be added; however, in addition to using the propensity score in logit risk units as described here, it may be used as a probability, for which the intercept is necessary.

Using Propensity Score for Comparisons

Once the propensity model is constructed and a propensity score is calculated for each patient, three common types of comparison are employed: matching, stratification, and multivariable adjustment.

The propensity score can be used as the sole criterion for matching pairs of patients ( Table 6-6 ). Although a number of matching strategies have been used by statisticians for many years, new optimal matching algorithms have arisen within computer science and operations research. These have been motivated by the need to optimally match volume of Intranet and Internet traffic to computer network configurations. In addition, Rubin (personal communication, 2008) has suggested matching with replacement vs. the usual “greedy” matching, which removes matched patients from further consideration. Indeed, matching can be bootstrapped, creating multiple matched comparison groups over which outcome can be averaged.

Table 6-6

Comparison of Patient Characteristics According to Long-Term Aspirin Use in Propensity-Matched Pairs ^a

Data from Gum and colleagues.

Patient Characteristic	ASA	No ASA
n	1351	1351
Men (%)	49	51
Age (years)	60	61
Smoker (%)	50	50
Resting heart rate (beats · min −1 )	77	76
Ejection fraction (%)	51	51

Key: ASA, Long-term aspirin use.

a Table illustrates ability of the propensity score to produce what appears to be a randomized study balancing both patient characteristics and n .

Rarely does one find exact matches. Instead, a patient is selected from the control group whose propensity score is nearest to that of a patient in the case group. If multiple patients are close in propensity scores, optimal selection among these candidates can be used. Remarkably, problems of matching on multiple variables disappear by compressing all patient characteristics into a single score (compare Table 6-6 with unmatched data in Table 6-3 ).

Tables 6-4 and 6-6 demonstrate that such matching works astonishingly well. The comparison data sets have all the appearances of a randomized study! The average effect of the comparison variable of interest is assessed as the difference in outcome between the groups of matched pairs. However, unlike a randomized study, the method is unlikely to balance unmeasured variables well, and this may be fatal to the inference.

Once patients are matched, it is important to diagnostically test the quality of matching. This can be accomplished visually by graphs of standardized differences ( Fig. 6-3 ). Differences that were substantial should virtually disappear. If they do not, it is possible that interaction terms (multiplicative factors rather than additive factors) may be required.

A graph of propensity scores for the groups is instructive ( Fig. 6-4 ). The scores for two treatments may nearly overlap, as they would for a randomized trial. On the other hand, there may be little overlap, as in Fig. 6-5 , and the comparison focuses on the center part of the spectrum of propensity score where there is substantial overlap (virtual equipoise).

Outcome can be compared within broad groupings of patients called strata or subclasses , according to propensity score. After patients are sorted by propensity score, they are divided into equal-sized groups. For example, they may be split into five groups, or quintiles (see Tables 6-4 and 6-5 ), but fewer or more groups may be used, depending on the size of the study. Comparison of outcome for the comparison variable of interest is made within each stratum. If a consistent difference in outcome is not observed across strata, intensive investigation is required. Usually, something is discovered about the characteristics of the disease, the patients, or their clinical condition that results in different outcomes across the spectrum of disease. For example, in their study of ischemic mitral regurgitation, Gillinov and colleagues discovered that the difference in survival between those undergoing repair vs. replacement progressively narrowed as complexity of the pattern of regurgitation increased and condition of the patient worsened ( Fig. 6-6 ). Apparent anomalies such as this give important insight into the nature of the disease and its treatment.

The propensity score for each patient can be included in a multivariable analysis of outcome. Such an analysis includes both the comparison variable of interest and the propensity score. The propensity score adjusts the apparent influence of the comparison variable of interest for patient selection differences not accounted for by other variables in the analysis.

Occasionally the propensity score remains statistically significant in such a multivariable model. This constitutes evidence that adjustment for selection factors by multivariable analysis alone is ineffective, something that cannot be ignored. It may mean that not all variables important for bias reduction have been incorporated into the model, such as when one is using a simple set of variables. It may mean that an important modulating or synergistic effect of the comparison variable occurs across propensity scores, as noted previously (e.g., the mechanism of disease may be different within quintiles). It may mean that important interactions of the variable of interest with other variables have not been accounted for, leading to a systematic difference identified by the propensity score. The collaborating statistician must investigate and resolve these possibilities. Understanding aside, this statistically significant propensity score has performed its intended function of adjusting the variable representing the group difference.

In some settings in which the number of events is small, the propensity score can be used as the sole means of adjusting for the variable representing the groups being compared.

Research Proposal

Because of the necessity for Institutional Review Board (Ethics Committee) oversight, but also because it is good science, every serious clinical study needs a formal proposal ( Box 6-6 ). This proposal serves to clarify and bring into focus the question being asked. A common mistake is to ask questions that are unfocused, or uninteresting, or overworked, or that do not target an area of importance. Marbán and Braunwald say, “Ask a bold question…about which you can feel passionate.” Brainstorming with fellow surgeons and collaborators is essential. The first deliverable is the research question, well debated.

The next step is to define clearly the inclusion and exclusion criteria for the study group (see “ Identify Study Group ” in Section III ). A common mistake is to define this group too narrowly, such that cases “fall through the cracks” or an insufficient spectrum is stipulated (see “ Continuity versus Discontinuity in Nature ” earlier in this section). The inclusive dates should be considered carefully. Readers will be suspicious if the dates are “strange”; did you stop just before a series of deaths? Whole years or at least half years dispel these suspicions. Similarly, suspicion arises when a study consists of a “nice” number of patients, such as “the first 100 or 1000 repairs.”

In defining the study group, particular care should be taken to include the denominator. For example, a study may be made of postoperative neurologic events, but it is also important to have a denominator to put these events into context. Or one may study a new surgical technique but be unable to compare it with the standard technique without a comparison group. A study of only numerators is the true definition of a retrospective study; if the denominator is included, it is a prospective or cohort study ( Box 6-7 ).

End points (results, outcomes) must be clearly defined in a reproducible fashion. Generally, every event should be accompanied by its date of occurrence. A common failing is that repeated end points (e.g., thromboembolism, assessments of functional status) are recorded only the first or most recent time they occur. This should never be done. Techniques to analyze repeated end points are available (see Longitudinal Outcomes in Section IV ).

Careful attention must be paid to the variables that will be studied. They should be pertinent to the study question (purpose, objective, hypothesis). A common failing is to collect values for too many variables such that quality suffers. This error usually arises in a reasonable and understandable way. The surgeon-investigator reasons that because the patient records must be reviewed, a number of other variables may as well be abstracted “while there.” Or realizing the full complexity of the clinical setting, the surgeon-investigator feels compelled to collect information on all possible ramifications of the study, even if it is quite peripheral to the focus of the study. This is termed the “Christmas tree effect,” meaning adding ornament upon ornament until they dominate what once was “just” a fine tree. There needs to be a balance between so sparse a set of variables that little can be done by way of risk factor identification or balancing characteristics of the group, and so rich a set of variables that the study flounders or insufficient care is given to the quality and completeness of relevant variables.

Study feasibility must then be assessed. A common failing is forgetting that if an outcome event is the end point, the effective sample size is the number of events observed (see Box 6-4 ). A study may have 1000 patients, but if only 10 events are observed, one cannot find multiple risk factors for those events.

It is wise at the outset to plan the data analysis . Often, for example, the setup for the analysis data set is specific to the methods of analysis. This has to be known by the data managers (see Appendix 6A).

A necessary step is review of the literature. Sifting through articles is often painful, but it should result in identifying those few key papers that are absolutely pertinent to the study. Unfortunately, the search is too often confined to recent literature, and this may result in “reinventing the wheel.”

For executing the study, some realistic time frame with deliverables should be established with collaborators. A common failing is not providing sufficient time for data verification and other aspects of data management that are the heart of a high-quality study. Actual analysis of data may consume one tenth the time of high-quality data preparation.

The completed formal research proposal becomes the second deliverable of a study. It is likely to be updated throughout the course of a study, and we advocate online tracking of each study, with periodic updates of the protocol as one of the tasks in project management.

Database Development and Verification

The next step for successful research is careful attention to the data themselves (see “ Extract Values for Variables ” in Section III ). If electronically available data are to be used, every variable must be defined both medically and at the database content level (see Section II ). If data are to be collected de novo, an appropriate database must be developed (see Fig. 6-7 ). Every variable must be in a format of one value per variable. These variables must follow a controlled vocabulary for analysis, not free text. The deliverable at this stage is a database ready for data to be collected and entered.

Data Collection

Research on existing databases for which blanket IRB approval has been secured may not require separate approval of each study. However, before any de novo data are gathered from medical records or by patient follow-up, separate IRB approval may be required.

There is generally a core set of variables (core data elements) that should be collected for each patient ( Box 6-8 ). In many cardiac surgical settings, these data elements are stipulated by regulatory agencies (e.g., the state of New York) or surgical societies (e.g., Society of Thoracic Surgeons National Database). They include demographics (note that it is essential to record patients’ date of birth rather than age because age can be calculated from date of birth to any chosen “time zero”), the cardiac procedure and possibly clinical symptoms and status at time of operation, past cardiac medical history (particularly prior cardiac procedures), disease etiology, coexisting cardiac defects, coexisting noncardiac morbidity (e.g., diabetes), laboratory measurements known to be consistently associated with clinical outcomes, findings of diagnostic testing, intraoperative findings, support techniques during operation, and factors related to experience (e.g., date of operation).

Box 6-8

Core Data Element Concept

Core data elements represent the most granular source information that can be logically combined or mapped in multiple ways to generate answers (values) to specific questions (variables). Schematically, this is shown in the following diagram for six core data elements (CDE) from sources a-f.

The diagram that follows answers two specific questions concerning use of antianginal medications.

Finally, a third question is asked that relates to a specific medication and requires a combination of temporal reasoning and medication class, prescription, and use.

Beyond these core variables, there will likely be a need for variables specific to a particular study. These should be identified and reproducibly defined. The danger is specifying too many variables; however, a thoughtfully compiled list adds depth to a study. Further, experienced investigators realize that in the midst of a study, it occasionally becomes evident that some variables require refinement, others collecting de novo, others rechecking, and others redefining. It is important to understand that when this occurs, the variables must be refined, collected, rechecked, or redefined uniformly for every patient in the study.

Clinical studies are only as accurate and complete as the data available in patients’ records. Therefore, cardiac surgeons and team members seriously interested in scientific progress must ensure their preoperative, operative, and postoperative records are clear, organized, precise, and extensive, so that information gathering from these records can be complete and meaningful. The records should emphasize description, and although they may well contain the conclusions of the moment, it is the description of basic observations that becomes useful in later analyses.

Verification

The first step in data verification is to enter values for each data element (variable) for 5 to 10 patients only. These reveal problems of definition, incomplete “pick lists,” missed variables, difficult-to-find variables that may not be worth the effort to locate, poor-quality variables/incomplete recording, lack of good definition, inconsistent recording, and questionable quality of observations. Once these issues are addressed, general data abstraction may proceed (see “ Verify Data ” in Section III ).

When all values for variables are in a computer database, formal verification commences. This can take three general forms: (1) value-by-value checking of recorded data against primary source documents, (2) random quality checking, and (3) automatic reasonableness checking. If a routine activity of recording core data elements (see Section III , Data ) is used, it is wise to verify each element initially to identify those that are rarely in error (these can be “spot checked” by a random process) and those that are more often in error. The latter are usually a small fraction of the whole and are often values requiring interpretation. These may require element-by-element verification.

When it is believed that data are correct (this is an iterative process with the above), they are checked for reasonableness of ranges, including discovery of inconsistencies among correlated values. For example, the database may indicate that a patient had a quadrangular resection of the mitral valve, but someone had failed to record that the posterior leaflet was prolapsing and had ruptured chordae, or the database records that a patient is 60 cm tall and weighs 180 kg; this is likely a problem of confused units of measurement (inches and pounds).

Data Conversion for Analysis

An often underappreciated, unanticipated, and time-consuming effort is conversion of data elements residing in a database to a format suitable for data analysis (see Analysis Data Set in Section III ). Even if the day comes that all medical information is recorded as values for variables in a computer-based patient record (see Section II , Computer-Based Patient Record ), this step will be unavoidable. Statistical procedures require data to be arranged in “columns and rows,” with each column representing values for a single variable (often in numeric format), and each row either a separate patient or multiple records on a single patient (as in repeated-measures longitudinal data analysis). Unfortunately, this conversion process may involve redundancy, such as the necessity to again document all variables and provide a key to the possible values for each.

This process nearly always involves creating additional variables from a single variable, such as a separate variable for each mutually exclusive etiology of cardiomyopathy. These polytomous variables (lists) are then converted to a series of dichotomous variables (best expressed as 0 for absence and 1 for presence of the listed value).

Some categorical variables are ordinal, such as NYHA functional classes. These may have to be reformulated as an ordered number sequence (e.g., 1-4). Variables recorded with units (e.g., weight in kilograms, weight in pounds) must be converted to a common metric.

Calculated variables are also formed. These include body surface area and body mass index from height and weight, z values (see Chapter 1 ) from measured cardiac dimensions, ejection fraction from systolic and diastolic ventricular volumes, intervals between date and time variables for which event indicator variables are created, and many other calculations. Because data conversion, creation of derived variables, and formation of calculated variables is time consuming and error prone, groups that conduct a large number of studies often store trusted, well-verified computer code to perform these operations on a repetitive basis.

Often information is coalesced from multiple databases, and these queries, concatenations, and joining functions transpire in this phase of the process. These otherwise arduous functions can, under some circumstances, be automated. Alternatively, a data warehouse composed of multiple disparate electronic data sources can be implemented and maintained and appears to the investigator as a single data source.

An important activity is managing sporadic missing data. If too many data are missing, the variable may be unsuitable for analysis (see “ Impute Values ” in Section III ). Otherwise, missing value imputation is necessary so that entire patients are not removed from analyses, the default option in many analysis programs.

Data Analysis

Specific data analysis methods will be described in Section IV . Here, we simply indicate how this aspect of the research process leads to success.

First, the analysis process leads to understanding of the “raw data,” often called exploratory data analysis. This understanding is gleaned from such analyses as simple descriptive statistics, correlations among variables, simple life tables for time-related events, cumulative distribution graphs of continuously distributed variables (see “ Descriptive Statistics ” in Section III ), and cluster analyses whereby variables with shared information content are identified.

Second, the analytic process attempts to extract meaning from the data by various methods akin to pattern recognition. Answers are sought for questions such as: Which variables relate to outcome and which do not? What inference can be made about whether an association is or is not attributable to chance alone? Might there be a causal relationship? For what might a variable associated with outcome be a surrogate?

What will be discovered is that answering such questions in the most clinically relevant way often outstrips available statistical, biomathematical, and algorithmic methodology! Instead, a question is answered with available techniques, but not the question. Some statisticians, because of insufficient continuing education, lack of needed statistical software, lack of awareness, failure of communication, or lack of time, may explore the data less expertly than required. One of the purposes of this chapter is to stimulate effective collaboration between cardiac surgeons and data analysis experts so that data are analyzed thoroughly and with appropriate methodology.

Interpreting Analyses

It is one thing for a statistician to provide a statistical inference; it is quite another for the cardiac surgeon, using that information, to draw meaningful interpretations that affect patient care.

Kirklin and Blackstone empirically found that the most successful way to embark on this interpretive phase of clinical research is to write on a clean sheet of paper the truest two or three sentences that capture the essence of the findings (and no more!). This important exercise produces an ultra-mini abstract for a paper (whether or not it is required by a journal) and provides the roadmap for writing the manuscript (see Scientific Paper in Section V ).

Communicating the Findings

A common error of the surgeon-investigator is to simply summarize the data instead of taking the important step of drawing meaningful clinical inferences from the data and analyses. He or she has not taken the vital step of asking (1) What new knowledge has been gleaned from the clinical investigation? (2) How can this new knowledge be incorporated into better patient care? (3) What do the data suggest in terms of basic research that needs to be stimulated? (4) How can I best communicate information to my local colleagues? (5) How can I best present this information to the cardiac surgical and cardiologic world at large?

Meaningful new knowledge may not be generated because the statistical inferences from data analyses are accepted as the final result. Instead, the results must be studied carefully and many questions asked. Often this will lead to additional analyses that increasingly illuminate the message the data are trying to convey. Graphical depictions are of particular importance in transforming mere numbers on computer printouts to insight. Depictions must lead beyond statistical inference to clinical inference. What have the data revealed about how to better care for patients? This question is the one best linked to the original purpose of the study. If the study has suggested ways to improve patient care, the next step is to put what has been learned into practice (see Section V ).

Most studies generate more new questions than they answer. Some of these new questions require additional clinical research. Others require the surgeon-investigator to stimulate colleagues in the basic sciences to investigate fundamental mechanisms of the disease process.

Because most surgeon-investigators are part of a group, an important facet of generating new knowledge is discussing with colleagues the results, statistical and clinical inferences, and implications of a study. Multiple points of view nearly always clarify rather than obscure their interpretation.

Finally, clinical research is not a proprietary activity. Yet, too often manuscripts fail to eventuate from research. One reason may be that an abstract was not accepted for a meeting, perhaps because the data were not thoroughly digested before its submission. Although abstract deadlines may be important mechanisms for wrapping up studies, they too often stifle a serious and contemplative approach to generating new knowledge. A second reason manuscripts do not get written is that the surgeon-investigator views the task as overwhelming. Possibly he or she has not developed an orderly strategy for writing. We provide some guidance for this in Section V . A third barrier to writing is time demands on the surgeon-investigator. Usually, this results from not making writing a priority in one's professional life. This is a decision that should be made early in one's surgical career. If dissemination of new knowledge is a desire, then writing must be made a high-priority part of one's lifestyle.

Ontology

If medical information is to be gathered and stored as values for variables, a medical vocabulary and organizing syntax must be available. A technical term for this is ontology.

In Greek philosophy, ontology meant “the nature of things.” Specifically, it meant what actually is (reality), not what is perceived (see “ Human Error ” in Section I ) or known (epistemology). In medicine of the 17th and 18th centuries, however, it came to mean a view of disease as real, distinct, classifiable, definable entities. This idea was adopted by computer science to embrace with a single term everything that formally specifies the concepts and relationships that can exist for some subject, such as medicine. An ontology permits sharing of information, such as a vocabulary of medicine (terms, phrases), variables, definitions of variables, synonyms, all possible values for variables, classification and relationships of variables (e.g., in terms of anatomy, disease, healthcare delivery), semantics, syntax, and other attributes and relationships.

An ontology for all of medicine does not yet exist. Efforts to develop a unified medical language, such as the Unified Medical Language System (UMLS) of the National Library of Medicine, are well underway and becoming increasingly formalized linguistically as ontologies.

Ontology is familiar to clinical researchers, who must always have a controlled vocabulary for values for variables, well-defined variables, and explicit interrelations among variables. Without these, there is no way to accurately interpret analyses or relate results to the findings of other investigators. However, a clinical study is a microscopic view of medicine; scaling up to all of medicine is daunting.

Perhaps, then, the simplest way of thinking about an ontology for the researcher is as dictionaries of variables and values and their organizational structure, and some mechanism to develop and maintain them. These attributes have collectively been called metadata (data about data) or a knowledge base , and metadata-base or knowledge-base management systems , respectively.

Information (Data) Model

An information (data) model is a specification of the arrangement of the most granular piece of information according to specific relationships and the organization of all of these into sets of related information. The objective of an information model is to decrease entropy—that is, to decrease the degree of disorder in the information and thereby increase efficiency of information storage and retrieval (performance).

Semistructured Information Model

A different kind of information model emerged from an important conference at UAB of leaders in the development of several different types of database as part of the CPR project. After review of the strengths and profound limitations of various information models, a novel approach was suggested by Kirklin and then formalized. He proposed that all information that provided context and meaning to a value for a variable be packaged together. He envisioned that such a complex data element should be able to reside as an independent self-sufficient entity. (In computer science terminology, this would be called a completely flattened data model .)

This idea has several meritorious implications. First, an electronic container for a collection of complex data elements could consist of a highly stable, totally generic repository for a CPR because it would be required to possess no knowledge of content of any data element. It could therefore manage important information storage and retrieval functions, implement data encryption for privacy and confidentiality, store knowledge bases used to construct the complex data elements and retrieve them, maintain audit trails, and perform all those functions of database management systems that are independent of data content. The second implication is that as medical knowledge increases, new entries would be made in the knowledge-base dictionaries. These would be updated, not the database structure. Not only would this ease database maintenance, it would enforce documentation in the knowledge base. The third implication, and the one most important for clinical research, is that no a priori limitations would be placed on relations; they could be of any dimensionality considered useful at the time data elements were retrieved for analysis. Thus, the electronic container is a single variable value-pair augmented with contextual documentation and capable of being modified as new or more knowledge accrues.

Essential characteristics of such an information repository are:

• ▪

  Self-documentation at the level of individual values for a variable (complex data element)

• ▪

  Self-reporting at the time of data element retrieval and potential

• ▪

  Self-displaying in a human-computer interface

• ▪

  Self-organizing

The latter is an important attribute for future implementation of what might be called “artificial intelligence” features of a CPR. These may be as simple as self-generation of alerts, solution of multivariable equations for decision support at the individual patient level, or intelligent data mining for undiscovered relations within the information.

About 1995, at the time these ideas were being developed at UAB, similar thinking was going on among computer scientists at Stanford University and the University of Pennsylvania, arising from different stimuli. They termed an information model of complex data elements that carried with them all attributes intended for self-documentation, self-reporting, and self-organizing semistructured data. This phrase meant that the data elements were fully structured, but no necessary relation of one data element to another was presupposed. The culmination of these efforts was a database for storing complex data elements called Lore and a novel query language for retrieving complex data elements called Lorel.

In the 1990s, it was recognized that the information structure suggested by Kirklin and the University of Pennsylvania and Stanford computer scientists could be conceptualized as a directed acyclic graph ( Fig. 6-8 ). At that time, another entity was also rapidly coming into existence with similar properties, but of global proportions: the World Wide Web (WWW, or simply the Web). A Web page is analogous to a complex data element, with an essential feature being that it is self-describing, so it can be retrieved. The Web is the infrastructure for these pages. It has no need to be aware of Web page content. The subject matter has no bounds. Not surprisingly, then, the tools developed for retrieving semistructured data were quickly adapted to what has become known as search engines for the Web. Like Dr. Kirklin's vision of complex data elements, information retrieved by a search engine can become related in ways never envisioned by the person generating it, because full structure is imposed only at the time of retrieval, not at the time of storage.

In 1998, the Lore scientists realized that the information model for semistructured data could be implemented in XML (extensible markup language). XML is a textual language for information representation and exchange, largely developed for document storage and retrieval but adaptable to values for variables.

At least on a conceptual basis, we believe a CPR can be formulated using a semistructured information model that will facilitate clinical research by not imposing restrictions at the time of storage, as relational and object models do.

Subsequently at Cleveland Clinic, investigators have been both harnessing and developing “Semantic Web” tools for data storage and manipulation, in part within the framework of the World Wide Web Consortium (W3C) and in part through ontologies built by Douglas Lenat of Cycorp. W3C is an international community that works with the public to develop Web standards. It is developing a suite of technologies that build on standards associated with the World Wide Web and provide a formal model for representing information in a manner that emphasizes the meaning of terms rather than their structure. It is a vision of how the existing infrastructure of the Web can be extended in such a way that machines can interpret the meaning of data involved in interactions over the Web.

This particular collection of standards is commonly referred to as Semantic Web technologies . They are built on a graph-based data model known as the Resource Description Framework (RDF), as well as a framework for describing conceptual models of RDF data in a particular domain known as the Ontology Web Language (OWL). It also includes a standard querying language called SPARQL .

RDF captures meaning as a collection of triples consisting of components analogous to those of an elementary sentence in natural language: subject, verb, object. Typically, terms in these sentences are resources identified by Uniform Resource Identifiers (URIs). URIs are global identifiers for items of interest (called resources ) in the information space of the Web. Collections of RDF triples constitute an RDF graph.

Many requirements outlined by the IOM as crucial for CPR systems are addressed by using RDF as a data format for patient record content. In particular, our ability to link with other clinical records can be facilitated when RDF is used in this way. Use of URIs as syntax for the names of concepts in RDF graphs is the primary reason for this. The meaning of terms used in a patient record (as well as the patient record itself or some part of it) can be made available over the Web in a (secure) distributed fashion for on-demand retrieval.

A judicious application of Semantic Web technologies can also lead to faster movement of innovation from the research laboratory to the clinic or hospital. In particular, it is envisioned that use of these technologies will improve productivity of research, help raise quality of health care, and enable scientists to formulate new hypotheses, inspiring research based on clinical experience.

Identify Study Group

The clinical research proposal (see Box 6-6 ) provides detailed specifications for the study group of interest. The medical specification must be translated into a formal query, generally using a query language that will be used to identify patients in the study group. Query engines include the now-familiar Bing and Google, and the search engines of the National Library of Medicine, including PubMed. For data managers familiar with relational databases, the Structured Query Language (SQL) is a universal query language.

It is frequently true that electronic sources of registry data do not narrow the study group as much as desired. This may require investigating a larger group of candidate patients and selecting by medical records review those who meet the study specifications.

If the semistructured information model is adopted (see “ Semistructured Information Model ” in Section II ) as we advocate, then at the present time, experts in query languages specific to this type of information must be consulted. However, because such information is stored in the same format as Web documents, the increasing sophistication, accuracy, and usability of Internet search engine technology will simplify this process.

The end result of a query is identification of a group of patients (or candidate patients) for the study. The major checks here are whether the patients indeed meet criteria for inclusion and exclusion and whether a sufficient number of patients (or a sufficient number of outcome events, as will be found in subsequent steps; see Box 6-4 ) are retrieved for a meaningful analysis.

Extract Values for Variables

A source, or sources, for obtaining values for the set of variables specified in the clinical research proposal must now be identified for the study group (see Fig. 6-4 and Box 6-6 ). Currently these are contained either in some electronic format (e.g., hospital information system) or in the paper record.