Clinical Trial Design Methodology and Data Analytic Strategies for Pain Outcome Studies

Case Report and Case Series

A case report is useful, for sharing with the medical community, novel, rare, or atypical features identified in medical practice in an individual patient. It has historical significance as a type of clinical study that dates back by millennia. However, since the mid-20th century, it has waned in terms of importance, in favor of more rigorous, statistically, methodologically relevant study designs. Nonetheless, publication guidelines such as Consensus-based Clinical Case Reporting Guideline Development have been developed by international groups of experts to promote the accuracy, transparency, and usefulness of case reports.

Currently, the case report remains a useful educational tool to generate new investigational questions and support learning methods rooted in critical thinking and problem solving. Case reports are also useful in pharmacovigilance because they facilitate sharing rare, unexpected, or long term adverse effects that may have gone undetected during clinical trials. Case reports contributed to the evidence that led to 18 of 22 drugs being withdrawn from the Spanish market between 1990 and 1999. ^, Important limitations remain, namely the lack of controls, uncertainty of a follow up, and the fact that it is difficult to generalize findings from a single patient.

A case series can help address some of these concerns by increasing the sample size. A case series describes the medical outcomes of a group of patients with a particular disease or disease-related outcome, often over time. As with a case report, a case series comes with the primary limitation of the lack of a control group and potential sampling bias. Risk ratios cannot be calculated based on such evidence.

An example of a case report, which represented the first reported occurrence of such an event, describes a patient who experienced an extremely rare and unpredictable thalamic stroke resulting in the disappearance of a pre-existing chronic pain condition, following which the patient’s chronic LBP completely disappeared. Such unique cases may provide insight into neuromodulation, neuroplasticity, and brain stimulation.

Cross-Sectional Study

A cross-sectional study determines the prevalence (i.e. the existing presence) of a disease at a given point or period in time. If a sample is randomly chosen, such a frequency survey provides a valid “snapshot” of the characteristics of the source population. It can provide an estimate of the magnitude of a problem and thus the significance and rationale for further investigation. Of note, a cross-sectional study’s measurements are taken at a single point in time, whereas longitudinal studies conduct several observations over time. A cross-sectional study cannot assess possible causality between a predictor variable (e.g. sex and socioeconomic status) and an outcome variable (e.g. pain intensity). While a cross-sectional study cannot assess the comparative outcomes (benefits and risks) of an intervention, the findings of a cross-sectional study can generate hypotheses for further ­studies. ^{, ,}

For instance, a cross-sectional study aiming to investigate the frequency and features of painful diabetic polyneuropathy enrolled 816 patients attending hospital diabetic outpatient clinics. Diabetic polyneuropathy was diagnosed in 36% of patients, strongly correlating with male sex, age, and severity of diabetes. A subset of these patients, 2.5% of the study population, with pure small-fiber polyneuropathy, was found to be unrelated to either the demographic variables or the severity of diabetes. On the contrary, painful polyneuropathy was correlated with the female sex and was diagnosed in 13% of the study population. As discussed, cross-sectional studies cannot determine causality. However, demonstrating that this form of neuropathic pain correlates with sex offers insights that may contribute to improving pain management. Cross-sectional studies can be used to observe or compare individuals and groups. For example, a cross-sectional study examining social health determinants in patients following total knee arthroplasty divided their participants into two groups based on whether they had chronic postsurgical pain. This allowed them the investigators to see that patients with a lower educational level had a three-fold higher association with chronic post-surgical pain.

Case-Control Study

A case-control study aimed to determine the relationship between a single outcome (disease) and one or more previous possible contributing factors (exposures). Case-control studies look or work backward, starting with the identified outcome or disease (exposure ß disease). Case-control studies are retrospective, relying upon existing data or recall of exposures, and are therefore generally less time consuming and less costly to perform than studies using a prospective design. This study design is particularly suited to outcomes that are rare or have a protracted onset. Cases are patients with the outcome of interest, whereas controls are patients without the outcome of interest. The controls must be drawn from the same source population (study base) as the cases, such that the controls are comparable to the cases in all important respects except for the outcome of interest. Thus controls must also be chosen independently of their exposure status.

Primary validity threats with a case-control study include selection bias (non-comparable controls without the disease) and recall bias (differential recollection of exposure among cases with the disease). Overmatching can also occur if the cases and controls are matched on a variable that is also associated with the exposure of interest. Thus if someone matches a factor that is affected by both exposure and a cause of disease, this will obscure disease ­association.

Because cases and controls are selected for analysis based on the presence or absence of outcomes, there is no true denominator in a case-control study. Therefore the event rate, relative risk, and risk ratio cannot be calculated. Instead, the odds ratio (the ratio of the odds of exposure in the cases and the odds of exposure in the controls OR) is conventionally generated as a measure of the association between exposure and disease in a case-control study. When the outcomes occur at an infrequent rate, the odds ratio should approximate the relative risk. ^,

A case-control study was performed to quantify the degree of overlap between facial pain and pain experienced in other parts of the body. Data were collected from participants in a large prospective study, including 424 cases of chronic facial pain and 912 controls. They reported pain at other locations in the body using a mailed questionnaire. The findings were that facial pain had the greatest overlap with headaches (OR = 14.2, 95% CI = 9.7–20.8), followed by neck pain (OR = 8.5, 95% CI = 6.5–11.0). For locations below the neck, the overlap decreased substantially (OR ≤ 4.4). A limitation of this study was its reliance on self-reported symptoms.

Nested Case-Control Study

A nested case-control study is a case-control study in which the cases and controls are selected from members of an existing cohort. The members of the base cohort are followed for a certain period until the specific outcome or end-point occurs. ^, The nested case-control design differs from the non-nested case-control design in that patients are selected from a well-defined cohort, for which data on all members can be obtained. This design makes it easier to satisfy the fundamental assumption of the case-control study: cases and controls represent unselected samples from the same study base. In addition, a nested case-control study can be designed to prevent attrition bias. If all data are collected prospectively before the outcome occurs, a nested case-control study is also unlikely to be affected by recall bias, even though the cohort does not need to be prospective.

The following is an example of a nested case-control study designed as a follow up to a 5-year cohort study, aiming to determine how headache types and patterns over time can be associated with the risk of developing temporomandibular disorder (TMD). The cohort study followed 2410 patients who were TMD-free at the beginning of the study. The ensuing nested case-control study matched the cohort of 248 incident TMD cases with 191 TMD-free controls. The findings showed that both headache prevalence and frequency increased among patients who developed TMD compared to controls. In particular, in terms of headache frequency, definite migraines among TMD cases increased tenfold, suggesting the usefulness of future studies in investigating whether the treatment of migraines may reduce the risk of developing TMD.

Cohort Study

A cohort study aims to determine the relationship between a single contributing factor (exposure) and one or more possible outcomes (diseases). Cohort studies look forward starting with the identified exposure (exposure à diseases). Cohort studies are, by definition, longitudinal. However, data collection can be prospective, retrospective, or ambispective (utilizing both retrospective and prospective data from the same study population). The key design element is that all participants in a cohort study are outcome or disease-free prior to exposure (intervention) and at the outset of data collection. The outcome data are then sequentially collected.

Prospective and retrospective cohort studies may be less prone to recall bias than case-control studies but are subject to differential attrition bias, whereby participants in one group exit the study at a disproportionate rate. Cohort studies are also vulnerable to confounding by indication, a form of selection bias in which a variable is a risk factor for a disease among the non-exposed individuals and is associated with the exposure of interest in the population from which the cases are derived, without being an intermediate step in the causal pathway between the exposure and the disease (e.g. a marked increase in gastrointestinal bleeding in patients prescribed a nonsteroidal anti-inflammatory drug [NSAID] plus a proton-pump inhibitor versus only an NSAID). ^,

Prospective cohort studies are also typically more time consuming and costlier than case-control or other observational studies, especially for rare outcomes (diseases) and those with a protracted onset. The cohort study allows for direct estimation of the incidence (new event) rate, relative risk, and relative risk ratio. ^, Of note, an RCT is a type of prospective cohort study with random assignment to exposures (interventions).

A prospective cohort study was undertaken to identify predictors of moderate to severe persistent postoperative pain (PPP) 6–12 months after total knee arthroplasty (TKA). The study population consisted of 300 patients undergoing primary unilateral TKA. Prior to surgery, a wide array of data was collected from the patients, including clinical information, psychological variables, quantitative sensory testing, and blood samples for genotyping, in addition to information on surgical factors related to the operation and measurements of acute postoperative pain. Thereafter, 6–12 months after surgery, the patients completed follow up questionnaires with minimal loss of participation, keeping the attrition bias low. All of this information was then analyzed using multivariate logistic regression to identify the predictors of PPP. The prevalence of moderate to severe PPP, 6 and 12 months after surgery, was 21% and 16%, respectively. At six months, 66% of patients were correctly classified as having moderate to severe PPP using a combination of the following predictive variables: preoperative pain intensity, expected pain, trait anxiety, and temporal summation. At 12 months, the same first three predictors correctly classified 66% of the patients.

Non-Randomized Controlled Trial

In a non-randomized controlled trial (NCT), assignment to treatment and control groups is performed by the investigators but not through a randomized process. In contrast to RCTs, an NCT lacks the critical component or randomization, making them more susceptible to biases and confounding factors. An inherent shortcoming of NCT is that it is difficult to determine whether differences in outcomes between interventions are because of the interventions themselves or other confounding variables.

Despite this important shortcoming, an NCT may be helpful in some situations in which RCTs are not feasible because of high cost, ethical considerations, lack of participants willing to be randomized, or other factors. NCTs can be useful for identifying potentially effective interventions that warrant an evaluation in RCTs. NCTs are sometimes included in meta-analyses and used to inform medical decision making, but research is needed to understand when NCTs are more likely to be reliable and when they are likely to be misleading.

One such experimental controlled trial that relied on a non-randomized design aimed to determine the benefit of a 10-week shoulder home-based exercise program (SHEP) in a group of elite wheelchair athletes. The participants were assigned to either the exercise or control group in a non-random, mixed-pair design, where each participant was matched to a control in terms of two variables: whether they used the wheelchair for daily activities or only for sports, and whether they experienced shoulder pain or not. This design allowed the researchers to compare the baseline and post-intervention changes more accurately, both in terms of shoulder pain and range of motion. The 10-week SHEP did not provide significantly different changes in either outcome between the experimental and control groups. The limitations relating to non-randomized allocation and small sample size apply when it comes to the interpretation of this study’s results in the context of similar studies.

Randomized Controlled Trial

An RCT is a type of controlled clinical trial based on participants being assigned to the possible exposures or interventions purely by play of chance (“flip of a coin”) using tools such as computer-generated randomization sequences. When successfully implemented, random allocation produces two or more treatment groups that are free of selection and confounding bias by both known and unknown factors, assuming that the sample size is sufficiently large and randomization is carried out successfully. When those involved in an RCT are blinded (masked) to the intervention each participant receives, allocation, treatment, and assessment biases are also reduced. Thus when properly designed and performed, an RCT is more likely to have internal validity. It accurately and reliably measures what it is intended to be measured, that is, the relative benefits and harms of two or more interventions.

Patient blinding is a critical factor contributing to the high quality and internal validity of a well-designed experimental trial. Moreover, double-blinding is achieved when both the patient and the investigator (if it is impossible to blind the investigator, the outcome assessor, data collectors, or data analyst may be blinded) are blinded to the treatment allocation. Successful double-blinding helps prevent response bias, such as observer bias and confirmation bias, which can be caused by the experimenters or the participants being influenced by their expectations of the results. In the case of either pharmacologic or non-pharmacologic interventions for chronic pain, blinding is rarely reported and has often been found to fail, primarily because of the large treatment effect sizes and high rates of adverse effects.

An example of an RCT trial on chronic LBP investigated whether adding an open-label placebo (OLP) to treatment as usual (TAU) could benefit patients. There are ethical concerns surrounding the study of placebo effects because of the widespread belief that deception is required in order for the placebo effect to take place. In this study, 97 participants consented to take OLP for three weeks following TAU treatment. The consenting participants were randomized to TAU or OLP, and the treatment assignment was revealed to both the participants and investigators. Only registered nurses who completed the assessments were blinded. The findings showed that OLP enhanced pain reduction by approximately 30% of the baseline pain and disability ratings. More specifically, OLP provided a pain reduction of 1.49 points on a 0–10 scale of pain intensity, compared to a 0.24-point change provided by TAU alone. This study has some important limitations, primarily in terms of reporting bias, because of the utilization of self-appraised outcomes combined with an open-label design. Additionally, the highly positive response to OLP may be explained by the encouraging educational information the participants were provided with placebo effects, combined with unconscious processes associated with physically opening pill bottles and taking pills. Expectations of pain relief may have encouraged participants to differentially interpret normal spontaneous fluctuations in pain. A double-blinded study design is important for reducing unwanted placebo effects related to expectations of benefits or harm (nocebo effects). An alternative strategy is the use of an active placebo, which produces side effects intended to persuade the patient that they are receiving the active compound.

An enriched enrollment randomized withdrawal (EERW) trial is a variation of the traditional RCT that has been introduced for studying chronic pain, particularly pharmacologic therapies. The first step of an EERW is the enrichment phase, where only the patients who respond to the drug and tolerate the side effects will progress to the next phase. In the withdrawal phase, these patients are then assigned, in a double-blind, randomized manner, into a group that will continue taking the study drug, and the control group will be switched, through titration, to a placebo or another comparator. ^, The EERW design may be considered an extreme form of efficacy study since it is intentionally designed to evaluate a highly selected patient population known to respond to and tolerate the treatment of interest. Furthermore, EERW studies reflect a highly selected population and are not representative of clinical practice, as the response to the drug cannot be predicted, and treatment would not normally be interrupted in patients who respond well. Additionally, patients withdrawn from treatment following randomization may be aware that they are no longer receiving the study drug, which may cause performance bias by influencing their response to the placebo. Therefore the interpretation and utility of EERW studies have been controversial.

Cluster Randomized Controlled Trial

A cluster randomized trial (CRT) is a study design in which individuals are randomized into groups; the group as a whole is randomized, and not the individual. Physicians, group practices, health plans, or even geographic regions (counties or states) can be defined as clusters. In a CRT, all individuals within a given cluster are assigned to the same study arm. The methodology of CRTs has been widely discussed. In evaluating a therapy, unlike a standard (individually randomized) RCT, a CRT may be better suited at replicating conditions of actual use in clinical practice to produce more generalizable findings.

CRT is often performed when individual randomization is not feasible. For example, if a clinician is implementing a new LBP evaluation and treatment study protocol, it would be challenging to randomize some patients in a clinic to one protocol and other patients to another protocol because there is a substantial amount of shared knowledge of the different protocols and overlap among clinic personnel. CRT can also offer cost and time efficiencies over individually randomized trials.

However, compared with individually randomized trials, CRTs are more complex to design, require more participants to obtain equivalent statistical power, and require more complex analyses (e.g. adjustment for the intracluster correlation coefficient of the cluster randomization). A CRT is also typically not blinded. Thus not surprisingly, the use of this trial design creates challenges that may undermine the validity of some findings. ^,

For example, a CRT investigated the benefits of a digital health psychological intervention (WebMAP) in comparison to usual care alone in children and adolescents with chronic pain. The 143 participants were patients selected from eight different clinics. The clustered randomization design that stems from the clinics determines whether they would initiate the WebMAP intervention two, four, six, or eight months after study commencement. The findings showed that neither WebMAP nor usual care showed significant changes in pain related disability and pain intensity over time. However, youths in the WebMAP group perceived a significantly higher global improvement both post-treatment and at follow up (p ≤ 0.01). Additionally, greater engagement with the WebMAP modules was associated with a significant reduction in both pain related disability and pain intensity. The clustered design of this study was used to implement the intervention. However, possible limitations are the potential differences in the amount or type of care provided at each clinic under usual care control. Nonetheless, the WebMAP intervention was provided through mobile access and was consistent among the groups.

Meta-Analysis

A systematic review involves a detailed and comprehensive search strategy to identify and appraise all relevant studies on a particular topic. A meta-analysis may then be performed to combine and compare the results of multiple scientific studies using statistical techniques to synthesize data into a single estimate or summary effect size. Moreover, when conducted rigorously, it may provide the strongest synthesis or summary of evidence. Meta-analyses typically feature data from published RCTs. However, they can also include non-randomized, observational, and even unpublished studies.

A conventional meta-analysis makes direct comparisons between trials in a pairwise manner. More recently, network meta-analysis has emerged, which incorporates evidence from indirect comparisons using logical inference (i.e. comparing A to C using studies that compare A to B and B to C). The first step of a systematic review is a specific and well-formulated hypothesis, followed by a thorough and extensive literature review. The inter-judge reliability of those involved in the review and selection of studies for inclusion is an important component of meta-analytic reports. The next step is to evaluate the quality and strength of evidence for each study, based on the robustness of the study design and minimization of bias. Pooling of studies in meta-analyses can be performed using a variety of statistical methods. This is discussed in the statistical section of this chapter.

Several forms of bias could arise in a meta-analysis. One of the most important is publication bias because studies with positive, statistically significant results are more likely to be published, promoted, and thus identified in literature searches. However, meta-analyses are also one of the ways that enable us to detect publication bias, which is not possible when looking at individual studies. The principal methods for detecting publication bias are typically based on the detection of small sample effects. This refers to the fact that the findings of larger studies are more precise and tend to cluster closer to the “true” point estimate, whereas smaller studies are less precise, with more variability in point estimates. “Small sample effects” refers to unbalanced estimates from studies with small sample sizes that can be a marker of publication bias. Small sample effects can be detected through graphical methods (funnel plots) or statistical methods (e.g. Egger test). Clinical heterogeneity is common among meta-analyses because confounding factors, uneven blinding, and other effect modifiers are inconsistent among studies. This reinforces the importance of stringent inclusion criteria and quality assessment using available guidelines.