Executive Summary
Synthetic control group derived from historical clinical trial data could augment smaller randomized trials and yield better information than single-arm studies, but this approach should not be viewed as a substitute for randomized trials where feasible, US FDA officials said at a Friends of Cancer Research meeting.
Use of a synthetic control arm drawn from historical clinical trial data could provide better information about a new investigational agent’s safety and efficacy than single-arm studies and allow sponsors to conduct randomized trials that are smaller, or with more patients assigned to the investigational drug.
However, such an approach should not be considered a replacement for randomized, controlled trials when they are feasible to do, US FDA officials said at the recent Friends of Cancer Research (FOCR) annual meeting.
The comments came during discussion of a white paper on development of a synthetic control arm, based on data drawn from historical clinical trials, to serve as a control group for a current-day trial. The white paper was developed by a multi-stakeholder working group consisting of representatives from FDA, the biopharma industry, data science sector, academia and the patient community.
While FDA officials generally were enthusiastic about the possibilities of using external controls derived from historical trials when randomized studies are difficult to do, they sounded a note of caution, including the need to ensure similarity of the patient population and standard of care between the historical data and current-day study of an investigational therapy.
“Using externally controlled data in places where today we are accepting single-arm studies is certainly better than single-arm studies,” said Rajeshwari Sridhara, director of FDA’s Division of Biometrics V. “But we should not … say this is going to be the new standard and don’t think about randomized, controlled studies.”
Randomization Challenges
Randomized trials, particularly for breakthrough therapy products approved under the accelerated approval pathway, may not be feasible for various reasons. Patients may be unwilling to enroll in, or continue on, confirmatory trials of the accelerated approval agent because of concern they will be assigned to what they view as suboptimal therapy in the control group. Also, trial results can be confounded by patients in the control arm crossing over to the investigational drug.
There also are some clinical settings, such as rare diseases, where randomized trials are not possible due to scarcity of patients or ethical concerns. In such cases, indications often are studied in single-arm trials.
“One approach to circumvent these challenges introduced by loss of equipoise is to consider the use of historical data to facilitate the conduct of clinical trials,” the white paper states.
Synthetic Control Derived From Pooled Studies
A panel at the FOCR meeting discussed two case studies that looked at whether data from the control arms of historical trials could mimic the results of a traditional, randomized control.
One case study focused on building a synthetic control arm based on patient-level data from historical studies in previously treated, advanced, non-small cell lung cancer (NSCLC), and comparing that arm to the control group of a current-day trial in the same disease.
Propensity score matching allows for the creation of “something that looks a lot like a randomized control for a setting where randomization is problematic.” – Medidata’s Davi
Patients in the synthetic control arm were selected according to baseline characteristics and prognostic factors to statistically match those of the randomized control patients in the more recent trial. Propensity score matching was used so that the distribution of baseline characteristics between the synthetic control arm and the randomized control arm were well balanced.
“This allows the apples-to-apples comparison that we need,” said Ruthie Davi, vice president of data science at Medidata Solutions Inc. “In this way, we create something that looks a lot like a randomized control for a setting where randomization is problematic.”
Medidata provides cloud-based solutions for clinical trials in the life sciences. The company previously has been involved in creating a synthetic control arm in the acute myeloid leukemia space. (See sidebar for story.)
In the NSCLC case study, patients in the randomized control arm who could not be matched with a patient from the historical pool were excluded from further analysis.
“This thought of excluding patients from the analysis goes against the intent-to-treat principle and may lead you to think there could be an imbalance between the two groups,” Davi acknowledged. “That would be true if this were a randomized, controlled trial. But this is not a randomized, controlled trial … and removing patients from this group actually improves balance rather than threatens it.”
After matching was complete, there was significant overlap in the overall survival curves of the historical control arm and the control arm in the current-day trial, suggesting that the synthetic control arm successfully replicated the results of the randomized control.
These results, Davi said, open the possibility that this approach could be used in future trials to augment or replace a randomized control when the latter would be difficult to do. “We think this is a very important step in mitigating the challenge we see with conducting concurrent control arms for these difficult-to-study indications,” she said.
Future work, however, is needed on this approach, Davi said. The working group plans to assess whether the investigational drug’s treatment effect can be replicated with the synthetic control arm by looking at whether the difference between the historical control and patients on investigational agent in the current day trial matches the difference between the randomized control and the treated patients.
In addition, more work is needed to tweak matching methods to reduce the proportion of patients who are not matched and are excluded from the analysis, Davi said. The approach needs to be tried in other indications as well, she said.
Single-Trial Historical Control
While the synthetic control arm example involved the use of data from multiple historical trials, Pallavi Mishra-Kalyani, a team leader in the Division of Biometrics V, presented results from a second case study that used external control data from a single melanoma study to identify a treatment effect.
The purpose was to determine whether the vemurafenib (Genentech Inc.’s Zelboraf) monotherapy arm in a vemurafenib vs. dacarbazine trial could match the control arm of a second study in which vemurafenib was compared to cobimetinib (Genentech’s Cotellic) plus vemurafenib.
The case study showed there was promise to using external data from one clinical trial to substitute for the control arm in another, Mishra-Kalyani said. She noted that the inclusion/exclusion criteria across the two trials were very similar, and the studies enrolled patients from a similar population.
Discussing considerations for using an external control in general, Mishra-Kalyani highlighted the need to be able to compare the endpoint between trials. In addition, temporality of the data are important, as even small lags in time between studies may make a big difference, she said.
Encouraging Results, But More Experience Needed
Joohee Sul, a medical officer in FDA’s Office of Hematology and Oncology Products, said she was surprised at how well the external controls matched up to the current-day controls, but that a larger body of experience with this approach is needed.
The slogan “reduce, reuse, recycle” came to mind during this project, Sul said. “I think it’s very helpful to think about trying to squeeze as much usefulness out of the existing clinical trial data that we have.”
FDA’s Sridhara cautioned about the need to be aware of temporal drifts in the control arm with respect to subsequent therapies and clinical practice.
FDA’s Sridhara said that while the two case studies suggest use of a synthetic or external control arm can work, caution is warranted given the impact that changes in clinical practice and standard of care could have on the relevance of historical trial data.
“We need to be aware of … temporal drifts or shifts that can happen in the control arm” with respect to subsequent therapies and changes in how available therapies are used in practice, Sridhara said. “While this is encouraging, I would say we have to examine a broader spectrum of clinical trials.”
Augment, Don’t Replace
Several speakers suggested a primary opportunity for use of a synthetic control approach may be found in therapeutic areas where the standard of care has changed little over the years.
“There are a lot of very fast-paced developments in oncology, but there are also groups of patients who have been taking the same treatment for year and years and years and years,” Mishra-Kalyani said.
FDA officials said that a synthetic control arm or external control could help augment new clinical data by allowing sponsors to reduce the number of subjects assigned to the control arm in a randomized trial, or to conduct smaller randomized trials.
“We don’t always require 1:1 randomization,” Mishra-Kalyani noted. “You could have a small, concurrent control arm that’s augmented with some historical controlled data, particularly data from previous clinical trials,” she said.
A major benefit of the synthetic control arm approach could be enabling a higher percentage of patients to get assigned to the investigational drug arm of a randomized trial, Davi said. “I know we do 2:1 randomization or maybe 3:1 randomization now, but we could do something even better than that, like 5:1 or 6:1, if we’re replacing with historical data as well.”
External or synthetic control arms fill the space between single-arm studies and randomized trials, Sridhara said.
“It is certainly better than single-arm studies, which we are already accepting,” Sridhara said. In addition, using prior clinical trial data to establish an external control is preferable to relying on other sources of evidence, such as case histories and the published literature, because “data that’s coming from a clinical trial is more curated already and we have a strict eligibility criteria, we know exactly how they measured endpoints, which may not be there in other sources of data.”
In diseases and populations where a randomized study can be done, it should be done, Sridhara said. “But if it’s not possible, or if it’s a place where the standard of care has remained the same forever … can we use that data to augment the concurrent control? Those are excellent places where you can look at that,” she said.
“I don’t want to preside over the death of the randomized trial.” – FDA’s Pazdur
The panel’s industry representative, Antoine Yver, global head of oncology research and development at Daiichi Sankyo Co. Ltd., said augmenting “is a fantastic concept” but industry and global regulators need to decide together what that means in this context and what regulators would be willing to accept.
“I don’t think we are ready to replace” randomized trials, Yver said. Instead, he suggested the synthetic control arm would be well suited to situations where randomized trials are problematic.
“We need to be a little bit bolder” in recognizing that this approach may eliminate randomizing subjects to less-than-optimal therapy and confounding due to crossover, he said.
Speaking from the audience, Oncology Center of Excellence Director Richard Pazdur said that while this approach may be appropriate in certain circumstances, “I don’t want to preside over the death of the randomized trial.”
“This needs further discussion in the statistical community, believe me, before we say we’re not going to be doing any more randomized trials,” he said.
Using Real World Data For External Controls
While the two case studies discussed at the FOCR meeting involved control arms derived from historical trials, FDA said it is considering issuing guidance on use of real-world data (RWD) to generate external control arms. (Also see “Real-World Evidence: US FDA Framework Emphasizes Data Fitness And Study Quality” – Pink Sheet, 9 Dec, 2018.)
FDA’s real-world evidence (RWE) framework released Dec. 6 discusses the potential use of single-arm trials with an external RWD control to support new effectiveness claims and the limitations of such an approach. “Collection of RWD on patients currently receiving other treatments, together with statistical methods, such as propensity scoring, could improve the quality of the external control data that are used when randomization may not be feasible or ethical, provided there is adequate detail to capture relevant covariates,” the framework states.
https://pink.pharmaintelligence.informa.com/PS124362/External-Control-A…
