Executive Summary
Emerging standards for regulatory use of RWE should resist the temptation to impose the standards and expectations of randomized clinical trials to a fundamentally different kind of evidence, Dreyer cautions.
As the 21st Century Cures Act’s vision for real-world evidence lumbers from legislation to application in the reality of regulatory decision-making, the Pink Sheet asked an expert in the use of real-world data, IQVIA Real-World & Analytic Solutions Global Chief of Scientific Affairs Nancy Dreyer, about the hurdles still ahead.
Dreyer has had a front-row seat to for the ongoing conversation about the role of RWE in FDA’s regulatory decisions as an active participant in the meetings and workshops that have been conducted by Duke-Margolis Center for Health Policy, the National Academies of Sciences, Engineering and Medicine, and the Friends of Cancer Research.
“We’re spending a lot of time talking with FDA about ‘how do you know when you can rely on real world evidence?’” Dreyer said. Indeed, she continued, that question is a “big one for everybody.”
“FDA is a big organization,” Dreyer said. “The people in surveillance and epidemiology are very comfortable with [RWE] and very sophisticated … more so than the other agencies and more so than many than most of the customers” in the pharmaceutical industry.
“The questions they are asking are all the right ones now. They have a lot of experience using health insurance claims through Sentinel and they have come to understand what outcomes they feel are reliable, which ones are not, which ones need more validation. And that’s where we’re going.”
The Office of New Drugs seems to have less comfort with RWE, she commented. Definitions and terminology remain hazy; Dreyer recalled how FDA officials at the Drug Information Association annual meeting characterized OND’s past experience with RWE of efficacy differently, with one speaker saying the agency has “never used real world data comparators in a decision” and a speaker in another session saying “we use external comparators, real world data comparators, for cardiovascular and oncology.”
Nonetheless, “FDA is consistent and clear” in its interactions with sponsors, Dreyer said. “The message they say is, come talk to us early. Come talk to us soon.”
Industry, however, has been reticent about bringing their RWE plans to FDA. “Pretty much every pharmaceutical company I work with, and it’s a very large number, doesn’t want to do that,” she said. “I’m not sure I understand quite why. But they don’t want to do that.”
[For further discussion of RWE, please tune into a free webinar on Feb. 14 where Pink Sheet and Scrip editors will talk to industry leaders about how they are approaching the field.]
Dreyer did not know of any sponsors who failed when they brought RWE efficacy proposals to FDA. “What I am seeing is, companies they get discouraged internally.”
“The biopharmaceutical world is a cautious group,” Dreyer observed. She noted her experience as head of IQVIA’s Center for Advanced Evidence Generation, which gets called on “every time somebody wants to do something novel or new.”
“We hear tremendous interest in pragmatic trials, in these external comparators,” she continued. “And what they always come down to is someone saying, ‘show me the precedents in my indication.’ And whatever you give them as precedents, they need something a little more specific.”
“I think it’s going to take actual cases” of RWE successfully supporting regulatory efficacy decisions to remove this widespread caution, Dreyer said. “This is a long-standing issue: when is data good enough?”
“What we are hearing from FDA” is that “sufficiency in quality depends on the data, and methods, and the questions.”
Regulators will ask different questions of RWE than payers ask, she said. RWE has been used in reimbursement decisions “for decades,” she said – “not that they will tell anybody exactly what they’ve been looking at.”
Now the regulatory audience has an opportunity to drive more of a focus on data quality, “because they are making different kinds of decisions.” Regulators “need more granularity in their data, and they need more detail than always the payers do. And the decisions that they are making will [rely on] different outcomes, than [the outcomes at the] top of the list for payers.”
RWE, Not RCT
“You see people tossing around words like ‘regulatory-grade’ data, or as Duke-Margolis talked about, ‘research-ready real-world data,’” she observed. However, she emphasized, the focus on “high quality” data can detract from the ways RWD and RWE can offer an alternative to the traditional randomized clinical trial paradigm, recalling how in early Duke-Margolis draft documents, “by the time they were describing what they meant by ‘high quality,’ you could have substituted ‘RCT.’”
“That’s not going to help us,” she said. “We need to make sure that we are not driving standards for real-world evidence that make it just like clinical trial data. Otherwise, we won’t be fulfilling the goal of the 21st Century Cures Act. We won’t be speeding drug development or bringing down the cost, if we can’t find a way to use the best of real-world evidence without turning it into a clinical trial.”
Dreyer pointed to treatment of missing data as an example of the differences between real-world and RCT settings. “The idea that you have no missing data makes no sense” for routine care settings. In a clinical trial, “if you are collecting data on a case report form, you fill out all the necessary boxes, and even then there’s a lot of querying to get things right,” she said. “When you’re using an electronic medical record, why would you expect it to be 100% complete for what your study needs?”
“Data curation, what it means, is a big black box,” Dreyer said. “I think that biopharmaceutical companies are worried that [real-world data] won’t be good enough.” And yet, “the short and long of every one of these real world exercises always comes down to the importance of not just the data, but the analytics. And sensitivity analysis is the single most important tool we have, I think, in our armament, because we say, ‘what if this missing data was 100% this way? Or 100% opposite? How wrong could we be? How much could that impact the results?’”
The IQVIA exec recalled one of the RWE test projects, “a very cool exercise” initiated by the Friends of Cancer Research to see if real-world data on advanced non-small cell lung cancer patients treated with immune checkpoint inhibitors matched randomized clinical trial data. The pilot project was presented July 10, 2018 at a Friends of Cancer Research-sponsored RWE conference. (Also see “US FDA Wants More Examples Of Real-World Data Use” – Pink Sheet, 5 Aug, 2018.)
FDA continues to be interested in the PD-1/L1 space as a testing ground for RWE; FDA Commissioner Scott Gottlieb recently revealed a project under the agency’s Information Exchange and Data Transformation (INFORMED) initiative to examine the impact of a labeling change from weight-based to flat dosing for two immune checkpoint inhibitors on community practices in collaboration with Flatiron Health Inc., a health information technology company acquired by Roche last year. (Also see “FDA’s Real World Study On Adoption Of PD-1 Dosing Change Aims To Inform Reviewers ” – Pink Sheet, 10 Feb, 2019.)
FOCR “put together six volunteering companies who had information, health insurance claims or medical record data, and we all followed a common protocol,” she said. “We agreed for this collaborative effort, not to go to any effort to clean up our data. No natural language processing, no mandatory reviews, just take what you got, apply these rules.” She noted that IQVIA contributed two datasets out of the six.
“Our data scientists aren’t 100% sure everyone compiled the cohorts the same way,” she said. “But what’s really amazing is that the real world data matched the RCT data pretty well, for overall survival… because as a real world data scientist, I know you don’t come to the doctor to tell them you’re dead. The idea of how we tweak and use real world data to figure out death, we did that pretty well. We were right around the same point as the RCTs.” Dreyer discussed the challenge that death poses for real-world data at the FOCR meeting. (Also see “Real-World Evidence Challenges: Death Among Toughest Data Points To Measure” – Pink Sheet, 14 Aug, 2018.)
“For time to treatment discontinuation, we were in general a little bit longer” than the RCTs, she said. Dreyer recalled that Amy Abernathy, then an executive at Flatiron Health, suggested a reason for that discrepancy: “in real world situations we’re seeing trends of people treating beyond progression, thinking that maybe just a little more of the drug would make a difference.”
Abernathy is now ascending to the number two position at FDA, principal deputy commissioner, where her deep experience with RWE is expected to inform agency policy. (Also see “US FDA Taps Abernethy As Principal Deputy Commissioner; Health IT Exec Brings “Real-World” Expertise” – Pink Sheet, 17 Dec, 2018.)
Privacy And Other Perils
Broad social and ethical concerns about privacy will pose barriers to widespread RWE adoption even as technological advances like electronic health records make RWD more accessible, Dreyer suggested. “I think the privacy issues are more important than the technology issues,” Dreyer said. She pointed to the issue of interoperability, which she called “everbody’s favorite buzzword about why things don’t work.” However, she said, “in my experience it’s more complicated than that.”
“You have to think about the willingness of a health system to share their data,” she continued. “If you assumed a world where everything was interoperable, then the question is, ‘why or under what circumstances would a health system be willing to share their data on patients?’”
“That part of the model we haven’t really solved,” Dreyer said. “I don’t think we’ve figured out the health system incentives for collaborating yet.” Interoperable systems can allow researchers to structure RWD with a common data model, but first you have to ask “why Harvard Medical or Mercy or any health system wants to put their data up against somebody else.”
“We have a lot of great research institutions that do wonderful research, but they’re not so good at collaborating on a regular basis in large scale research,” she said. “An abundance of caution…outweighs the benefits of sharing, because you worry that you will be violating patient privacy, you worry about reuse of the data or uncontrolled use of the data, so it’s easier to say no.”
FDA’s regulatory decisions on efficacy that relied on RWE have been exclusively in rare diseases, where decisions have incorporated data that falls short of the RCT paradigm more as a matter of necessity than choice. (Also see “A Baker’s Dozen Of US FDA Efficacy Approvals Using Real World Evidence” – Pink Sheet, 7 Aug, 2018.)
But ultra-orphan diseases also produce special privacy challenges, because RWD could potentially expose patients. “Pretty much any ethical or institutional review board” will have concerns about putting data into the public sphere in such rare disease cases, Dreyer said, because “if you put out so much data on them, it will violate their privacy.”
Dreyer used FDA’s approval EMD Serono Inc.’s Bavencio (avelumab) for metastatic Merkel cell carcinoma to illustrate the privacy conundrum. The single-arm trial that supported the BLA was conducted with many exclusion criteria, including requirements that patients be free of distant metastases and be new to systemic therapy, that further constricted the already-small pool of people with the rare skin cancer. She referred to 11 years worth of real-world data from Europe from 2004-2015, which were used to construct the external comparator.
“They started out with 971 patients with metastatic merkel cell carcinoma, and by the time they had applied all the exclusion criteria, they had 34,” she said. “But that was what the single arm trial was, so you had to match them. There’s not much you can do in terms of modeling or simulations for those.”
[For more on FDA’s use of RWE in the approvals of Bavencio and another oncologic, Amgen Inc.’s Blincyto, see the Pink Sheet’s Drug Review Profile of the products.]
The promise of precision medicine, which allows for more targeted patient populations, means that such privacy concerns will only increase, she said. “We have the competing interests of patient privacy, which everyone certainly deserves, versus the need to make decisions on smaller and smaller subsets.”
Nonetheless, the patient perspective can drive use of RWE, especially for unmet needs. “We see that patients don’t want to be randomized,” Dreyer said. “If there’s no treatment, and your choice is flip of a coin I get in the study or not, well, doctors are going to look for other ways to get their patients the study drug.”
FDA’s accelerated approval of Blincyto (blinatumomab) for relapsed and refractory Philadelphia chromosome-negative B-precursor acute lymphoblastic leukemia, relied on a single-arm Phase II and a historical comparator of patients using the standard of care, she noted.
“They were doing their Phase III RCT, it was just taking a very long time to enroll because patients didn’t want to go in it, doctors didn’t want to put their patients in it, because [the drug] looked so good,” she observed.
Blincyto’s accelerated approval was later converted into full approval on the basis of the Phase III TOWER study, which confirmed the efficacy suggested by the Phase II data and RWE comparator. (Also see “When Will Real World Evidence Be Persuasive? FDA’s Temple Offers Perspective” – Pink Sheet, 29 Jun, 2017.)
European Perspective
When discussing Bavencio, Dreyer praised the European Medicines Agency opinion’s treatment of the external comparators. The EMA “made a big point of calling them benchmarks, not comparators,” said, because “this drug did look like for some people it made a huge difference, and they needed a benchmark just to make sure it wasn’t hearsay, that is in fact what you’re seeing in the real world.”
In general, Dreyer said her “opinion has been that the EMA was ahead” of the US in its use of RWE. “They are really not behind us, that’s for sure.” She praised the EMA’s adaptive licensing pilot, completed in 2016, which explored adaptive pathways that would allow a drug to be approved earlier based on limited data for restricted populations where its benefit-risk balance could be favorable; real-world data would be used to support the product’s wider use.
By the end of the two-year pilot, six applications had progressed to receive scientific advice from EMA in parallel with health technology assessment (HTA) bodies. (Also see “Adaptive Pathways: Companies Advised To Consider Hurdles As EMA Accepts More Applications” – Pink Sheet, 5 Aug, 2016.)
The EMA adaptive licensing pilot study “was just a great formal way to say, you’ve got a high unmet need, come to us,” Dreyer said. “You could bring a product early on, for a very limited indication, high unmet need and go forward.”
Discussions with HTA bodies, patients and “other downstream stakeholders would have to take place early on during drug development so as to make sure their demands were also met.”
EMA continues to explore what it calls “medicines adaptive pathways to patients,” or MAPPs, under the ADAPT SMART initiative with more than 30 international partners from across the medical system. (Also see “Adaptive Pathways: A Solution For Worried Payers in Europe” – Pink Sheet, 4 May, 2017.)
Taking The Plunge With Label Expansions
Dreyer predicted “a lot more use” of RWE to support label expansions for previously approved drugs, “where we know we’ve watched very closely to see the toxicity of the product and the broad safety profile.”
The 21st Century Cures Act directs FDA to issue guidance on use of RWE to support new indications for marketed products and to satisfy post-marketing study requirements by December 2021. The agency reached a milestone in the preparation for that guidance with the “Framework for FDA’s Real-World Evidence Program” released in December. (Also see “Real-World Evidence: US FDA Framework Emphasizes Data Fitness And Study Quality” – Pink Sheet, 9 Dec, 2018.) and (Also see “US FDA Is Hesitant About Using Observational Studies In Real-World Evidence Framework ” – Pink Sheet, 6 Dec, 2018.)
An established toxicity profile for a marketed drug “doesn’t mean it’s going to be safe for everyone,” she said, but importantly, “you have some level of comfort with the product.”
“Then,” Dreyer said, “the question is, ‘do we want to spend our money on drug development with pretty expensive trials where they will be hard to recruit and very very slow, or do we want to try some of these regulatory alternative methods of getting your comparators?”
The process of codifying terms and standards for RWE will not be easy and will require sponsors and regulators alike to take risks. Nonetheless, Dreyer concluded, “let’s use this opportunity to advance medical decision-making.”
https://pink.pharmaintelligence.informa.com/PS124737/RealWorld-Evidence…
