On February 4, 2020, the U.S. secretary of health and human services declared that emergency use of diagnostics for SARS-CoV-2 was justified, triggering emergency authority for the Food and Drug Administration (FDA) to grant an emergency use authorization (EUA) for a device if it reasonably believes that it may be effective, rather than waiting to grant full approval when it has reasonable assurance that the device is safe and effective. This mechanism expedites access to accurate diagnostic tests during emergencies, when information gaps and false results may adversely affect patient care and public health decision making.
The EUA process enabled molecular diagnostic tests to be developed, validated, and deployed within weeks rather than several months to over a year, as traditionally required. In January, the agency had begun engaging with commercial manufacturers of diagnostic test kits and laboratories to help foster test development. To streamline submissions, the agency developed an EUA template with recommendations on validating a molecular diagnostic test for SARS-CoV-2 and outlined the required information. By July 31, the FDA had authorized 163 Covid-19 diagnostic tests.
The EUA template also streamlined submission paperwork. A typical submission seeking full FDA approval for a test is about 1000 pages for laboratories and about 2000 pages for commercial manufacturers that distribute tests. The template reduced commercial manufacturers’ EUA submissions to 100 to 200 pages and laboratory submissions to about 40 pages, of which only about half were generated solely to meet FDA requirements, and most of those consisted of data rather than text.
Submissions for full approval and those for EUAs differ primarily in the extent and type of evidence required. For a Covid-19 EUA, the FDA initially permitted test performance to be demonstrated by a computer analysis indicating the percentage of identity matches with publicly available SARS-CoV-2 sequences that could be detected by the proposed molecular assay and cross-reactivity with other respiratory pathogens and by testing contrived samples. Developers could “spike” human specimens, such as sputum, with different amounts of extracted SARS-CoV-2 RNA or live or inactivated virus to assess viral detection, rather than using patient specimens. Validation could thus be completed rapidly once viral RNA or virus became available. However, this approach was less likely than use of patient specimens to accurately characterize test performance.
As positive patient samples became more readily available, the FDA transitioned to requiring the use of samples of known SARS-CoV-2 status. Once it became feasible to do so, we began requiring postauthorization validation using samples containing levels of virus ranging from low to high, to harmonize assessment and better establish true performance. Though ideally a standard set of patient samples would be used to establish relative performance among tests, implementing such a scheme is challenging because of the limited amount of sample obtained from each patient.
We therefore developed a reference panel of samples with different amounts of inactivated virus and samples whose SARS-CoV-2 status is unknown to developers. We are distributing this panel to makers of EUA-authorized molecular assays to allow determination of relative limits of detection (LODs). Once an international reference standard is finalized, the panel can be anchored to it and assessments can be updated to represent absolute LOD.
We believe that this approach provides a workable solution for assessing tests’ comparative performance. The requirement to assess authorized tests with FDA-sanctioned reference panels has been a condition of authorization for all SARS-CoV-2 tests to date. In addition, the Reagan–Udall Foundation for the FDA, in collaboration with Friends of Cancer Research, has started the Covid-19 Diagnostics Evidence Accelerator, which will leverage real-world data to elucidate the performance of diagnostic tests.
Although some Covid-19 test laboratories and kit developers had experience with previous emergencies, some developers and health care professionals were unfamiliar with the EUA process, which created misunderstandings and confusion. During the week of February 24, several laboratory groups sought permission to use laboratory-developed tests (LDTs) for patient care without obtaining an EUA. They cited unmet needs due to challenges with the rollout of the test from the Centers for Disease Control and Prevention (CDC) and the lack of alternative tests. Some labs suggested that they would not pursue test development, believing that EUA validation was too burdensome; others were unaware that the EUA pathway was available.
Recognizing the continued need for rapid expansion of testing capabilities, the FDA issued a policy on February 29 indicating that we would not enforce certain requirements for qualified laboratories that had commenced testing after validating a test, provided that the lab notified the FDA of its intention to test patient samples and submitted the validation data in an EUA request within 15 business days. We thus put labs developing their own Covid-19 tests on the honor system and prioritized early access over independent confirmation of accuracy.
Although this approach resulted in earlier test availability, the EUA’s less-rigorous evidence standard, coupled with delayed FDA review, allowed the use of several LDTs that ultimately proved to have performance problems or to be poorly validated. In analyzing 125 EUA requests from laboratories, we identified 82 with design or validation problems, and several have been denied authorization. In the majority of cases, the FDA worked with the laboratories to correct the issues and permit continued testing. Similar problems were seen with commercial manufacturers.
The process also revealed that some clinicians have a limited understanding of test performance. No test is 100% accurate, and performance can vary within populations. Covid-19 diagnostic tests may be less accurate in asymptomatic or low-risk populations and in persons who shed little virus or are early or late in the course of illness. Even if a test were 98% sensitive and 99% specific, it would still produce a false negative result in 2 of every 100 people infected. If we test 5 million Americans daily and only 1% of them have Covid-19, a total of 1000 positive cases will be missed, which increases the risk of spread, and another 49,500 people will receive false positive results. False positive results can be burdensome to public health officials who are tasked with contact tracing and other public health activities, and many people may be unnecessarily quarantined. An understanding of the positive and negative predictive value of a test should be factored into clinical decision making and patient counseling. To mitigate the impact of false results, all Covid-19 tests authorized to date have been made available only by prescription, so that clinicians can interpret results for patients. However, several nonprescription tests are currently under development.
As we have adapted our policies to changing circumstances, we have learned lessons that should inform our response to future outbreaks. First, we believe that the U.S. government should work with international partners to establish a plan for sharing clinical specimens as soon as a public health threat emerges. This effort could be aided by having appropriate international agreements in place in advance.
Second, when a public health threat warrants large-scale testing, it would be more effective to authorize a small number of well-designed, well-developed, and validated tests run on common high-throughput platforms, followed by a few point-of-care tests, all of which are manufactured in large quantities, than to simultaneously develop and authorize scores of diagnostics. Such diffuse efforts are an inefficient use of resources.
A handful of test designs could be developed or identified by the U.S. government alone or in collaboration with the testing community, then manufactured by the CDC, preset contract manufacturers, commercial manufacturers, and laboratories, which would speed up development, validation, authorization, production, launch, and testing. Use of contract manufacturers at the outset can greatly increase test availability. The FDA could proactively create validation protocols in collaboration with the test-developer community for commonly anticipated pathogen and sample types before an outbreak and modify them as necessary once an outbreak occurred. This approach would permit faster validation.
Third, we need common approaches to validating test design and performance, regardless of whether there is an emergency. Our experience with Covid-19 highlights the need for a common legislative framework to ensure that all clinical tests are accurate and reliable.
Fourth, the clinical community should understand test performance and how to use that information in patient care. Tests should be accompanied by clear, standardized, comprehensible information on performance for clinicians and patients. Training and continuing education can enhance physicians’ understanding of test performance, selection, interpretation, and clinical usefulness.
As we continue to address Covid-19, it’s important to reflect on what we’ve learned to better prepare ourselves for current and future epidemics.
