Summary

Brittany McKelvey, PhD, Director of Regulatory Affairs at Friends of Cancer Research (Friends), presented at the 2024 San Antonio Breast Cancer Symposium in San Antonio, Texas. The poster titled, “Agreement Across 10 Artificial Intelligence Models in Assessing HER2 in Breast Cancer Whole Slide Images: Finding from the Friends of Cancer Research Path Project,” evaluated the variability in biomarker assessment across 10 computational  pathology tools.  

In a video interview with Oncology Learning Network, Dr. McKelvey elaborated on the study’s methodology and findings. Friends of Cancer Research distributed over 1,000 whole slide images from patients with breast cancer to developers of 10 different computational pathology models. Developers provided their HER2 scoring, and the results presented focused on the seven models that provided the American Society of Clinical Oncology (ASCO)/College of American Pathologists (CAP) HER2 scores, which categorizes HER2 status as 0, 1+, 2+, or 3+, and is used to inform treatment decisions.  

In collaboration with the National Cancer Institute (NCI) Biometric Research Program, analyses conducted evaluated the levels of agreement in HER2 assessment among models. The findings revealed that the models demonstrated a high level of agreement in identifying cases with HER2 scores of 3+ and, to a lesser extent, 0. However, there was greater variability in cases scored as 1+ and 2+. These results are consistent with the published literature highlighting similar challenges with pathologists’ manual scoring and determination of HER2 status. 

The next phase of this research partnership will focus on identifying the specific factors that contribute to the observed variability, such as how the models are trained, and inform the development of approaches for alignment and assessment of models to ensure patients receive reliable and accurate results.  

At Friends February 4, 2025, meeting. “Advancing the Future of Diagnostics and Regulatory Innovations” final findings will be presented alongside a panel to discuss policy implications and provide recommendations for advancing the development of reference sets.

To register to attend the conference virtually or in-person please visit: https://friendsofcancerresearch.org/event/advancing-the-future-of-diagnostics-and-regulatory-innovations/.  

For more information on Friends’ Digital PATH Project and accompanying research, please visit https://friendsofcancerresearch.org/digital-pathology/.