by Christos Evangelou, MSc, PhD – Medical Writer and Editor

Artificial intelligence (AI) can transform cancer diagnosis through digital pathology; however, AI models can learn to recognize patterns specific to a dataset rather than actual disease characteristics. Researchers from Trinity College Dublin and their collaborators have developed an innovative solution to this problem. In a study published in the Journal of Pathology Informatics, they demonstrate how a statistical technique called ComBat harmonization can prevent AI models from learning these misleading patterns while preserving their ability to detect true histological features.

“Batch effects in digital pathology datasets arise due to variations in how tissues are processed, stained, and scanned at different clinical sites,” noted Pierre Murchan, the lead author of the study. “When models inadvertently learn these confounding features, their performance can be inflated on internal datasets, resulting in models that fail to generalize to external datasets or real-world applications.”

The Challenge of Hospital-Specific Patterns and Batch Effects in Digital Pathology

When pathology slides are prepared and digitized, subtle differences in tissue processing, staining, and scanning procedures can create patterns specific to each hospital or tissue source site. These technical variations are commonly referred to as batch effects and can lead AI models to make predictions based on these technical artifacts rather than on true biological features, limiting the generalizability and reliability of digital pathology algorithms.

Murchan explained the motivation behind this study:

“We needed to address batch effects in large digital pathology datasets, specifically when data originates from multiple different sources. It had previously been shown that batch effects in digital pathology can behave as confounders when models are trained to predict a feature that might be associated with the site at which a WSI originates.”

ComBat Harmonization: A Solution for Batch Effects in Digital Pathology

The researchers adapted ComBat harmonization, a statistical method originally developed for genetic data analysis, to address batch effects in large digital pathology datasets.¹ The technique works by modeling and removing hospital-specific patterns from the features that AI models extract from medical images, while preserving clinically relevant information. They tested their approach using three different AI architectures on colon and stomach cancer datasets from The Cancer Genome Atlas (TCGA).

“While ComBat was originally developed to address batch effects in microarray data, it has widely been applied to the field of radiomics. However, it had not yet been applied to digital pathology,” said Murchan. “Our goal was to investigate whether ComBat could extend to deep learning-derived features to mitigate batch effects in digital pathology.”

The team processed thousands of whole-slide images from multiple hospitals to extract features using three different AI architectures: a standard ImageNet-pretrained model (INPT-ResNet50), a pathology-specific self-supervised model (CCL-ResNet50), and a hybrid transformer-CNN architecture (CTransPath).¹ They then applied ComBat harmonization to the features extracted by these models to remove hospital-specific patterns. The team also trained new AI models to predict various clinical characteristics and compared model performance before and after harmonization.

Impact of ComBat Harmonization on AI Models

Before harmonization, AI models could predict the hospital of origin of WSIs with over 95% accuracy, indicating that they were learning hospital-specific patterns.¹ After applying ComBat harmonization, this accuracy dropped to approximately 50%, which is equivalent to random guessing. According to the authors, this dramatic reduction indicates that harmonization successfully mitigated batch effects and hospital-specific patterns when extracting features from the WSIs.

Despite the reduced ability of the models to predict tissue source sites, they maintained their ability to predict clinically relevant features after harmonization. For example, the models retained high accuracy in predicting microsatellite instability status, an important biomarker for immunotherapy response in colorectal cancer.¹ The models also maintained their ability to detect certain genetic mutations, such as BRAF and TP53, demonstrating that the harmonization preserved histological and biological features while removing technical artifacts.¹

The study also showed that race was highly associated with tissue source site in both TCGA cohorts. ComBat harmonization significantly reduced the predictability of race from patch embeddings, potentially offering a method to mitigate demographic biases in AI models.¹

However, some predictions that were previously thought to be reliable became less accurate after harmonization, suggesting they may have been partially based on hospital-specific patterns rather than true biological features.¹

“An unexpected finding was that, while ComBat harmonization effectively reduced predictability of the clinical site at which a sample was processed, its impact varied depending on the feature extraction model used,” Murchan noted. “Despite being pretrained on large-scale and diverse pathology datasets, we found that some foundation models for digital pathology still extract features that allow downstream machine learning models to inadvertently learn site-specific characteristics related to the clinical site where the slides were processed.”

Implications and Future Directions

According to Murchan, harmonizing digital pathology datasets using ComBat can result in models that are more likely to learn true histological features that are consistent across datasets and clinical settings.

“This can lead to more robust models that generalize better to new data, thus increasing their potential for clinical adoption and integration into routine pathology workflows,” he said.

Murchan outlined the potential applications of ComBat in healthcare:

“When a model is deployed to a new clinical center, calibration of the model is required to account for potential domain shifts. By applying ComBat harmonization, new data could be standardized to match the characteristics of the original training data, effectively mitigating domain shift and enhancing model performance across clinical sites.”

This approach is particularly valuable for small healthcare facilities.

“One of the key advantages of ComBat is its ability to correct for batch effects even in the presence of relatively small sample sizes, which is particularly valuable for smaller clinical centers or in studies focused on rare diseases, where data is often limited,” Murchan noted.

The research team plans to further validate the ability of ComBat harmonization to enhance model generalizability by applying it to larger cohorts and a broader range of biomarkers.

“Expanding the scope of the study to include various cancer types and histological features will be crucial to assess the robustness of this approach,” Murchan added.

This study received financial support from the Science Foundation Ireland (SFI) through the SFI Centre for Research Training in Genomics Data Science.

References

1. Murchan P, Ó Broin P, Baird AM, Sheils O, P Finn S. Deep feature batch correction using ComBat for machine learning applications in computational pathology. J Pathol Inform. 2024;15:100396. Published 2024 Sep 12. doi:10.1016/j.jpi.2024.100396