New Study Highlights the Role of Physical Color Calibration in Enhancing Digital Pathology Diagnostics by AI

A groundbreaking study has addressed a major challenge in digital pathology: the impact variations in color consistency across different scanners has on AI reliability.

A new international study, supported by PathQA, has shown that physical color calibration can solve a key challenge in digital pathology AI: inconsistencies in whole slide images (WSIs) caused by different scanners. These variations potentially compromise diagnostic accuracy and reduce concordance with pathologists that limits widespread adoption, but this research demonstrates how PathQA’s Sierra ICC profiles can standardize scanner outputs and robustly improve reliability.

The study, led by Assistant Professor Kimmo Kartasalo and Xiaoyi Ji of Karolinska Institutet in Stockholm, used AI to evaluate prostate cancer biopsies from multiple hospitals across three countries and found significant improvements in AI diagnostic agreement with pathologists when color calibration was applied. For example, in a cohort from Stavanger University Hospital, agreement between pathologists and diagnostic assessments by AI improved dramatically, with Cohen’s kappa values rising from 0.439 to 0.619. Similar results were seen at Karolinska University Hospital, where Cohen’s kappa values increased from 0.354 to 0.738. Assistant Professor Kartasalo reflected on the variability in AI performance being a well-known challenge in digital pathology, stemming from inconsistencies in input data across different institutions:

“Uncontrolled variability in the characteristics of input data is a major challenge for most AI algorithms. For example, diagnostic AI models trained with data from one hospital may function in unexpected ways when processing data collected from another clinic. This can pose a severe risk to patient safety or require costly evaluations and adjustments of the AI models when such systems are introduced for clinical use. In digital pathology, the focus has so far been on designing increasingly complex AI systems to handle the variability in the appearance of digitized tissue samples, which leads to various new complications.”

The work also revealed that AI performance could be more robustly maintained by color standardization than some computational normalization methods when challenged with datasets of differing sample numbers, representing the real-world of differing patient populations, and that further benefit to AI was found even after foundation model training when direct scanner correction was also applied. Dr Rick Salmon, CEO of PathQA, commented:

“This paper demonstrates that by correcting and standardizing inter-hospital, multi-vendor scanner variability, PathQA’s Sierra ICC profiles provided a practical, scalable, and regulation-friendly solution for ensuring consistent and reliable digital pathology workflows with explainable benefits to AI. This research represents a major step forward in advancing diagnostic reliability and patient safety when utilising AI across diverse clinical settings”.

The findings highlight an important shift in the approach to improving AI reliability in pathology. Instead of solely relying on increasingly complex AI models to mitigate variations in digital pathology data, this study underscores the effectiveness of standardizing image acquisition itself. By integrating physical color calibration with AI development, institutions can enhance diagnostic consistency and safeguard patient outcomes.

“In this study, we showed that by using the Sierra color calibration technology to standardize the properties of digital scans obtained with different scanners, much of the variation in the image data can be eliminated. We observed that this leads to more consistent AI performance in diagnosing and grading prostate cancer across data from different clinical sites. We believe adopting calibration technologies to improve the quality and consistency of data collection processes should be pursued in parallel with further improvements in AI system design. This will help ensure the safety and reliability of diagnostic AI systems as they are being deployed in real-world environments,” says Xiaoyi Ji, PhD student at Karolinska Institutet and main author of the study.

Key Benefits of Physical Color Calibration in Digital Pathology and AI demonstrated by this research

Scalability

Physical color calibration with PathQA’s Sierra ICC profiles provided a universal approach to standardize whole slide images (WSIs) across various clinical sites and scanner models.
Unlike computational normalization methods, it did not require large datasets or site-specific retraining to retain diagnostic accuracy of AI, making AI more accessible as a potential product for smaller clinics and research facilities.
Ensured consistent performance across diverse setups, supporting broader implementation in digital pathology workflows.

Reliability

Reduced variability caused by differences in scanners, enhancing the consistency of diagnostic results.
Demonstrated significant improvements in agreement with pathologists (Cohen’s kappa), particularly in challenging datasets such as those from Karolinska and Stavanger hospitals.
Will mitigate issues like scanner aging, hardware changes, and software updates, ensuring dependable diagnostics over time through routine recalibration.

Explainability

Uses quantifiable metrics and standards, such as International Color Consortium (ICC) profiles, to create a transparent and traceable calibration process.
Aligns image outputs with a universal standard (absolute, true color), avoiding opaque transformations seen in computational techniques like GANs.
Enhances predictability and trust by ensuring uniform image characteristics, improving communication and concordance among clinicians and researchers.

Regulation

Aligns with FDA guidelines for WSI precision and accuracy, attractive for clinical use and regulatory approval.
Simplifies compliance by providing a universal standard, avoiding the need for complex retraining or time-consuming revalidation processes.
Facilitates inter-lab quality assurance and data sharing, addressing regulatory concerns about consistency and reproducibility across institutions.

Conclusion

By addressing challenges in scalability, reliability, explainability, and regulation, physical color calibration establishes itself as a critical tool for consistent, high-quality, and regulation-friendly digital pathology and AI workflows.

The success of this study was made possible through the invaluable contributions of an international network of clinical collaborators. The researchers extend their gratitude to teams at Stavanger University Hospital, Aarhus University Hospital, Karolinska University Hospital, and University of Turku for their dedication to advancing digital pathology. Their efforts in providing high-quality clinical data and expertise have been instrumental in demonstrating the impact of color calibration on AI reliability. By working together across institutions and countries, this collaboration marks a significant step toward more standardized, accurate, and scalable AI-driven diagnostics in pathology.