by Christos Evangelou, MSc, PhD – Medical Writer and Editor
The rate of clinical adoption of digital pathology for diagnosis has increased drastically over the past few years. Despite numerous studies to validate the performance of whole slide imaging (WSI) in facilitating accurate and reliable diagnosis, limited efforts have been made to systematically compare the diagnostic performance of digital pathology to that of light microscopy.
In a recent study, researchers from the University of Copenhagen conducted a systematic review of studies comparing the diagnostic performance of WSI to that of light microscopy in terms of diagnostic test accuracy, diagnostic concordance, and degree of overdiagnosis.1 The results of the study suggest that WSI provides an overall high diagnostic accuracy, although the degree of overdiagnosis could not be determined.
Commenting on the implications of their findings, Olsi Kusta noted: “With healthcare systems and pathology departments suffering from staff shortage, the increasing workload and complexity of examinations further complicates the efficiency and accuracy of diagnostic work. Digital pathology presents some opportunities to partially address these challenges by enabling working remotely, facilitating virtual and potentially faster consultations with colleagues, allowing the interactive and remote supervision of residents using digital solutions, and opening new avenues for the use of AI algorithms for computer-assisted diagnosis.”
Olsi Kusta is a PhD fellow at the University of Copenhagen and the first author of the study. The study was published in the Journal of Pathology Informatics.
Study Rationale: Determining the Diagnostic Performance of WSI
“As digital pathology is increasingly being introduced in European and US pathology departments, more validation studies are being conducted to assess their diagnostic performance. However, few systematic reviews have compared the findings of these validation studies,” said Kusta.
However, these reviews have mostly focused on the diagnostic concordance between WSI and light microscopy. The team “wanted to make the comparison between WSI and light microscopy more robust.” Hence, they introduced diagnostic test accuracy indicators and the degree of overdiagnosis, in addition to concordance, to evaluate the performance of WSI.
Approach: Systematic Review of Current Evidence
The systematic review included validation studies comparing the diagnostic performance of WSI with that of light microscopy. All the included studies were published between 2010 and 2021. From the screened articles, 12 peer-reviewed studies met the selection criteria and were used to qualitatively assess the diagnostic accuracy, diagnostic concordance, degree of overdiagnosis, and intraobserver variability of WSI.
Commenting on the novelty of their approach, Kusta said: “This review took place within an interdisciplinary collaboration, involving expertise in pathology, evidence-based medicine, clinical reasoning, and medical sociology. This approach for evaluating the diagnostic performance of WSI is novel.”
He explained that the introduction of overdiagnosis as an indicator of diagnostic performance was a result of interdisciplinary deliberation because traditional accuracy measurements (i.e., diagnostic accuracy and diagnostic concordance) are insufficient for capturing the performance of high-resolution technologies, such as WSI.
WSI Is Not Inferior to Light Microscopy in Diagnostic Accuracy
The team analyzed the data extracted from the 12 studies that met their inclusion criteria and compared the diagnostic performance of WSI to that of light microscopy in terms of diagnostic accuracy, diagnostic concordance, and interobserver variability. They found that the diagnostic accuracy of WSI was not inferior to that of light microscopy. In seven of the 12 studies, the sensitivity of WSI ranged from 86% to 100%, while its specificity was 75%–100%. Additionally, WSI provided positive predictive values ranging from 92% to 99% and negative predictive values ranging from 75% to 100%. However, the application of WSI for intraoperative cancer staging using frozen tissue sections provided specificity and negative predictive values of only 75%.
Although WSI and light microscopy showed similar overall performance, only four studies directly compared the performance of the two methods. Relatively low negative predictive values were reported in a study of pancreatic pathology. However, both WSI and light microscopy performed poorly in this respect (negative predictive value: 51% for light microscopy and 52% for WSI). Similarly, a study on breast cancer diagnosis showed that both WSI and light microscopy provided low average performance for the diagnosis of ductal carcinoma in situ (57.1% for WSI vs. 69.6% for light microscopy) and atypia (27.8% vs. 37.8%). However, both WSI and light microscopy yielded high predictive values for the identification of benign tumors without atypia (95.7% vs. 97.1%) and invasive breast cancer (97.2% vs. 97.7%).
Moreover, the diagnostic concordance of WSI was not inferior to that of light microscopy, with values ranging from 86% to 98.35%. However, only six studies measured diagnostic concordance; therefore, future studies are needed to determine the diagnostic concordance of WSI.
Observer variability was assessed in six studies, although there was significant variation in observer variability measures among the studies. The percentage of intraobserver variability ranged from 73% to 100% for both WSI and light microscopy.
Lack of Evidence on Overdiagnosis
Although overdiagnosis is an important indicator of the performance of a diagnostic assay, the degree of overdiagnosis was not assessed in any of the 12 studies included in the review.
The non-inclusion of overdiagnosis in traditional accuracy measurements has important clinical implications. “Since most studies did not report the degree of overdiagnosis, sensitivity and predictive values may be artificially inflated,” Kusta explained. These findings suggest that studies assessing the performance of WSI or other technologies with diagnostic applications should also include assessments of overdiagnosis.
Remaining Challenges
Despite accumulating evidence confirming the high overall diagnostic accuracy of WSI, the degree of overdiagnosis remains unclear. Additionally, the findings of this study suggest high heterogeneity in practice among pathology subspecialties, which might affect diagnostic performance.
“Pathology subspecialties are highly heterogeneous, and the diagnostic performance of WSI is often influenced by the study design, sample preparation and type, and diagnostic complexity,” Kusta noted. “The high heterogeneity of pathology as a medical discipline might have practical implications on the laboratory and diagnostic work. These variations in practice could affect the implementation and use of digital pathology,” he added.
Another open question is how AI can influence the diagnostic performance of WSI. “In this study, we excluded studies involving the use of AI or machine learning applications for digital pathology because we wanted to assess the performance of WSI only as a visual technology used by pathologists.” Nevertheless, AI solutions are increasingly being adopted in pathology practice, and future studies are warranted to assess the diagnostic performance of AI algorithms.
References
- Kusta O, Rift CV, Risør T, Santoni-Rugiu E, Brodersen JB. Lost in digitization – A systematic review about the diagnostic test accuracy of digital pathology solutions. J Pathol Inform. 2022;13:100136. doi:https://doi.org/10.1016/j.jpi.2022.100136
