Evaluating Advancements in AI Algorithms for White Blood Cell Identification in Hematologic Malignancies

by Christos Evangelou, MSc, PhD – Medical Writer and Editor

Artificial intelligence (AI) systems have been developed to automate and improve the accuracy of blood film analysis. One of these AI systems is Techcyte, a web-based platform that can analyze blood films scanned by any digital microscope and provide white blood cell (WBC) differentials and blast identification. Techcyte uses deep machine learning to refine its algorithms and increase its accuracy over time. But how much has Techcyte improved since its first release in 2019?

A team of researchers at the Calvary Mater Newcastle Hospital and New South Wales Health Pathology in Australia conducted a comparative evaluation of three consecutive versions of Techcyte AI released in 2019, 2020, and 2022.1 They analyzed the same 124 abnormal blood films, including 64 acute and 22 chronic leukemias, by manual microscopy and by Techcyte AI and compared the results for each WBC class and blast detection.

“Our study showed that the Techyte AI algorithms have improved significantly over time and are now much better at correctly identifying different white blood cell types and malignant blasts in abnormal blood films,” said Associate Professor Lisa F Lincz, GDipClinEpid, PhD, who was the first and corresponding author of this study.

The report was published in the International Journal of Laboratory Hematology.

 

Rationale: Assessing Improvements in AI-assisted WBC Identification

Blood film analysis is a common laboratory test that involves examining the shape, size, and number of different types of white blood cells (WBC) under a microscope. This can help diagnose various blood disorders, such as leukemia, anemia, and infections. However, manual microscopy is time consuming, subjective, and prone to errors. Therefore, several AI algorithms have been developed recently to assist pathologists in blood film analysis.

“Given the large geographic area of Australia and the ever-increasing demand on the health care system, there is a vital need to adopt new technologies that allow for more efficient use of limited resources,” said Dr. Lincz. “Digital pathology promises to increase connectivity between remote areas and specialized pathology labs, while AI has the potential to streamline current processes to maximize diagnostic outputs.”

To assess improvements in AI-assisted WBC identification over the years, the research team compared consecutive versions of Techcyte AI released in 2019, 2020, and 2022.1

“We chose to trial Techcyte because it was one of the few available online platforms at the time, and since it didn’t require special slide makers or scanners, it would be easy to introduce into existing infrastructure. But before making any large investment, we wanted to see just how good it was and if it would really improve over time,” explained Dr. Lincz.

 

Approach

To evaluate the three AI algorithms, the team used 124 digitized abnormal peripheral blood films, including cases of acute and chronic leukemias.1 The WBC differentials derived from the AI algorithms were correlated with those obtained using manual microscopy. Lin’s concordance coefficients and sensitivity and specificity of blast identification were used to determine the best AI version.

“We uploaded a set of abnormal peripheral blood film slides to the Techcyte server in 2019 for our first study analyzing the accuracy of white blood cell differentials by the original Techcyte AI algorithm. For the current study, we re-ran the same images through the next 2 AI releases and compared the results to the current gold standard of manual microscopy,” noted Dr. Lincz.

 

Improved Cell Identification

The researchers found that Techcyte AI correlations with manual microscopy improved with each version, ranging from 0.50–0.90 for the 2019 version (AI1), 0.66–0.86 for the 2020 version (AI2), and 0.71–0.91 for the 2022 version (AI3).1 They also found that AI3 had significantly better concordance with manual microscopy than AI1 for identification of neutrophils, total granulocytes, immature granulocytes, and promyelocytes. AI3 was also significantly better than AI2 for identification of total granulocytes and promyelocytes.

One of the notable differences between the versions was the removal of the “other” cell category, which was used by AI1 to label cells that it could not identify.1 This category accounted for 0.2%–70% of cells in 81 out of 124 slides analyzed by AI1, with higher rates of unclassified cells in non-leukemic films and those with red blood cell abnormalities. These unclassified cells were omitted from the analysis, which may have biased the results towards the cells that AI1 was confident to classify. In contrast, AI2 and AI3 did not have this category and classified all cells into the differential results.

 

Improved Sensitivity for Blast Detection

The researchers also evaluated the performance of Techcyte AI for blast detection, which is crucial for diagnosing and monitoring leukemias. They found that Techcyte AI maintained high sensitivity for blast identification in malignant films, which improved from 97% for AI1 to 98% for AI2 and to 100% for AI31.1 This means that Techcyte AI correctly identified all the slides that had blasts, as confirmed by manual microscopy. However, the specificity of blast identification, which measures the ability to correctly identify slides that did not have blasts, decreased from 24% for AI1 to 14% for AI2 and to 12% for AI3.1 This means that Techcyte AI also identified some slides as having blasts when they did not, resulting in false positives.

The researchers explained that the high rate of false positives may be due to the difficulty of distinguishing blasts from other immature cells, such as promyelocytes and monoblasts, especially in blood films with low blast counts. They also suggested that the low specificity may be clinically more desirable than low sensitivity, as it would prompt further review of suspicious slides and avoid missing a cancer diagnosis.

 

Looking Ahead

The study demonstrated that Techcyte AI has shown significant improvement in cell identification over time and has maintained high sensitivity for blast identification in malignant films. The researchers concluded that Techcyte AI is a promising tool for blood film analysis, especially in remote and regional areas where hematopathology expertise is limited. They also highlighted the advantages of Techcyte AI over other commercial systems, such as its flexibility to process data from any stained slide and any scanner and its web-based platform that allows multiple users to easily access the online remote server.

However, the study also had some limitations, such as the small sample size, the lack of manual reassignment of cells that needed manual reclassification by a morphologist, and the use of only one scanner. The researchers acknowledged that these factors may have affected the accuracy and generalizability of the results.

They also noted that the performance of Techcyte AI on pediatric samples and other types of blood disorders has yet to be evaluated. They recommended that future studies should include larger and more diverse samples, manual reassignment of cells, and comparison of different scanners and AI systems.

“We are currently planning to repeat the study on other platforms for comparison. Ultimately, we would like to test similar tools developed for bone marrow analysis, as this is where there is the most potential benefit for time-saving AI screening,” said Dr. Lincz.

She added that digital pathology and AI are not intended to replace pathologists but merely make their jobs easier and quicker, and this is highly dependent on the accuracy of the initial AI readings.

“Knowing that AI is not only good to begin with but also constantly improving makes it a worthwhile investment for diagnostic laboratories. For patients in remote areas where specialists are not on hand, digital pathology allows instant access to dedicated pathologists, enabling more rapid diagnoses,” she said.

The study was funded by New South Wales Health Pathology and the Haematology Department of Calvary Mater Newcastle, and the Hunter Medical Research Institute Precision Medicine Research Program.

 

References

  1. Lincz LF, Makhija K, Attalla K, Scorgie FE, Enjeti AK, Prasad R. A comparative evaluation of three consecutive artificial intelligence algorithms released by Techcyte for identification of blasts and white blood cells in abnormal peripheral blood films. Int J Lab Hematol. Published online October 3, 2023. doi:10.1111/ijlh.14180

Share This Post

Leave a Reply