by Christos Evangelou, MSc, PhD – Medical Writer and Editor
Technical advancements in artificial intelligence (AI) and machine learning have contributed to the rapid evolution of computational pathology. AI algorithms can augment various digital pathology processes, including object counting, biomarker scoring, tissue classification, and outcome prediction.
To ensure the performance and accuracy of AI algorithms when used for diagnostic purposes, human experts must train machine learning models using large volumes of well-annotated images. Nonetheless, there is limited guidance on the annotation or labeling of pathology images intended for the training of machine learning algorithms.
In a recent study, researchers from IMP Diagnostics (Porto, Portugal), the Institute for Systems and Computer Engineering, Technology and Science (INESC TEC; Porto, Portugal), and the Faculty of Engineering of the University of Porto developed a practical guide to help pathologists and AI developers with annotation of pathology images.1
“An important take-home message from our report is how important it is to establish frequent interactions and iterations between the pathologists/annotation team and the machine learning experts,” said Diana Montezuma Felizardo, MD, head of Research & Development Unit at IMP Diagnostics and corresponding author of the study. “Truly taking the time to talk with each other allows for easy corrections and improvements of the models,” she added.
Commenting on the importance of publishing a guide on annotation of pathology images, Dr. Montezuma said: “We believe that describing our experience with annotating pathology images will help others doing annotations for machine learning development in pathology. Having better and more robust annotations will help improve the performance of models and algorithms.”
The report was published in Modern Pathology.
Developing Experience-based Guidelines on Pathology Image Annotation
Although training machine learning algorithms for digital pathology is key to the accuracy of AI-assisted diagnosis, there are no guidelines on the annotation of training pathology images.
“When our team at IMP Diagnostics, in collaboration with INESC TEC, first started doing annotations for computational pathology applications, we found there was a general lack of guidance in this area, and almost no published articles and very scarce information was available online,” Dr. Montezuma noted.
She added that after two years working in this field, they thought it could be valuable to share their insights and experiences and “hopefully help other pathologists and researchers initiating their annotation endeavors.” To this end, the team published a report describing their experience in annotating pathology images and providing a practical guide for developing annotation strategies for AI-assisted digital pathology.
According to the team’s experience, the methods used for image annotation may vary depending on the diagnostic application or study objectives; however, similar issues may arise across different annotation types and settings.1 They also pointed out that regardless of the setting, image annotation should be a collaborative effort between pathologists, engineers, machine learning researchers, and computer specialists. Team collaboration and efficient communication between the members of the multidisciplinary team are key to the success of the annotation process.
The guidelines also emphasize that a flexible and iterative rather than a rigid approach should be used for the annotation of pathology images.1 In addition, the team noted that an important aspect of ground truth quality assessment is that the ground truth should be rigorous and defined at the beginning of the project.
“The work was innovative as it thoroughly describes our experience annotating for the development of classification models in pathology, and there was a general lack of published information specifically on this topic. It was written as a practical and simple guide, in a user-friendly manner, which we feel is also quite unique in medical literature,” said Dr. Montezuma.
Using Hardware for Pathology Image Annotation
In the report, the authors described their experience using hardware to annotate pathology images. Using a computer and a mouse or pen/pad was reported as potentially tiresome and requires a learning curve to be able to draw accurately, although these are readily available and affordable.1 Computers with touch screens are easy to use for image annotation, although annotation software is not optimized for touch screens and often needs to be adjusted. Although expensive and not optimized for image annotation, digital drawing boards are easy to use and offer a helpful alternative for pathology image annotation.
Addressing Common Issues with Image Annotation
The authors also provided practical suggestions to address common issues with image annotation. They advise pathologists to carefully consider their aims and objectives as well as the amount of data before deciding which type of image annotation and machine learning strategy they will use.
“From our experience, choosing a hardware and software solution is a matter of trial and error. We recommend testing different options before committing to a hardware or software solution,” Dr. Montezuma noted.
The authors also cautioned image annotators to carefully consider the format of the annotation files and how easy it is to delete annotations when choosing annotation software. Finally, the authors warned that complementary strategies may be needed to reduce the cost and time required for image annotation. These may include computer-aided interactive annotations and weakly supervised machine learning algorithms.
Looking Ahead
“In this article, we shared our own experience, which is mostly derived from developing classification algorithms focusing on neoplastic pathology. We are hoping that this article will prompt other groups to share their experiences with image annotation, namely, working on other tasks, such as prognostic prediction. But there are multiple applications of computational pathology that will require different approaches,” said Dr. Montezuma.
The team at IMP Diagnostics is currently following up on their previous pathology projects focusing on colorectal and cervix uteri but is also developing new gastric pathology applications. “We have so far mainly worked on manual annotations, and there are other works taking advantage of semi-automated tools. It would be interesting to see these aspects in other publications,” Dr. Montezuma added.
References
- Montezuma D, Oliveira SP, Neto PC, et al. Annotating for Artificial Intelligence Applications in Digital Pathology: A Practical Guide for Pathologists and Researchers. Mod Pathol. 2023:100086. doi:https://doi.org/10.1016/j.modpat.2022.100086
