A novel deep learning approach for predicting cell types directly from H&E-stained tissue samples could streamline pathology workflows, according to research presented by Sayat Mimar, PhD, data scientist at the University of Florida College of Medicine.[1] This study demonstrates the potential of AI to bridge the gap between traditional histology and advanced molecular imaging techniques.

“The cell prediction pipeline will be able to quickly inform clinicians of the quantitative measure of the cell type distribution within a tissue sample,” Mimar explained in an interview. “Both the cell type distribution and the morphology of each cell type are useful for clinicians in diagnosing or monitoring intestinal diseases.”

The research team developed and validated a semantic segmentation network that can identify 12 cell types in intestinal tissue samples using bright-field microscopy images. The system achieved a 4.5-fold improvement in balanced accuracy compared with random baseline predictions.

The team used 32 frozen intestinal tissue sections from four donors, encompassing both small and large intestine regions. The researchers mapped over 850,000 cells in the training set and approximately 170,000 cells in the test set. Ground truth annotations were generated using CODEX, which enables the visualization of more than 40 protein markers at the single-cell resolution. The system’s architecture combines a ResNet-50 encoder for feature extraction with a DeepLab V3 plus decoder, enabling the model to learn both fine details and larger-scale tissue architecture patterns.

During his presentation, Mimar emphasized the practical implications of this technology: “Imagine you have a histology image but no access to molecular imaging. Our vision is to create a pipeline where you upload your histology image and receive a detailed output showing the locations of different cell types and distribution of cell states.”

Mimar added that the technology’s clinical potential extends beyond cell type identification. “If we can quantify different cell types in the intestine along with their location and prevalence, we will have a more nuanced understanding of the intestinal microenvironment than H&E alone,” he explained. “Subtle shifts in immune cell infiltration, stromal alterations, and epithelial dysregulation are observed in disease and thus critical in diagnosing and monitoring.”

One notable aspect of the project is its focus on accessibility. “This technology can be integrated with our ‘CompRePS’ system for democratized and accessible computational resources,” Mimar stated. “Clinical and research questions can be investigated regardless of the person or place, as long as they have an internet connection.”

A key challenge in training the segmentation network is the phenotype and spatial niche of rare cells. “Paneth cells and neuroendocrine cells are both less common intestinal epithelial cells that are interspersed between the more common enterocytes and goblet cells,” Mimar explained. “The difference in histological appearance is subtle, especially in frozen tissue samples.”

To address this challenge, the team plans to supplement the training data with formalin-fixed paraffin-embedded samples and implement class weights and oversampling during training to mitigate class imbalance issues. This approach could improve the ability of the model to distinguish between similar cell types in different spatial contexts.

The research team is also working on expanding the applications of this technology to other organs. Preliminary work on kidney and lung tissue samples has shown promising results, with kidney cell type prediction achieving 78% accuracy across ten cell types. The team envisions creating a comprehensive framework that could transfer learning between different organ systems. This expansion to various organs could help address the limitations in distinguishing between certain cell types, particularly those with similar morphological features or spatial distributions.Sayat Mimar, PhD, data scientist at the University of Florida College of Medicine, USA.

[1] Sayat Mimar, Cell type prediction for intestine tissue samples from bright-field histology via deep learning. Presented at SPIE 2025 Digital and Computational Pathology conference, February 18, 2025; San Diego, CA.