by Christos Evangelou, MSc, PhD – Medical Writer and Editor
Researchers at the University of Houston (Texas, USA), the University of Texas McGovern Medical School (Texas, USA), and the National Institute of Neurological Disorders and Stroke (Maryland, USA) have developed a visual search engine for exploring and analyzing multiplex brain tissue images without programming. The search engine lets scientists query a slide the way they might search for images online and retrieve communities of biologically similar cells and/or cellular neighborhoods within seconds.
“Our work opens the door to a new way of analyzing spatial proteomics data that is interactive and driven by visual queries that result in the retrieval of similar cells or cellular neighborhoods,” said Badrinath Roysam, D.Sc., Hugh Roy and Lillie Cranz Cullen University Professor at the University of Houston and corresponding author of the study.
The study was published in Scientific Reports.
Study Rationale
Roysam explained that brain tissue is not easy to study. A single whole-slide image from a modern multiplex immunohistochemistry protocol can capture 30 or more protein markers simultaneously, across tens of millions of cells, at sub-micron resolution. The result is an image file that can exceed 40,000 × 30,000 pixels and contain more biological information than any pipeline-based analysis tool can efficiently extract.
Spatial proteomics, measuring protein distributions in their full spatial and histological context, was declared the Method of the Year by Nature Methods in 2024. What has lagged is the ability to analyze spatial proteomics datasets without specialized programming skills.
“The newly emerged field of spatial proteomics is revolutionizing the study of cells and tissues in health and disease,” Roysam said in an interview with Pathology News. “Such high-resolution, high molecular detail data are challenging to analyze and require sophisticated programming. We were motivated to develop an alternative programming-free, query-driven approach based on the concept of a visual search engine.”
The research team led by Roysam developed mViSE, a multiplex visual search engine, now available as an open-source plug-in for QuPath, a widely used digital pathology viewer.
How mViSE Works
mViSE operates the same way a visual similarity search does. A user clicks on a cell or tissue region of interest, and the system retrieves the most similar cells or multicellular neighborhoods elsewhere in the image.
Making that simple interaction work on massive, 30-channel fluorescence data required solving several non-trivial problems. Standard image encoders, including recently published foundation models for computational pathology, are designed for three-channel RGB images. They cannot handle the combinatorial protein expression captured in multiplex data.
“Our work represents an intuitive, visual way of tackling the challenging task of making sense of massive spatial proteomics images,” Roysam noted.
Rather than trying to process all proteins simultaneously, the researchers organized the markers into panels, each capturing a distinct aspect of tissue architecture. Panel 1 covers major cell types (NeuN, S100β, IBA1, Olig2, RECA1). Panel 2 focuses on glial components, and Panel 3 captures myeloarchitecture. Panels 4 through 6 address neuronal subtypes, cell phenotypic status, and vasculature, respectively.
For each panel, mViSE trains a dedicated Vision Transformer (ViT)-based encoder in a fully unsupervised manner, with no human annotations required. The encoder learns to place visually and molecularly similar tissue patches close together in a high-dimensional feature space. Training is guided by a combined loss function that merges contrastive learning with triplet loss, a combination the authors found necessary to distinguish brain regions with subtly different cytoarchitectures, including the individual cortical layers.
The training process generates visual confirmation along the way. Community maps (color-coded visualizations of how the encoder has grouped tissue patches) allow researchers to watch the model learn brain structure over successive training epochs. By epoch 50, the community maps clearly reflect the known anatomy of the rat brain.
A Surprising Finding
Once trained, mViSE can be queried interactively. A user clicking on an oligodendrocyte retrieves hundreds of similar oligodendrocytes distributed across the slide, and their locations are overlaid on a fitted brain atlas. Clicking on a cortical pyramidal neuron returns cells that are concentrated in cortical regions. The same query mechanism works for vessel-proximal cell pairs, GABAergic interneurons, cholinergic neurons, and multicellular niches.
What surprised the researchers was how much emerged from the retrievals without any explicit instruction to find it.
“At first glance, this may seem boring,” Roysam said, “but what surprised us is the fact that the retrievals revealed and accurately delineated the major cortical cell layers, brain regions, sub-regions, and cyto-vascular niche patterns, all with visual confirmation.”
Delineating cortical layers is technically demanding even with purpose-built pipelines. mViSE achieved an average Intersection over Union (IoU) score of 0.70 across all six cortical layers, with true positive rates of 0.80 to 0.90. When the team compared this against five commonly used foundation models (DINO V3, BEiT V3, Prov-GigaPath, UNI2, and Titan), mViSE outperformed all of them across every cortical layer. For L1, mViSE scored an IoU of 0.78, compared to 0.67 for BEiT V3 and 0.69 for DINO V3 and Prov-GigaPath.
mViSE also outperformed the other foundation models in deeper layers. At L4, mViSE achieved 0.49 IoU while the rest achieved an IoU of 0.25 to 0.32. The reason, the authors argue, is that mViSE encodes each imaging channel independently, preserving channel-level information that is lost when multiplex data is compressed into RGB.
Single-Cell Retrieval Performance
For single-cell searches using 75 × 75 pixel patches centered on individual nuclei, the system achieved a mean Top-1 retrieval accuracy of 0.90 and a Top-5 accuracy of 0.96 across the five major brain cell types. NeuN-positive neurons were retrieved with a Top-1 score of 0.97. Performance was slightly lower for RECA1-positive endothelial cells (Top-1: 0.83), which the authors attribute to the spatial complexity introduced by vessel-proximal structures.
The system can also handle “proximal cell pair” queries, retrieving patches where two specific cell types appear in close spatial proximity, such as astrocyte-oligodendrocyte pairs or vessel-proximal microglia, without the need for custom-built pipelines.
Practical Considerations and Future Work
Training is computationally intensive. A 5-channel panel with ~110,000 training samples required roughly six hours on two NVIDIA A6000 GPUs. Dataset preparation for a full 30-channel slide took approximately 1.5 hours. The authors recommend overnight training on a GPU cluster.
For researchers without GPU access, the team also developed a “quick search” tool that runs entirely within QuPath. It uses cell morphometrics and protein expression statistics computed by QuPath itself, allowing flexible queries by molecular markers, morphological features, and neighborhood profiles. The output is a ranked list of similar cells along with a similarity matrix.
The authors also acknowledged that the study focused on healthy rat brain tissue, where the known anatomy provides a clear reference for validation.
“Our work is not directly applicable to clinical studies at this time; it is more relevant to pre-clinical and basic studies of brain tissue,” Roysam noted.
He explained that applying mViSE to diseased, injured, or human brain tissue would require retraining and, for quantitative validation, ground truth annotation.
The team is already extending the system toward comparative analysis across datasets.
“We are currently extending this method to seek out differences between datasets, for example, healthy vs. diseased, treated vs. untreated, drug-responsive vs. unresponsive,” Roysam said.
That direction could eventually make mViSE useful for evaluating drug candidates, tracking disease progression, or characterizing treatment responses in tissue.
All code, training data, a representative 36-channel TIF file, and a step-by-step tutorial are publicly available through GitHub and Zenodo.
*The study received financial support from the National Institutes of Neurological Disorders and Stroke.
References
- Huang L, Mills R, Mandula S, et al. mViSE: A visual search engine for analyzing multiplex IHC brain tissue images (spatial proteomics). Sci Rep. 2026;16(1):10245. doi:10.1038/s41598-026-40620-5
