The Tissue Image Analytics (TIA) Centre hosts a seminar series for those interested in computational pathology. The series invites researchers and leaders in computational pathology and related fields to present their work and foster thought-provoking discussions.

This seminar features Andrew Zhang from Harvard Medical School, Boston, USA.

Abstract

The field of medical AI has traditionally focused on isolated clinical domains, such as chest X-rays or pathology slides. These data modalities have the advantage of being plentiful, relatively standard in format, and information dense. But by looking at only individual snapshots of a patient’s medical journey, are we missing the forest for the trees? Outside of the world of curated medical imaging datasets, there are petabytes of clinical data existing in the electronic health record (EHR) as unstructured or semi-structured longitudinal events. These include diagnoses, symptoms, lab tests, clinical reports, medications, and more, each tied to a specific timepoint in the course of a patient’s healthcare journey. With our capacity to train increasingly larger and more powerful AI models, the next frontier is to build a virtual patient: a model that understands and makes predictions on the entire EHR. In this talk, I will introduce APOLLO, a major effort towards modeling whole patients: advancing the state of the art not only in data size (25 billion events from 7.2 million patients) but also in data breadth (structured data, clinical text, and images) and downstream capabilities. To evaluate the potential of this model, we created a large-scale benchmark consisting of over 300 patient retrieval and clinical forecasting tasks and showed that APOLLO patient embeddings generalize across medical domains and encode both retrospective and prognostic information about each patient. APOLLO has exciting applications not only in disease risk prediction but also as a multimodal medical search engine and unlocks the possibility of biomarker discovery from the entire EHR.

Bio

Andrew Zhang is a 4th-year PhD student in the Harvard-MIT Health Sciences and Technology Program, working in the lab of Dr. Faisal Mahmood at Harvard Medical School. Prior to his PhD, Andrew studied cell biology and computer science at Harvard. His work centers on advancing self-supervised representation learning at multiple biological and temporal scales: from encoding a single pathology slide to encoding entire longitudinal medical histories.