Introducing BiomedParse, a groundbreaking foundation model for biomedical image analysis

Image analysis is fundamental for clinical diagnostics and biomedical discovery. In this video, we introduce BiomedParse, a biomedical foundation model for holistic image analysis that can jointly conduct recognition, detection, and segmentation for 64 major object types across 9 imaging modalities. BiomedParse adopts a novel image parsing framework for joint learning across the three interdependent tasks. Users can conduct image analysis simply by specifying objects through a unified text prompt. To pretrain BiomedParse, we harness GPT-4 for data synthesis from existing segmentation datasets and created the first dataset for biomedical image parsing, comprising over six million triples of image, object description, and segmentation mask. BiomedParse substantially outperforms prior state-of-the-art methods, often by large margin, especially for the most challenging complex and irregular-shaped objects. BiomedParse is an all-in-one tool for biomedical image analysis on all major image modalities, paving the way for efficient and accurate image-based biomedical discovery.