Project InnerEye Open-Source Software for Medical Imaging AI
Radiotherapy planning workflow using CT images
An important potential application for machine learning in medical imaging is to assist clinicians for image preparation and planning tasks in radiotherapy cancer treatment workflows. Our latest research, published in JAMA Network Open, shows how AI can augment and accelerate clinicians’ ability to perform radiotherapy planning 13 times faster. For details of how you can make use of the InnerEye OSS tools for radiotherapy planning workflows with CT images see our Getting started section. Support is via GitHub Issues on the relevant repositories only.
Medical imaging research
The InnerEye-DeepLearning Toolkit allows you to develop your own models for different applications. This is enabled by using a configuration-based approach and making the process of training models at scale easy. The InnerEye OSS tools may be used for developing classification, regression, and sequence models using only images as inputs, or a combination of images and non-imaging data as input. This supports typical use cases on medical data where measurements, biomarkers, or patient characteristics are often available in addition to images. The ability to bring-your-own-model setup means that this toolkit can be used for a variety different tasks. The InnerEye OSS has different levels of abstraction and allows you to build any model with bring-your-own model or pre-defined models with configuration files. We have examples for segmentation, classification, and sequence models that can take images or multiple modalities. Use of this toolkit requires medical imaging knowledge and data science skills.
Support is via GitHub Issues on the InnerEye-DeepLearning toolkit repo here (opens in new tab).
Bring-your-own-model
The InnerEye-DeepLearning Toolkit (opens in new tab) makes it easy to use pre-configured neural networks, such as UNet3D, or bring-your-own-models (opens in new tab). This requires dedicated research expertise and effort. You can read more details about our research work in our publications here (opens in new tab). Support is via GitHub Issues on the InnerEye-DeepLearning toolkit repo here (opens in new tab).
Radiology and surgery planning with CT images
The InnerEye-DeepLearning Toolkit (opens in new tab) makes it easy to use pre-configured neural networks, such as UNet3D, or bring-your-own-models (opens in new tab) for CT images. Potential use cases for this are quantitative radiology and surgery planning. Use of the Gateway and Inference Services will require additional software engineering effort by you. Support for these use cases is via GitHub Issues on the relevant repositories only.
Magnetic Resonance, X-Ray, OCT images
We have had some positive, but limited, experience with MR, X-ray and OCT images. You can bring-your-own-models (opens in new tab) to make it easier to further develop and deploy them. This requires dedicated research expertise and effort to pursue these, and having the relevant, annotated clinical data for the algorithm’s training and optimization. Use of InnerEye OSS tools should significantly increase your overall productivity once you have become familiar with them and make use of Azure Machine Learning. Support is via GitHub Issues on the InnerEye -DeepLearning toolkit repo here (opens in new tab).
Magnetic Resonance imaging reconstruction
Research into accelerating MR image reconstruction could potentially lead to reduced patient stress and provider costs. Training and evaluating machine learning for this task is challenging due to the large data volume of raw MR data. The InnerEye-DeepLearning toolkit enables researchers to use Azure Machine Learning to train and evaluate models in hours, that would otherwise take days. This makes use of the bring-your-own-model setup for training models with very large datasets. Documentation for working with FastMRI models is here. This requires dedicated research expertise and effort. Support is via GitHub Issues on the InnerEye DeepLearning toolkit repo here.
Self-supervised model pre-training
The InnerEye-DeepLearning toolkit (opens in new tab) contains a capability for self-supervised learning. The code in the InnerEye/ML/SSL (opens in new tab) folder allows you to train self-supervised models using SimCLR (opens in new tab) or BYOL (opens in new tab). This code runs as a » bring-your-own-model» self-contained module (see docs/bring_your_own_model.md (opens in new tab)). This requires dedicated research expertise and effort. Support is via GitHub Issues on the InnerEye DeepLearning toolkit repo here (opens in new tab).