Project InnerEye Open-Source Software for Medical Imaging AI
Frequently Asked Questions
-
The InnerEye-DeepLearning toolkit (opens in new tab) is an open-source project that makes it easier to train high- performance medical imaging machine learning models and deploy your models using Azure Machine Learning (opens in new tab). It is based on over a decade of research at Microsoft Research Cambridge and continues to be used and developed by MSR teams. We’re excited to see how researchers and partners make use of this. You can get started here – InnerEye-DeepLearning/README.md at main · microsoft/InnerEye-DeepLearning (github.com) (opens in new tab).
-
The InnerEye-Gateway (opens in new tab) comprises Windows services that act as a DICOM Service Class Provider. After an Association Request to C-STORE a set of DICOM image files, these will be anonymised by removing a user-defined set of identifiers and passed to a web service running InnerEye-Inference. Inference will then pass them to an instance of InnerEye-Deeplearning running on Azure to execute InnerEye-DeepLearning (opens in new tab) models. The result is downloaded, deanonymized and passed to a configurable DICOM destination. All DICOM image files, and the model output, are automatically deleted immediately after use. The gateway should be installed on a machine within your DICOM network that is able to access a running instance of InnerEye-Inference. You can get started here – InnerEye-Gateway/README.md at main · microsoft/InnerEye-Gateway (github.com) (opens in new tab).
-
InnerEye-Inference (opens in new tab) is an AppService webapp in Python to run inference on medical imaging models trained with the InnerEye-DeepLearning toolkit. You can also integrate this with DICOM using the InnerEye-EdgeGateway (opens in new tab). You can get started here – InnerEye-Inference/README.md at main · microsoft/InnerEye-Inference (github.com) (opens in new tab).
-
If you have any problems, find issues in the code, or have a feature request, then please create an issue on GitHub (opens in new tab). We monitor these issues and will look to respond via GitHub.
-
We have explored the use of different algorithms over many years for medical imaging, including decision trees and deep neural networks. The InnerEye Deep Learning Toolkit makes it easy to use pre-configured neural networks, such as UNet3D, or bring-your-own-models (opens in new tab). You can read more details about our MSR work in our research publications (opens in new tab).
-
The InnerEye-DeepLearning toolkit (opens in new tab) can use GPUs for training and inference, and other technologies made available in Microsoft Azure. The amount of computing power required depends on the model being used. The toolkit makes it easy to scale out computations using Azure Machine Learning. As an example, for large segmentation models we need GPUs with 16GB or more. In Azure we have successfully tested inference on virtual machines with 4 GPUs and 16GB per GPU. We can use smaller virtual machines but this might affect the performance of the model and should be tested carefully on a case-by-case basis.
-
Yes. The InnerEye-DeepLearning Toolkit (opens in new tab) has been designed with usability and flexibility at its core, built on PyTorch and making extensive use of Microsoft Azure. The InnerEye-Deep Learning toolkit takes full advantage of Azure to provide GPUs for training, secure and scalable data storage. Azure Machine Learning is used for scaling clusters 0 to N compute nodes to train models on multiple GPUs. Our toolkit uses Azure Machine Learning to manage DevOps for ML (MLOps), including experiment traceability, experiment transparency model reproducibility, model management, model deployment, integration with Git and Continuous Integration (CI). In addition, the toolkit supports more advanced ML development features including cross-validation, hyperparameter tuning, building ensemble models, comparing new and existing models, and creating new models easily via a configuration-based approach, and inheriting from an existing architecture. For more details, see Azure Machine Learning – ML as a service | Microsoft Azure (opens in new tab)
-
The InnerEye-DeepLearning Toolkit (opens in new tab) allows you to develop your own models for different applications. This is enabled by using a configuration-based approach, and making the process of training models at scale easy. The InnerEye OSS tools may be used for developing classification, regression, and sequence models using only images as inputs, or a combination of images and non-imaging data as input. This supports typical use cases on medical data where measurements, biomarkers, or patient characteristics are often available in addition to images. We have had some positive, but limited, experience with MR, X-ray and OCT images. You can bring-your-own-models (opens in new tab) to make it easier to further develop and deploy them. We have examples (opens in new tab) for segmentation, classification, and sequence models that can take images or multiple modalities. It requires dedicated research expertise and effort to pursue these, and having the relevant, annotated clinical data for the algorithm’s training and optimization. Our technology is not designed for use in non-solid cancers such as leukemia.
-
The answer to this question depends highly on the task at hand. For instance, for the training of segmentation models, over 200 CT scan images are used per application for a reliable performance in the our JAMA Network Open paper Evaluation of Deep Learning to Augment Image-Guided Radiotherapy for Head and Neck and Prostate Cancers | Head and Neck Cancer | JAMA Network Open | JAMA Network (opens in new tab)
-
Support for these OSS tools is via GitHub Issues on the relevant repositories. We have released the InnerEye tools at no-cost as open-source software on GitHub under an MIT license to make these machine learning developments and technical components available to the community. These tools are open-source research projects and not Microsoft products. They take advantage of Microsoft Azure to make it easier to develop and deploy medical imaging models. If you have any feature requests, or find issues in the code, please create an issue on GitHub (opens in new tab). We monitor these issues and will look to respond via GitHub.
-
Yes, we welcome contributions and suggestions for our InnerEye OSS projects. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com (opens in new tab). When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA. This project has adopted the Microsoft Open Source Code of Conduct (opens in new tab). For more information see the Code of Conduct FAQ (opens in new tab) or contact [email protected] (opens in new tab) with any additional questions or comments.
-
No. We have released the InnerEye tools at no-cost as open-source software on GitHub under an MIT license to make these machine learning developments and technical components available to the community. These tools are open-source research projects and not Microsoft products. They take advantage of Microsoft Azure to make it easier to develop and deploy medical imaging models. For more information about Microsoft products, see Azure for Healthcare—Healthcare Solutions | Microsoft Azure (opens in new tab)
-
There are many organizations around the world building on these open-source tools for research and towards helping patients – see our News and Features page for more details (opens in new tab). Healthcare providers, companies, and partners may build on this toolkit to develop their own ML products and services using Microsoft Azure. Any use beyond research is subject to testing and regulatory approval as appropriate, such as FDA clearance, CE marking, or in-house exemption controls. We’re excited to see how people and organizations build on this to improve patient care. For more information about Microsoft products, see Azure for Healthcare—Healthcare Solutions | Microsoft Azure (opens in new tab)
-
Yes. We have released the InnerEye OSS tools at no-cost as open-source software on GitHub under an MIT license so healthcare providers, companies, and partners can use these to develop their own ML products and services. Any use beyond research is subject to testing and regulatory approval as appropriate, such as FDA clearance, CE marking, or in-house exemption controls. We’re excited to see how people and organizations build on this to improve patient care. For more information about Microsoft products, see Azure for Healthcare—Healthcare Solutions | Microsoft Azure (opens in new tab)
-
No. We have released the InnerEye tools at no-cost as open-source software on GitHub under an MIT license to make these machine learning developments and technical components available to the community. These tools are open-source research projects and not Microsoft products. They take advantage of Microsoft Azure to make it easier to develop and deploy medical imaging models. Healthcare providers, companies, and partners may build on these OSS projects to develop their own ML products and services. Any use beyond research is subject to testing and regulatory approval as appropriate, such as FDA clearance, CE marking, or in-house exemption controls. We’re excited to see how people and organizations build on this to improve patient care. For more information about Microsoft products, see Azure for Healthcare—Healthcare Solutions | Microsoft Azure (opens in new tab)
Disclaimer: The InnerEye Deep Learning Toolkit, Inner Eye-Gateway and InnerEye-Inference (collectively the “Research Tools”) are provided AS-IS for use by third parties for the purposes of research, experimental design and testing of machine learning models. The Research Tools are not intended or made available for clinical use as a medical device, clinical support, diagnostic tool, or other technology intended to be used in the diagnosis, cure, mitigation, treatment, or prevention of disease or other conditions. The Research Tools are not designed or intended to be a substitute for professional medical advice, diagnosis, treatment, or judgment and should not be used as such. All users are responsible for reviewing the output of the developed model to determine whether the model meets the user’s needs and for validating and evaluating the model before any clinical use. Microsoft does not warrant that the Research Tools or any materials provided in connection therewith will be sufficient for any medical purposes or meet the health or medical requirements of any person.