A compilation of research being done at Microsoft on accelerating biodiversity surveys with AI
We apply machine learning tools to a variety of image sources – including motion-triggered camera traps, aerial cameras, and microphones – to accelerate ecologists’ workflows. Our team spans Microsoft Research, Microsoft AI for Earth (opens in new tab), and the AI for Good Research Lab (opens in new tab).
Code
- Accelerating the processing of images from motion-triggered camera traps (opens in new tab)
- Species classification from handheld photos (opens in new tab)
- Active learning for wildlife detection in aerial images (opens in new tab)
- Detecting seals in aerial imagery (opens in new tab) (w/NOAA Fisheries (opens in new tab))
- Using machine learning to detect beluga whale calls in hydrophone recordings (opens in new tab) (w/NOAA Fisheries (opens in new tab))
- Multi-species bioacoustic classification (opens in new tab) (w/Sieve Analytics (opens in new tab))
Data
- In collaboration with many partners, we maintain a repository of labeled conservation images, the Labeled Information Library of Alexandria: Biology and Conservation (opens in new tab), aka “LILA”, pronounced “lie-la”.
Demos
- Species classification from handheld photos (opens in new tab)
- Object detection in camera trap images (opens in new tab)
APIs
- AI for Earth Species Classification API (opens in new tab)
- AI for Earth Camera Trap Image Processing API (opens in new tab)
Publications and publication-like things
- Juan M. Lavista Ferres, Derek E. Lee, Md Nasir, Yu-Chia Chen, Avleen S. Bijral, Fred B. Bercovitch, Monica L. Bond. Social connectedness and movements among communities of giraffes vary by sex and age class (opens in new tab). Science Direct | October 2021
- Zhong M, Taylor R, Bates N, Christey D, Basnet H, Flippin J, Palkovitz S, Dodhia R, Ferres JL. Acoustic detection of regionally rare bird species through deep convolutional neural networks (opens in new tab). Ecological Informatics. 2021 May 28:101333.
- Zhong M, Torterotot M, Branch TA, Stafford KM, Royer JY, Dodhia R, Lavista Ferres J. Detecting, classifying, and counting blue whale calls with Siamese neural networks (opens in new tab). The Journal of the Acoustical Society of America. 2021 May 6;149(5):3086-94.
- Gupta G, Kshirsagar M, Zhong M, Gholami S, Ferres JL. Comparing recurrent convolutional neural networks for large scale bird species classification (opens in new tab). Scientific reports. 2021 Aug 24;11(1):1-2.
- Robinson C, Ortiz A, Hughey L, Stabach JA, Ferres JM. Detecting Cattle and Elk in the Wild from Space (opens in new tab). arXiv preprint arXiv:2106.15448. 2021 Jun 29.
- Kellenberger B, Tuia D, Morris D. AIDE: Accelerating Image‐Based Ecological Surveys with Interactive Machine Learning (opens in new tab). Methods in Ecology and Evolution, September 2020.
- Norouzzadeh M, Morris D, Beery S, Joshi N, Jojic N, Clune J. A deep active learning system for species identification and counting in camera trap images (opens in new tab). Methods in Ecology and Evolution, October 2020.
- LeBien J, Zhong M, Campos-Cerqueira M, Velev JP, Dodhia R, Ferres JL, Aide TM. A pipeline for identification of bird and frog species in tropical soundscape recordings using a convolutional neural network (opens in new tab). Ecological Informatics. 2020 Jun 8:101113.
- Zhong M, LeBien J, Campos-Cerqueira M, Dodhia R, Ferres JL, Velev JP, Aide TM. Multispecies bioacoustic classification using transfer learning of deep convolutional neural networks with pseudo-labeling (opens in new tab). Applied Acoustics. 2020 Sep 1;166:107375.
- Shashidhara BM, Mehta D, Kale Y, Morris D, Hazen M. Sequence Information Channel Concatenation for Improving Camera Trap Image Burst Classification (opens in new tab). arXiv preprint arXiv:2005.00116. 2020 Apr 30.
- Zhong M, Castellote M, Dodhia R, Lavista J, Keogh M, Brewer A. Beluga whale acoustic signal classification using deep learning neural network models (opens in new tab). The Journal of the Acoustical Society of America 147, 1834 (2020).
- Beery S, Liu Y, Morris D, Piavis J, Kapoor A, Meister M, Perona P. Synthetic examples improve generalization for rare classes (opens in new tab). Winter Conference in Applications of Computer Vision (WACV), Aspen, CO, 2020.
- Zhong M, Castellote M, Dodhia R, Lavista Ferres J, Keogh M, Brewer A. Improving passive acoustic monitoring applications to the endangered Cook inlet beluga whale (opens in new tab). The Journal of the Acoustical Society of America. 2019 Oct;146(4).
- Norouzzadeh M, Morris D, Beery S, Joshi N, Jojic N, Clune J. A deep active learning system for species identification and counting in camera trap images (opens in new tab). arXiv preprint arXiv:1910.09716. 2019 Oct 22.
- Beery S, Morris D, Yang S. Efficient Pipeline for Camera Trap Image Review (opens in new tab). KDD Workshop Data Mining and AI for Conservation (DMAIC), Anchorage, AK, August 2019.
- Kaeser-Chen C, Birch T, Chou K, Gadot T, Adam H, Belongie S, Robertson T, Fegraus E, Morris D. Towards Ethical Deployment of AI for Conservation Systems (opens in new tab). KDD Workshop Data Mining and AI for Conservation (DMAIC), Anchorage, AK, August 2019.
- Beery S, Morris D, Yang S, Simon M, Norouzzadeh M, Joshi N. Efficient Pipeline for Automating Species ID in new Camera Trap Projects (opens in new tab). Biodiversity Information Science and Standards. 2019 Jun 19;3:e37222.
Blogs and blog-like things
- “Wildlife Protection Solutions helps protect the wildest places with Microsoft AI for Earth (opens in new tab)”. Microsoft Customer Story.
- “Using Artificial Intelligence to Identify Endangered Beluga Whales (opens in new tab)”. NOAA Fisheries News.
- “Developing Artificial Intelligence to Find Ice Seals and Polar Bears from the Sky (opens in new tab)”. NOAA Fisheries News.
- “Helping Scientists Protect Beluga Whales with Deep Learning (opens in new tab)”. Azure and AI for Earth on Medium.
- “Accelerating biodiversity surveys with Azure Machine Learning (opens in new tab)”. Azure and AI for Earth on Medium.
- “Artificial intelligence makes a splash in efforts to protect Alaska’s ice seals and beluga whales (opens in new tab)”. Microsoft News.
Other work at Microsoft
Whoever “we” are (i.e., the “we” who maintain this page), “we” are not the only ones at Microsoft working in this area. Here are some other great projects that our Microsoft colleagues have worked on in the biodiversity space:
- Counting Puffins with AI (opens in new tab) (with SSE Renewables)
- Accelerating camera trap workflows (opens in new tab) (with the Snow Leopard Trust (opens in new tab))
- Accelerating seabird surveys with active learning (opens in new tab) (with Conservation Metrics (opens in new tab))
- Accelerating seabird surveys with Azure ML Workbench (opens in new tab) (with Conservation Metrics (opens in new tab))
- Accelerating poacher detection (opens in new tab) (with Peace Parks (opens in new tab))
- Accelerating marine video surveys (opens in new tab) (with the Northern Territory)
- Accelerating giraffe surveys (opens in new tab) (with the Wild Nature Institute (opens in new tab)) (paper (opens in new tab))
- Accelerating drone-based wildlife surveys and land cover mapping (opens in new tab) (with Kakadu National Park)
- WWF/Microsoft Hackathon to user computer vision to identify illegal pangolin products in online marketplaces (opens in new tab) (with WWF)
- WWF/Microsoft Hackathon to use computer vision to identify illegal pangolin products in online marketplaces (opens in new tab) (with WWF)
- Microsoft/Heathrow collaboration that uses computer vision to screen for illegal wildlife products at Heathrow airport (opens in new tab) (with Heathrow) (video (opens in new tab))