Reducing AI’s Carbon Footprint
Pursuing more efficient AI
下载
XtremeDistil
2022年4月
XtremeDistil framework for distilling/compressing massive multilingual neural network models to tiny and efficient models for AI at scale.
LoRA
2022年4月
This repo contains the source code of the Python package loralib and several examples of how to integrate it with PyTorch models, such as those in HuggingFace. We only support PyTorch for now. See our paper for a detailed description…
Maximal Update Parametrization (μP)
2022年3月
Maximal Update Parametrization (μP) and Hyperparameter Transfer (μTransfer), in association with the paper: Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer
Archai – Reproducible Rapid Research for Network Architecture Search
2020年10月
Archai is a platform for Neural Network Search (NAS) that allow you to generate efficient deep networks for your applications. Archai aspires to accelerate NAS research by enabling easy mix and match between different techniques while ensuring reproducibility, self-documented hyper-parameters…
UniLM – Unified Language Model Pre-training
2019年10月
Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities.