Microsoft at ICLR 2022

Adversarial Retriever-Ranker for dense text retrieval

Hang Zhang, Yeyun Gong, Yelong Shen, Jiancheng Lv, Nan Duan, Weizhu Chen

PriorGrad: Improving Conditional Denoising Diffusion Models with Data-Dependent Adaptive Prior

Sang-gil Lee, Heeseung Kim, Chaehun Shin, Xu Tan, Chang Liu, Qi Meng, Tao Qin, Wei Chen, Sungroh Yoon, Tie-Yan Liu

Taming Sparsely Activated Transformer with Stochastic Experts

Simiao Zuo, Xiaodong Liu, Jian Jiao, Young Jin Kim, Hany Hassan Awadalla, Ruofei Zhang, Tuo Zhao, Jianfeng Gao

SQuant: On-the-Fly Data-Free Quantization via Diagonal Hessian Approximation

Cong Guo, Yuxian Qiu, Jingwen Leng, Xiaotian Gao, Chen Zhang, Yunxin Liu, Fan Yang, Yuhao Zhu, Minyi Guo

Modular Lifelong Reinforcement Learning via Neural Composition

Jorge A. Mendez, Harm van Seijen, Eric Eaton

Efficient Self-supervised Vision Transformers for Representation Learning

Chunyuan Li, Jianwei Yang, Pengchuan Zhang, Mei Gao, Bin Xiao, Xiyang Dai, Lu Yuan, Jianfeng Gao

Boosting the Certified Robustness of L-infinity Distance Nets

Bohang Zhang, Du Jiang, Di He, Liwei Wang

Provably Filtering Exogenous Distractors using Multistep Inverse Dynamics

Yonathan Efroni, Dipendra Misra, Akshay Krishnamurthy, Alekh Agarwal, John Langford

Retriever: Learning Content-Style Representation as a Token-Level Bipartite Graph

Dacheng Yin, Xuanchi Ren, Chong Luo, Yuwang Wang, Zhiwei Xiong, Wenjun Zeng

Learning Disentangled Representation by Exploiting Pretrained Generative Models: A Contrastive Learning View

Xuanchi Ren, Tao Yang, Yuwang Wang, Wenjun Zeng

Variational oracle guiding for reinforcement learning

Dongqi Han, Tadashi Kozuno, Xufang Luo, Zhao-Yun Chen, Kenji Doya, Yuqing Yang, Dongsheng Li

Target-Side Input Augmentation for Sequence to Sequence Generation

Shufang Xie, Ang Lv, Yingce Xia, Lijun Wu, Tao Qin, Tie-Yan Liu, Rui Yan

GNN is a Counter? Revisiting GNN for Question Answering

Kuan Wang, Yuyu Zhang, Diyi Yang, Le Song, Tao Qin

On the Connection between Local Attention and Dynamic Depth-wise Convolution

Qi Han, Zejia Fan, Qi Dai, Lei Sun, Ming-Ming Cheng, Jiaying Liu, Jingdong Wang

Simple GNN Regularisation for 3D Molecular Property Prediction and Beyond

Jonathan Godwin, Michael Schaarschmidt, Alexander Gaunt, Alvaro Sanchez-Gonzalez, Yulia Rubanova, Petar Veličković, James Kirkpatrick, Peter Battaglia

Learning to Downsample for Segmentation of Ultra-High Resolution Images

Chen Jin, Ryutaro Tanno, Thomy Mertzanidou, Eleftheria Panagiotaki, Daniel C. Alexander

Pretraining Text Encoders with Adversarial Mixture of Training Signal Generators

Yu Meng, Chenyan Xiong, Payal Bajaj, Saurabh Tiwary, Paul Bennett, Jiawei Han, Xia Song

Generalized Kernel Thinning

Raaz Dwivedi, Lester Mackey

No Parameters Left Behind: Sensitivity Guided Adaptive Learning Rate for Training Large Transformer Models

Chen Liang, Haoming Jiang, Simiao Zuo, Pengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu Chen, Tuo Zhao

Minimax Optimality (Probably) Doesn't Imply Distribution Learning for GANs

Sitan Chen, Jerry Li, Yuanzhi Li, Raghu Meka

Cross-Trajectory Representation Learning for Zero-Shot Generalization in RL

Bogdan Mazoure, Ahmed Ahmed, Patrick MacAlpine, Devon Hjelm, Andrey Kolobov

Synchromesh: Reliable code generation from pre-trained language models

Gabriel Poesia, Alex Polozov, Vu Le, Ashish Tiwari, Gustavo Soares, Chris Meek, Sumit Gulwani

Efficient Computation of Deep Nonlinear Infinite-Width Neural Networks that Learn Features

Greg Yang, Michael Santacroce, Edward J. Hu

Evaluating Distributional Distortion in Neural Language Modeling

Benjamin LeBrun, Alessandro Sordoni, Timothy J. O’Donnell

Triangle and Four Cycle Counting with Predictions in Graph Streams

Justin Y. Chen, Talya Eden, Piotr Indyk, Honghao Lin, Shyam Narayanan, Ronitt Rubinfeld, Sandeep Silwal, Tal Wagner, David Woodruff, Michael Zhang

TAPEX: Table Pre-training via Learning a Neural SQL Executor

Qian Liu, Bei Chen, Jiaqi Guo, Morteza Ziyadi, Zeqi Lin, Weizhu Chen, Jian-Guang Lou

Transformer Embeddings of Irregularly Spaced Events and Their Participants

Hongyuan Mei, Chenghao Yang, Jason Eisner

Differentially private fine-tuning of language models

Da Yu, Saurabh Naik, Arturs Backurs, Sivakanth Gopi, Huseyin Inan, Gautam Kamath, Janardhan (Jana) Kulkarni, Yin Tat Lee, Andre Manoel, Lukas Wutschitz, Sergey Yekhanin, Huishuai Zhang

Learning to Complete Code with Sketches

Daya Guo, Alexey Svyatkovskiy, Jian Yin, Nan Duan, Marc Brockschmidt, Miltos Allamanis

Recursive Disentanglement Network

Yixuan Chen, Yubin Shi, Dongsheng Li, Yujiang Wang, Mingzhi Dong, Yingying Zhao, Robert Dick, Qin Lv, Fan Yang, Li Shang

Sampling with Mirrored Stein Operators

Jiaxin Shi, Chang Liu, Lester Mackey

Creating Training Sets via Weak Indirect Supervision

Jieyu Zhang, Bohan Wang, Xiangchen Song, Yujing Wang, Yaming Yang, Jing Bai, Alexander Ratner

Optimal Transport for Causal Discovery

Ruibo Tu, Hedvig Kjellstrom, Kun Zhang, Cheng Zhang

Knowledge Infused Decoding

Ruibo Liu, Guoqing Zheng, Shashank Gupta, Radhika Gaonkar, Chongyang Gao, Soroush Vosoughi, Milad Shokouhi, Ahmed Awadallah

Provably Filtering Exogenous Distractors using Multistep Inverse Dynamics

Yonathan Efroni, Dipendra Misra, Akshay Krishnamurthy, Alekh Agarwal, John Langford

Missingness Bias in Model Debugging

Saachi Jain, Hadi Salman, Eric Wong, Pengchuan Zhang, Vibhav Vineet, Sai Vemprala, Aleksander Madry

Learning to Dequantise with Truncated Flows

Shawn Tan, Chin-Wei Huang, Alessandro Sordoni, Aaron Courville

Towards Building A Group-based Unsupervised Representation Disentanglement Framework

Tao Yang, Xuanchi Ren, Yuwang Wang, Wenjun Zeng, Nanning Zheng

Distribution Compression in Near-Linear Time

Abhishek Shetty, Raaz Dwivedi, Lester Mackey

Towards Deployment-Efficient Reinforcement Learning: Lower Bound and Optimality

Jiawei Huang, Jinglin Chen, Li Zhao, Tao Qin, Nan Jiang, Tie-Yan Liu

BEiT: BERT Pre-Training of Image Transformers

Hangbo Bao, Li Dong, Songhao Piao, Furu Wei

DEPTS: Deep Expansion Learning for Periodic Time Series Forecasting

Wei Fan, Shun Zheng, Xiaohan Yi, Wei Cao, Yanjie Fu, Jiang Bian, Tie-Yan Liu

Gradient Information Matters in Policy Optimization by Back-propagating through Model

Chongchong Li, Yue Wang, Wei Chen, Yuting Liu, Zhi-Ming Ma, Tie-Yan Liu

Node-GAM: Neural Generalized Additive Model For Interpretable Deep Learning

Chun-Hao Chang, Rich Caruana, Anna Goldenberg

BEiT: BERT Pre-Training of Image Transformers

Hangbo Bao, Li Dong, Songhao Piao, Furu Wei

BEiT: BERT Pre-Training of Image Transformers

Hangbo Bao, Li Dong, Songhao Piao, Furu Wei

Learning to Extend Molecular Scaffolds with Structural Motifs

Krzysztof Maziarz, Henry Jackson-Flux, Pashmina Cameron, Finton Sirockin, Nadine Schneider, Nikolaus Stiefl, Marwin Segler, Marc Brockschmidt

Exploiting Class Activation Value for Partial-Label Learning

Fei Zhang, Lei Feng, Bo Han, Tongliang Liu, Gang Niu, Tao Qin, Masashi Sugiyama

Explaining Point Processes by Learning Interpretable Temporal Logic Rules

Shuang Li, Mingquan Feng, Lu Wang, Abdelmajid Essofi, Yufeng Cao, Junchi Yan, Le Song

Learning to Extend Molecular Scaffolds with Structural Motifs

Krzysztof Maziarz, Henry Jackson-Flux, Pashmina Cameron, Finton Sirockin, Nadine Schneider, Nikolaus Stiefl, Marwin Segler, Marc Brockschmidt

Exploiting Class Activation Value for Partial-Label Learning

Fei Zhang, Lei Feng, Bo Han, Tongliang Liu, Gang Niu, Tao Qin, Masashi Sugiyama

Explaining Point Processes by Learning Interpretable Temporal Logic Rules

Shuang Li, Mingquan Feng, Lu Wang, Abdelmajid Essofi, Yufeng Cao, Junchi Yan, Le Song

Unsupervised Learning of Full-Waveform Inversion: Connecting CNN and Partial Differential Equation in a Loop

Peng Jin, Xitong Zhang, Yinpeng Chen, Sharon X. Huang, Zicheng Liu, Youzuo Lin

Learning Object-Oriented Dynamics for Planning from Text

Guiliang Liu, Ashutosh Adhikari, Amir-massoud Farahmand, Pascal Poupart

Axiomatic Explanations for Visual Search, Retrieval, and Similarity Learning

Mark Hamilton, Scott Lundberg, Stephanie Fu, Lei Zhang, William T. Freeman

Unsupervised Semantic Segmentation by Distilling Feature Correspondences

Mark Hamilton, Zhoutong Zhang, Bharath Hariharan, Noah Snavely, William T. Freeman

Mirror Descent Policy Optimization

Manan Tomar, Lior Shani, Yonathan Efroni, Mohammad Ghavamzadeh

Anti-Concentrated Confidence Bonuses for Scalable Exploration

Jordan T. Ash, Cyril Zhang, Surbhi Goel, Akshay Krishnamurthy, Sham M. Kakade

Unsupervised Learning of Full-Waveform Inversion: Connecting CNN and Partial Differential Equation in a Loop

Peng Jin, Xitong Zhang, Yinpeng Chen, Sharon X. Huang, Zicheng Liu, Youzuo Lin

Learning Object-Oriented Dynamics for Planning from Text

Guiliang Liu, Ashutosh Adhikari, Amir-massoud Farahmand, Pascal Poupart

Axiomatic Explanations for Visual Search, Retrieval, and Similarity Learning

Mark Hamilton, Scott Lundberg, Stephanie Fu, Lei Zhang, William T. Freeman

Unsupervised Semantic Segmentation by Distilling Feature Correspondences

Mark Hamilton, Zhoutong Zhang, Bharath Hariharan, Noah Snavely, William T. Freeman

Mirror Descent Policy Optimization

Manan Tomar, Lior Shani, Yonathan Efroni, Mohammad Ghavamzadeh

Anti-Concentrated Confidence Bonuses for Scalable Exploration

Jordan T. Ash, Cyril Zhang, Surbhi Goel, Akshay Krishnamurthy, Sham M. Kakade

Automatic Loss Function Search for Predict-Then-Optimize Problems with Strong Ranking Property

Boshi Wang, Jialin Yi, Hang Dong, Bo Qiao, Chuan Luo, Qingwei Lin 林庆维

LoRA: Low-Rank Adaptation of Large Language Models

Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen