Scaling Laws for Pre-training Agents and World Models
Tim Pearce, Tabish Rashid, Dave Bignell, Raluca Stevenson, Sam Devlin, Katja Hofmann
November 2024
Return to Microsoft Research Lab – Cambridge
Game Intelligence
Publications
Publications by Year
-
Learning Safety Constraints from Demonstrations with Unknown Rewards
David Lindner, Xin Chen, Sebastian Tschiatschek, Katja Hofmann, Andreas Krause
International Conference on Artificial Intelligence and Statistics | April 2024
Toward Human-AI Alignment in Large-Scale Multi-Player Games
Sugandha Sharma, Guy Davidson, Khimya Khetarpal, Anssi Kanervisto, Udit Arora, Katja Hofmann, Ida Momennejad
arXiv:2402.03575 | February 2024
Published by Microsoft
Transformer Neural Autoregressive Flows
Massimiliano Patacchiola, Aliaksandra Shysheya, Katja Hofmann, Richard E Turner
January 2024
-
Comparing the Efficacy of Fine-Tuning and Meta-Learning for Few-Shot Policy Imitation
Massimiliano Patacchiola, Mingfei Sun, Katja Hofmann, Richard E Turner
Conference on Lifelong Learning Agents | November 2023
Contrastive Meta-Learning for Partially Observable Few-Shot Learning
Adam Jelley, Amos Storkey, Antreas Antoniou, Sam Devlin
ICLR | May 2023
Imitating Human Behaviour with Diffusion Models
Tim Pearce, Tabish Rashid, Anssi Kanervisto, Dave Bignell, Mingfei Sun, Raluca Stevenson, Sergio Valcarcel Macua, Shanzheng Tan, Ida Momennejad, Katja Hofmann, Sam Devlin
ICLR 2023 | May 2023
Trust Region Bounds for Decentralized PPO Under Non-stationarity
Mingfei Sun, Sam Devlin, Jacob Beck, Katja Hofmann, Shimon Whiteson
International Conference on Autonomous Agents and Multiagent Systems (AAMAS) | May 2023
Best Paper Award
Navigates Like Me: Understanding How People Evaluate Human-Like AI in Video Games
Stephanie Milani, Arthur Juliani, Ida Momennejad, Raluca Stevenson, Jaroslaw Rzepecki, Alison Shaw, Gavin Costello, Fei Fang, Sam Devlin, Katja Hofmann
CHI | April 2023
-
Go-Explore Complex 3D Game Environments for Automated Reachability Testing
Cong Lu, Raluca Stevenson, Johan Verwey
IEEE Transactions on Games | December 2022, pp. 1-6
Uni[MASK]: Unified Inference in Sequential Decision Problems
Micah Carroll, Orr Paradise, Jessy Lin, Raluca Georgescu, Mingfei Sun, Dave Bignell, Stephanie Milani, Katja Hofmann, Matthew Hausknecht, Anca Dragan, Sam Devlin
2022 Neural Information Processing Systems | November 2022
Imitating Playstyle with Dynamic Time Warping Imitation
Mark Ferguson, Sam Devlin, Daniel Kudenko, James Alfred Walker
International Conference on the Foundations of Digital Games (FDG) | September 2022
How Humans Perceive Human-like Behavior in Video Game Navigation
Evelyn Zuniga, Stephanie Milani, Guy Leroy, Jaroslaw Rzepecki, Raluca Stevenson, Ida Momennejad, Dave Bignell, Mingfei Sun, Alison Shaw, Gavin Costello, Mikhail Jacob, Sam Devlin, Katja Hofmann
CHI 2022 | April 2022
Deterministic and Discriminative Imitation (D2-Imitation): Revisiting Adversarial Imitation for Sample Efficiency
Mingfei Sun, Sam Devlin, Katja Hofmann, Shimon Whiteson
AAAI 2022 | February 2022
-
Strategically Efficient Exploration in Competitive Multiagent Reinforcement Learning
Robert Loftin, Aadirupa Saha, Sam Devlin, Katja Hofmann
Conference on Uncertainty in Artificial Intelligence (UAI) 2021 | July 2021
Navigation Turing Test (NTT): Learning to Evaluate Human-Like Navigation
Sam Devlin, Raluca Stevenson, Ida Momennejad, Jaroslaw Rzepecki, Evelyn Zuniga, Gavin Costello, Guy Leroy, Ali Shaw, Katja Hofmann
2021 International Conference on Machine Learning | July 2021
Source code available at: https://github.com/microsoft/NTT
Exploration in Approximate Hyper-State Space for Meta Reinforcement Learning
Luisa Zintgraf, Leo Feng, Cong Lu, Maximilian Igl, Kristian Hartikainen, Katja Hofmann, Shimon Whiteson
2021 International Conference on Machine Learning | July 2021
TeachMyAgent: a Benchmark for Automatic Curriculum Learning in Deep RL
Clément Romac, Rémy Portelas, Katja Hofmann, Pierre-Yves Oudeyer
2021 International Conference on Machine Learning | July 2021
Acting with Style: Towards Designer-centred Reinforcement Learning for the Video Games Industry
Batu Aytemiz, Mikhail Jacob, Sam Devlin
CHI 2021 Workshop on Reinforcement Learning for Humans, Computer, and Interaction (RL4HCI) | May 2021
Deep Interactive Bayesian Reinforcement Learning via Meta-Learning
Luisa Zintgraf, Sam Devlin, Kamil Ciosek, Shimon Whiteson, Katja Hofmann
20th International Conference on Autonomous Agents and Multiagent Systems | May 2021
Difference Rewards Policy Gradients
Jacopo Castellini, Sam Devlin, Frans A. Oliehoek, Rahul Savani
20th International Conference on Autonomous Agents and Multiagent Systems | May 2021
Evaluating the Robustness of Collaborative Agents
Paul Knott, Micah Carroll, Sam Devlin, Kamil Ciosek, Katja Hofmann, Anca Dragan, Rohin Shah
20th International Conference on Autonomous Agents and Multiagent Systems | May 2021
The MineRL 2020 Competition on Sample Efficient Reinforcement Learning using Human Priors
William H. Guss, Mario Ynocente Castro, Sam Devlin, Brandon Houghton, Noboru Sean Kuno, Crissman Loomis, Stephanie Milani, Sharada Mohanty, Keisuke Nakata, Ruslan Salakhutdinov, John Schulman, Shinya Shiroshita, Nicholay Topin, Avinash Ummadisingu, Oriol Vinyals
January 2021
-
“It’s Unwieldy and It Takes a Lot of Time.” Challenges and Opportunities for Creating Agents in Commercial Games
Mikhail Jacob, Sam Devlin, Katja Hofmann
16th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment | October 2020
AIIDE 2020 BEST PAPER AWARD
Best paper award winner.
Player Style Clustering without Game Variables
Mark Ferguson, Sam Devlin, Daniel Kudenko, James Walker
FDG ’20: International Conference on the Foundations of Digital Games | September 2020
Variational Integrator Networks for Physically Meaningful Embeddings
Steindór Sæmundsson, Alexander Terenin, Katja Hofmann, Marc Peter Deisenroth
Twenty-Third International Conference on Artificial Intelligence and Statistics (AISTATS) | June 2020
Combining No-regret and Q-learning
Ian A. Kash, Michael Sullins, Katja Hofmann
2020 Adaptive Agents and Multi-Agents Systems | May 2020
A Novel Individually Rational Objective In Multi-Agent Multi-Armed Bandit: Algorithms and Regret Bounds
Aristide Tossou, Christos Dimitrakakis, Jaroslaw Rzepecki, Katja Hofmann
International Conference on Autonomous Agents and Multiagent Systems (AAMAS) | May 2020
AMRL: Aggregated Memory For Reinforcement Learning
Jacob Beck, Kamil Ciosek, Sam Devlin, Sebastian Tschiatschek, Cheng Zhang, Katja Hofmann
Eighth International Conference on Learning Representations (ICLR) | April 2020
Conservative Uncertainty Estimation By Fitting Prior Networks
Kamil Ciosek, Vincent Fortuin, Ryota Tomioka, Katja Hofmann, Richard Turner
Eighth International Conference on Learning Representations (ICLR) | April 2020
VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learning
Luisa Zintgraf, Kyriacos Shiarlis, Maximilian Igl, Sebastian Schulze, Yarin Gal, Katja Hofmann, Shimon Whiteson
Eighth International Conference on Learning Representations (ICLR) | April 2020
Rolling Horizon Evolutionary Algorithms for General Video Game Playing
Raluca Gaina, Sam Devlin, Simon M. Lucas, Diego Perez-Liebana
IEEE Transactions on Games | March 2020
-
Generalization in Reinforcement Learning with Selective Noise Injection and Information Bottleneck
Maximilian Igl, Kamil Ciosek, Yingzhen Li, Sebastian Tschiatschek, Cheng Zhang, Sam Devlin, Katja Hofmann
Thirty-third Conference on Neural Information Processing Systems (NeurIPS) | December 2019
Better Exploration with Optimistic Actor-Critic
Kamil Ciosek, Quan Vuong, Robert Loftin, Katja Hofmann
33rd Conference on Neural Information Processing Systems (NeurIPS) | December 2019
Reinforcement Learning: Past, Present, and Future Perspectives
Thirty-third Conference on Neural Information Processing Systems (NeurIPS) | December 2019
Tutorial
Successor Uncertainties: Exploration and Uncertainty in Temporal Difference Learning
David Janz, Jiri Hron, Przemysław Mazur, Katja Hofmann, Jose Miguel Hernandez Lobato, Sebastian Tschiatschek
Thirty-third Conference on Neural Information Processing Systems (NeurIPS) | December 2019
The MineRL Competition on Sample Efficient Reinforcement Learning using Human Priors
William H Guss, Cayden Codel, Katja Hofmann, Brandon Houghton, Noboru Sean Kuno, Stephanie Milani, Sharada Mohanty, Diego Perez Liebana, Ruslan Salakhutdinov, Nicholay Topin, Manuela Veloso, Philip Wang
Thirty-third Conference on Neural Information Processing Systems (NeurIPS) Competition track | December 2019
Minecraft as AI Playground and Laboratory
Proceedings of the Annual Symposium on Computer-Human Interaction in Play (ChiPlay) | November 2019
Opening keynote (extended abstract)
Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environments
Rémy Portelas, Pierre-Yves Oudeyer, Cédric Colas, Katja Hofmann
Third Conference on Robot Learning (CoRL) | October 2019
MazeExplorer: A Customisable 3D Benchmark for Assessing Generalisation in Reinforcement Learning
Luke Harries, Sebastian Lee, Jaroslaw Rzepecki, Katja Hofmann, Sam Devlin
IEEE Conference on Games | August 2019
A Generalized Framework for Self-Play Training
Daniel Hernandez, Kevin Denamganai, Yuan Gao, Peter York, Sam Devlin, Spyridon Samothrakis, James Walker
IEEE Conference on Games | August 2019
Win or Learn Fast Proximal Policy Optimisation
Dino Ratcliffe, Katja Hofmann, Sam Devlin
IEEE Conference on Games | August 2019
Fast Context Adaptation via Meta-Learning
Luisa Zintgraf, Kyriacos Shiarlis, Vitaly Kurin, Katja Hofmann, Shimon Whiteson
Thirty-sixth International Conference on Machine Learning (ICML) | June 2019
Customizing Scripted Bots: Sample Efficient Imitation Learning for Human-like Behavior in Minecraft
Brian Broll, Matthew Hausknecht, Dave Bignell, Adith Swaminathan
The Adaptive and Learning Agents (ALA) Workshop at AAMAS 2019 | May 2019
The Multi-Agent Reinforcement Learning in MalmÖ (MARLÖ) Competition
Diego Perez-Liebana, Katja Hofmann, Sharada Prasanna Mohanty, Noboru Sean Kuno, Andre Kramer, Sam Devlin, Raluca D. Gaina, Daniel Ionita
1901.08129 | January 2019
Published by Arxiv
-
How Players Speak to an Intelligent Game Character Using Natural Language Messages
Fraser Allison, Ewa Luger, Katja Hofmann
Transactions of the Digital Games Research Association | December 2018, Vol 4(2): pp. 1-49
Combining No-regret and Q-learning (2018)
Ian Kash, Katja Hofmann
The 14th European Workshop on Reinforcement Learning (EWRL 2018) | October 2018
Also presented at the AAAI-19 workshop on Reinforcement Learning in Games.
Learning good policies from suboptimal demonstrations
Yuxiang Li, Ian Kash, Katja Hofmann
The 14th European Workshop on Reinforcement Learning (EWRL 2018) | October 2018
Advancements in Dueling Bandits
Yanan Sui, Masrour Zoghi, Katja Hofmann, Yisong Yue
Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence | July 2018
Depth and nonlinearity induce implicit exploration for RL
Justas Dauparas, Ryota Tomioka, Katja Hofmann
ICML workshop on Exploration in RL | July 2018
TrueSkill 2: An improved Bayesian skill rating system
Tom Minka, Ryan Cleven, Yordan Zaykov
MSR-TR-2018-8 | March 2018
Published by Microsoft
-
A Deep Learning Approach for Joint Video Frame and Reward Prediction in Atari Games
Felix Leibfried, Nate Kushman, Katja Hofmann
ICML workshop on Principled Approaches to Deep Learning | August 2017
Spontaneous Interactions with a Virtually Embodied Intelligent Assistant in Minecraft
Fraser Allison, Ewa Luger, Katja Hofmann
CHI EA ’17 Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems | May 2017
The Atari Grand Challenge Dataset
Vitaly Kurin, Sebastian Nowozin, Katja Hofmann, Lucas Beyer, Bastian Leibe
MSR-TR-2017-61 | May 2017
Published by Microsoft
-
Memory Lens – How Much Memory Does an Agent Use?
Christoph Dann, Katja Hofmann, Sebastian Nowozin
The 13th European Workshop on Reinforcement Learning (EWRL) 2016 | December 2016
Also presented at the Interpretable ML for Complex Systems Workshop at NeurIPS 2016
Decoding multitask DQN in the world of Minecraft
Lydia T. Liu, Urun Dogan, Katja Hofmann
The 13th European Workshop on Reinforcement Learning (EWRL) 2016 | December 2016
Also presented at the 11th Women in Machine Learning Workshop and the Deep Reinforcement Learning Workshop at NeurIPS 2016.
The Malmo Platform for Artificial Intelligence Experimentation
Matthew Johnson, Katja Hofmann, Tm Hutton, David Bignell, Katja Hofmann
25th International Joint Conference on Artificial Intelligence (IJCAI-16) | July 2016
Publications by Research Area
-
Scaling Laws for Pre-training Agents and World Models
Tim Pearce, Tabish Rashid, Dave Bignell, Raluca Stevenson, Sam Devlin, Katja Hofmann
November 2024
Learning Safety Constraints from Demonstrations with Unknown Rewards
David Lindner, Xin Chen, Sebastian Tschiatschek, Katja Hofmann, Andreas Krause
International Conference on Artificial Intelligence and Statistics | April 2024
Toward Human-AI Alignment in Large-Scale Multi-Player Games
Sugandha Sharma, Guy Davidson, Khimya Khetarpal, Anssi Kanervisto, Udit Arora, Katja Hofmann, Ida Momennejad
arXiv:2402.03575 | February 2024
Published by Microsoft
Transformer Neural Autoregressive Flows
Massimiliano Patacchiola, Aliaksandra Shysheya, Katja Hofmann, Richard E Turner
January 2024
Comparing the Efficacy of Fine-Tuning and Meta-Learning for Few-Shot Policy Imitation
Massimiliano Patacchiola, Mingfei Sun, Katja Hofmann, Richard E Turner
Conference on Lifelong Learning Agents | November 2023
Contrastive Meta-Learning for Partially Observable Few-Shot Learning
Adam Jelley, Amos Storkey, Antreas Antoniou, Sam Devlin
ICLR | May 2023
Trust Region Bounds for Decentralized PPO Under Non-stationarity
Mingfei Sun, Sam Devlin, Jacob Beck, Katja Hofmann, Shimon Whiteson
International Conference on Autonomous Agents and Multiagent Systems (AAMAS) | May 2023
Best Paper Award
Imitating Human Behaviour with Diffusion Models
Tim Pearce, Tabish Rashid, Anssi Kanervisto, Dave Bignell, Mingfei Sun, Raluca Stevenson, Sergio Valcarcel Macua, Shanzheng Tan, Ida Momennejad, Katja Hofmann, Sam Devlin
ICLR 2023 | May 2023
Navigates Like Me: Understanding How People Evaluate Human-Like AI in Video Games
Stephanie Milani, Arthur Juliani, Ida Momennejad, Raluca Stevenson, Jaroslaw Rzepecki, Alison Shaw, Gavin Costello, Fei Fang, Sam Devlin, Katja Hofmann
CHI | April 2023
Go-Explore Complex 3D Game Environments for Automated Reachability Testing
Cong Lu, Raluca Stevenson, Johan Verwey
IEEE Transactions on Games | December 2022, pp. 1-6
Uni[MASK]: Unified Inference in Sequential Decision Problems
Micah Carroll, Orr Paradise, Jessy Lin, Raluca Georgescu, Mingfei Sun, Dave Bignell, Stephanie Milani, Katja Hofmann, Matthew Hausknecht, Anca Dragan, Sam Devlin
2022 Neural Information Processing Systems | November 2022
Imitating Playstyle with Dynamic Time Warping Imitation
Mark Ferguson, Sam Devlin, Daniel Kudenko, James Alfred Walker
International Conference on the Foundations of Digital Games (FDG) | September 2022
Deterministic and Discriminative Imitation (D2-Imitation): Revisiting Adversarial Imitation for Sample Efficiency
Mingfei Sun, Sam Devlin, Katja Hofmann, Shimon Whiteson
AAAI 2022 | February 2022
Strategically Efficient Exploration in Competitive Multiagent Reinforcement Learning
Robert Loftin, Aadirupa Saha, Sam Devlin, Katja Hofmann
Conference on Uncertainty in Artificial Intelligence (UAI) 2021 | July 2021
Navigation Turing Test (NTT): Learning to Evaluate Human-Like Navigation
Sam Devlin, Raluca Stevenson, Ida Momennejad, Jaroslaw Rzepecki, Evelyn Zuniga, Gavin Costello, Guy Leroy, Ali Shaw, Katja Hofmann
2021 International Conference on Machine Learning | July 2021
Source code available at: https://github.com/microsoft/NTT
TeachMyAgent: a Benchmark for Automatic Curriculum Learning in Deep RL
Clément Romac, Rémy Portelas, Katja Hofmann, Pierre-Yves Oudeyer
2021 International Conference on Machine Learning | July 2021
Exploration in Approximate Hyper-State Space for Meta Reinforcement Learning
Luisa Zintgraf, Leo Feng, Cong Lu, Maximilian Igl, Kristian Hartikainen, Katja Hofmann, Shimon Whiteson
2021 International Conference on Machine Learning | July 2021
Acting with Style: Towards Designer-centred Reinforcement Learning for the Video Games Industry
Batu Aytemiz, Mikhail Jacob, Sam Devlin
CHI 2021 Workshop on Reinforcement Learning for Humans, Computer, and Interaction (RL4HCI) | May 2021
Difference Rewards Policy Gradients
Jacopo Castellini, Sam Devlin, Frans A. Oliehoek, Rahul Savani
20th International Conference on Autonomous Agents and Multiagent Systems | May 2021
Deep Interactive Bayesian Reinforcement Learning via Meta-Learning
Luisa Zintgraf, Sam Devlin, Kamil Ciosek, Shimon Whiteson, Katja Hofmann
20th International Conference on Autonomous Agents and Multiagent Systems | May 2021
Evaluating the Robustness of Collaborative Agents
Paul Knott, Micah Carroll, Sam Devlin, Kamil Ciosek, Katja Hofmann, Anca Dragan, Rohin Shah
20th International Conference on Autonomous Agents and Multiagent Systems | May 2021
The MineRL 2020 Competition on Sample Efficient Reinforcement Learning using Human Priors
William H. Guss, Mario Ynocente Castro, Sam Devlin, Brandon Houghton, Noboru Sean Kuno, Crissman Loomis, Stephanie Milani, Sharada Mohanty, Keisuke Nakata, Ruslan Salakhutdinov, John Schulman, Shinya Shiroshita, Nicholay Topin, Avinash Ummadisingu, Oriol Vinyals
January 2021
“It’s Unwieldy and It Takes a Lot of Time.” Challenges and Opportunities for Creating Agents in Commercial Games
Mikhail Jacob, Sam Devlin, Katja Hofmann
16th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment | October 2020
AIIDE 2020 BEST PAPER AWARD
Best paper award winner.
Player Style Clustering without Game Variables
Mark Ferguson, Sam Devlin, Daniel Kudenko, James Walker
FDG ’20: International Conference on the Foundations of Digital Games | September 2020
Variational Integrator Networks for Physically Meaningful Embeddings
Steindór Sæmundsson, Alexander Terenin, Katja Hofmann, Marc Peter Deisenroth
Twenty-Third International Conference on Artificial Intelligence and Statistics (AISTATS) | June 2020
Combining No-regret and Q-learning
Ian A. Kash, Michael Sullins, Katja Hofmann
2020 Adaptive Agents and Multi-Agents Systems | May 2020
A Novel Individually Rational Objective In Multi-Agent Multi-Armed Bandit: Algorithms and Regret Bounds
Aristide Tossou, Christos Dimitrakakis, Jaroslaw Rzepecki, Katja Hofmann
International Conference on Autonomous Agents and Multiagent Systems (AAMAS) | May 2020
VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learning
Luisa Zintgraf, Kyriacos Shiarlis, Maximilian Igl, Sebastian Schulze, Yarin Gal, Katja Hofmann, Shimon Whiteson
Eighth International Conference on Learning Representations (ICLR) | April 2020
Conservative Uncertainty Estimation By Fitting Prior Networks
Kamil Ciosek, Vincent Fortuin, Ryota Tomioka, Katja Hofmann, Richard Turner
Eighth International Conference on Learning Representations (ICLR) | April 2020
AMRL: Aggregated Memory For Reinforcement Learning
Jacob Beck, Kamil Ciosek, Sam Devlin, Sebastian Tschiatschek, Cheng Zhang, Katja Hofmann
Eighth International Conference on Learning Representations (ICLR) | April 2020
Rolling Horizon Evolutionary Algorithms for General Video Game Playing
Raluca Gaina, Sam Devlin, Simon M. Lucas, Diego Perez-Liebana
IEEE Transactions on Games | March 2020
Generalization in Reinforcement Learning with Selective Noise Injection and Information Bottleneck
Maximilian Igl, Kamil Ciosek, Yingzhen Li, Sebastian Tschiatschek, Cheng Zhang, Sam Devlin, Katja Hofmann
Thirty-third Conference on Neural Information Processing Systems (NeurIPS) | December 2019
Better Exploration with Optimistic Actor-Critic
Kamil Ciosek, Quan Vuong, Robert Loftin, Katja Hofmann
33rd Conference on Neural Information Processing Systems (NeurIPS) | December 2019
Reinforcement Learning: Past, Present, and Future Perspectives
Thirty-third Conference on Neural Information Processing Systems (NeurIPS) | December 2019
Tutorial
Successor Uncertainties: Exploration and Uncertainty in Temporal Difference Learning
David Janz, Jiri Hron, Przemysław Mazur, Katja Hofmann, Jose Miguel Hernandez Lobato, Sebastian Tschiatschek
Thirty-third Conference on Neural Information Processing Systems (NeurIPS) | December 2019
The MineRL Competition on Sample Efficient Reinforcement Learning using Human Priors
William H Guss, Cayden Codel, Katja Hofmann, Brandon Houghton, Noboru Sean Kuno, Stephanie Milani, Sharada Mohanty, Diego Perez Liebana, Ruslan Salakhutdinov, Nicholay Topin, Manuela Veloso, Philip Wang
Thirty-third Conference on Neural Information Processing Systems (NeurIPS) Competition track | December 2019
Minecraft as AI Playground and Laboratory
Proceedings of the Annual Symposium on Computer-Human Interaction in Play (ChiPlay) | November 2019
Opening keynote (extended abstract)
Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environments
Rémy Portelas, Pierre-Yves Oudeyer, Cédric Colas, Katja Hofmann
Third Conference on Robot Learning (CoRL) | October 2019
MazeExplorer: A Customisable 3D Benchmark for Assessing Generalisation in Reinforcement Learning
Luke Harries, Sebastian Lee, Jaroslaw Rzepecki, Katja Hofmann, Sam Devlin
IEEE Conference on Games | August 2019
A Generalized Framework for Self-Play Training
Daniel Hernandez, Kevin Denamganai, Yuan Gao, Peter York, Sam Devlin, Spyridon Samothrakis, James Walker
IEEE Conference on Games | August 2019
Win or Learn Fast Proximal Policy Optimisation
Dino Ratcliffe, Katja Hofmann, Sam Devlin
IEEE Conference on Games | August 2019
Fast Context Adaptation via Meta-Learning
Luisa Zintgraf, Kyriacos Shiarlis, Vitaly Kurin, Katja Hofmann, Shimon Whiteson
Thirty-sixth International Conference on Machine Learning (ICML) | June 2019
Customizing Scripted Bots: Sample Efficient Imitation Learning for Human-like Behavior in Minecraft
Brian Broll, Matthew Hausknecht, Dave Bignell, Adith Swaminathan
The Adaptive and Learning Agents (ALA) Workshop at AAMAS 2019 | May 2019
The Multi-Agent Reinforcement Learning in MalmÖ (MARLÖ) Competition
Diego Perez-Liebana, Katja Hofmann, Sharada Prasanna Mohanty, Noboru Sean Kuno, Andre Kramer, Sam Devlin, Raluca D. Gaina, Daniel Ionita
1901.08129 | January 2019
Published by Arxiv
How Players Speak to an Intelligent Game Character Using Natural Language Messages
Fraser Allison, Ewa Luger, Katja Hofmann
Transactions of the Digital Games Research Association | December 2018, Vol 4(2): pp. 1-49
Combining No-regret and Q-learning (2018)
Ian Kash, Katja Hofmann
The 14th European Workshop on Reinforcement Learning (EWRL 2018) | October 2018
Also presented at the AAAI-19 workshop on Reinforcement Learning in Games.
Learning good policies from suboptimal demonstrations
Yuxiang Li, Ian Kash, Katja Hofmann
The 14th European Workshop on Reinforcement Learning (EWRL 2018) | October 2018
Advancements in Dueling Bandits
Yanan Sui, Masrour Zoghi, Katja Hofmann, Yisong Yue
Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence | July 2018
Depth and nonlinearity induce implicit exploration for RL
Justas Dauparas, Ryota Tomioka, Katja Hofmann
ICML workshop on Exploration in RL | July 2018
TrueSkill 2: An improved Bayesian skill rating system
Tom Minka, Ryan Cleven, Yordan Zaykov
MSR-TR-2018-8 | March 2018
Published by Microsoft
A Deep Learning Approach for Joint Video Frame and Reward Prediction in Atari Games
Felix Leibfried, Nate Kushman, Katja Hofmann
ICML workshop on Principled Approaches to Deep Learning | August 2017
Spontaneous Interactions with a Virtually Embodied Intelligent Assistant in Minecraft
Fraser Allison, Ewa Luger, Katja Hofmann
CHI EA ’17 Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems | May 2017
The Atari Grand Challenge Dataset
Vitaly Kurin, Sebastian Nowozin, Katja Hofmann, Lucas Beyer, Bastian Leibe
MSR-TR-2017-61 | May 2017
Published by Microsoft
Memory Lens – How Much Memory Does an Agent Use?
Christoph Dann, Katja Hofmann, Sebastian Nowozin
The 13th European Workshop on Reinforcement Learning (EWRL) 2016 | December 2016
Also presented at the Interpretable ML for Complex Systems Workshop at NeurIPS 2016
Decoding multitask DQN in the world of Minecraft
Lydia T. Liu, Urun Dogan, Katja Hofmann
The 13th European Workshop on Reinforcement Learning (EWRL) 2016 | December 2016
Also presented at the 11th Women in Machine Learning Workshop and the Deep Reinforcement Learning Workshop at NeurIPS 2016.
The Malmo Platform for Artificial Intelligence Experimentation
Matthew Johnson, Katja Hofmann, Tm Hutton, David Bignell, Katja Hofmann
25th International Joint Conference on Artificial Intelligence (IJCAI-16) | July 2016
-
How Humans Perceive Human-like Behavior in Video Game Navigation
Evelyn Zuniga, Stephanie Milani, Guy Leroy, Jaroslaw Rzepecki, Raluca Stevenson, Ida Momennejad, Dave Bignell, Mingfei Sun, Alison Shaw, Gavin Costello, Mikhail Jacob, Sam Devlin, Katja Hofmann
CHI 2022 | April 2022
Acting with Style: Towards Designer-centred Reinforcement Learning for the Video Games Industry
Batu Aytemiz, Mikhail Jacob, Sam Devlin
CHI 2021 Workshop on Reinforcement Learning for Humans, Computer, and Interaction (RL4HCI) | May 2021
How Players Speak to an Intelligent Game Character Using Natural Language Messages
Fraser Allison, Ewa Luger, Katja Hofmann
Transactions of the Digital Games Research Association | December 2018, Vol 4(2): pp. 1-49
Spontaneous Interactions with a Virtually Embodied Intelligent Assistant in Minecraft
Fraser Allison, Ewa Luger, Katja Hofmann
CHI EA ’17 Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems | May 2017
Publications by Type
-
Learning Safety Constraints from Demonstrations with Unknown Rewards
David Lindner, Xin Chen, Sebastian Tschiatschek, Katja Hofmann, Andreas Krause
International Conference on Artificial Intelligence and Statistics | April 2024
Comparing the Efficacy of Fine-Tuning and Meta-Learning for Few-Shot Policy Imitation
Massimiliano Patacchiola, Mingfei Sun, Katja Hofmann, Richard E Turner
Conference on Lifelong Learning Agents | November 2023
Contrastive Meta-Learning for Partially Observable Few-Shot Learning
Adam Jelley, Amos Storkey, Antreas Antoniou, Sam Devlin
ICLR | May 2023
Trust Region Bounds for Decentralized PPO Under Non-stationarity
Mingfei Sun, Sam Devlin, Jacob Beck, Katja Hofmann, Shimon Whiteson
International Conference on Autonomous Agents and Multiagent Systems (AAMAS) | May 2023
Best Paper Award
Imitating Human Behaviour with Diffusion Models
Tim Pearce, Tabish Rashid, Anssi Kanervisto, Dave Bignell, Mingfei Sun, Raluca Stevenson, Sergio Valcarcel Macua, Shanzheng Tan, Ida Momennejad, Katja Hofmann, Sam Devlin
ICLR 2023 | May 2023
Navigates Like Me: Understanding How People Evaluate Human-Like AI in Video Games
Stephanie Milani, Arthur Juliani, Ida Momennejad, Raluca Stevenson, Jaroslaw Rzepecki, Alison Shaw, Gavin Costello, Fei Fang, Sam Devlin, Katja Hofmann
CHI | April 2023
Uni[MASK]: Unified Inference in Sequential Decision Problems
Micah Carroll, Orr Paradise, Jessy Lin, Raluca Georgescu, Mingfei Sun, Dave Bignell, Stephanie Milani, Katja Hofmann, Matthew Hausknecht, Anca Dragan, Sam Devlin
2022 Neural Information Processing Systems | November 2022
Imitating Playstyle with Dynamic Time Warping Imitation
Mark Ferguson, Sam Devlin, Daniel Kudenko, James Alfred Walker
International Conference on the Foundations of Digital Games (FDG) | September 2022
How Humans Perceive Human-like Behavior in Video Game Navigation
Evelyn Zuniga, Stephanie Milani, Guy Leroy, Jaroslaw Rzepecki, Raluca Stevenson, Ida Momennejad, Dave Bignell, Mingfei Sun, Alison Shaw, Gavin Costello, Mikhail Jacob, Sam Devlin, Katja Hofmann
CHI 2022 | April 2022
Deterministic and Discriminative Imitation (D2-Imitation): Revisiting Adversarial Imitation for Sample Efficiency
Mingfei Sun, Sam Devlin, Katja Hofmann, Shimon Whiteson
AAAI 2022 | February 2022
Strategically Efficient Exploration in Competitive Multiagent Reinforcement Learning
Robert Loftin, Aadirupa Saha, Sam Devlin, Katja Hofmann
Conference on Uncertainty in Artificial Intelligence (UAI) 2021 | July 2021
Navigation Turing Test (NTT): Learning to Evaluate Human-Like Navigation
Sam Devlin, Raluca Stevenson, Ida Momennejad, Jaroslaw Rzepecki, Evelyn Zuniga, Gavin Costello, Guy Leroy, Ali Shaw, Katja Hofmann
2021 International Conference on Machine Learning | July 2021
Source code available at: https://github.com/microsoft/NTT
TeachMyAgent: a Benchmark for Automatic Curriculum Learning in Deep RL
Clément Romac, Rémy Portelas, Katja Hofmann, Pierre-Yves Oudeyer
2021 International Conference on Machine Learning | July 2021
Exploration in Approximate Hyper-State Space for Meta Reinforcement Learning
Luisa Zintgraf, Leo Feng, Cong Lu, Maximilian Igl, Kristian Hartikainen, Katja Hofmann, Shimon Whiteson
2021 International Conference on Machine Learning | July 2021
Acting with Style: Towards Designer-centred Reinforcement Learning for the Video Games Industry
Batu Aytemiz, Mikhail Jacob, Sam Devlin
CHI 2021 Workshop on Reinforcement Learning for Humans, Computer, and Interaction (RL4HCI) | May 2021
Difference Rewards Policy Gradients
Jacopo Castellini, Sam Devlin, Frans A. Oliehoek, Rahul Savani
20th International Conference on Autonomous Agents and Multiagent Systems | May 2021
Deep Interactive Bayesian Reinforcement Learning via Meta-Learning
Luisa Zintgraf, Sam Devlin, Kamil Ciosek, Shimon Whiteson, Katja Hofmann
20th International Conference on Autonomous Agents and Multiagent Systems | May 2021
Evaluating the Robustness of Collaborative Agents
Paul Knott, Micah Carroll, Sam Devlin, Kamil Ciosek, Katja Hofmann, Anca Dragan, Rohin Shah
20th International Conference on Autonomous Agents and Multiagent Systems | May 2021
“It’s Unwieldy and It Takes a Lot of Time.” Challenges and Opportunities for Creating Agents in Commercial Games
Mikhail Jacob, Sam Devlin, Katja Hofmann
16th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment | October 2020
AIIDE 2020 BEST PAPER AWARD
Best paper award winner.
Player Style Clustering without Game Variables
Mark Ferguson, Sam Devlin, Daniel Kudenko, James Walker
FDG ’20: International Conference on the Foundations of Digital Games | September 2020
Variational Integrator Networks for Physically Meaningful Embeddings
Steindór Sæmundsson, Alexander Terenin, Katja Hofmann, Marc Peter Deisenroth
Twenty-Third International Conference on Artificial Intelligence and Statistics (AISTATS) | June 2020
Combining No-regret and Q-learning
Ian A. Kash, Michael Sullins, Katja Hofmann
2020 Adaptive Agents and Multi-Agents Systems | May 2020
A Novel Individually Rational Objective In Multi-Agent Multi-Armed Bandit: Algorithms and Regret Bounds
Aristide Tossou, Christos Dimitrakakis, Jaroslaw Rzepecki, Katja Hofmann
International Conference on Autonomous Agents and Multiagent Systems (AAMAS) | May 2020
VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learning
Luisa Zintgraf, Kyriacos Shiarlis, Maximilian Igl, Sebastian Schulze, Yarin Gal, Katja Hofmann, Shimon Whiteson
Eighth International Conference on Learning Representations (ICLR) | April 2020
Conservative Uncertainty Estimation By Fitting Prior Networks
Kamil Ciosek, Vincent Fortuin, Ryota Tomioka, Katja Hofmann, Richard Turner
Eighth International Conference on Learning Representations (ICLR) | April 2020
AMRL: Aggregated Memory For Reinforcement Learning
Jacob Beck, Kamil Ciosek, Sam Devlin, Sebastian Tschiatschek, Cheng Zhang, Katja Hofmann
Eighth International Conference on Learning Representations (ICLR) | April 2020
Generalization in Reinforcement Learning with Selective Noise Injection and Information Bottleneck
Maximilian Igl, Kamil Ciosek, Yingzhen Li, Sebastian Tschiatschek, Cheng Zhang, Sam Devlin, Katja Hofmann
Thirty-third Conference on Neural Information Processing Systems (NeurIPS) | December 2019
Better Exploration with Optimistic Actor-Critic
Kamil Ciosek, Quan Vuong, Robert Loftin, Katja Hofmann
33rd Conference on Neural Information Processing Systems (NeurIPS) | December 2019
Reinforcement Learning: Past, Present, and Future Perspectives
Thirty-third Conference on Neural Information Processing Systems (NeurIPS) | December 2019
Tutorial
Successor Uncertainties: Exploration and Uncertainty in Temporal Difference Learning
David Janz, Jiri Hron, Przemysław Mazur, Katja Hofmann, Jose Miguel Hernandez Lobato, Sebastian Tschiatschek
Thirty-third Conference on Neural Information Processing Systems (NeurIPS) | December 2019
The MineRL Competition on Sample Efficient Reinforcement Learning using Human Priors
William H Guss, Cayden Codel, Katja Hofmann, Brandon Houghton, Noboru Sean Kuno, Stephanie Milani, Sharada Mohanty, Diego Perez Liebana, Ruslan Salakhutdinov, Nicholay Topin, Manuela Veloso, Philip Wang
Thirty-third Conference on Neural Information Processing Systems (NeurIPS) Competition track | December 2019
Minecraft as AI Playground and Laboratory
Proceedings of the Annual Symposium on Computer-Human Interaction in Play (ChiPlay) | November 2019
Opening keynote (extended abstract)
Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environments
Rémy Portelas, Pierre-Yves Oudeyer, Cédric Colas, Katja Hofmann
Third Conference on Robot Learning (CoRL) | October 2019
MazeExplorer: A Customisable 3D Benchmark for Assessing Generalisation in Reinforcement Learning
Luke Harries, Sebastian Lee, Jaroslaw Rzepecki, Katja Hofmann, Sam Devlin
IEEE Conference on Games | August 2019
A Generalized Framework for Self-Play Training
Daniel Hernandez, Kevin Denamganai, Yuan Gao, Peter York, Sam Devlin, Spyridon Samothrakis, James Walker
IEEE Conference on Games | August 2019
Win or Learn Fast Proximal Policy Optimisation
Dino Ratcliffe, Katja Hofmann, Sam Devlin
IEEE Conference on Games | August 2019
Fast Context Adaptation via Meta-Learning
Luisa Zintgraf, Kyriacos Shiarlis, Vitaly Kurin, Katja Hofmann, Shimon Whiteson
Thirty-sixth International Conference on Machine Learning (ICML) | June 2019
Customizing Scripted Bots: Sample Efficient Imitation Learning for Human-like Behavior in Minecraft
Brian Broll, Matthew Hausknecht, Dave Bignell, Adith Swaminathan
The Adaptive and Learning Agents (ALA) Workshop at AAMAS 2019 | May 2019
Combining No-regret and Q-learning (2018)
Ian Kash, Katja Hofmann
The 14th European Workshop on Reinforcement Learning (EWRL 2018) | October 2018
Also presented at the AAAI-19 workshop on Reinforcement Learning in Games.
Learning good policies from suboptimal demonstrations
Yuxiang Li, Ian Kash, Katja Hofmann
The 14th European Workshop on Reinforcement Learning (EWRL 2018) | October 2018
Advancements in Dueling Bandits
Yanan Sui, Masrour Zoghi, Katja Hofmann, Yisong Yue
Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence | July 2018
Depth and nonlinearity induce implicit exploration for RL
Justas Dauparas, Ryota Tomioka, Katja Hofmann
ICML workshop on Exploration in RL | July 2018
A Deep Learning Approach for Joint Video Frame and Reward Prediction in Atari Games
Felix Leibfried, Nate Kushman, Katja Hofmann
ICML workshop on Principled Approaches to Deep Learning | August 2017
Spontaneous Interactions with a Virtually Embodied Intelligent Assistant in Minecraft
Fraser Allison, Ewa Luger, Katja Hofmann
CHI EA ’17 Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems | May 2017
Memory Lens – How Much Memory Does an Agent Use?
Christoph Dann, Katja Hofmann, Sebastian Nowozin
The 13th European Workshop on Reinforcement Learning (EWRL) 2016 | December 2016
Also presented at the Interpretable ML for Complex Systems Workshop at NeurIPS 2016
Decoding multitask DQN in the world of Minecraft
Lydia T. Liu, Urun Dogan, Katja Hofmann
The 13th European Workshop on Reinforcement Learning (EWRL) 2016 | December 2016
Also presented at the 11th Women in Machine Learning Workshop and the Deep Reinforcement Learning Workshop at NeurIPS 2016.
The Malmo Platform for Artificial Intelligence Experimentation
Matthew Johnson, Katja Hofmann, Tm Hutton, David Bignell, Katja Hofmann
25th International Joint Conference on Artificial Intelligence (IJCAI-16) | July 2016
-
Scaling Laws for Pre-training Agents and World Models
Tim Pearce, Tabish Rashid, Dave Bignell, Raluca Stevenson, Sam Devlin, Katja Hofmann
November 2024
Transformer Neural Autoregressive Flows
Massimiliano Patacchiola, Aliaksandra Shysheya, Katja Hofmann, Richard E Turner
January 2024
The MineRL 2020 Competition on Sample Efficient Reinforcement Learning using Human Priors
William H. Guss, Mario Ynocente Castro, Sam Devlin, Brandon Houghton, Noboru Sean Kuno, Crissman Loomis, Stephanie Milani, Sharada Mohanty, Keisuke Nakata, Ruslan Salakhutdinov, John Schulman, Shinya Shiroshita, Nicholay Topin, Avinash Ummadisingu, Oriol Vinyals
January 2021
-
Toward Human-AI Alignment in Large-Scale Multi-Player Games
Sugandha Sharma, Guy Davidson, Khimya Khetarpal, Anssi Kanervisto, Udit Arora, Katja Hofmann, Ida Momennejad
arXiv:2402.03575 | February 2024
Published by Microsoft
The Multi-Agent Reinforcement Learning in MalmÖ (MARLÖ) Competition
Diego Perez-Liebana, Katja Hofmann, Sharada Prasanna Mohanty, Noboru Sean Kuno, Andre Kramer, Sam Devlin, Raluca D. Gaina, Daniel Ionita
1901.08129 | January 2019
Published by Arxiv
TrueSkill 2: An improved Bayesian skill rating system
Tom Minka, Ryan Cleven, Yordan Zaykov
MSR-TR-2018-8 | March 2018
Published by Microsoft
The Atari Grand Challenge Dataset
Vitaly Kurin, Sebastian Nowozin, Katja Hofmann, Lucas Beyer, Bastian Leibe
MSR-TR-2017-61 | May 2017
Published by Microsoft
-
Go-Explore Complex 3D Game Environments for Automated Reachability Testing
Cong Lu, Raluca Stevenson, Johan Verwey
IEEE Transactions on Games | December 2022, pp. 1-6
Rolling Horizon Evolutionary Algorithms for General Video Game Playing
Raluca Gaina, Sam Devlin, Simon M. Lucas, Diego Perez-Liebana
IEEE Transactions on Games | March 2020
How Players Speak to an Intelligent Game Character Using Natural Language Messages
Fraser Allison, Ewa Luger, Katja Hofmann
Transactions of the Digital Games Research Association | December 2018, Vol 4(2): pp. 1-49