By Noburu Kuno, Senior Research Program Manager; Scarlet Schwiderski-Grosche (opens in new tab), Principal Research Program Manager
As we move from narrow AI to more general AI, it will be important to instill machines with the ability to work together with both other agents and humans. Project Malmo (opens in new tab), which is built on the popular multiplayer game Minecraft, is an AI research tool for investigating how to train intelligent agents to collaborate. Our recent Project Malmo Collaborative AI Challenge (opens in new tab) asked teams to solve a game using collaborative agents, a project designed to push the state of the art of collaborative AI.
About Microsoft Research
Advancing science and technology to benefit humanity
Katja Hofmann, the lead and visionary for Project Malmo, summarizes the intent of this challenge: “In Minecraft, the possibilities for creation are endless. Project Malmo adds the ability to try different methods and approaches to teaching an agent to work within the Minecraft framework. The challenge then gave teams a specific task to demonstrate an agent’s ability to predict or learn whether to collaborate and how to collaborate successfully. From this challenge, we learned lots about various strategies for developing collaborative AI, such as planning-based approaches, deep neural network-based approaches, and co-evolution approaches.”
We were gratified that more than 80 teams comprising postgraduate students from 26 countries entered the challenge which required the teams to create and train agents to play a collaborative minigame — catch the pig — in which players work together to achieve a common goal. We selected this challenge because it mirrors the game theoretic “stag hunt,” a classic example that models the trade-offs between choosing to work together or go solo. We wanted to see how participants would approach this problem, what algorithms and strategies perform well today, and identify promising directions for future research.
Each team had to submit their code to GitHub (opens in new tab), send in a write-up about their approach, and create a video showing their agent in action. Prizes for winning the challenge included a placement at the invitation-only Microsoft Research AI Summer School (opens in new tab) and/or Microsoft Azure for Research grants (opens in new tab), worth up to $20,000 USD. Winners were selected based upon several criteria, including the ability of their agents to achieve high game scores consistently, and the novelty and creativity of the teams’ approaches.
Today we’re pleased to announce the winners with three teams winning prizes in both categories.
Microsoft Research AI Summer School placement prize winners
This prize provides winners with a slot at our Microsoft Research AI Summer School, where PhD students work alongside Microsoft Research scientists in Cambridge. Attendees learn general research skills, get to know the MSR Cambridge Lab and enjoy the opportunity to network with leading AI research leaders.
First place:
Second place:
- Village People (opens in new tab) (Romania)
Third place:
- The Danish Puppeteers (opens in new tab) (Denmark)
Runner up:
- AASMA (opens in new tab) (Portugal)
Microsoft Azure for Research grant prize winners
This prize provides additional computing resources to the student teams for their future research efforts. Microsoft Azure for Research supports hundreds of researchers worldwide when they need cloud-based memory, a cluster with thousands of cores, a big data platform, an Internet of Things solution or open-source machine learning at scale.
First place:
- HogRider (opens in new tab) (Singapore)
Second place (tie):
- HelloJason (opens in new tab) (Mainland China)
- Bacon Gulch (UK)
Third place (tie):
- The Danish Puppeteers (Denmark)
- Village People (Romania)
Said Bo An, assistant professor of Nanyang Technological University and supervisor of the HogRider team: “The Malmo testbed reflects all the important characteristics of collaborative AI: incomplete information, partial/noisy observation, co-existence of cooperation and competition, sequential decision making, and more. To solve the challenge, we needed to coherently integrate techniques from different sub-fields together. We’re looking forward to applying our Azure for Research grant to future efforts.”
We thank the teams that entered, and congratulate the winners on their achievement.
What we’ve learned from this challenge
The Project Malmo challenge was designed to move the understanding of collaborative AI forward. Did it do so? The answer is an unqualified yes. Here’s some of what we learned:
- There is no single approach to this problem that was clearly superior to the others. If you look at the leaderboard performance (game scores) alone, many different approaches achieved a very similar result.
- Don’t devalue the work that’s gone before. Many teams achieved their success through a combination of both classic AI approaches and incorporating newer heuristics and models.
- Sometimes the biggest learning is what question to ask next. One of the biggest problems in collaborative AI is the ability for an agent to, on-the-fly, adjust to new information in a dynamic environment. By introducing agents of unknown character and goals that the agent had to collaborate with, the teams were required to solve for this level of dynamism in real time. As a result, this challenge helped identify promising directions for additional research in helping AI agents adapt to unknown conditions.
- Everything matters. Some teams spent more time on problem analysis, others on coding efficiency and still others on tuning their models. All three of these areas showed promise for future research efforts.
Related links:
Artificial intelligence