PhD Scholarship Programme in EMEA

Selected PhD Projects – 2020

The following applications were selected for funding starting in the academic year 2020–2021.

• Better Sample Efficiency of Reinforcement Learning

  Supervisor: Amos Storkey, University of Edinburgh, UK
  MSR Supervisor: Katja Hofmann

  Summary: Deep reinforcement learning (RL) has had huge empirical success and is a major enabling technology for many applications of AI. However, recent RL algorithms still require millions of samples to obtain good performance. Since obtaining environment interactions is often costly and since challenging environments are rarely static, this inhibits many practical applications.  This project will investigate ways of reducing this cost, aiming to find more sample-efficient RL algorithms. We aim for the algorithms to be deployable in realistic settings, where agents use deep networks to represent knowledge about the environment. Improving sample efficiency of RL has immediate applications to Microsoft’s efforts in applying RL games. It is also likely to lead to improved performance of other systems making automated decisions.

• Designing Conversational User Interfaces for More Effective Mental Health Status Reporting

  Supervisor: Gavin Doherty, Trinity College, Dublin
  MSR Supervisor: Anja Thieme

  Summary: Health self-report or self-monitoring activities, such as mood logging, are a central part of many treatments for mental disorders. Mood logs need to be recorded regularly to be beneficial, yet people can find it difficult to stay engaged with them over time. Mood logging is an example of a health status reporting task, and while mental health is a huge issue, management and self-management of many other health conditions also involve some form of health status reporting. We propose that health status reporting systems based on speech and conversational user interfaces (CUI) have potential advantages in terms of accessibility, engagement, and disclosure. Speech as a modality can be natural and convenient for users, and may support users to disclose their feelings and help create a reporting experience that is engaging and lightweight. The act of speaking aloud might also increase the sense of unburdening associated with mood disclosure, and help provide an experience which is cathartic. The aim of this PhD is to explore the feasibility of conversational user interfaces (CUIs) as a means for supporting health self-reports of mood logging, while gathering design based insight for the development of such systems. The PhD research will establish the feasibility of this approach to mood logging, together with a detailed exploration of the design space, identification of appropriate strategies, and a carefully designed best-of-breed example of a conversational health status reporting interface. The results will inform the development of a set of design guidelines for health status CUI’s. The provision of richer, natural language based health status reporting data also opens up opportunities for further research on the application of machine learning to mental health status data.

• Enabling Long-tail Hardware: Bringing reproducibility to low volume device manufacture

  Supervisor: Joe Finney, Lancaster University, UK
  MSR Supervisor: Steve Hodges

  Summary: This project proposes the creation of new tools and processes that will enable embedded hardware devices to be successfully manufactured in low volumes – thousands of units and fewer. This is important in a world where citizen developers are becoming empowered to design new hardware solutions ranging from innovative interactive devices to IoT sensing and control systems. Recent advances in DevOps offerings from Microsoft and others have enabled these citizen developers to deploy software-only solutions at any scale, but there are no equivalent “hardware DevOps” tools which facilitate deployment of novel hardware. By engaging with Microsoft Research and other industrial partners, this PhD project will start by identifying requirements common to the reproducible manufacture of a range of embedded hardware devices. From these hardware tooling design patterns that support low volume yet reproducible manufacturing will be proposed, built and evaluated. The ultimate aim is to create a set of freely available open tools and processes that others can use and build upon, to unlock a long tail of hardware devices.

• Human avatars as co-workers

  Supervisor: Thomas Bohné, University of Cambridge, UK
  MSR Supervisor: Sean Rintel

  Summary: Our project will be the world’s first research endeavour to study how artificial intelligence (AI) influences the behaviour of humans when they interact with new forms of virtual humans. Our research is motivated by recent technical developments enabling digital representations of humans that are visually and behaviourally high-fidelity (up to indistinguishable) from real humans. The potential impact of these virtual humans on society is expected to be substantial and cannot be understood without new research. Our research team is specifically interested in Avatar-Agent-Hybrids, which can be controlled either by a human or by an AI. Avatar-Agent-Hybrids offer new research opportunities in which humans cannot distinguish if the digital representations they interact with are human or artificial agents. As this type of interaction is likely to affect especially how humans work, our project focuses on work-related situations. By developing a novel experimental setup in virtual reality (VR), we will simulate – in real-time – realistic social and work contexts in which we can study the effects of visually and behaviourally indistinguishable human avatars on humans and their social interaction. This will allow our research team to gain ground-breaking and fundamentally new cognitive and behavioural insights into the effects of AI on humans.

• Local Forward Model Learning for Sample-Efficient Sequential Decision Making in Open-World 3D Games

  Supervisor: Diego Perez Liebana, Queen Mary University of London, UK
  MSR Supervisor: Sam Devlin

  Summary: The latest breakthroughs in game playing AI have primarily focused on the application of Reinforcement Learning (RL) and Statistical Forward Planning (SFP) methods, such as Monte Carlo Tree Search (MCTS) and Rolling Horizon Evolutionary Algorithms (RHEA), to games. Advances in Go, Chess, Shogi, Atari and Starcraft have been achieved by combining Deep Learning and different forms of model-free or model-based RL, including variants of MCTS. Model-based and SFP algorithms require access to an internal model of the game that allows agents to reason about the future, enabling a more data-efficient and flexible behaviour than those learnt with model-free Deep RL. However, in many real world applications and complex games (such as many created in the games industry), this model is non-existent, not available, or computationally too expensive to use. There is a prominent line of research that currently aims at automatically learning these models from interactions with the environment, showing that learning a forward model provides an important boost for action decision making. However, the scenarios normally employed in the literature are simple. Using the Malmo platform, this proposal focuses on approximating forward models in complex games by learning local interaction functions, to then investigate the use of SFP algorithms in these domains. Specifically, this research aims at (1) learning local forward models in 3D environments with partial observability; (2) implement SFP methods that would plan over said learnt models; and (3) investigate how the presence of non-stationary policies of other agents affect both model learning and SFP performance.

• Measurement of birefringence of 3D structures * Funded by Optics for the Cloud – Outreach

  Supervisor: Matthew Foreman, Imperial College, UK
  MSR Supervisor: James Clegg

  Summary: Measuring birefringence in 3D is important for many applications including optical data storage and cell biology. The current state-of-the-art techniques use methods that are designed to measure thin (2D) structures but have been adapted in ad-hoc ways and combined with advanced processing to recover 3D information. Remarkably, there is an important and basic research question that still exists: is there a physically correct way to isolate and measure the birefringence of a thin 2D layer that is contained within a 3D volume, and therefore correctly reconstruct a full 3D volume? This PhD research project will answer this question.

• Phase Change Photonic Computing – from primitives to systems

  Supervisor: Harish Bhaskaran, University of Oxford, UK
  MSR Supervisor: Francesca Parmigiani

  Summary: The use of functional materials that can accumulate information to carry out both memory and computing tasks in-situ is a growing field. Using optical integration onto silicon chips provides a unique opportunity to combine the benefits of silicon scaling and the wavelength multiplexing of optics onto a single platform. In this proposal, we will significantly expand our initial work on carrying out non-von Neumann computations (such as Vector-Matrix Multiplication directly in hardware) to a larger matrix to prove a lab scale demonstration of the potential for such hardware in real-world computing tasks. In addition, this proposal will aim to benchmark such computations against existing state-of-the-art in terms of energy and operational speed.

• Predicting Failure of Machine Learning Models in Image Guided Clinical Decision Support

  Supervisor: Ben Glocker, Imperial College London, UK
  MSR Supervisor: Ozan Oktay

  Summary: Being able to detect and communicate when a predictive model fails is of utmost important in system that are used for clinical decision making. In this PhD research project, novel mechanisms for failure detection and inferring causes for failure will be developed and tested on large scale population data. It is expected that the research leads to high impact innovation that is critical for deploying learned based systems and services in healthcare settings.

• Predicting population level behaviour by single-cell characterisation of genetic devices

  Supervisor: James Locke, University of Cambridge, UK
  MSR Supervisor: Andrew Phillips

  Summary: Programming biological systems in cell populations is a key goal of synthetic biology. To achieve this, it is critical to understand how synthetic genetic circuits behave in individual cells. This is because gene expression and growth can vary substantially between individual cells, which can impact the function of synthetic gene circuits. This PhD will combine mathematical modelling, machine learning, and synthetic biology techniques to predict the population level behaviour of gene circuits from single cell gene expression data. We will use single cell time-lapse microscopy and microfluidics to characterise already constructed synthetic gene circuit components for programming self-organised cell behaviour (Aim 1). Next, we will use modelling and machine learning to design and build spatial signalling and patterning systems based on our single cell data (Aim 2). Finally, we will test our modelling predictions using custom microfluidics that allows spatial signalling between cells (Aim 3). We aim to develop a series of robust spatial patterning circuits, using the data inferred from our single cell experiments to guide our synthetic biology efforts.

• Probabilistic Tree Search for Data-Efficient Active Learning

  Supervisor: Philipp Hennig, Eberhard Karls Universität Tübingen, Germany
  MSR Supervisor: Cheng Zhang

  Summary: Tree search is a classic computational task and yet still relevant for machine learning applications. Phrased as (sequential) active and reinforcement learning, it is arguably the most extreme case of “small-data AI”, since it requires learning in an exponentially large decision space from linearly limited data. Recent successes in Go and related domains proved that statistically motivated best-first search strategies, guided by good heuristics, can drastically outperform the classic rule-based algorithms. Even contemporary best-first algorithms, however, are derived with local worst-case (bandit) formalisms that do not consider structure in the search domain, and only propagate a single point estimate up the tree. We propose a research project to develop a probabilistic formalism for tree search. The resulting algorithmic framework will include classic touchstones as corner cases and make clear connections to other formalisms for sequential decision making in ML. It will be able to leverage structure of the search domain through kernels and other similarity measures and use probabilistic decision theory to locally improve search efficiency. The project proposal allocates significant research time for foundational theoretical and algorithmic research, complemented by application to practical domains of relevance for MSR. A suitable PhD student has already been identified.

• Quantifiably Secure Architectures

  Supervisor: Stefanos Kaxiras, Uppsala University, Sweden
  MSR Supervisor: Boris Kopf

  Summary: This proposal addresses speculative side-channel attacks, i.e., Spectre-type attacks, at the architectural level. Protecting software from malicious attacks is already a daunting task without having to worry about fundamental guarantees at the architectural level. As a community, we were complacent that if security at the software level fails at least security at the architectural level will safeguard our most valuable secrets (e.g., secure enclaves, Intel SGX), keep adversaries contained/confined (e.g., process and address space separation), etc. This conviction was shattered in January 2018 when speculative side-channel attacks were revealed. These attacks break software and hardware barriers that are commonly raised against adversaries. This characteristic sets these attacks apart from any past threat. It is, therefore, of utmost urgency to address them. We simply cannot reason about computer security without being able to reason about the architectural foundations of security.

• Regions and Types for the Secure Foundations of Cloud-First, Confidential Computing

  Supervisor: Sophia Drossopoulou, Imperial College London, UK
  MSR Supervisor: Matthew Parkinson

  Summary: Adoption of the cloud requires trust, but software vulnerabilities can quickly erode that trust. There is a real drive in the industry to make memory safety vulnerabilities a thing of the past. Project Verona is a highly ambitious research project to make that a reality for the infrastructure we build for the cloud. This Ph.D. scholarship will apply world-class research on semantics and type systems to the design of Project Verona. This will provide the secure foundations we need for trusting the cloud.

• The Pluripotency Program in Human Embryonic Stem Cells

  Supervisor: Graziano Martello, University of Padua, Italy
  MSR Supervisor: Sara-Jane Dunn

  Summary: This proposed project draws from a longstanding interdisciplinary collaboration between the two named supervisors and Dr. Boyan Yordanov, which has to-date led to high impact publications in the domain of stem cell research. The project makes use of our previous work and extends the research focus to the domain of human pluripotency – the unique embryonic state in which a stem cell is poised to generate all lineages of the adult body. This is a timely project, as the stem cell field is beginning to learn more about the critical regulators of pluripotency in human, and how we can leverage the power of reprogramming and differentiation to generate cells for translational and pharmaceutical research. The aim of the proposed project is to uncover the biological program governing pluripotency in humans, and to utilise this understanding to inform reprogramming protocols, as well as our understanding of stem cell differentiation.

• Unified Framework for Numerical Processing with Guaranteed Efficiency

  Supervisor: Amir Shaikhha, University of Oxford, UK
  MSR Supervisor: Tom Minka

  Summary: This proposal aims to build a unified compilation-based framework for processing of a wide range of numerical processing tasks with guaranteed efficiency. The key idea is to express such tasks in terms of a class of algorithms with strong asymptotic guarantees thanks to their nice algebraic structures. The techniques developed in this proposal will be integrated into the Infer.NET framework in order to make them applicable for all the existing applications developed using this framework, as well as new applications for automatic differentiation and abstract interpretation.

• Using AI to Develop Joint Attention in Blind Children

  Supervisor: Oussama Metatla, University of Bristol, UK
  MSR Supervisor: Cecily Morrison

  Summary: Children born blind often have substantial social difficulties thought to stem from challenges in building up basic attentional processes that are usually developed as babies through vision. It is likely that spatialized audio could provide an alternative channel whereby visual social information is translated into spatial information to address these fundamental development challenges which have life-long consequences for social interaction and learning. This PhD would build on the Project Tokyo system and the initial findings of research done with the aim of understanding the long-term impact of the technology. The PhD is broken down into three parts: 1) Establishing appropriate longitudinal mechanisms for measuring joint attention in blind children; 2) Designing the human-AI partnership experience for long-term usage based on the initial Tokyo study; and 3) Evaluating the effects of long-term use of this technology on spatial attention in a cohort of blind children.

• Low Shot Realistic Human Rendering from Partial Information *Funded by Cognition

  Supervisor: Tomasz Trzciński, Warsaw University of Technology, Poland
  MSR Supervisors: Marek Kowalski, Nate Kushman

  Summary: High quality human-like animations are required for many different applications. In video gaming many popular games involve controlling and interacting with humanish characters. In AR and VR a core component is realistic rendering and animation of other people’s avatars. In filmmaking, CGI often involves generating footage for sequences that involve the human characters in the film.The common methods used for creating and animating digital characters are both time consuming and difficult to scale. The creation process usually begins with a 3D scan of a real person’s face or body performing various actions. The scan requires specialized equipment and needs to be further processed by artists. To animate the model in a realistic way an actor needs to perform the desired sequences in a motion capture studio. This effort is extremely time consuming and expensive.