Next-Generation Systems in the AI Era: Microsoft Research Asia StarTrack Scholars 2025 Shapes System Innovation in the AI Era
AI is advancing at an extraordinary pace, driven by groundbreaking innovations in computer systems. As we rapidly move towards an era centered around AI computing, these systems have significantly benefited from advancements in AI technology. At this pivotal moment, Microsoft Research Asia is committed to the synergistic development of AI and system innovation, fostering a series of pioneering technological breakthroughs.
If you are an aspiring researcher with a zeal for exploring next-generation systems in the AI era, we invite you to apply to the Microsoft Research Asia StarTrack Scholars Program. Applications are now open for the 2025 program. For more details and to submit your registration, visit our official website: Microsoft Research Asia StarTrack Scholars Program – Microsoft Research
An Era of AI-Centric Computing: Change and Innovation
Over the past decade, advancements in computer systems have been pivotal in fueling the extraordinary breakthroughs in AI. For instance, modern AI accelerators like GPUs have transformed deep neural networks from concept to reality. In addition, the growing ecosystems such as NVIDIA CUDA, TensorFlow/XLA, PyTorch, and OpenAI Triton have enabled the rise of large language models (LLMs), unveiling AI’s astonishing capabilities.
As AI technologies advance at an unprecedented rate, the world is undergoing a rapid transition into an AI-centric computing era. This transition is permeating every layer of our digital infrastructure — from smartphones and personal computers to data centers, and from operating systems to video conferencing platforms. In this era, computing foundations across domains are being reshaped to embrace AI as a central element, driving the need for fundamental innovation across multiple dimensions in AI and computer systems.
The Relationship between AI and System Innovations
At Microsoft Research Asia, we believe the evolution of AI is intrinsically linked with advances in systems research. This synergy is evident in our recent progress. For example, our exploration on next-generation neural networks like RetNet and LongRoPE heavily relies on nnScaler, our system innovations in deep learning training frameworks, paired with insights from our extensive AI compiler expertise. Our passion for exploring future neural models and hardware trends have guided us toward software-hardware co-design, paving the way for next-generation hardware.
Our speculations on storage systems in the AI era also place us among the first to investigate vector stores, now a vital component of diverse AI applications. Additionally, our work on agentic AI has driven the development of Parrot, a new LLM service system optimized for agents. Going forward, we are committed to furthering next-generation system designs through AI and system co-innovation.
Meanwhile, as an essential infrastructure for modern society, computer systems like data centers can benefit profoundly from advancements in AI. Ensuring the security and reliability of these systems is crucial yet developing secure and resilient infrastructures is a complex task requiring specialized expertise. Leveraging AI to simplify and enhance the development of secure, reliable systems could dramatically lower barriers and elevate standards across the industry.
Insights and Innovations: StarTrack Scholars’ Experiences
The essence and impact of the Microsoft Research Asia StarTrack Scholars Program are vividly captured through the firsthand experiences of StarTrack Scholars:
Luo Mai (opens in new tab), Assistant Professor in the School of Informatics (opens in new tab) at the University of Edinburgh (opens in new tab), reflects on his visit:
“MSRA is a unique institution, bringing together exceptional talent in system research and machine learning. During my visits, I had the privilege of collaborating with world-leading researchers across these fields, which inspired me to define an ambitious research agenda for the future.
At MSRA, my research focused on tackling the cost and sustainability challenges of modern machine learning (ML) systems. We explored leveraging emerging computing technologies to enhance the energy efficiency of ML systems, optimize resource utilization, and reduce expensive overheads.
To better align the architectural characteristics of novel platforms with existing ML systems, our team developed innovative solutions rooted in core principles. These solutions significantly improve the operational efficiency of energy-intensive ML models, showcasing the immense potential of emerging technologies.
Looking forward, we aim to build a comprehensive system stack optimized for these emerging technologies. We believe our work will accelerate the democratization of ultra-efficient machine learning systems and lay the foundational groundwork for the next generation of computing.
I wholeheartedly recommend MSRA to young, talented researchers with a vision to drive impactful change through their work. MSRA serves as a powerful catalyst, accelerating progress and turning groundbreaking ideas.”
Cheng Tan (opens in new tab), assistant professor of Khoury College of Computer Sciences (opens in new tab) at Northeastern University (opens in new tab), shares:
“During my time at MSRA in the summer of 2024, I was immersed in discussions about large language models (LLMs). This experience made me realize that LLMs are ushering in a new era, one that challenges computer scientists to adapt and redefine their contributions.
What really is an LLM? Technically, an LLM is a deep neural network—a piece of code and data. But LLMs differ fundamentally from traditional programs. While typical programs are “problem solvers” designed for solving specific tasks, like sorting arrays, managing files and dirs, or serving web contents, LLMs are on a different level. They are what I would call “meta-solvers”—programs capable of generating other problem-solving programs. Instead of solving problems directly, LLMs possess a more generalized capability: they can generate solvers to solve problems.
LLMs may not be the first program with this capability, but they are, so far, the most effective, on par with human beings. The idea of a program that can generate other useful programs thrills me. This represents a form of “self-reference”, a concept foundational to some of the most profound cores in computational theory, such as Gödel’s incompleteness theorems and Alan Turing’s halting problem proof. Both demonstrate the inherent limits of formal systems using self-reference, revealing boundaries to what can be computed or proven.
I wonder if, as close approximations of human reasoning, LLMs might ultimately reveal the limits of human problem-solving capabilities themselves. And, if so, we will accept that our mind is fundamentally limited.
I am currently working on three major projects in collaboration with MSRA.
- The first project addresses correctness in large language model (LLM) training. As much of the community’s focus has traditionally been on performance, our team aims to go beyond traditional research approaches, by enhancing confidence in the correctness of LLM training, mitigating many potential bugs observed in previous training systems. The system is largely complete, and we are now drafting the paper.
- The second project focuses on the robustness of vector databases. The current standard metric for vector databases, “average recall”, fails to capture variations between skewed and even results. For instance, an average recall of 90% could mean (a) all ten queries return 90% accuracy, or (b) nine queries have perfect accuracy while one query has zero accuracy. We introduce a new metric to capture this critical issue, and plan to submit a paper early next year.
- The third project improved the performance of VMs through reinforcement learning. By developing novel methods, the team aims to optimize the performance of misbehaved cloud VMs, thereby improving user satisfaction. We are now drafting the paper for this work.
Overall, I feel welcomed and supported within the MSRA systems group. I look forward to the possibility of visiting MSRA again in the future!”
Addressing the challenge of advancing AI systems to transition into the AI-centric computing era requires a cross-disciplinary approach to innovation. Researchers are invited to join in this effort. Through collaborative exploration, the objective is to shape the future of computing in the AI era, driving advancements for systems and applications that will define the future.
Microsoft Research Asia StarTrack Scholars advocates an open attitude, encouraging dialogue and joint experimentation with researchers from various disciplines to discover viable solutions. Now visit our official website to know more: Microsoft Research Asia StarTrack Scholars Program – Microsoft Research
Theme Team:
Fan Yang, Principal Researcher Manager, Microsoft Research Asia
Chieh-Jan Mike Liang, Principal Researcher, Microsoft Research Asia
Ting Cao, Principal Research Manager, Microsoft Research Asia
Xian Zhang, Senior Researcher, Microsoft Research Asia
Li Lyna Zhang, Senior Researcher, Microsoft Research Asia
Lingxiao Ma, Senior Researcher, Microsoft Research Asia
Shijie Cao, Senior Researcher, Microsoft Research Asia
Shuai Lu, Research SDE, Microsoft Research Asia
Baotong Lu, Researcher, Microsoft Research Asia
Ran Shu, Senior Researcher, Microsoft Research Asia
Zhirong Wu, Senior Researcher. Microsoft Research Asia
If you have any questions, please email Ms. Yanxuan Wu, program manager of the Microsoft Research Asia StarTrack Scholars Program, at [email protected]