By Rob Knies, Managing Editor, Microsoft Research
In the summer of 2008, the leadership at Microsoft Research Redmond (opens in new tab) reorganized an existing set of research groups with a refreshed, more encompassing mandate: reinventing all aspects of software development. The revamped area, Research in Software Engineering (opens in new tab) (RiSE), headed by Wolfram Schulte, includes a diverse set of topics such as experimental software engineering, human interactions in programming, software reliability, programming-language design and implementation, and theorem proving. He recently found time to discuss the new area and the challenges it is addressing:
Q: How is RiSE going to address all of those topics?
on-demand event
Microsoft Research Forum Episode 4
Learn about the latest multimodal AI models, advanced benchmarks for AI evaluation and model self-improvement, and an entirely new kind of computer for AI inference and hard optimization.
Schulte: As our Web site shows, you’ll see that it is organized into 10 broad research areas, ranging from the development process itself all the way to the underlying runtimes and system software the code eventually runs on. Our research projects span multiple areas. The important thing is to address the set of current challenges we see: global software development, multicore and Web concurrency, reliability, security, and energy-efficient computing. These challenges are interdependent and so their solutions must be as well.
Q: Concurrency is a hot topic these days. What is RiSE doing in that area?
Schulte: With multicore machines, most of our programs are parallel. Today, we need a way to express parallelism, and we need analysis tools to find errors related to concurrency.
Wolfram Schulte
To give .NET programmers a much more reliable way to program multi-core machines, Daan Leijen (opens in new tab) recently developed a library of high level concurrency abstractions, called the Task Parallel Library (TPL), which will ship as part of the Parallel Extensions to .NET (opens in new tab) in .NET 4.0.
Now that we can express parallelism, we need to have analysis tools to make sure there are no races, deadlocks, or performance bottlenecks. To address some of these problems, Madan Musuvathi (opens in new tab) and Shaz Qadeer (opens in new tab) have developed CHESS (opens in new tab), which uncovers data races by controlling the scheduler for the different threads in a program. CHESS will be presented during the Professional Developers Conference (opens in new tab) [PDC].
But in the long run, our existing concurrency models of programming, with threads and locks, are not adequate. If you think about the way that our abstractions have evolved, we started with assembly language, we went to C, and we have managed languages today. Threads and locks should be seen as the assembly-language level. We need better abstractions and are looking to do more here.
Q: Static analysis has always been a strong point of MSR. What is new in that area?
Schulte: Yes, we have had a lot of success in static analysis. For instance, Tom Ball (opens in new tab) and colleagues built the Static Driver Verifier (opens in new tab) toolkit, which helps eliminate blue screens from drivers in Windows (opens in new tab) and improves the reliability of the platform. Manuel Fähndrich contributed to the SAL project, the annotation language for the static checkers PREfix and PREfast (opens in new tab), which won Microsoft’s Chairman’s Award for Excellence in Engineering in 2007. Rustan Leino (opens in new tab) has led the Spec# (opens in new tab) project, a research effort that has created the world’s leading program-verification system for object-oriented programming.
Now, a spinoff from our previous work is making its way into .NET. At PDC, the Code Contracts (opens in new tab) project will be shown. It has been developed in part by Mike Barnett (opens in new tab) and Francesco Logozzo. It provides a language-independent way for programmers to record their design decisions for any .NET language. We then have tools that mine those contracts to generate proper documentation, to check them at runtime, or even to statically verify code relative to the contracts. In a sense, code contracts and the associated tools act as grammar checking for programmers, while today’s compilers are like spell checkers.
Q: What does the virtual team for testing work on?
Schulte: Testing is in desperate need of automation: It continues to cost as much as 50 percent of the development effort. We have recently combined automated testing with static analysis based on research Patrice Godefroid (opens in new tab) started while he was at Bell Labs.
Another PDC presentation will be the Pex (opens in new tab) project, headed by Nikolai Tillmann and Peli de Halleux (opens in new tab). Pex builds on Patrice’s ideas and brings them to a new level. It performs an intelligent exploration of software to systematically discover its behavior, thereby forcing it into the possibly faulty corner cases that manual testing is often unable to find. Even without guidance, Pex often automatically reaches high code coverage. As a side effect, Pex immediately generates concrete test cases. The test cases can be used for immediate debugging or as regression suites.
Q: What does the virtual team for experimental software engineering look at?
Schulte: This is an interesting virtual team because it spans two Microsoft Research labs. It includes Nachi Nagappan (opens in new tab) in Redmond and Brendan Murphy (opens in new tab) of Microsoft Research Cambridge (opens in new tab). They measure Microsoft’s internal software-development process to make better predictions on how the next similar project will run. The team analyzes the development process and resulting software artifacts to understand how likely it is that a particular piece of code still has defects. If you do those analyses and have the right statistical models, you can also do better risk estimation and resource allocation. Their focus is not just at the software-system level, but also, more holistically, at the team/project level, for instance, the impact of organizational structure on software quality.
Q: This all sounds very applied. Are any teams looking at more foundational issues?
Schulte: Certainly! But foundations often pay off in very applied ways. We are doing amazing work on automated theorem proving. Nikolaj Bjørner (opens in new tab) and Leo de Moura (opens in new tab) have created Z3, which has won most of the categories in the worldwide competition for Satisfiability Modulo Theorem solvers. This is not only a theoretical achievement, but also provides the underlying engine for most of our program analysis and testing tools.
A true foundational result is recent work by Yuri Gurevich (opens in new tab): he has solved a logic problem that has been open for more than 70 years. Yuri has proven Church’s thesis (opens in new tab), which tries to rigorously define what computability means.
Q: People in your area are also looking at the human side of programming. What is the research there?
Schulte: Ultimately, our code is all developed by people. If we don’t understand how people collaborate with their teammates, we might miss a huge opportunity to help the overall development process.
Previous work by Rob DeLine and his team has had impact by mining all of the information related to a development project to help new developers who have just joined a product team. During PDC, Andrew Begel (opens in new tab) will show one incarnation of this work, Deep Intellisense.
One interesting future question is: How can we develop a code base when engineers are distributed over different locations and over different time zones? It’s the question of global software development. Can we provide tools to enable them to work as if they were in the next room?
Q: Then you have a virtual team looking at programming languages themselves. What is going on there?
Schulte: You can either create a new programming language or impose rules for using an existing programming language. We are working on both.
For members of the RiSE group, the passion to improve software engineering runs red-hot.
Programmers appreciate type systems because they provide partial guarantees for your program, for instance, that the variables only hold values of a proper type. But the type systems for existing languages such as C# and Visual Basic are very restrictive. We are exploring much more expressive type systems. For instance, we can guarantee that processes have to obey a certain protocol if they want to exchange messages with each other. The Singularity (opens in new tab) project followed this approach with the static analysis enabled by type system extensions to C#.
But we also need to address all the systems already written. In fact, we can improve them a lot just by changing the underlying runtime systems. Using the traditional engineering techniques of redundancy and randomization, Ben Zorn (opens in new tab) has created RobustHeap, a memory allocator resilient against buffer overruns. Trishul Chilimbi (opens in new tab) uses program analysis from the performance point of view. He has built various low-overhead profiler and data-relocation tools that help find performance bottlenecks in code, whether it’s sequential code or contention problems in concurrent code. And he is focused on extending this work to improve software energy efficiency.
Q: Another hot topic these days is Web programming. Are you looking into that?
We have three great projects that will be presented during PDC. Ben Livshits (opens in new tab) and Emre Kiciman (opens in new tab) have developed AjaxView (opens in new tab), the first system that provides a view of the overall performance of a Web application. This includes the server, the client, and the latency on the wire in between. They also developed Doloto (opens in new tab), which rewrites a JavaScript application so that individual pieces can be downloaded on demand, thus reducing the wait before a Web application can begin running. Finally, there is BAM (opens in new tab), developed by Ethan Jackson (opens in new tab), which enables cloud applications to be specified as logic programs independently from implementation technologies and service connectivity. Complete, runnable, programs are then generated with the single push of a button.
Q: How would you like to engage the research community outside of Microsoft?
Schulte: We already have lots of interactions with the academic community. We host many visiting researchers and Ph.D. interns. We participate in conferences and co-author papers with researchers outside of Microsoft. But I would also like to have more interaction with the broader developer and tester community.
Q: Your team is playing a larger role in this year’s PDC. What are your goals for that event?
Schulte: The most important goal is engaging in a discussion with developers who work on the Microsoft platform. Going to academic conferences, we get a lot of exposure to one set of people, but not to those people who would just like to use our research ideas without necessarily going into the details. They just want to see how they work. By going to the PDC, we hope we can attract lots of developers and testers and engage with them in a discussion about whether they like what they are seeing. Maybe we misunderstand their main problem. Maybe it’s the build system, not the programming environment per se, that is the most difficult to use for them. That might reprioritize our research efforts.
We’ll have several tools as downloads available for the PDC, either from DevLabs (opens in new tab), where prototype tools for Microsoft Visual Studio (opens in new tab) can be downloaded and explored, or from the Microsoft Research Web site, where we make tools available for non-commercial use to get feedback.
Q: With the RiSE initiative, how long will it take before you can evaluate what you have accomplished? What are you hoping to see once you get there?
Schulte: We believe we can make major advances in each of the research areas we are looking at. For instance, in concurrency we think that a new programming model will emerge that can guarantee programs will behave correctly by construction. For testing, we think we can extend the impact we have had on finding security bugs for codecs to whole applications.
Research isn’t on a time schedule. Addressing the right research questions, then coming up with solutions, and then looking for opportunities to harness those ideas—that’s what we would like to do.
Programming languages and software engineering