SigmaDolphin: Automated Math Word Problem Solving

Building a computer system to automatically solve math word problems written in natural language. SigmaDolphin is a project initiated in early 2013 at Microsoft Research Asia, with the primary goal of building a computer intelligent system with natural language understanding and reasoning capacities. We focus on the application of automatic problem solving, i.e., automatically solving problems (especially math word problems) written in natural language.

Motivation

Computer programs can complete many tasks much more effectively and efficiently than human beings, such as calculating the product of two large numbers, or finding all occurrences of a string in a long text. However, the performance of computers on many intelligent tasks is still low. For example, in a chatting scenario, computers often generate irrelevant or incorrect responses; and we can easily find ridiculous results in automatic machine translation; and it is still a very challenging task for state-of-the-art computer programs to solve even primary-school-level math word problems.

A well trained person can apparently outperform computers on the above intelligent tasks. The main reason, we believe, lies in our amazing power of natural language understanding and reasoning (especially common sense reasoning) over state-of-the-art artificial intelligent (AI) systems.

Once we are able to empower computers with the ability of natural language understanding and reasoning, we may see significant performance improvements on text processing tasks like question answering and machine translation. In addition, it may open doors to new applications which are impractical so far because of the limited intelligent level of current computer systems.

Present Targeting Scenario: Math Word Problem Solving

We drive our system design and implementation by the scenario of automatically solving math word problems. It is an ideal starting point to our research because of the following reasons:

1. Math word problems typically contain concise text with clear semantics
2. They fundamentally cover a wide range of domains (considering the fact that math word problems are often simplified versions of real problems from various domains)
3. It is relatively easy to measure performance: A lot of problems have been compiled for education purpose; one can often unambiguously determine whether the solutions of a math problem is correct or not.

Research Topics

The following research topics are highly related to our project.

1. Meaning representation: How to represent the semantic meaning of natural language text
2. Semantic parsing: How to parse natural language text into structured semantic representation
3. Reasoning: How to perform reasoning based on the structured semantic representation as well as common sense knowledge

Datasets