How to Make a First Accomplishment in the NLP Field

Natural Language Processing (NLP) is a technology to help computer process or understand human languages. It includes:

1. Syntactic–semantic Analysis: Conduct word segmentation, the tagging of parts of speech, named entity recognition and linking, syntactic analysis, semantic role recognition, and polysemy disambiguation for a given sentence.
2. Information extraction: Extract important information from a given text, such as the time, the place, the people, the event, the cause, the result, numbers, dates, currency, proper nouns, etc. Put simply, it is to understand who did what at what time, for what reason, and with what results. This is related to entity identification, time extraction, causal relationship extraction, and other key technologies.
3. Text mining: This includes text clustering, classification, information extraction, abstracts, emotional analysis, and the visualization and interactive expression of mined information and knowledge. Currently, the mainstream technology is all based on statistical machine learning and deep learning.
4. Machine translation: Automatically translate text from an input source language to another language. Based on different input media, this can be sorted into text translation, voice translation, sign language translation, graphic translation, etc. Machine translation, from the earliest rule-based method, to the statistics-based method 20 years ago, to today’s neural network (encoding-decoding) method, is gradually transforming into a more accurate system.
5. Information retrieval: Index large-scale documents. Different weights can be given to various words in the document to build a simple index, or the technologies from the above 1, 2, and 3 can be used to build a more in-depth index. In the query process, the system will analyze the input query expression such as a search term or a sentence, and then search for matching documents from the index, and then sort the candidate documents based on a sorting mechanism, and finally output the top-ranked document.
6. Question answering system: Q&A system provides a precise answer to a question expressed through a natural language. This requires a certain amount of semantic analysis of natural language queries, including entity links and relationship recognition, to form logical expressions, and then it searches for possible answers in the knowledge database, finally locating the best answer through a sorting mechanism.
7. Conversation system: The system completes a task through a series of dialogues, conducting chatting and question answering with the user. This is related to technology such as understanding user intent, general chat engines, Q&A engines, and dialogue management. In addition, in order to reflect context, there needs to be an ability to conduct multiple rounds of dialogue. At the same time, in order to reflect the individual, there needs to be development of user portraits and personalized, user-based replies.

With the impressive results of deep learning in image and speech recognition, people have high hopes for the value of deep learning in NLP. In addition, the success of AlphaGo renders the research and application of artificial intelligence extremely popular. Natural language processing as a cognitive intelligence in the field of artificial intelligence has become the focus of attention. Many graduate students are entering the field of natural language, hoping to display their talents in artificial intelligence in the future. However, people frequently encounter problems. As the saying goes, everything is difficult in the beginning. If the first attempt is successful, the student will build confidence, learn the ropes, and do better and better. Otherwise, the student may become discouraged and even leave the field. I will provide my personal advice here, and I hope these views can lead to deeper discussion.

Advice #1: How can I learn my first skill in the NLP field?

My advice is: Find an open source project, such as a machine translation or a deep learning project. Understand the mission of the open source project, compile the demonstration program issued by the project, and try to obtain the same results as the project demonstration program. Try to gain an in-depth understanding of the algorithm of the demonstration program for the open source project and implement the algorithm of this demonstration program, and then follow the standard test set provided by the project to test your own program. If your results are different from the ones shown by the project, then you need to carefully check your program and make changes repeatedly until the results are basically the same. If you still can’t get the right results, bravely contact the project author. On this foundation, see if you can further improve the algorithm or implementation to obtain results even better than the demonstration program.

Advice #2: How do I select a good topic?

For graduate students in the engineering field, topics are often given by the teachers. You need to take a practical and solid hands-on approach. You might not need much theoretical innovation, but you need to have strong abilities in realizing your ideas and in comprehensive innovation. However, for academically oriented graduate students, in order to obtain first-class research results, the research topic must have a certain degree of innovation. My advices are as follows.

• First, find a research field you like. Find a recent set of ACL conference proceedings, and look for a field you like. When you are looking for a topic, pay greater attention to the blue ocean field, which is a relatively new field and thus easier to produce results in.
• Thoroughly research the current state of development in this field, including the following: in terms of methodology, look into whether there is a relatively clear mathematical system and machine learning system; in terms of data, look into whether there is a widely recognized standard training dataset and test dataset; in terms of research teams, look into whether there are any well-known teams or people participating in them. If you obtain uncertain conclusions for any of the above aspects, then beginners might not be able to easily enter this field.
• After you have decided on entering a field, follow Advice #1 and look for open source projects or tools in this field. Carefully study the existing branches and methods as a way to enter the field.
• Carefully read the newest articles published in the field, especially those written by top specialists in the field. Upon gaining an in-depth understanding of the existing work, explore any parts that can be overturned, improved, integrated, or migrated. Note that when conducting experiments, don’t be greedy and remember that each experiment only needs to verify one idea. After each experiment, analyze your errors and find the reasons behind them.
• For successful experiments, further explore how you can improve the algorithm. Please note that experimental data must be industry-recognized data.
• Compare with existing algorithms to see whether you could draw generally applied conclusions. If there is one, go write an article. Otherwise, you should change to a different topic.

Advice #3: How do I write my first paper?

Following the last question, if your idea is good and has been supported by experiments, then you can begin to write your first paper.

• Decide on the paper topic. When deciding, try not to use “… system” or “… research and practice.” Try to avoid overly long topics that aren’t good at reflecting the main idea. Topics should be specific, have depth, and highlight the algorithm.
• Write the abstract. Highlight the important issues in the paper, the method you used, and what advantages it has compared with existing work. Explain what level of expertise the experimental results have reached and what problems they solved.
• Write the introduction. First explain the background of the work, the definition of the issue, and what importance it holds. Introduce the existing methods for tackling this problem and what advantages they have. Then, explain the flaws or challenges of the current methods. State what method (or who’s work) inspired you, and what new method you propose. List the aspects of research you worked on. Explain each aspect of your research in sub-categories, then state the conclusions of your experiment. Following this, write the contributions of your work, usually no more than three. You can then talk about the organization of your paper and its focus. Sometimes you have too much content and not enough space, so you can just introduce the most important aspects of your work and don’t need to cover everything.
• Related work. Comb through the relevant work according to the various research methods. Cover the three most important methods at most and give a brief introduction to each. Introduce their principles and explain their limitations.
• Allow for two chapters to introduce your work. The first chapter is on the description of the algorithm. This includes the problem definition, mathematical notation, and algorithm description. The main formulas in the paper should all be here. Sometimes you need to provide a concise derivation process. If you drew on someone else’s theory and algorithm, provide clear citations. Because they are generally based on machine learning or deep learning methods, you need to introduce your model training method and decoding method. The second chapter is on experimentation. You usually need to state the purpose of the experiment, what you are testing, the experimental method, the source of your data, and the scale of the experiment. Preferably use publicly tested data to make it easier for others to repeat your work. Provide the required technical parameters for each experiment and report the experimental results. At the same time, in order to make comparisons with existing work, you need to refer to the results of existing work, and when necessary, you might need to reproduce important works and report the results. Use your experimental data to do the talking and show that your method is better than others’. Analyze the results of your work and compare them to the work of others, looking at their respective advantages and disadvantages and explaining the reasons for them. For areas that are not good enough, analyze the problem and list it as work for the future.
• Once again summarize the contribution of this paper. Sum up the theories and methods, and also explain the experimental contributions and conclusions. The conclusion needs to be convincing for readers and point out future directions of study.
• List out all the important related work papers. Remember, you’ll essentially lose hope of being accepted if you miss out any important reference.
• Finish the first draft and then edit it three times.
• Ask someone on the same project team to critically review your paper, looking at the novelty of the algorithm, the level of innovation, the experimental scale, and the conclusions. Further improve on areas that are weak and focus on strengthening the depth of the algorithm and the innovation of the work.
• Ask people from different project teams to review your paper. If they don’t understand, then it means the readability of your paper isn’t good enough. You need to fix the paper’s structure, improve the wording, and increase readability.
• If you are submitting to international conferences such as ACL, it would be a good idea to ask an English professional or native English speaker to review your paper.

About the Author

Dr. Ming Zhou is assistant managing director of Microsoft Research Asia, where he also leads the Natural Language Computing. He is the chair of the Chinese Computer Federation’s (CCF) Chinese Information Technology Committee and an executive member of the Chinese Information Processing Society. Also, he serves as PhD advisors at universities such as Harbin Institute of Technology, Tianjin University, Nankai University, Shandong University, Beihang University, etc.

Dr. Zhou received his B.S. in computer engineering from Chongqing University in 1985, and his M.S. degree and Ph.D. in computer science from Harbin Institute of Technology in 1988 and 1991. He did post-doctoral work at Tsinghua University from 1991 to 1993, later became an associate professor there. From 1996-1999, during a sabbatical, he worked for Kodensha Ltd. Co. in Japan as the leader of the Chinese-Japanese machine translation project. In 1998, he designed the famous Chinese-Japanese machine translation software product J-Beijing in Japan.

He joined the natural language group at Microsoft Research China (now Microsoft Research Asia) in September 1999 as researcher. He became the manager of this group in 2001. His group created Chinese-English, and Cantonese-Chinese-English machine translation engine for MS Translator and Skype Translator. Recently, his group worked closely with Microsoft product teams to create well-known chat-bot products in China (Xiaoice), Japan (Rinna) and the US (Tay) with total of 100 million users. He has presented and published over 150 papers at top conferences (including 55+ ACL papers) and NLP journals, and obtained over 50 international patents.