An empirical study of release note production and usage in practice

The release note is one of the most important software artifacts that serves as a bridge for communication among stakeholders. Release notes contain a set of crucial information, such as descriptions of enhancements, improvements, potential issues, development, evolution, testing, and maintenance of projects throughout the whole development lifestyle. A comprehensive understanding of what makes a good release note and how to write one for different stakeholders would be highly beneficial. However, in practice, the release note is often neglected by stakeholders and has not to date been systematically investigated by researchers. In this paper, we conduct a mixed methods study to investigate the use of release notes in practice. We first conducted a large-scale empirical study of 32,425 release notes from 1,000 GitHub projects to understand current contents and information found in real-world release notes. We then performed interviews with 15 practitioners and an online survey with 314 respondents to investigate how key stakeholders perceive release notes. From the analysis of these data, we summarized eight categories of information that are normally documented in release notes in GitHub projects. We found that stakeholders consider that well-formed release notes have a positive impact on software development, such as software evolution. We concluded 28 statements grouped into eight topics based on stakeholders’ opinions. There exist significant discrepancies between different stakeholders on how release notes should be written and used. Our study provides new insights on release notes and facilitates stakeholders to better take advantage of them during software development.