“It would work for me too”: How Online Communities Shape Software Developers’ Trust in AI-Powered Code Generation Tools

Software developers commonly engage in online communities to learn about new technologies. As revolutionary AI-powered code generation tools such as GitHub Copilot emerge, many developers are uncertain about how to trust them. While we see the promise of online communities in helping developers build appropriate trust in AI tools, we know little about how communities shape developers’ trust in AI tools and how community features can facilitate trust in the design of AI tools. We investigate these questions through a two-phase study. Through an interview study with 17 developers, we unpack how developers in online communities collectively make sense of AI code generation tools by developing proper expectation, understanding, strategies, and awareness of broader implications, as well as how they leverage community signals to evaluate AI suggestions. We then surface design opportunities and conduct 11 design probe sessions to explore the design space of integrating a user community to AI code generation systems. We conclude with a series of design recommendations.