UBC research team wins CSCW Impact Recognition Award for paper on GitHub communication
Communication is key. Not only does it help with understanding, but it is the pathway to better understanding and ultimately, improvement. And especially when it comes to communicating about something done predominantly online, like coding.
Online collaborative code writing tools like GitHub are wonderful things for programmers and help to evolve software. But as a tool for communication, is GitHub actually sufficient? When it comes to pull requests, a team of UBC Computer Science researchers think that there is room for improvement, and their research has won them an award.
Pull requests let you tell others about code changes you've pushed to a branch in a repository on GitHub. Once a pull request is opened, you can discuss and review the potential changes with collaborators and add follow-up commits before your changes are merged into the codebase.
UBC CS faculty members Dongwook Yoon and Ivan Beschastnikh, UBC ECE faculty member Sid Fels, and three UBC students explained their findings in the award-winning paper, “@alex, this fixes #9”: Analysis of Referencing Patterns in Pull Request Discussions.
This paper was recently published in the proceedings of the top-tier HCI conference Computer-Supported Cooperative Work (CSCW) 2021, where it won an Impact Recognition Award. It was one of 13 papers given the award, out of a total 1,000+ papers submitted across CSCW 2021. The specific award is given to recognize “strong examples of work that demonstrate or have clear potential to demonstrate real-world or practical impact.”
This paper bridges a knowledge gap in Human Computer Interaction and Software Engineering. Specifically, it identifies the kinds of information developers talk about when discussing software code. And it examines how well the existing user interfaces support such communication.
The main issue is detailed in their abstract, “When discussing proposed changes in a Pull Request (PR) discussion, stakeholders often reference a wide variety of information objects for establishing shared awareness and common ground. Previous work has not considered how the referential behavior impacts collaborative software development via PRs.”
They studied and collected data on the types of references and expressions used on GitHub, and determined that the interface does not support those references. They also hypothesised that this gap may be adversely affecting the outcomes of pull requests. The paper includes design proposals that would effectively adapt the interface to support more effective communication (referencing methods) in pull request discussions.
Millions and millions of pull requests
Considering that GitHub reported that more than 87 million PRs were created on its platform between 2018 and 2019, the impact of this group’s research has the potential to be astounding.
The work began as a CS Master’s thesis for the first author on the paper, Ashish Chopra, and took nearly two years. The second author, Morgan Mo, was an undergraduate intern at the time, and another team member, Samuel Dodson was a PhD student working at UBC School of Information and is now an assistant professor at the State University of New York at Buffalo. The other faculty collaborator was Sidney Fels, a Professor in the Department of Electrical and Computer Engineering who heads up the UBC ViDeX video learning project, a collaboration that includes Yoon.
The collaboration of this group of researchers came about through the UBC Computer Science Designing for People group and Software Practices Lab. As Dongwook explains, “This community is very multi-disciplinary. It’s not only about technology or collaboration tools, it’s a lot about humans and society around technology, and pervades everything from anthropology, psychology, and sociology, to gender studies and critical race theories.”
Ivan goes on to explains the nature of the study, “It was basically a qualitative study to understand the behaviour of people so we can build a more a suitable interface for people to discuss and collaborate about code.”
The research has far-reaching implications for coding collaboration at a global level through platforms like GitHub. If the communication element of pull requests is supported by an interface that makes communicating effortless, succinct, and easier to collaborate, the ultimate end result is better, and faster conclusions about the underlying software.
Dongwook says, “This project is exactly at the intersection of human computer interaction and software engineering. In the human computer interaction component, we first sought to understand what people talk about through pull requests. Knowing first, what these coding discussion topics are like in nature, is fundamental to developing the right framework to facilitate those discussions.”
Ivan and Dongwook speculated on why they may have won the award from CSCW. “We imagine that the committee thought our work has significant direct impact potential because we're dealing with a real set of people including developers, and we're dealing with a real platform that is used by millions. Changes to the interface will substantially influence outcomes,” Dongwook said.
Ivan added, “I will also say that we’ll end up with better software and get to create it faster. Collaborators will be able to improve the quality of the software and rate it more quickly. So, productivity will improve. In addition, there is scientific understanding to be gained from just figuring out how people collaborate and what they refer to when they work on code together.”
Dongwook agreed and said, “The enhanced productivity is through clearer communication so communication is key. Because when you can clearly understand why people talk about certain references, you can help make it easier for them to do so.”
A surprising win
Dongwook said, “In my work on systems research, I have always endeavoured to see if my research has real world impact. So it comes as a bit of a surprise that this purely empirical paper won the award, which tells me it’s not about the mode – it’s about the end result.”
Ivan concurred, “All our papers on this topic have been empirical papers. For me, the intersection is important and it’s very fruitful, showing how UBC has strength in both groups – human computer interaction and computer systems.”
The research continues
More work needs to be done to bring these findings into effect, so further collaboration and research is being conducted by members of the team. “We have further studies in action,” said Ivan. “The work is continuing amongst members of the original research group. We are far from done, but yes, this is a fantastic start.”
You bet it is. Coders everywhere thank you Dongwook, Ivan and the research team.
About the ACM Conference on Computer-Supported Cooperative Work and Social Computing (CSCW)
This is the premier venue for research in the design and use of technologies that affect groups, organizations, communities, and networks. Bringing together top researchers and practitioners, CSCW explores the technical, social, material, and theoretical challenges of designing technology to support collaborative work and life activities.
Learn more about the UBC CS researchers:
Ivan Beschastnikh
Dongwook Yoon