Semantic Merge Tools

Problem

Current merge tools are oblivious to the semantic of the code changes that they integrate. As a consequence, they might integrate conflicting changes without warning developers, like when two people independently change the behavior of dependent parts of the code.

Such missed conflicts are hardly detected by project tests or code reviews, and end up escaping to system users, with potentially severe negative consequences. When users report such defects, developers might have a hard time to understand the underlying causes and fix the code.

These problems compromise development productivity and software quality.

Solution

To detect such conflicts at integration time, we are studying two techniques for building semantic merge tools. First, we are developing static analyses to detect interference among the changes to be integrated. Second, we are using test generation tools for generating tests that reveal such interference.

Results

Unit-test generation tools can help detect semantic conflicts

In our ICSME 2020 paper we propose and implement test execution criteria for automatically detecting semantic conflicts, and evaluate them with unit tests generated by Randoop and Evosuite. We find evidence that these tools can generate tests that reveal interference (that is, potential semantic conflicts) between developers’ changes that have to be merged. We observe no false positives, but observe a significant rate of false negatives.

This suggests that semantic merge tools based on unit test generation, as we propose, can help developers to detect semantic conflicts early, avoiding them to reach end users as failures. The associated benefits are likely achieved with small false-positive costs. However, with the current limited capacity of the test generation tools, developers cannot solely rely on such semantic merge tools for detecting semantic conflicts.