Mutation testing for R with mutator

Mon, 13 Jul 2026 11:27:31 +0200

mutator is an automated mutation testing tool for the R language. Mutation testing measures how good a test suite actually is: instead of trusting code coverage, it introduces small, systematic changes (mutants) into the source code and checks whether the tests notice.

For instance, given a function that tests if (x > 0), mutator generates a mutant where > is replaced with <. If the test suite still passes on the mutant, the mutant survived — a sign that the tests are not exercising that behaviour. If a test fails, the mutant is killed. The ratio of killed mutants to the total is the mutation score, a direct measure of test suite effectiveness.

mutator works at the level of the abstract syntax tree and ships a range of mutation operators — arithmetic, comparison, and logical operators, constant and return-value replacements, and statement deletions. It plugs into both testthat and non-testthat test layouts, can run mutation testing directly in continuous integration through a reusable GitHub Actions workflow, and can even flag likely equivalent mutants using LLMs.

Because running a whole test suite once per mutant is expensive, mutator puts a lot of effort into making it fast: parallel test execution, coverage-guided test selection (running only the tests that cover each mutated line), in-process package loading with pkgload to avoid an install per mutant, and self-calibrated per-mutant timeouts to prevent hangs.

mutator is developed at PRL-PRG with Assanali Amandykov, and was presented at useR! 2026.

Source code: github.com/PRL-PRG/mutator
Reference documentation: prl-prg.github.io/mutator

Testing | Pierre Donat-Bouillud

Mutation testing for R with mutator