mutate4py

Mutation testing for Python. Discovers mutation sites, applies each one, runs your tests, and reports killed, survived, and uncovered mutations — with an embedded-in-source manifest so differential reruns survive a clone with zero CI setup.

Originally a Python port of unclebob/mutate4go, with the user-facing contract cross-checked against unclebob/clj-mutate. Where Python forced a divergence — coverage acquisition, the manifest hash, parallel worker isolation — the reasoning is recorded in docs/adr/.

Install

uvx mutate4py        # run without installing
uv tool install mutate4py

Usage

mutate4py path/to/file.py --lcov lcov.info

pytest is the only supported test runner, invoked directly (never through a shell). Pass extra pytest arguments with --pytest-args "ARGS", e.g. --pytest-args "-x -k calc".

Generate lcov.info with coverage.py:

pytest --cov --cov-branch --cov-report=lcov:lcov.info

See mutate4py --help for the full flag set.

Multiple targets & glob patterns

The positional argument is zero or more targets — literal paths or glob patterns, expanded with glob.glob(..., recursive=True):

mutate4py 'src/**/*.py' --check-manifest

How many paths that resolves to decides the run shape (ADR 0017): one path runs exactly like today (single-file or directory dispatch); two or more run as a single union batch — one baseline, one exit code, .py files under every root deduped by realpath:

mutate4py src/mutate4py/__main__.py src/mutate4py/_workspace.py --check-manifest

uv workspace autodiscovery

Run mutate4py with no positional argument inside a uv workspace and it finds its own targets: it climbs from the current directory to the nearest pyproject.toml declaring [tool.uv.workspace], then processes that root (recursively) plus every directory matched by members that has its own pyproject.toml:

# pyproject.toml at the workspace root
[tool.uv.workspace]
members = ["packages/*"]
exclude = ["packages/legacy"]

cd my-workspace && mutate4py --check-manifest   # no target — discovers packages/*

exclude is honored on top of what members requires, pruning both the member list and the workspace root's recursive walk. A directory matched by members but missing its own pyproject.toml is skipped rather than erroring — a deliberate divergence from uv itself, since mutate4py's job is finding Python files, not validating the workspace. No workspace found (or no pyproject.toml at all) is a usage error, exit 2, naming the path it inspected. Full rationale in ADR 0017.

Directory mode

Point mutate4py at a directory instead of a file and every .py file under it is processed in turn. The walk prunes __pycache__, venv, node_modules, and any dot-directory (.git, .venv, …); build/ and dist/ are left walkable (issue #22 — previously only __pycache__ was pruned). This works for --scan, --update-manifest, --check-manifest, and scored runs alike.

mutate4py src/ --check-manifest

--exclude PATTERN drops files from that walk. It is repeatable, and a file is skipped if it matches any pattern — never scanned, never reported, never able to fail the run:

mutate4py src/ --check-manifest \
  --exclude '**/__init__.py' \
  --exclude '**/migrations/**' \
  --exclude '**/vendor/**'

Patterns are matched case-sensitively on every platform, against the whole path as walked — i.e. built from the path you passed, so src/ yields src/pkg/mod.py. --exclude shares its glob dialect with positional targets and uv workspace members/exclude (ADR 0017): * matches exactly one path segment and never crosses /; ** matches zero or more segments, but only when it stands alone as a whole /-bounded component (glued to literal text, e.g. foo**bar, it degrades to an ordinary same-segment wildcard). Two consequences worth knowing:

• * stays within one path segment, and the path always has the walked directory prefixed on it (see the next point) — so --exclude '*.py' matches nothing: it can never match a string containing /. Even a file directly inside the target (src/a.py) needs 'src/*.py' or '**/*.py', and '**/vendor/**' (not '*/vendor/*') to match vendor/ wherever it sits.
• A bare basename never matches: the path always has the target directory prefixed on it (even for a file sitting directly inside the target), so '__init__.py' matches nothing at any depth. Always prefix it, e.g. '**/__init__.py'.

Excluded files are silent by default; --verbose prints one Excluded: <path> line each. If the exclusions leave nothing to process — or the directory holds no .py files at all — the command prints error: no Python files to process. and exits 2, rather than passing vacuously. --exclude also applies to a single-file target: a target that matches is not analysed, and exits 2 the same way.

Django projects

Parallel Workers (--max-workers) each get their own test database automatically: mutate4py supplies pytest-django with the per-Worker identity it normally only gets from pytest-xdist, so no conftest.py change is required and Workers never collide on one test database during migration.

Pass pytest-django's own --reuse-db through --pytest-args if you want it — mutate4py can't turn it on for you, because it changes what's on disk between runs (a kept-around test database) in a way only you can judge safe for your project and environment:

mutate4py polls/models.py --max-workers 4 --pytest-args="-q --reuse-db" --lcov lcov.info

A migration-signal count in your terminal is not evidence that --reuse-db skipped migrations: pytest-django still runs migrations to create (or verify) each test database, reused or not. The saving from reuse is skipping the teardown and recreation at the end of the run, not the migration step itself.

Framework bootstrap (django.setup()) runs once per Worker, before any Mutant, and imports every INSTALLED_APPS model module as a side effect. That has one consequence worth predicting rather than discovering: a Mutant in an app-loaded module (e.g. models.py) is already in sys.modules by the time its Worker primes, so it degrades to the subprocess executor — one fresh pytest process per Mutant, same as --no-fork — instead of the warm forking path. The Mutant is still killed correctly; only the speed of getting there changes. A Mutant in a module outside app loading (a plain utility module Django never imports through INSTALLED_APPS) keeps the warm path. Net effect: Django projects get the warm path least on the modules mutation testing cares about most (your models), which is correct, not a defect.

Per-Mutant plugin cost

Every Mutant runs pytest as a fresh session, whether the interpreter is warm or cold — no execution model removes plugin cost at collection or session scope. A warm interpreter (the forking executor) skips re-importing plugins, but their session-scope hooks still fire on every invocation; measured against one advisory plugin in this project, that cost was ~1.6s cold and still ~0.8s warm.

Two plugins mutate4py neutralises automatically for Mutant runs only (never Baseline, which needs the real numbers to classify correctly):

• pytest-cov, via --no-cov if it's importable — coverage instrumentation is pure cost here, because the run already holds coverage and per-Mutant coverage is never consumed.
• pytest-benchmark, via --benchmark-disable if it's importable — benchmark timing is unreliable under mutation testing regardless.

--no-cov and --benchmark-disable are each plugin's own designed-for-this override, not -p no:<plugin>: blocking a plugin outright deregisters its own options too, so a project's own addopts = "--cov=..." would turn into a pytest "unrecognized arguments" error. Any other plugin is left untouched — some are load-bearing for correctness, and mutate4py has no way to know which.

The remaining cost — pytest's own bootstrap plus whatever unknown plugins still hook in — is measured once per run, at Baseline time, with one extra --collect-only pass (no test body runs), and printed in the Mutation Report:

Per-Mutant overhead: 0.82s
Hint: per-Mutant overhead is high relative to your test suite; audit pytest
plugins with --pytest-args (e.g. -p no:<plugin>).

The hint fires once overhead reaches half of the Baseline's own duration. To audit which plugin is responsible, pass -p no:<plugin> through --pytest-args one plugin at a time and re-run — that flag does fully deregister a plugin, which is exactly what you want for an audit even though mutate4py can't use it automatically for the two it neutralises by default.

Worker provisioning cost and when parallelism pays off

Each parallel Worker (--max-workers >= 2) is a full tree copy of the working directory, provisioned with uv venv/uv sync so its editable install resolves to its own copy (see "How it differs from mutate4go" below). That provisioning cost is paid once per Worker per run, not once per Mutant — so it's fixed overhead that a run's total Site count must amortize.

For a handful of selected Sites, that fixed cost can exceed what parallelism saves; for a target file with many selected Sites, splitting the work across Workers wins even after paying it. There's no universal threshold — it depends on your test suite's own per-invocation cost — so if --max-workers isn't paying off, check the Mutation workers: <n> line against the Selected mutation sites: count in the run header and compare a serial run's wall time against a parallel one before assuming parallelism should always be on.

Building a test-context db

--test-contexts needs a coverage.py context db to read; --build-test-contexts OUTPUT_PATH builds one:

mutate4py --build-test-contexts contexts.db --pytest-args '-q'
mutate4py src/ --test-contexts contexts.db

It runs every test pytest would collect (scoped by --pytest-args, same as any other run) in its own isolated coverage run session, then combines the per-test data files into OUTPUT_PATH. That one-session-per-test approach costs one pytest startup per test, but it's the only sound way to build the db: a single shared pytest --cov-context=test session silently drops every test after the first to touch a shared line, so a line covered by several tests would narrow to only one of them. See ADR 0021 for the full failure mode and why it rules out the faster alternative.

--build-test-contexts is a no-run mode like --scan or --check-manifest: it builds the db and exits, accepts no positional PATH target (it builds a db for whatever pytest collects, not for a mutation target), and can't be combined with an execution option.

Test selection and the static outcome

With --test-contexts, most Mutants get narrowed to just the tests that cover their line. A static outcome — the line executed only under coverage.py's whole-run context, e.g. module-level constants, imports, or class headers — is not a narrowing failure: mutate4py runs the full test set verbatim for that one Mutant, same as it would without --test-contexts at all. A run with several static Mutants will show a longer tail than a fully narrowed one; that's expected, not a defect. See ADR 0018 for the full three-case model (narrowed / static / hard-error on disagreement) and why the third case aborts instead of silently falling back.

How it differs from mutate4go ([PY])

• --max-workers uses clone-per-worker, not tree-copy+cwd — mutate4go's tree-copy model is unsound under Python editable installs (pip install -e .), so each worker gets its own uv-provisioned venv instead.
• Coverage is acquired explicitly — --lcov / --cov-cmd / --reuse-coverage (Python has no universal -coverprofile equivalent).
• Manifest hash is structural (ast.unparse()), so reformatting and comment edits don't trigger a re-test, but any behavior-affecting edit does.
• Operators are localized to Python: adds and/or, True/False, and the identity/membership negation flips is/is not and in/not in.

Develop

Python ≥ 3.11, stdlib ast (zero runtime deps), packaged with hatchling.

uv sync
uv run mutate4py --help
uv run pytest