# `mix mutagen`
[🔗](https://github.com/iautom8things/mutagen_ex/blob/v0.2.0/lib/mix/tasks/mutagen.ex#L1)

# mix mutagen

## Synopsis

    MIX_ENV=test mix mutagen [--scope <target>] [--tests <target|auto>] [--since REF] [--site ID] [--summary PATH] [--summary-max N] [--escalate] [--escalate-max-files N] [--escalate-budget-ms N] [--timeout-ms N] [--no-fail-fast] [--seed N] [--json PATH] [--stream] [--no-progress] [--unsafe-json-outside-project] [--max-sites N] [--budget-ms N] [--max-concurrency N] [--no-host-app] [--no-test-alias] [--boot-timeout-ms N] [--no-source] [--include-host-output]

Run mutation testing against one or more scope targets, judged by a chosen
set of tests. The final JSON document goes to stdout or `--json <path>`.
It lists every mutation and its outcome together with run metadata.

## Reading the summary

    MIX_ENV=test mix mutagen --since origin/main --summary mutagen.md --json mutagen.json

The JSON and Markdown reports give advice for one run. Check first marks
functions with only uncovered sites or with at least three evaluated
high-confidence mutants and no kills. Worth a look lists remaining
high-confidence survivors. Low-confidence survivors stay in the detailed
results. Timeouts are counted as unevaluated.

Copy a printed `--site <id>` command to rerun one site with wider
automatic test discovery. Use the same checkout.

## Audit mode

    MIX_ENV=test mix mutagen --scope MyApp.Billing --tests auto --json billing.json --summary billing.md
    jq '.summary.by_function[]' billing.json
    find lib/my_app -name '*.ex' | while IFS= read -r file; do
      MIX_ENV=test mix mutagen --scope "$file" --tests auto --json "audit/$file.json" --summary "audit/$file.md"
    done
    find audit -name '*.json' -exec cat {} + | jq -rs 'sort_by(.summary.counts.kill_rate // 2)[] | [.summary.counts.kill_rate, ([.scope[].module] | join(", "))] | @tsv'

## Running in CI

Check out full history (`fetch-depth: 0`). A shallow checkout aborts.
Set `MUTAGEN_BASE` to the pull request's base commit.

    MIX_ENV=test mix mutagen --since "$MUTAGEN_BASE" --json mutagen.json --summary mutagen.md

Append the Markdown to `$GITHUB_STEP_SUMMARY` when present and upload
the JSON. Use `continue-on-error: true` for aborts. See the README for a GitHub Actions
example. The unmapped list shows changed lines outside runtime mutation
sites, including attributes and macros. Each run is advisory.

## Host test setup

Use `MIX_ENV=test` or explicitly select a custom test environment.
The task compiles test support, loads runtime config, and runs the host
test alias entries before the test task. It then starts the host OTP
application and requires `test_helper.exs` once from each configured
`:test_paths` entry with ExUnit autorun disabled.

Coverage, baseline, and each mutant use the helper's ExUnit options.
Formatters run on each pass. The report is written before after-suite
callbacks run. Each callback gets its own boot timeout. Cleanup errors
and timeouts go to stderr. A call to `System.halt/1` in the helper
prevents reporting.

The default report contains source excerpts and structured facts. Host
diagnostic bodies go to stderr. Abort documents include a fixed hint and
the step, exception and location when available. Source excerpts may
contain secrets committed in the repository. Pattern redaction of opted-in
host diagnostics is best-effort.

`--json PATH` is the guaranteed report channel. The file contains exactly
one JSON document, or NDJSON with `--stream`, regardless of host output.

Without `--json`, Mutagen writes the report to stdout and routes the
`:user` device and known console logger handlers to stderr. The VM can
write directly to file descriptor 1, including when an OTP logger
handler crashes. In-process I/O routing cannot intercept those writes.
Use `--json PATH` and `--summary PATH` in CI.

Mix can also write to stdout before the task starts. Do not publish stderr.

## Flags

  * `--scope <target>` selects a file ending in `.ex`, a module, or
    `Module.Name.function/arity`. Repeat it to add targets. Required
    unless you use `--since` or `--site`.
  * `--tests <target>` selects a test file, a `file:line` reference, or
    `tag:<name>`. Repeat it to add targets, or use `auto` on its own.
    Required unless you use `--since` or `--site`.
  * `--summary <path>` writes Markdown to a file alongside the JSON report.
    The files must have distinct paths. As with `--json`, the path must stay
    within the project root unless `--unsafe-json-outside-project` is set.
    Use a filename instead of `-`.
  * `--summary-max <int>` caps the Worth a look list. Default 5, minimum 0.
  * `--site <id>` selects one report site. It defaults to automatic tests.
  * `--since <ref>` plans the runtime mutations touched since the merge
    base with the given ref. `--scope` intersects the diff when supplied.
    Test selection defaults to `auto`.
  * `--tests auto` selects conventional test files and, with `--since`,
    test files with code changes and new test files.
  * `--escalate` searches for more tests for surviving or uncovered sites.
    `--no-escalate` is accepted with no effect for one release.
  * `--escalate-max-files <int>` caps files per function in each pass. Default 20.
  * `--escalate-budget-ms <int>` limits time spent on the extra passes. Default
    120000. Zero removes the limit for runs without `--site`. A single-site
    run caps this budget at the larger of 30 seconds and ten times the
    narrow baseline.
  * `--boot-timeout-ms <int>` sets the deadline for host setup after
    compilation and the deadline for each cleanup callback. Each
    coverage and baseline pass gets the greater of this timeout and
    ten times `--timeout-ms`. Default `300000`.
  * `--no-source` omits site source excerpts, file and function from
    aggregate and stream results and compile errors. Scope, test and
    coverage paths, module names and source-derived IDs remain.
  * `--include-host-output` includes captured compiler and test stderr
    and abort messages after best-effort redaction and truncation.
  * `--no-test-alias` skips the host test alias pre-steps. Mutagen ignores
    the arguments on the `test` entry and the return values of alias
    functions. It warns if the alias has entries and no `test` entry.
  * `--timeout-ms <int>` overrides the per-mutant deadline in milliseconds.
    By default the deadline is three times the baseline duration, with a
    5000 ms floor. Mutagen warns if a requested timeout is shorter than
    the baseline.
  * `--no-fail-fast` runs the full cited suite for each mutant.
    By default, the mutation phase
    stops executing tests on the first failure. A failure recorded
    before the deadline kills the mutant even if the full suite
    would time out. Baseline and coverage have no failure cap.
    The JSON field `meta.fail_fast` records the mode and defaults to
    `true` on argument errors.
    Survivor and kill equivalence assumes the tests run independently.
    Tests that set up or clean up state for other tests depend on
    ExUnit's ordering when the seed changes. Skipping one can change
    later mutants' verdicts, warnings, and taint, even when a later
    mutant runs every test. Use `--no-fail-fast` for these suites.

    After cited files load, Mutagen sets the seed, serial execution,
    failure cap and resolved filters. It keeps host formatters,
    adds its own when collecting failures and restores the host
    settings after each phase. Other host options stay active.
  * `--seed <int>` — ExUnit seed, propagated to every test-running phase.
    Default `0`. See Constraints.
  * `--json <path>` — write the final JSON document to `<path>` instead
    of stdout. The document always ends with a single newline. The path
    is canonicalised before any mutation runs: `..` segments and NUL
    bytes are refused at parse time, and the resolved path must stay
    inside the project root unless `--unsafe-json-outside-project` is
    also passed. Symlinks whose target escapes the project root are
    refused.
  * `--unsafe-json-outside-project` — opt-in to writing `--json` output
    outside the project root. CI integrations targeting an artifacts
    directory above the project root pass this flag; everyday use
    should leave it off. When set, a one-shot warning naming the
    resolved target lands on stderr at run start.
  * `--stream` emits per-site NDJSON lines with `start` and `end` envelopes
    on the JSON sink. The last line is the aggregate document without a
    `kind` field.
  * `--no-progress` suppresses the human-readable progress feed on stderr.
    By default, the feed appears when stderr is a TTY.
  * `--max-sites <int>` — upper bound on enumerated mutation sites
    for one run. Default `10000`. The enumerator aborts with
    `abort_reason: "too_many_sites"` before the runner starts when the cap is
    exceeded; narrow `--scope` (or raise the cap) to proceed.
  * `--budget-ms <int>` — aggregate wall-clock budget for the
    mutation phase, in milliseconds. Optional (default: unbounded;
    the per-site `--timeout-ms` is still enforced). When the budget
    elapses the runner stops dispatching new sites and emits a
    `truncated: true` partial report.
  * `--max-concurrency <int>` — per-site mutation dispatch cap. Default
    `1` (fully serial, v1.0-equivalent). Values greater than `1` are
    experimental and add warnings to the report and stderr because real
    ExUnit/:cover backends can produce incorrect kill/survive
    classification and corrupted coverage under parallel dispatch. The
    report records the limit as `meta.max_concurrency`.
  * `--no-host-app` — skip starting the host OTP application in the
    runtime preamble. Reserved for self-test sandboxes and libraries
    that intentionally need a minimal boot; the default is to start
    the host app so Repo/Sandbox, PubSub, Mox, and supervised test
    support are available. Runtime config and the helper still load when
    the app is skipped.

## Caps

  * `--scope` and `--tests` each accept at most 100 occurrences. Excess
    is refused at parse time with `abort_reason: "too_many_targets"`.
  * `--max-sites` caps enumerated mutation sites (default 10_000).
  * `--budget-ms` (optional) caps aggregate wall-clock for the
    mutation phase. The per-site `--timeout-ms` still applies.

## Examples

    # Single file scope, single test file
    MIX_ENV=test mix mutagen --scope lib/foo.ex --tests test/foo_test.exs

    # Multiple scopes, tag-based tests, custom timeout
    MIX_ENV=test mix mutagen --scope MyApp.Foo --scope lib/bar.ex \
                --tests tag:fast --timeout-ms 10000

    # Module-and-function scope, deterministic seed, redirected output
    MIX_ENV=test mix mutagen --scope MyApp.Foo.bar/1 --tests test/foo_test.exs \
                --seed 42 --json out/mutagen.json

## Constraints

  * Tests run **serially** (`max_cases: 1`) in every phase. There is no
    `--parallel` flag in v1 (see
    `mutagen.decision.serial_execution_and_seed`).
  * `--seed` controls **ExUnit test ordering** only. Mutation enumeration
    order is independent of the seed.
  * `mutagen_ex` runs **in-process** (same BEAM as your suite); see
    Caveats for the consequences (state drift, no self-mutation).

Requires Elixir 1.18.0 or later and Erlang/OTP 27 or later.

## Exit Codes

  * `0` means Mutagen wrote a report, including incomplete runs and runs
    with surviving mutants.
  * `2` means Mutagen aborted and wrote an error document to stdout or
    `--json <path>`. The string field `abort_reason` identifies the error.

A host callback calling `System.halt/1` decides the exit code. During
after-suite cleanup it can leave a completed report with `aborted: false`
and a non-zero exit.

## JSON Schema Pointer

Each aggregate result and NDJSON `"result"` line carries `confidence`.
The rating applies to the whole mutator. `with_swap`, `pipeline` and
`guard_drop` rate low. The rest rate high. Both levels count toward
aggregate totals and the kill rate.

The aggregate's `meta.timings_ms` records elapsed time for boot, coverage,
baseline, mutation and the full run. When comparing otherwise identical
reports, exclude this field.

See the README for examples from the CLI and
`.spec/specs/json_schema.spec.md` for the schema contract.

A survivor means “the tests you selected miss it”. First rerun with a
wider `--tests` selection before writing a test.

## Known Caveats

  * **State drift on `use SomeModule`.** Modules using compile-time DSLs
    can drift between baseline and mutation runs; the JSON `warnings`
    array names the affected modules.
  * **Macro mutation slowdown.** Mutations inside macro-heavy modules
    recompile dependents; expect longer per-site runs.
  * **Equivalent mutants.** Some mutations are semantically equivalent
    to the original; they survive every test by definition. This is a
    known limit of mutation testing, not a tool bug.
  * **Mutation IDs use scheme 2.** Formatting and unrelated edits preserve
    IDs when the file path, enclosing definition, normalized expression,
    and occurrence among identical sites stay the same. Both JSON and
    NDJSON use report schema version `"2"`. IDs and reports from earlier
    versions cannot be compared with version 2. Compare the schema, ID
    scheme, and Elixir and OTP versions before comparing IDs. Formatter
    migrations (`mix format --migrate`) can rewrite the syntax and change IDs.
    When an identical expression is added earlier in a function, later IDs
    can shift. Persist and compare `function` and `before_source` alongside IDs.
  * **`--no-json` is not supported in v1.** Pretty terminal output is
    deferred to v1.1 (see `mutagen.decision.no_pretty_output_v1`); use
    `jq` for now.
  * **`--seed` controls ExUnit ordering only.** It does not seed mutation
    enumeration. Given the same source, enumeration order is stable.
  * **`--scope` colon syntax is unsupported.** `file.ex:Module` is rejected
    with `abort_reason: "colon_syntax_unsupported"` (see
    `mutagen.decision.scope_syntax_simplified`).
  * **Self-mutation is refused.** `--scope MutagenEx.*` or `Mix.Tasks.Mutagen`
    exits with `abort_reason: "self_mutation_refused"` (see
    `mutagen.decision.self_mutation_refused`).
  * **`--json` paths are canonicalised inside the project root.** Paths
    with `..` segments or NUL bytes exit with
    `abort_reason: "unsafe_json_path"` at parse time. Symlinks whose target
    escapes the project root exit the same way at canonicalisation
    time, before any mutation runs. CI workflows writing artifacts
    outside the project root must pass `--unsafe-json-outside-project`
    explicitly; a one-shot stderr warning then names the resolved
    target.
  * **`--max-concurrency > 1` is experimental.** Parallel dispatch shares
    ExUnit.Server, the Code.Server, and `:cover` across per-site tasks.
    It can produce incorrect kill/survive classification and corrupted
    coverage on real ExUnit/:cover backends; the safe path is the
    default, `--max-concurrency 1`.
  * **`tag:NAME` charset is bounded.** `--tests tag:NAME` must match
    `~r/\A[a-z][a-z_0-9]{0,63}\z/` — lowercase ASCII, digits, or `_`,
    up to 64 chars, with a lowercase first character. Targets outside
    this charset are refused with `abort_reason: "invalid_tag_name"` before any
    test resolution runs. This is the atom-table-DOS bound (see
    `mutagen.cli.r11`): CI loops like
    `mix mutagen --tests tag:$(uuidgen)` cannot grow the BEAM atom table.
  * **Caps on input and output volume.** `--scope` and `--tests` each
    accept at most 100 occurrences (`abort_reason: "too_many_targets"`).
    `--max-sites` caps enumerated mutation sites (default 10_000;
    `abort_reason: "too_many_sites"`). `--budget-ms` (optional) caps the
    aggregate mutation wall-clock and yields a `truncated: true`
    partial report when exhausted.

# `dispatch`

```elixir
@type dispatch() :: %{
  optional(:cli) =&gt; module(),
  optional(:otp_release) =&gt; String.t(),
  optional(:elixir_version) =&gt; String.t(),
  optional(:project_root) =&gt; Path.t(),
  optional(:clock) =&gt; (atom() -&gt; integer()),
  optional(:scope) =&gt; module(),
  optional(:tests) =&gt; module(),
  optional(:ast_cache) =&gt; module(),
  optional(:coverage) =&gt; module(),
  optional(:enumerator) =&gt; module(),
  optional(:baseline) =&gt; module(),
  optional(:mutation) =&gt; module(),
  optional(:reporter_ok) =&gt; module(),
  optional(:reporter_error) =&gt; module(),
  optional(:io) =&gt; module(),
  optional(:reporter) =&gt; {module(), atom()},
  optional(:pipeline) =&gt; {module(), atom()}
}
```

Pluggable collaborators for the mix task's state machine. Each entry is a
**module atom** the task uses instead of a hard-coded call, so every
error path AND the happy path can be unit-tested by injecting a stub
module that captures its arguments and returns canned shapes.

Per bw mutagen-wrd.33 (closes F9), the 11 production phase keys carry
module atoms (not `{module, function}` tuples). The Mix task calls
`mod.callback(args)` directly; the callback for each slot is defined
by a dedicated behaviour under `MutagenEx.Pipeline.*Facade`.

The keys correspond to the orchestration stages of
`mutagen.mutation_pipeline`:

  * `:cli` — `mod.parse(argv)` (`MutagenEx.Pipeline.CliFacade`,
    default `MutagenEx.CLI`).
  * `:scope` — `mod.resolve(target, opts)`
    (`MutagenEx.Pipeline.ScopeFacade`, default
    `MutagenEx.ScopeResolver`).
  * `:tests` — `mod.resolve(tests, opts)`
    (`MutagenEx.Pipeline.TestsFacade`, default
    `MutagenEx.TestSelector`).
  * `:ast_cache` — `mod.load(files, opts)`
    (`MutagenEx.Pipeline.AstCacheFacade`, default
    `MutagenEx.AstCache`).
  * `:coverage` — `mod.run(input)`
    (`MutagenEx.Pipeline.CoverageFacade`, default
    `MutagenEx.CoverageRunner`).
  * `:enumerator` — `mod.enumerate(cache, scope, covered, opts)`
    (`MutagenEx.Pipeline.EnumeratorFacade`, default
    `MutagenEx.MutationEnumerator`).
  * `:baseline` — `mod.run(input)`
    (`MutagenEx.Pipeline.BaselineFacade`, default
    `MutagenEx.Baseline`).
  * `:mutation` — `mod.run(input)`
    (`MutagenEx.Pipeline.MutationFacade`, default
    `MutagenEx.MutationRunner`).
  * `:reporter_ok` — `mod.emit_report(report)`
    (`MutagenEx.Pipeline.ReporterOkFacade`, default
    `MutagenEx.JsonReporter`).
  * `:reporter_error` — `mod.emit_error(report, reason)`
    (`MutagenEx.Pipeline.ReporterErrorFacade`, default
    `MutagenEx.JsonReporter`).
  * `:io` — `mod.emit(iodata, exit_code, Config.t())` sink for the
    final document (`MutagenEx.Pipeline.IoFacade`, default
    `MutagenEx.Pipeline.DefaultIo` — writes to stdout or
    `Config.json_path` and halts the VM with `exit_code`).

S1 shipped a two-key legacy shape (`:reporter`, `:pipeline`) for
early CLI testing. Those keys carry `{module, function}` tuples and
route through the legacy code path (`run_legacy/2`), which uses
`apply/3` to dispatch — that is the explicit back-compat fallback
carve-out of `mutagen-wrd.33`'s "no apply/3 in the Mix task"
contract.

# `run`

```elixir
@spec run([String.t()], dispatch()) ::
  :ok
  | {:aborted, atom(), MutagenEx.JsonReporter.Report.t()}
  | {:error, MutagenEx.CLI.reason(), map()}
```

Test seam: run the task with a custom dispatch table.

Production code calls `run/1`, which threads through `default_dispatch/0`.
Tests pass a partial dispatch via `run/2` — any key absent from the
passed map falls back to the default.

The return value mirrors what `run/1` does observably:

  * `:ok` on a successful parse and pipeline run that produced a
    success-shape JSON document.
  * `{:aborted, reason, %Report{}}` on any abort path. The reporter and
    io collaborators have already been invoked with the iodata + exit
    code; the return value is for tests that want to assert which abort
    reason fired without parsing the emitted JSON.

`run/1` itself does not return one of these — it calls `System.halt/1`
via the default io collaborator on every path. `run/2` lets tests observe
without halting the test VM by overriding the `:io` collaborator.

---

*Consult [api-reference.md](api-reference.md) for complete listing*
