All notable changes to mutagen_ex are recorded here.
The format follows Keep a Changelog; this project uses Semantic Versioning.
The released and Unreleased sections describe user-visible behavior. The engineering log is preserved as an appendix at the end of this file for traceability.
[Unreleased]
0.2.0 - 2026-11-20
First Hex release. Mutagen changes one expression at a time and runs the selected tests to show which changes they catch.
Breaking changes since the development snapshot
- Reports use schema
"2". Mutation IDs use a new scheme. Start a fresh comparison when moving from earlier output. - Exit 0 means the run produced a report, including partial runs and survivors. Exit 2 means the tool aborted. Check completeness in the report.
- Use
--json PATHfor machine-readable output. Stdout is best effort because host code and the VM can write to it directly.
Features
- Audit a module to rank its functions by kill rate. Use
--summary PATHfor a Markdown report and--site IDto rerun a finding. - Use
--since REFto inspect changed code since the merge base. Mutagen selects relevant tests automatically. Use--escalateto try more tests. - Runs load the host test setup and report startup failures. Per-mutant deadlines and an optional run budget bound the mutation phase.
- Execution probes distinguish confirmed misses from sites where coverage cannot tell whether an expression ran.
Known limits
- A survivor may be equivalent to the original code. Review it before writing a test. Automatic selection can miss relevant tests.
- Some expressions, including guards and patterns, cannot host execution probes. Mutagen reports uncertainty when it lacks execution evidence.
- Run serially for reliable results. Parallel mutants share VM state and can produce incorrect verdicts.
0.1.0 (unpublished development snapshot, 2026-05-13)
Version 0.1.0 was never published on Hex or tagged.
Added
mix mutagenMix task — the sole CLI entry point.--scope <target>(required, repeatable): file path, module name, orModule.fun/arity.--tests <target>(required, repeatable): test file path,file:line, ortag:<name>.--timeout-ms <int>(default5000): per-mutation wall-clock budget.--seed <int>(default0): ExUnit seed, propagated to every test-running phase.--json <path>: redirect the final JSON document from stdout to a file.
- Orchestration state machine for the run pipeline: CLI → scope → tests → AST cache → coverage → enumeration → baseline → mutation → reporter. Every phase is dispatch-table-driven for test substitution.
- Single-process, serial-execution model (no worker pool, no shelled subprocess).
- Coverage phase using
:cover, scoped to in-scope modules, with the:cover_servertorn down cleanly between runs. - Mutation enumeration restricted to covered lines, with content-addressed
mutation IDs so they stay stable across
mix formatruns. - Mutators present in the snapshot:
arith,compare,boolean,case_drop,else_removal,withblock_with_swap,withblock_else_removal,literal. (See Known limitations for theliteralgap.) - Mutation runner with per-site sandboxed compile, ExUnit re-run, and
module restoration. Per-site outcomes classify as
:killed,:survived,:timeout,:error, or:compile_error. - JSON reporter emitting the v1 document shape (success and error
variants), terminated by a single newline. Golden fixtures under
test/mutagen_ex/golden/serve as the schema-by-example. - Exit-code discipline:
0on completion (even at 0.0 kill rate), non-zero on any abort. Every non-zero exit emits an error-JSON document. - Refusals (each with a stable
reason:atom)::missing_scope,:missing_tests,:invalid_timeout— bad input.:flag_not_supported_in_v1— explicit reject of--no-json.:colon_syntax_unsupported— explicit reject offile.ex:Module.:self_mutation_refused— refuses to mutateMutagenEx.*orMix.Tasks.Mutagen.:baseline_red— aborts before any mutation phase if the cited tests do not all pass against unmodified source.
mix help mutagenoutput with Synopsis, Flags, Examples, Constraints, Exit Codes, JSON Schema Pointer, and Known Caveats sections.README.mdwith quick-start, flag reference, exit-code table, JSON-schema pointer, and known-limitations list.
Known limitations
Known in the development snapshot:
- File-cited
--tests(a bare_test.exspath) produced a filter that excluded every test. - The production mix task wired the mutation runner with an empty test module list.
- A per-site timeout could leave the code server holding a module-load lock, occasionally corrupting the tail of a run.
- The
:case_dropmutator on a guarded base case classifies askilled(aCaseClauseError) rather thantimeout. - The
literalmutator never fired, because atomic literals were wrapped in a way the mutator did not recognise.
(Several of these were fixed in 0.2.0.)
Internal development log
Archive-installed CLI runtime self-heal. Fixed the archive adoption path where invoking
mix mutagenfrom a host project without:mutagen_exin deps, but with the task installed globally viamix archive.install, could crash with aMatchErroratlib/mix/tasks/mutagen.ex:236instead of emitting the CLI's structured error surface. The runtime preamble now gates repair on theApplication.ensure_all_started(:mutagen_ex)failure tuple, callsMix.Local.append_archives/0, retries, then defensively scansMix.path_for(:archives)for an archive ebin containingmutagen_ex.appbefore a final retry. If the archive remains unrecoverable, the task emits error JSON withabort_reason: "runtime_load_failed"rather than leaking a traceback. Regression coverage lives intest/integration/archive_install_test.exsunder tag:archive_integration; the spec recordsmutagen.cli.r16, scenariosmutagen.cli.s14c/mutagen.cli.s14d, and verificationmutagen.cli.v11. The existingpath:/ Hex / git dependency adoption paths are preserved byte-for-byte because they succeed on the first runtime-start attempt and never enter the archive repair branch..25epic capstone: bench harness, large-scope memory test, and measured speedup. The.25AST/perf epic (S1–S6) is complete. This entry summarises what the epic shipped, the perf number the bench harness produced before vs. after the epic, and the byte- identity contract the refactor preserves.Bench harness —
scripts/helper_scripts/bench_ast_perf.exsis no longer a skeleton: it drives the realMix.Tasks.Mutagen.run/2pipeline against thewrd25_200sitesfixture (S1), captures wall-clock +:erlang.memory/0snapshots + SHA-256 of the emitted NDJSON, supports--baseline <path>/--compare <path>for before/after scoring, and uses a stable tmp-ebin so the SHA is reproducible across invocations. Two minor wrd25 fixture asserts (mix4,chain2) were corrected so baseline runs green; the fixture'slib/is unchanged.Measured speedup (wrd25_200sites, scope
lib/arith_dense.ex, Elixir 1.19.5 / OTP 28):metric before ( 978a995, pre-.25 main merge)after ( 78b022f, post-.25.6)sites/run 87 87 wall_ms 2179.53 1314.00 per_site_us 25052.05 15103.49 Wall-clock speedup: 1.66×. Per-site speedup: 1.66×. This is below the epic's documented 2–4× target — see the follow-up note below.
Byte-identity: the SHA-256 of the full NDJSON document differs between commits (state-drift warning text and "redefining module" paths embed run-time process state), but when those advisory diagnostics are normalised the documents are byte-identical:
3534443176dd9e2c673b21703085b1b02b2ddac0f2d6e3f7598542d5723df57aon both commits. The mutation results — site IDs, before/after source slices, killed/survived verdicts, kill_rate — are byte-for-byte the same; the .25 refactor changes timing, not outcomes. The full per-commit document SHAs (advisory:mix run scripts/helper_scripts/bench_ast_perf.exsreproduces them):- before:
d878975d19112378f3598cb32c452c7c55b90b810156316335f13c9e5e8bee85 - after:
46292518ccecca161161f278e3df58dc05d1031a47b9bff9ff7137af499b8b2a
Speedup-shortfall triage — the measured 1.66× falls short of the spec's lower bound. Trace through the epic's deliverables:
- S2 (Ast lift) removed duplicated helpers — wins compile time and code clarity, not per-site runtime.
- S3 (AstCache categorised load) cut redundant
File.read/1for cited test files in the Baseline async-warning path — runs once per pipeline, sub-millisecond impact at this scope. - S4 (head-atom dispatch table) pre-filters mutators per node
— the
O(nodes × catalog)→O(nodes × applicable)win is real but proportional to catalog size; with 10 mutators it's a 2-3× win on the enumeration phase, which is a small slice of wall time vs. the per-site test-execution cost. - S5 (batched grouped-by-file prewalk) replaced per-site
Macro.prewalk/2withO(depth)path descent — the headline win. Its impact isO(sites × file_ast_size)saved, but on the wrd25 fixture the file AST is small (~40 lines/file), so the win is modest. On a real codebase with multi-thousand-line scope files the win compounds. - S6 (BeamCache binary-swap restore) replaced
Code.compile_quoted/2on the cached AST with:code.load_binary/3. This is per-site and should be a strong contributor; the bench shows it delivered.
The bench fixture exposes hot AST machinery, but the per-site cost is dominated by the test-execution phase (one
ExUnit.run/0per mutation, with:coverinstrumentation) — and that's untouched by.25.Mutators.normalize/1(out of scope per the epic's Out of Scope) and the per-site ExUnit boot also dominate. A follow-up considered replacing the per-site ExUnit boot with a longer-lived test runner to close the gap to the 2–4× target.Large-scope memory test — new
test/mutagen_ex/memory_test.exs(tagged:e2e_slow) generates a 1000-site synthetic fixture inSystem.tmp_dir!/0at runtime (NOT committed to disk), drives the full pipeline against it with a 60s--budget-mscap, and asserts the run completes without OOM. The assertion is deliberately weak per the epic's Out of Scope (intent) — it passes either on a clean completion (aborted: false) or a graceful budget-cap (truncated: true); a non-graceful abort (any reason outsidebudget_exceeded/too_many_sites) fails the test. Heap-size assertions are not pinned (brittle across BEAM versions).Epic summary — one line per feature shipped:
MutagenEx.Asthelper lift: canonicalalias_to_module/1,find_module_body/2,node_line/1AstCache.load/2categorised input + Baseline cache consumerMutagenEx.Mutators.Dispatchhead-atom table + enumerator pre-filter- Batched grouped-by-file prewalk for the per-site AST swap
MutagenEx.BeamCacheper-run ETS + binary-swap restore- Bench harness, memory test, this CHANGELOG entry
- before:
BeamCache per-run ETS + binary-swap restore.
MutagenEx.MutationRunner.run/1no longer restores viaCode.compile_quoted/2on the cached AST. Restore is now a binary swap via:code.load_binary/3against a per-runMutagenEx.BeamCachesnapshot: a:set, :publicETS table owned byrun/1itself (created at entry, deleted inafter— no GenServer, no supervisor child, no cross-invocation staleness). A serial snapshot pre-pass runs BEFOREasync_stream_nolink/4dispatch and captures every scoped module's currently-loaded.beamvia:code.get_object_code/1, closing the TOCTOU window two parallel workers would otherwise open. Snapshot order vs. cover instrumentation matters: the pre-pass runs AFTER:cover.compile_directory/1, so the cached binary IS the cover-instrumented binary and restore preserves coverage between sites. The newMutagenEx.Test.CodeServerFacadebehaviour (with default implMutagenEx.Test.CodeServer) is the test seam — mirrors the existingMutagenEx.Test.CompilerFacadeshape exactly. Thewith_restore/4wrapper's signature and external contract are preserved; only its internal restore call moved offCode.compile_quoted/2.MutagenEx.Application's supervisor child list is unchanged. Revised requirementmutagen.mutation_pipeline.r6+ new scenarioss16/s17+ verification stubv9. Two new decisions:mutagen.decision.per_run_beam_cache(ETS-via-cfg, not GenServer) andmutagen.decision.code_server_facade(testability seam).Batched grouped-by-file prewalk for the per-site AST swap.
MutagenEx.MutationRunner.run/1now pre-computes a per-file path index once before the per-site loop begins (oneMacro.prewalk/2per distinct file incfg.sites, regardless of how many sites that file carries). Each site's swap then descends along the cached path inO(depth)viaapply_swap_at_path/3instead of running a freshMacro.prewalk/2over the whole file AST per site. Path entries are keyed bysite.id(content-addressed) so duplicate-position sites do not collide. Bare-literal sites (Literalmutator on bare integers/booleans;ResultTupletargeting bare booleans) carry no static AST coordinates and fall back to the existing ambient-threading walker (replace_bare_site/2); the legacy per-siteMacro.prewalkis also retained as a defensive fallback. Byte-identity property pinned bytest/mutagen_ex/mutation_runner_batched_test.exs: for every site, the mutated file AST produced by the batched path equals the file AST that an unconditional per-siteMacro.prewalk/2would have produced. Closes F16 (HIGH, F-PERF-02). New requirementmutagen.mutation_pipeline.r16+ scenarioss14/s15+ verification stubv8.
Added
- Head-atom dispatch table for the mutator catalog. New module
MutagenEx.Mutators.Dispatchcarries a static, order-preserving mapping from AST head atom (:+,:case,:with, …) to mutator list, plus an:anybucket for mutators that match non-3-tuple shapes (Literalfor bare scalars /{:__block__, _, [v]}wrappers;ResultTuplefor{:ok, _}/{:error, _}2-tuples).MutagenEx.MutationEnumerator.try_mutators/6consultsDispatch.mutators_for_node/1to pre-filter the catalog before callingmatch?/1per node —O(nodes × catalog)becomesO(nodes × applicable_mutators). No mutator behaviour change; the ten mutator modules are untouched. Permutagen.decision.static_mutator_dispatch. The:dispatch_modeoption onenumerate/4(:head_atomdefault,:legacyfor the equivalence test) is an internal test seam — it is NOT documented in the public API and NOT exposed viamix mutagen. New requirementmutagen.mutation_enumeration.r9and equivalence testtest/mutagen_ex/head_atom_dispatch_test.exslock the correctness (same mutators consulted per node) and order- preservation (byte-identity / r1) properties.
Changed
AstCache.load/2accepts categorised input. A newopts[:categories]map (e.g.categories: %{scope: scope_files, test: test_files}) lets callers partition the flatfileslist for diagnostic visibility without changing the cache entry shape. Entries remain{Macro.t(), String.t()}2-tuples permutagen.decision.ast_cache_facade_preserved; thePipeline.AstCacheFacade@callback load/2signature is preserved verbatim. Per-category counts are logged at debug level. Thephase_ast_cachestep ofmix mutagennow loads scope files AND cited test files (test_filter.files) in one pass. New requirementmutagen.coverage.r9. Closes F18 / F40.Baseline.detect_async_modules/1consumes the AST cache when provided. WhenBaseline.run/1's input map carries:ast_cache, the async-warning path looks each cited test file up viaAstCache.get/2and consumes the cached{ast, _source}directly — no re-read of test files from disk. On cache miss the implementation falls back toFile.read/1+ parse (pre-.25behaviour) and logs the miss for diagnostics.- Shared AST helpers lifted into
MutagenEx.Ast. Thealias_to_module/1,find_module_body/2, andnode_line/1helpers — previously duplicated acrossMutagenEx.ScopeResolver,MutagenEx.MutationEnumerator,MutagenEx.MutationRunner, andMutagenEx.TestModuleDiscovery— now live in one canonical module (lib/mutagen_ex/ast.ex). Behaviour is byte-identical to the donors (pinned bytest/mutagen_ex/ast_donor_equivalence_test.exs). Closes F21 / CF10. New subjectmutagen.astcarries the contract; see.spec/specs/ast.spec.md. ScopeResolversorts the default `lib//.ex` wildcard result* before searching for module-shaped targets. Closes the F30 / CF7 determinism risk:Path.wildcard/1's order is file-system-dependent, so two hosts could otherwise pick a different match. New requirementmutagen.scope_resolution.r9.
Added
wrd25 bench fixture at
scripts/helper_scripts/bench_fixtures/wrd25_200sites/— a small self-contained Mix project (5 modules + colocated tests) used by the.25AST-perf bench harness skeleton (scripts/helper_scripts/bench_ast_perf.exs, completed in S6) and by the determinism safety-net test (test/mutagen_ex/determinism_test.exs, re-targeted from the lane fixture).Parallel mutation loop, NDJSON streaming, and progress feedback. The mutation runner now dispatches per-site work through
Task.Supervisor.async_stream_nolink/4underMutagenEx.TaskSupwith:ordered: true, so results are collected in input order regardless of which task body finished first. Configurable via the new--max-concurrency <int>flag (default1for v1.0-equivalent serial execution; set explicitly toSystem.schedulers_online()to opt in). New requirement:mutagen.mutation_pipeline.r15.:telemetryevents at well-defined points:run.start/stop,coverage.start/stop,baseline.start/stop,enumeration.stop, and per-sitesite.start/stop. Superseded and removed (see the[Unreleased]Removed section): the producer-only event API and the:telemetryruntime dependency were dropped so the archive install is dependency-free; per-site observation now rides the runner's:on_site_completedcallback.--streamenables NDJSON-per-site emission via the newMutagenEx.JsonStreamer. Each line carries a"kind"discriminator ("start","result","compile_error","end") and a per-site wire shape byte-equal to the equivalent entry in the aggregate document. New requirements:mutagen.json_schema.r10,mutagen.json_schema.r11.--no-progresssuppresses the human-readable per-site progress feed on stderr; the default is auto-on when stderr is a TTY, auto-off otherwise. See the newMutagenEx.Progressmodule.- New dependency:
:telemetry ~> 1.0(lightweight, no transitive deps). The runner emits events through the standard:telemetry.execute/3and:telemetry.span/3surfaces. The runtime is now dependency-free.
Security
- Bound stderr / exception /
Macro.to_stringoutput flowing into the JSON report; added opt-in:redactconfig knob. Stderr captured during per-mutation runs, exception messages from compile/restore paths, and Elixir source slices in compiler diagnostics flowed verbatim intomutation.results[].warnings[],mutation.compile_errors[].message, and abort-detail message fields. Combined with the lax--json <path>write, secrets archived in those reports became file-system-resident. Fixed by:- A new
MutagenEx.JsonReporter.Sanitizermodule truncates every free-form text field at a 4 KiB cap, appending the literal marker... <N bytes truncated>when truncation occurs. Truncation splits on a codepoint boundary so the emitted string is always valid UTF-8. New requirement:mutagen.json_schema.r10. - Opt-in
:redactapplication config (a list of%Regex{}or binary regex sources, read viaApplication.get_env(:mutagen_ex, :redact, [])) — each match in any sanitized field is replaced with the literal[REDACTED]. Redaction runs BEFORE truncation so secrets near or past the 4 KiB cap are still replaced rather than silently dropped. Default:redactis[](no-op). New requirement:mutagen.json_schema.r11. - Wired into all sanitization choke points:
MutagenEx.AstCachefile-read + parse error messages;MutagenEx.Baselinetest-file load + ExUnit.run failure messages;MutagenEx.MutationRunnercompile-error / restore-failure / unrecoverable-restore-failure messages and per-result warnings (viacompose_warnings/1).
- A new
Performance
Macro.to_string/1computed once per mutation result. The Mix task'srender_result/1(lib/mix/tasks/mutagen.ex) calledMacro.to_string(r.original_ast)twice per result — once forbeforeand once forbefore_source— wasting a full AST-to-string walk on every reported mutation. The render path now computesMacro.to_string(r.original_ast)exactly once and aliases the resulting binary into bothbeforeandbefore_source. Reference identity (:erts_debug.same/2) is the falsifiability check. Verbatim source-slice extraction forbefore_source(per the contract documented inlib/mutagen_ex/ast_cache.ex) is deferred to a follow-up since it requires extending theSitestruct to carry{end_line, end_column}metadata. New requirement:mutagen.json_schema.r12.Bound atom creation on CLI input. Three call sites converted attacker- or typo-controlled strings to atoms with no upper bound, making CI loops like
mix mutagen --tests tag:$(uuidgen)a fast atom-table sink (atoms are not GC'd; the default cap is ~1M). Fixed by:MutagenEx.CLInow applies atag:NAMEcharset gate at the front door (~r/\A[a-z][a-z_0-9]{0,63}\z/); inputs outside the charset return{:error, :invalid_tag_name, _}before any test resolution runs. New requirement:mutagen.cli.r11.MutagenEx.TestSelector.resolve/2fortag:NAMEno longer callsString.to_atom(name). Instead it walks the test corpus and compares the user'sNAMEstring againstAtom.to_string/1of each@tag :ATOMliteral found in the parsed AST; the matched AST-derived atom flows intoinclude:. New requirement:mutagen.test_selection.r7.MutagenEx.ScopeResolver.resolve/2forModule.NameandModule.Name.fun/aritytargets no longer callsString.to_atomon user input. Module matching uses canonical-string comparison ("Elixir.Foo.Bar"vs.Atom.to_string/1of each ASTdefmoduleatom); function-name matching uses string comparison against AST-deriveddef-head atoms. The matched module atom comes from the AST.details.moduleon:module_not_foundis now the canonical string (e.g."Elixir.Nope");details.functionon:function_not_foundis the user's string segment. New requirement:mutagen.scope_resolution.r8.
Changed
Test-seam call sites swapped to behaviour-backed facades. The eleven
apply(Map.get(cfg, :facade, ProductionMod), :fun, [args])call sites inMutagenEx.Baseline,MutagenEx.CoverageRunner,MutagenEx.MutationRunner, andMutagenEx.MutationRunner.MutationLoopnow dispatch through named behaviours instead ofapply/3with no compile-time leverage. New facade modules underlib/mutagen_ex/test/:MutagenEx.Test.ExUnitFacade(defaults toMutagenEx.Test.ExUnit, delegates toExUnit.configure/1,ExUnit.run/0).MutagenEx.Test.ExUnitServerFacade(defaults toMutagenEx.Test.ExUnitServer, delegates toExUnit.Server.add_module/2).MutagenEx.Test.CaptureIoFacade(defaults toMutagenEx.Test.CaptureIo, delegates toExUnit.CaptureIO.with_io/2).MutagenEx.Test.CompilerFacade(defaults toMutagenEx.Test.Compiler, delegates toCode.compile_quoted/2).MutagenEx.Test.CoverFacade(defaults toMutagenEx.Test.Cover, delegates to:cover.start/0,:cover.stop/0,:cover.compile_beam/1,:cover.analyse/3).
Existing facade-using tests continue to swap modules (not function names) and pass without changes. The
:compilerconfig key also accepts a legacy{module, function}tuple for back-compat with older stubs; new callers should pass a behaviour- implementing module atom.
Fixed
- Scenario 7 (
:ecto_user_scenario) un-skipped. The end-to-end Scenario 7@tag :skipintest/mutagen_ex/end_to_end_test.exswas long blamed on a:cover+ Ecto-style DSL interaction. The coverage spike (direct reproduction inscripts/helper_scripts/spike_19_repro.exs) showed every macro-injected callback —__schema_kind__/0,field/2-generatedname/0andage/0, thebirthday/1arithmetic helper, and the persisted:lane_schema_kindattribute — survives:cover.compile_beam/1->:cover.stop/0->:code.purge/1->:code.load_file/1byte-for-byte. The actual baseline-red came from one assertion intest/fixtures/lane_project/test/lane_fixture/ecto_user_test.exs:30(assert :registered in Keyword.get_values(attrs, :lane_schema_kind)) that fails becausepersist: trueattributes serialise their value wrapped in a list —Keyword.get_values/2returns[[:registered]], not[:registered], and:registered in [[:registered]]isfalse. The test fails identically with or without:cover. The assertion was rewritten to flatten the values plus an explicitKeyword.fetch!(attrs, :lane_schema_kind) == [:registered]check, Scenario 7's@tag :skipwas removed, and README "Known limitations" item 5 was marked resolved. The Spike-I bytecode-identical-restore invariant is now exercised end-to-end against the hand-rolled DSL.
Security
--json <path>is canonicalised before any mutation runs. Closes F7 (HIGH, Security reviewer H2):mix mutagen --json <path>previously accepted any binary, which combined with the in-process compile-and- execute pipeline (mutagen.decision.in_process_pipeline) was an arbitrary-file-write primitive — a malicious mutated test could redirect the report into/etc/,~/.ssh/, or any symlinked path. Two layers of check now run before any mutation phase:- Parse-time pure-string check refuses paths with NUL bytes or any
..segment (abort_reason: "unsafe_json_path"). - Filesystem canonicalisation expands every component through
File.read_link/1and refuses paths whose fully-resolved target escapes the project root (alsoabort_reason: "unsafe_json_path"). CI integrations that need to write outside the project root pass--unsafe-json-outside-projectexplicitly; that flag emits a one-shot stderr warning naming the resolved target. New invariant:mutagen.cli.r10.
- Parse-time pure-string check refuses paths with NUL bytes or any
Resource caps on input and output volume. Closes F28 / CF11 (MEDIUM consensus from Security M1 + Performance F-PERF-07/12): the CLI previously accepted unlimited
--scope/--testsrepetition, the enumerator materialised every site in memory before the runner started, and there was no aggregate wall-clock budget — three avenues for runaway resource use.--scopeand--testseach cap at 100 occurrences; the 101st is refused at parse time withabort_reason: "too_many_targets".--max-sitescaps enumerated mutation sites (default 10_000). Exceeding the cap aborts the pipeline withabort_reason: "too_many_sites"BEFORE the mutation runner starts.--budget-msis an optional aggregate wall-clock budget for the mutation phase. When the budget elapses the runner stops dispatching new sites and emits atruncated: truepartial JSON report (aborted: false; the per-site--timeout-msstill bounds the in-flight site). New invariants:mutagen.cli.r12,mutagen.cli.r13,mutagen.mutation_enumeration.r7.
Added
MutagenEx.JsonPath— single home for the--jsonpath-safety contract.validate_literal/1is the pure-string check the CLI parser calls at parse time;canonicalize/2is the filesystem-aware check the mix task calls before any mutation phase. The project root is resolved through its own symlinks (handles macOS/var -> /private/var); the inside-root comparison uses the canonical form.--unsafe-json-outside-projectflag onmix mutagen. Boolean opt-in that bypasses the inside-root check while still resolving symlinks. Lands onConfig.unsafe_json_outside_project.MutagenEx.Application— a one-for-one supervisor (MutagenEx.Supervisor) whose only child is a namedTask.Supervisorregistered asMutagenEx.TaskSup. Declared viamod: {MutagenEx.Application, []}inmix.exs, so the supervision tree starts whenever:mutagen_exboots — for both themix mutagenCLI path and library callers depending on:mutagen_exdirectly. New invariant:mutagen.mutation_pipeline.r13.test/mutagen_ex/supervision_test.exsexercising start_link / terminate cycles, recursive-kill propagation across an OTP-supervised descendant subtree and an untrapped linked grandchild, and concurrent-caller rejection ofCoverageRunner.run/1. Coversmutagen.mutation_pipeline.r13,mutagen.mutation_pipeline.r14,mutagen.coverage.r1, andmutagen.coverage.r8..spec/decisions/supervision_tree.mddocumenting the Application + namedTask.Supervisorchoice, the singleton-ownership contract for:cover_server/ExUnit.Server, and the v2 won't-haves (out-of-process workers,:cover.start/0race arbiter).test/support/disk_snapshot_helper.exs— disk-snapshot diffing helper used by the r11 / r7 "no disk writes" tests. Captures byte-identity acrosslib/**,_build/**/*.{beam,app},cover/**, host project config (mix.exs,mix.lock,.formatter.exs), andmutagen_ex_- attributable entries underSystem.tmp_dir!().
Changed
Mix.Tasks.Mutagen.phase_scope/3now accumulates per-target scope records in O(n) overall instead of O(n²). The previousacc ++ recordspattern was quadratic in the cumulative record count — a measurable cost on broad--scopeworkloads. The new accumulator prepends chunks and flattens once via:lists.append(:lists.reverse(acc)). Closes F-PERF-12 (the half-overlap with F20). Order of records is preserved; a new regression test (test/mutagen_ex/mix_task_phase_scope_test.exs) asserts both ordering and a 2_000 ms ceiling for 10_000 records across 2_000 targets.MutationRunner.safe_compile_quoted/3andMutationLoop.capture_stderr/2now useExUnit.CaptureIO.with_io/3— which returns{closure_result, captured_io}directly — instead ofcapture_io/2paired with amake_ref/Process.put/Process.getsmuggle to extract the closure's return value. Removes the dependency on the (undocumented) fact thatcapture_io/2runs its closure in the calling process and deletes the per-call process-dictionary lifetime bookkeeping. Specmutagen.mutation_pipeline.r9updated to namewith_io/3; behavior is byte-identical (same stderr capture, same suppression, same warning attachment). The:capture_ioseam used by tests now requireswith_io/3(test stubsCaptureIoStubandRaisingCaptureIOupdated in lock-step).The r11 disk-write test in
mutation_runner_test.exsand the r7 disk-write test incoverage_runner_test.exsnow assert byte-identity acrosslib/,_build/,cover/, host project config, andmutagen_ex_-attributable tmp entries — not justlib/**/*.ex. Closes Testing-reviewer F13: the original tests would have shipped green if the runner accidentally rewrotemix.exs, dropped artifacts under_build/, or wrote a coverage report tocover/. Themutagen.mutation_pipeline.r11andmutagen.coverage.r7spec statements were updated to name the broader surface explicitly, with an allowed-write list ofNONE.MutationRunner.run_one_site/3now routes the loaded-mutation window through a newwith_restore/4lifecycle helper that mirrorsMutagenEx.CoverageRunner.with_cover_lifecycle/2. A raise, throw, or exit inside the window (e.g. a misbehaving:capture_ioseam) now triggers restore before the exception re-propagates. The original{kind, value, stacktrace}is preserved viareraise/2/:erlang.raise/3; restore failure during propagation is swallowed insidesafe_restore/3and never masks the original cause. New invariant:mutagen.mutation_pipeline.r12.The
:compile_errorbranch now surfaces defensive-restore failure as:unrecoverable_restore_failure(with both the restore failure and the original:compile_errormessage indetails.message) instead of silently discarding it via_ = restore(...). Permutagen.mutation_pipeline.r6.MutationLoop.run/1spawns its per-site task viaTask.Supervisor.async_nolink(MutagenEx.TaskSup, ...)instead ofTask.async/1. The two-phase cancel keepsTask.shutdown(task, cancel_grace_ms)for the mailbox-drain window, then escalates toTask.Supervisor.terminate_child(MutagenEx.TaskSup, task.pid)— which propagatesProcess.exit(_, :shutdown)through the task's link tree, reaping untrapped linked grandchildren and OTP-supervised descendant subtrees synchronously with respect to the direct task pid and asynchronously for the descendant fan-out. AProcess.demonitor(task.ref, [:flush])follows to prevent stale:DOWNmessages accumulating in the runner's mailbox across timeout sites. The.13:code.purge/1post-timeout settle pass is retained on top. Permutagen.mutation_pipeline.r4/mutagen.mutation_pipeline.r14.CoverageRunner.run/1's:cover_already_runningrejection message now namesMutagenEx.TaskSupas the documented singleton owner of:cover_serverandExUnit.Serverduring a MutagenEx mutation cycle, and points readers at.spec/decisions/supervision_tree.md. Permutagen.coverage.r1/mutagen.coverage.r8.The C1/C2 integration spikes under
test/mutagen_ex/integration/are now tagged:spikeand excluded from the defaultmix testrun.test/test_helper.exscallsExUnit.start(exclude: [:e2e_slow, :spike]). The spikes remain the gating decision artifact for the in-process pipeline (permutagen.decision.in_process_pipeline) and are runnable on demand viamix test --only spike. Defaultmix testwall-clock drops from ~24s to ~1s; the smoke gate is no longer dominated by ~500 cover lifecycles every run.The C2 spike's iteration count is now controlled by the
MUTAGEN_SPIKE_ITERATIONSenv var (default10, original gating value100). The invariants —failures == 0, process growth ≤ 50, memory growth ≤ 1.5× — hold at any positive N; the count is a cycle-time knob, not a contract. C1's 100-iteration count is unchanged because its loop count IS the contract (restore fidelity measured across cycles).README.mddocuments the:spikeopt-in and theMUTAGEN_SPIKE_ITERATIONSknob under a new "Test suite gates" subsection.
Notes
- User-visible cancellation latency per timeout-classified site is
dominated by
cancel_grace_ms(100 ms by default). The supervisor's:shutdown_timeout(5_000 ms) bounds only a hypothetical scenario in which the per-site task itself traps:shutdown; in practice the task dies on the signal in microseconds and grandchild teardown is asynchronous afterwards. - Closes F2-arch for the common grandchild classes (untrapped linked
processes, OTP-supervised subtrees). Hand-rolled trap-exit'd
GenServers that swallow
{:EXIT, _, :shutdown}info messages remain out of reach; ther7snapshot-delta machinery continues to detect these as advisory growth signals.
Removed
lib/mutagen_ex.exandtest/mutagen_ex_test.exs(mix newplaceholders) are deleted. TheMutagenExnamespace is owned by its submodules; no top-level module body is necessary.- The README's
.13caveat ("re-run if the tail of the run looks wrong" after a timeout-classified mutation) is removed. The.18supervised teardown +:code.purgesettle pass + recursive shutdown through the link tree close the underlying Code.Server hang condition.