Pre-mutation test pass: do the cited tests pass on unmodified code?
Contract: mutagen.mutation_pipeline
r1, r2.
Responsibilities (and what it deliberately is NOT)
- r1.
run/1runs the cited tests once with no mutation applied. If any test fails, return{:error, :baseline_red, %{failures: [{module, name}, ...], passed: int}}. The orchestrator then routes the failure into the universal error-JSON shape. - r2. Runs ExUnit serially with the configured seed and an
unbounded failure count. Detects
async: truetest modules in the cited filter and includes warnings in the report.
Baseline does NOT own:
- Per-mutation timeouts (no timeout wrapping; if the baseline hangs,
the user can Ctrl-C — per
mutagen.decision.mutation_loop_private). - Stdout/stderr suppression (baseline runs ExUnit's own formatters by
default; the orchestrator can
ExUnit.CaptureIO-wrap the call if it doesn't want the banner). - Cover instrumentation (that's
MutagenEx.CoverageRunner). - Loading test files (that's the orchestrator: see
MutagenEx.CoverageRunnerfor the precedent; baseline does it once because tests can run only once perrequire_fileperr10).
Why baseline re-registers cited test modules
ExUnit.Server consumes its registered-module list at every
ExUnit.run/0 invocation. The use ExUnit.Case
__after_compile__ hook that performs the initial registration only
fires when the cited test file is freshly evaluated, and
Code.require_file/1 is one-shot per path in a BEAM lifetime.
When the coverage phase runs first it loads the cited test files,
fires the hook, and consumes the registered modules during its own
ExUnit.run/0. By the time baseline runs, the registry is empty and
Code.require_file/1 on the same paths is a cached no-op — without
explicit re-registration, baseline's ExUnit.run/0 would see zero
modules and return %{total: 0, failures: 0}, silently classifying
every red baseline as green (mutagen-wrd.37).
The fix mirrors what MutagenEx.MutationRunner.MutationLoop already
does per site: explicitly call ExUnit.Server.add_module/2 for each
cited test module before invoking ExUnit.run/0. After loading the
cited files, baseline discovers cases through
MutagenEx.TestModuleDiscovery.discover/1 unless the caller provided
:test_modules. The :ex_unit_server seam defaults to
MutagenEx.Test.ExUnitServer and lets tests record the calls.
Input shape
A map with:
:seed— non-negative integer.:test_filter—%MutagenEx.TestSelector.TestFilter{}.
Optional:
:ex_unit— module implementingMutagenEx.Test.ExUnitFacade(test seam). Defaults toMutagenEx.Test.ExUnit, which delegates to the realExUnit. Tests pass their own module with the same callback surface.:test_loader—(path -> any)overridingCode.require_file/1.:test_modules—[{module(), MutagenEx.TestModuleDiscovery.module_cfg()}]. Re-registered withExUnit.Server.add_module/2beforeExUnit.run/0so the cited modules appear in the registry that the prior phase'sExUnit.run/0drained. Defaults to[](preserves pre-mutagen-wrd.37 behaviour for callers that haven't wired the production payload through; in that mode baseline is vulnerable to theCode.require_file/1cache hazard whenever it runs after anotherExUnit.run/0in the same BEAM).:ex_unit_server— module implementingMutagenEx.Test.ExUnitServerFacade(the registration seam). Defaults toMutagenEx.Test.ExUnitServer.:ast_cache—MutagenEx.AstCache.t(). Optional. When present,detect_async_modules/1(the r2 async-warning path) looks each cited test file up in the cache viaAstCache.get/2and consumes the cached{ast, _source}directly — no re-read from disk. On:error(cache miss), the implementation falls back to the pre-.25File.read/1path as a safety net and logs the miss. When:ast_cacheis absent, the fall-back path is the only path (preserves pre-.25behaviour for callers that haven't wired the cache through yet). Seemutagen.coverage.r9.
Summary
Functions
Run the baseline. Returns {:ok, %{passed:, failed: 0, failures: [], warnings: [...]}} on full success or {:error, :baseline_red, %{failures: [...], passed: n}} on any failing cited test.
Types
@type error_reason() ::
:baseline_red | :ex_unit_run_failed | :test_file_load_failed | :invalid_input
Reasons the baseline can return.
@type ok_result() :: %{ passed: non_neg_integer(), failed: non_neg_integer(), failures: [{module(), String.t() | atom()}], warnings: [String.t()] }
Successful baseline result.
Functions
@spec run(map()) :: {:ok, ok_result()} | {:error, error_reason(), map()}
Run the baseline. Returns {:ok, %{passed:, failed: 0, failures: [], warnings: [...]}} on full success or {:error, :baseline_red, %{failures: [...], passed: n}} on any failing cited test.