mod repro_report

module repro_report

Machine-readable scores for the reproduction grind, and the ratchet that keeps them moving one way.

The reproduction harness compares an emitted bump against the file a maintainer merged, byte for byte, except for an allowance: the lines a maintainer added by hand that no mechanical bump can invent. The length of that allowance is the miss count for the case, so it is the number the grind reports and the number that has to shrink as the tool improves.

Two things follow. The allowance count leaves the test binary as a JSON artifact the scoreboard reads instead of a human transcribing it, and a checked-in ratchet file pins every count, so an allowance that grows fails the suite and an allowance that shrinks fails until the smaller number is recorded.

Scoring itself is not repeated here: the ladder and its normalization live in crate::miner, and this module applies the allowance and then asks that code for the rung.

Functions

fn check_case_against_ratchet(ratchet: &ReproRatchet, score: &ReproCaseScore) -> Vec<RatchetViolation>

Check one scored case against the committed ratchet.

fn check_ratchet(ratchet: &ReproRatchet, scores: &[ReproCaseScore]) -> Vec<RatchetViolation>

Check a whole collected run: every case, plus the ratchet’s own total.

fn read_case_scores(dir: &Path) -> Result<Vec<ReproCaseScore>, ReproReportError>

Read every case score in dir, ordered by case name.

fn render_scoreboard_table(scores: &[ReproCaseScore]) -> String

Render collected scores as the scoreboard’s own table shape.

The grind pastes this under a scored entry rather than transcribing numbers out of test output by hand.

fn score_with_allowance(emitted: &str, target: &str, allowance: &[&str]) -> ScoredReproduction

Apply allowance to target and score emitted against the rest.

Each allowance line is removed from the target once, at its first occurrence, which is the rule the harness has always used. An entry that is missing from the target, or that the emit now carries itself, comes back as stale rather than silently forgiving nothing.

fn write_case_score(dir: &Path, score: &ReproCaseScore) -> Result<PathBuf, ReproReportError>

Write one case score into dir as <case>.json, creating dir.

One file per case rather than one shared file: the harness writes these from tests that run in parallel, and a per-case path needs no lock and no merge.

Enums

enum RatchetViolation

A way the live suite and the committed ratchet disagree.

Grew

The case forgives more lines than the ratchet records.

case: String

Case name.

declared: usize

Allowance the case declared.

recorded: usize

Count the ratchet records.

Improved

The case forgives fewer lines than the ratchet records, which is progress that has to be written down before the suite goes green again – otherwise the recorded number drifts upward of reality and stops catching the next regression.

case: String

Case name.

declared: usize

Allowance the case declared.

recorded: usize

Count the ratchet records.

Unrecorded

The case is not in the ratchet at all.

case: String

Case name.

declared: usize

Allowance the case declared.

Stale

An allowance entry that forgives nothing.

case: String

Case name.

detail: String

What is stale and what to do about it.

TotalMismatch

total disagrees with the sum of the per-case counts.

recorded: usize

The total field as committed.

summed: usize

The sum of the per-case counts.

Traits implemented

impl std::fmt::Display for RatchetViolation

enum ReproReportError

Reading or writing a score artifact or ratchet file failed.

Io

The artifact directory or file could not be read or written.

path: PathBuf

The path being read or written.

source: std::io::Error

The underlying filesystem error.

Json

A score artifact or ratchet file did not parse as its JSON shape.

path: PathBuf

The file that failed to parse.

source: serde_json::Error

The underlying parse error.

enum StaleAllowance

An allowance entry that no longer earns its place.

Both shapes mean the same thing for the ratchet: the count for this case can come down, and the entry should be deleted rather than left to hide a future regression behind an unused exception.

AbsentFromTarget

The line is not in the merged target at all, so the allowance describes a maintainer addition that is no longer there.

line: String

The allowance entry, as declared.

AlreadyEmitted

The bump emits the line itself now, so nothing needs forgiving.

line: String

The allowance entry, as declared.

Implementations

impl StaleAllowance

Functions

fn explain(&self) -> String

What to do about it, in one sentence.

fn line(&self) -> &str

The allowance entry this refers to.

Structs and Unions

struct ReproCaseScore

One reproduction case, scored.

case: String

Case name. Stable across runs, and the key the ratchet is keyed by.

pull_request: Option<u32>

The upstream pull request, when the case was mined from one. None for the frozen fixture pairs, whose PR numbers are not recorded in the repository.

source: String

Source recipe the bump started from.

target: String

The merged file the emit is scored against.

score: ReproScore

The rung, decided after the allowance is applied.

allowance: Vec<String>

Lines the harness forgives, as declared by the case.

stale_allowance: Vec<StaleAllowance>

Allowance entries that no longer earn their place.

residual_lines: usize

How many raw diff lines remain once the allowance is applied. Zero for EXACT; nonzero for SEMANTIC says how much comment and whitespace difference the score forgave.

Implementations

impl ReproCaseScore

Functions

fn miss_count(&self) -> usize

The miss count for the scoreboard: how many hand-added lines this case still cannot reproduce.

struct ReproRatchet

The committed allowance counts: how many hand-added lines each case is still permitted to miss.

This file only ever moves down. Its diff is the record of every allowance change the grind has made, which is why the counts live here rather than only in test source.

note: String

Why the file exists and which way it moves, for whoever opens it first.

cases: BTreeMap<String, usize>

Allowance count per case name.

total: usize

Sum over cases, so a single number tracks the whole grind and a hand edit that forgets to update it is caught.

Implementations

impl ReproRatchet

Functions

fn allowance_for(&self, case: &str) -> Option<usize>

The recorded count for case, if it has one.

fn from_json(text: &str, path: &Path) -> Result<Self, ReproReportError>

Parse a ratchet file.

fn read(path: &Path) -> Result<Self, ReproReportError>

Read a ratchet file from disk.

fn total_violation(&self) -> Option<RatchetViolation>

total against the sum of the per-case counts.

struct ScoredReproduction

A scored comparison before it becomes an artifact.

score: ReproScore

The rung, from crate::miner::score_reproduction, comparing the emitted text against the target with the allowance removed.

stale: Vec<StaleAllowance>

Allowance entries that no longer earn their place.

residual: Vec<String>

The raw diff that remains, one -/+ line per difference.

target_without_allowance: String

The target with each allowance line removed once, in order: the text the emit is actually compared against.