mod eb_parse

module eb_parse

Parse EasyBuild easyconfig (.eb) files into structured candidates.

Easyconfigs are a restricted Python DSL. This module evaluates that subset (assignments, lists/tuples/dicts, SYSTEM, local_* and other name refs) and resolves EasyBuild-style %(…)s templates derived from name / version / versionsuffix / toolchain — matching EasyBuild’s EasyConfigParser plus the core template set used for fixture goldens under fixtures/parser_hardcases/.

Functions

fn candidate_matches_dep(c: &Candidate, dep: &ResolvedDep) -> bool

Whether a universe candidate satisfies a resolved dep (name + version + optional versionsuffix + optional per-dep toolchain). Cross-toolchain dependencies crossing from a parent toolchain to a core toolchain are first-class here — unlike filter_toolchain which keeps only the policy toolchain.

When dep.toolchain is None, any toolchain with a matching name/version/suffix is accepted (legacy identity match). Prefer candidate_matches_dep_for_recipe so implicit deps follow EasyBuild’s minimal-toolchain search within the recipe generation.

fn candidate_matches_dep_for_recipe(c: &Candidate, dep: &ResolvedDep, hierarchy: &crate::hierarchy::ToolchainHierarchy) -> bool

Like candidate_matches_dep, but when the dep has no explicit toolchain pin, only non-SYSTEM candidates in hierarchy count for a non-SYSTEM recipe. EasyBuild may minimize an implicit dependency to a generation member such as GCCcore, but it only selects SYSTEM when the dependency tuple says so explicitly. Explicit fourth-tuple pins still match exactly, including explicitly selected cross-generation dependencies.

fn check_recipe_deps(recipe: &ResolvedEasyconfig, universe: &[Candidate]) -> RecipeDepCheck

Check that every runtime/build dep of recipe appears as a candidate in universe (any tree layer already merged). Does not run the SAT solver — this is the packaging/robot completeness gate used before eb.

Implicit deps must match a non-SYSTEM hierarchy member of the recipe toolchain (derived from the robot universe when possible), so neither an older-generation candidate nor a SYSTEM fallback can false-pass.

fn checksum_structure_findings(recipe: &ResolvedEasyconfig) -> Vec<String>

Structural findings for the checksums list or top-level dict.

Real EasyBuild (get_checksum_for + verify_checksum + Cargo lifecycle):

  • Cargo.``__init__`` appends each crates tarball onto cfg['sources'].

  • check_checksums_for then builds filenames via chain(sources, data_sources) then patches, so the list order is declared sources, crate tarballs, data_sources, patches.

  • A list of checksums is positional by index. A list entry may be a plain MD5/SHA256 string, a (type, value) tuple, alternatives, or a dict looked up by artifact basename at that index; one dict may map several names for the same slot (multi-arch), not several sources.

  • A top-level checksums dict is name-keyed and ignores index.

fn easyblock_sites(src: &str) -> Result<(ResolvedEasyconfig, EasyblockSites), ParseError>

The recipe plus the easyblock each extension and component resolves to.

fn easyblock_sites_for_arch(src: &str, arch: &str) -> Result<(ResolvedEasyconfig, EasyblockSites), ParseError>

As easyblock_sites, with an explicit ARCH value.

fn easyconfig_basename(name: &str, version: &str, tc: &Toolchain, versionsuffix: Option<&str>) -> String

Conventional basename: Name-{versionprefix}{version}-Toolchain-tcver.eb (+ versionsuffix). version is the bare recipe version; pass the candidate’s versionprefix separately so callers never fold the install stem into version.

fn easyconfig_basename_with_prefix(name: &str, versionprefix: &str, version: &str, tc: &Toolchain, versionsuffix: Option<&str>) -> String

Like easyconfig_basename, with an EasyBuild versionprefix.

Toolchain spelling matches EasyBuild 5.4.2 module/filename identity: only the exact name system omits the toolchain segment (dummy / System keep theirs).

fn easyconfig_letter_dir(name: &str) -> String

EasyBuild-style letter directory for name (ExamplePkg → e).

fn existing_versions(query: &ExistingVersionsQuery<'_>) -> Vec<String>

Versions of query.name already present at query.generation, sorted newest first by the same comparison the miner uses.

This is the producer for crate::miner::is_backfill’s existing_versions: without it a caller has to hand-build that list, which is why backfill detection had no production call site.

fn filter_toolchain(cands: &[Candidate], tc: &Toolchain) -> Vec<Candidate>

Keep only candidates built at exactly this toolchain.

The narrow form, for callers that mean one level and nothing under it. A solve wants filter_toolchain_hierarchy instead, since a recipe’s own build dependencies routinely sit one level down.

fn filter_toolchain_hierarchy(cands: &[Candidate], tc: &Toolchain, members: &[Toolchain]) -> Vec<Candidate>

Candidates built against exactly this toolchain. Keep the policy toolchain and everything below it in its hierarchy.

A stack is not built at one toolchain. A recipe at GCC-15.2.0 takes its CMake from GCCcore-15.2.0 and its licence file from SYSTEM, exactly as EasyBuild’s minimal-toolchain search does, so a universe filtered to the policy toolchain alone cannot satisfy the dependencies of its own members and reports them as missing packages.

members is the hierarchy for the policy toolchain, lowest first. Passing an empty slice reduces this to filter_toolchain.

fn lock_from_candidates(cands: &[Candidate], generation_label: Option<String>, engine: &str) -> StackLock

Build a lock from an already-selected candidate set.

The caller has done the choosing; this records it.

fn merge_candidates_with_precedence(layers: &[Vec<Candidate>]) -> Vec<Candidate>

Merge candidate layers with later-layer precedence: when two candidates share the same name + version + versionprefix + toolchain + versionsuffix, the later layer wins (overlay). Distinct installable variants remain separate candidates.

Used for site overlays on top of an upstream easyconfigs tree.

fn packaging_gate(recipe: &ResolvedEasyconfig, required_configopts: &[&str]) -> Result<(), Vec<String>>

Packaging gate: checksums present, moduleclass set, and optional required configopts substrings (e.g. -Dwith_tests=false).

A missing easyblock is not an error: EasyBuild derives the easyblock from the software name when the recipe omits it (OpenMPI -> EB_OpenMPI, GCC -> EB_GCC, …), which is how the majority of upstream recipes are written. Only recipes whose name does not map to a software-specific easyblock need to declare one, and the recipe author — not this gate — makes that call.

fn parse_easyconfig_file(path: &Path) -> Result<Candidate, ParseError>

Parse one .eb file into a solver-facing Candidate.

fn parse_easyconfig_file_for_arch(path: &Path, arch: &str) -> Result<Candidate, ParseError>

As parse_easyconfig_file, with an explicit ARCH value.

fn parse_easyconfig_tree(root: &Path) -> Result<ParseTreeResult, ParseError>

Walk a directory tree for *.eb and parse all easyconfigs.

Unparseable files are skipped (not fatal): they appear in ParseTreeResult::skipped so callers can report coverage without aborting a real multi-thousand-file tree on the first bad recipe.

Each real directory is read once, however many symlinks lead to it.

fn parse_easyconfig_tree_for_arch(root: &Path, arch: &str) -> Result<ParseTreeResult, ParseError>

As parse_easyconfig_tree, with an explicit ARCH value.

fn parse_easyconfig_trees(roots: &[&Path]) -> Result<ParseTreeResult, ParseError>

Parse multiple easyconfig trees and merge with later-path precedence. Skipped paths from every tree are retained.

fn parse_easyconfig_trees_for_arch(roots: &[&Path], arch: &str) -> Result<ParseTreeResult, ParseError>

As parse_easyconfig_trees, with an explicit ARCH value.

fn pr_title(ecs: &[ResolvedEasyconfig]) -> String

{class}[{toolchain}] Name vX, then w/ surrounded by spaces and the resolved versionsuffix.

fn resolve_easyconfig_file(path: &Path) -> Result<ResolvedEasyconfig, ParseError>

Resolve one .eb file to fully expanded fields.

fn resolve_easyconfig_file_for_arch(path: &Path, arch: &str) -> Result<ResolvedEasyconfig, ParseError>

As resolve_easyconfig_file, with an explicit ARCH value.

fn resolve_easyconfig_file_reporting(path: &Path) -> Result<(ResolvedEasyconfig, Vec<SkippedStatement>), ParseError>

As resolve_easyconfig_file, also returning the skipped statements.

fn resolve_easyconfig_file_reporting_for_arch(path: &Path, arch: &str) -> Result<(ResolvedEasyconfig, Vec<SkippedStatement>), ParseError>

As resolve_easyconfig_file_reporting, with an explicit ARCH value.

fn resolve_easyconfig_str(src: &str) -> Result<ResolvedEasyconfig, ParseError>

Resolve easyconfig source text to fully expanded fields (no filesystem path).

Tolerant: a statement the parser cannot model is skipped so the rest of the recipe still resolves. Use resolve_easyconfig_str_reporting when you need to know what was skipped.

fn resolve_easyconfig_str_for_arch(src: &str, arch: &str) -> Result<ResolvedEasyconfig, ParseError>

As resolve_easyconfig_str, with an explicit ARCH value.

fn resolve_easyconfig_str_reporting(src: &str) -> Result<(ResolvedEasyconfig, Vec<SkippedStatement>), ParseError>

Resolve easyconfig text, also returning the statements the parser skipped.

resolve_easyconfig_str is this without the second value; the parse is identical, so asking for the report costs nothing and changes nothing.

fn resolve_easyconfig_str_reporting_for_arch(src: &str, arch: &str) -> Result<(ResolvedEasyconfig, Vec<SkippedStatement>), ParseError>

As resolve_easyconfig_str_reporting, with an explicit ARCH value.

fn resolve_local_files(src: &str) -> Result<(ResolvedEasyconfig, Vec<LocalFile>), ParseError>

The recipe and every file it expects EasyBuild to find rather than download: patches of the recipe, its extensions and its components, sources no URL fetches, and patch files named only in checksums.

An extension’s or component’s options are templated with that entry’s own name and version, which is what EasyBuild does, so '%(name)s-%(version)s_fix.patch' inside an exts_list entry names the extension’s file and not the parent’s.

fn validate_lock_deps(lock: &StackLock, cands: &[Candidate]) -> Result<(), String>

Check that every dependency of every locked package is itself locked.

A lock that omits a transitive dependency installs a stack that cannot build, so this is worth checking before one is trusted.

fn version_field_to_req(version: &str) -> String

Map EasyBuild dependency version field to a solver requirement string.

Enums

enum LocalFileKind

Which field named a LocalFile.

Patch

A patches or postinstallpatches entry.

Source

A sources entry with no download URL and no git_config.

ChecksumPatch

A patch file named only as a checksums key, the way Python’s patch_ctypes_ld_library_path is.

enum LocalFileOwner

Whose options name a LocalFile.

Recipe

The easyconfig’s own patches, postinstallpatches, sources, or checksums.

Extension

An exts_list entry.

name: String

Extension name.

version: String

Extension version.

Component

A components entry of a Bundle.

name: String

Component name.

version: String

Component version.

enum ParseError

Why an easyconfig could not be read.

Io(String, std::io::Error)

The file could not be read.

The operating-system error is part of the message. It is not also a source, or {:#} prints that text a second time.

Parse(String, String)

The file could not be understood. Carries the path and the reason, including a line number where the failure was positional.

Structs and Unions

struct EasyblockSites

Easyblock choices a recipe makes beyond the top-level easyblock line.

Extensions and bundle components carry their own class, or inherit one from exts_defaultclass, exts_classmap, or default_easyblock. The top-level class stays on ResolvedEasyconfig::easyblock.

exts_defaultclass: Option<String>

exts_defaultclass, when the easyconfig sets one.

default_easyblock: Option<String>

default_easyblock for components that name no class of their own.

exts_classmap: BTreeMap<String, String>

exts_classmap: extension name to easyblock class.

extensions: Vec<NamedEasyblock>

One row per exts_list entry that parsed.

components: Vec<NamedEasyblock>

One row per components entry that parsed.

struct ExistingVersionsQuery<'a>

Which versions of one package a universe already carries at one toolchain generation.

Struct-shaped like the other request types in this crate so the three fields cannot be transposed at a call site.

universe: &'a [Candidate]

Universe from parse_easyconfig_tree or parse_easyconfig_trees, live checkout or historical merge-base tree alike.

name: &'a str

Package name to match, exactly.

generation: &'a Toolchain

Toolchain generation to match, name and version.

struct LocalFile

A file EasyBuild has to find on disk because nothing downloads it.

filename: String

File name after the owner’s templates are applied.

kind: LocalFileKind

The field that named it.

owner: LocalFileOwner

The recipe, extension, or component whose options name it.

alt_location: Option<String>

alt_location of a patch dict: EasyBuild searches under this name instead of the software name, and skips the easyconfig’s directory.

struct MissingDep

One missing (or unmatched) dependency from a packaging/robot check.

name: String

Dependency that could not be matched.

version: String

Version the recipe asked for.

versionsuffix: Option<String>

Versionsuffix required, when one was.

toolchain: Option<Toolchain>

Toolchain required, when the entry pinned one.

role: String

Runtime vs build-time role in the recipe.

reason: String

Why nothing matched: absent entirely, or present at the wrong version or toolchain.

struct NamedEasyblock

An extension or bundle component, and the easyblock it names itself.

name: String

Extension or component name.

class_name: Option<String>

easyblock in the entry’s dict. Absent when the entry names none.

struct ParseTreeResult

Result of walking an easyconfig tree: successes + skipped unparseable files.

candidates: Vec<Candidate>

Everything that parsed.

skipped: Vec<SkippedEasyconfig>

Everything that did not, kept so a tree is never silently partial.

Implementations

impl ParseTreeResult

Functions

fn coverage(&self) -> f64

Coverage fraction parsed / (parsed + skipped); 1.0 when the tree is empty.

fn merge_with_precedence(mut self, other: ParseTreeResult) -> ParseTreeResult

Merge another result (later candidates override on identity; skips append).

fn parsed_count(&self) -> usize

How many parsed successfully.

fn skip_count(&self) -> usize

How many easyconfigs failed to parse.

struct RecipeDepCheck

Result of checking that a recipe’s deps exist somewhere in a robot universe.

recipe: String

Easyconfig that was checked.

name: String

Package it declares.

version: String

Version it declares.

toolchain: Toolchain

Toolchain it targets.

easyblock: Option<String>

Easyblock it uses, when it sets one.

configopts: Option<String>

Configure flags, after template expansion.

moduleclass: Option<String>

Moduleclass, when it sets one.

checksum_count: usize

How many checksum entries it carries.

missing: Vec<MissingDep>

Dependencies the robot tree does not provide.

found: Vec<String>

Dependencies that were matched.

unverified_toolchain: Option<String>

Set when the recipe’s toolchain generation could not be resolved, so matching ran without a hierarchy to filter by.

Every entry in found is then a name-and-version match that ignores toolchain, which can pair a recipe with a dependency from an unrelated generation (the EB_MAINT_CROSS_GEN shape). missing staying empty is not evidence the recipe resolves in EasyBuild; callers must say so rather than report a clean result.

Implementations

impl RecipeDepCheck

Functions

fn ok(&self) -> bool

Whether every dependency was found.

fn toolchain_verified(&self) -> bool

Whether the matches carry toolchain evidence, i.e. a hierarchy was known.

Distinct from Self::ok on purpose: ok answers “was anything left unmatched”, this answers “is a match worth anything”.

struct ResolvedDep

One resolved dependency entry (2–4 element EasyBuild dependency tuple).

name: String

Dependency name.

version: String

Raw version field after template/local resolution (may be 1.2.3 or >=1.2).

versionsuffix: Option<String>

Versionsuffix required, when the entry names one.

toolchain: Option<Toolchain>

Per-dependency toolchain override (None = inherit the easyconfig toolchain).

struct ResolvedEasyconfig

Fully resolved easyconfig fields (templates and locals applied).

Solver-facing co-selection uses Self::to_candidate. Packaging / contribution checks also use the optional metadata fields below (easyblock, configopts, moduleclass, …).

name: String

Package name.

version: String

Package version after template expansion.

versionsuffix: Option<String>

Versionsuffix, when the easyconfig sets one.

versionprefix: Option<String>

EasyBuild versionprefix after the common parse boundary (evaluated env value, then templates). None when unset or empty.

toolchain: Toolchain

Toolchain this recipe builds against.

dependencies: Vec<ResolvedDep>

Runtime dependencies.

builddependencies: Vec<ResolvedDep>

Build-time-only dependencies.

exts_list: Vec<ResolvedExt>

Bundled extensions.

easyconfig_path: String

Path of the source .eb when parsed from disk (empty for in-memory text).

easyblock: Option<String>

EasyBuild easyblock class name (MesonNinja, CMakeMake, …).

configopts: Option<String>

Meson/CMake/configure flags string after template expansion.

moduleclass: Option<String>

EasyBuild moduleclass (chem, lib, tools, …).

homepage: Option<String>

Homepage URL when set.

checksums: Vec<String>

Source checksums list (strings), when present — used for packaging gates.

sources_count: usize

Number of entries in sources (0 when the field is absent).

data_sources_count: usize

Number of entries in data_sources (0 when absent).

source_urls: Vec<String>

Expanded source_urls entries, used to classify which artifact the recipe actually downloads.

source_files: Vec<SourceFile>

Resolved sources entries as (filename, download_filename): the name EasyBuild saves the artifact under and the name it asks each source_urls entry for, which differ when the entry is a dict with a download_filename. A source_urls key inside the dict replaces the recipe-level list for that entry, as it does in EasyBuild.

crate_source_names: Vec<String>

Filenames Cargo easyblocks append onto sources from each crates tuple (name-version.tar.gz, or name-version-revprefix.tar.xz for git crates). Empty when crates is absent.

data_source_names: Vec<String>

Filenames from data_sources (string or dict filename/name).

patch_names: Vec<String>

Patch file names from patches (tuple/dict entries reduced to the name).

checksum_entry_keys: Vec<Vec<String>>

Per-checksums-LIST-ENTRY dict keys (empty inner vec for plain-string entries). Parallel to the checksums list entries, not the flattened values, so a multi-arch dict entry stays one entry. Empty when checksums is a top-level filename dict rather than a list.

checksums_by_filename: BTreeMap<String, String>

Checksums the recipe states by filename, for list entries written as {'name': 'sha…'} and for a top-level checksums dict.

checksums_top_level_dict: bool

checksums is a single filename→digest dict (EasyBuild looks up by name and ignores list index).

has_components: bool

The recipe sets components: it bundles other packages, whatever its easyblock is called.

component_checksums_complete: bool

Every parsed component source, patch, and postinstall patch carries a checksum.

Implementations

impl ResolvedEasyconfig

Functions

fn to_candidate(&self) -> Candidate

Map into the solver-facing Candidate / DepReq shapes.

struct ResolvedExt

One exts_list entry after resolution.

name: String

Extension name.

version: String

Extension version.

patches: Vec<String>

Patch files the entry’s options name ('patches': [...]). They sit beside the easyconfig like its own patches and travel with it.

struct SkippedEasyconfig

One easyconfig path that could not be parsed into a candidate.

path: String

Easyconfig that was skipped.

error: String

Why it could not be parsed.

struct SkippedStatement

Resolve easyconfig source text to fully expanded fields (no filesystem path). One statement the tolerant parser could not model, and why.

The parser skips an unrepresentable statement so a single oddity does not cost the whole easyconfig. That is the right default, but it means a recipe can parse “successfully” while a line the author cared about was dropped. These records are what --strict reports.

line: usize

1-based line the statement starts on.

message: String

Why it could not be parsed, already carrying its own line prefix.

text: String

The statement text as written, trimmed.

Implementations

impl SkippedStatement

Functions

fn reason(&self) -> &str

The reason with the parser’s own line N: position prefix stripped, so a caller that prints its own position does not repeat it.

Traits implemented

impl std::fmt::Display for SkippedStatement
struct SourceFile

One sources entry, resolved against the recipe’s templates.

filename: String

Name the artifact is saved under, which checksums is keyed by.

download_filename: String

Name requested from each source URL; equals filename unless the entry sets download_filename.

source_urls: Vec<String>

Per-entry source_urls, when the dict entry carries its own.

git: bool

The entry fetches a git checkout (git_config), so no download URL names it and no tarball digest can be computed ahead of the build.