mod eb_parse¶
- module eb_parse¶
Parse EasyBuild easyconfig (
.eb) files into structured candidates.Easyconfigs are a restricted Python DSL. This module evaluates that subset (assignments, lists/tuples/dicts,
SYSTEM,local_*and other name refs) and resolves EasyBuild-style%(…)stemplates derived from name / version / versionsuffix / toolchain — matching EasyBuild’sEasyConfigParserplus the core template set used for fixture goldens underfixtures/parser_hardcases/.Functions
- fn candidate_matches_dep(c: &Candidate, dep: &ResolvedDep) -> bool¶
Whether a universe candidate satisfies a resolved dep (name + version + optional versionsuffix + optional per-dep toolchain). Cross-toolchain dependencies crossing from a parent toolchain to a core toolchain are first-class here — unlike
filter_toolchainwhich keeps only the policy toolchain.When
dep.toolchainis None, any toolchain with a matching name/version/suffix is accepted (legacy identity match). Prefercandidate_matches_dep_for_recipeso implicit deps follow EasyBuild’s minimal-toolchain search within the recipe generation.
- fn candidate_matches_dep_for_recipe(c: &Candidate, dep: &ResolvedDep, hierarchy: &crate::hierarchy::ToolchainHierarchy) -> bool¶
Like
candidate_matches_dep, but when the dep has no explicit toolchain pin, only non-SYSTEM candidates inhierarchycount for a non-SYSTEM recipe. EasyBuild may minimize an implicit dependency to a generation member such as GCCcore, but it only selects SYSTEM when the dependency tuple says so explicitly. Explicit fourth-tuple pins still match exactly, including explicitly selected cross-generation dependencies.
- fn check_recipe_deps(recipe: &ResolvedEasyconfig, universe: &[Candidate]) -> RecipeDepCheck¶
Check that every runtime/build dep of
recipeappears as a candidate inuniverse(any tree layer already merged). Does not run the SAT solver — this is the packaging/robot completeness gate used beforeeb.Implicit deps must match a non-SYSTEM hierarchy member of the recipe toolchain (derived from the robot universe when possible), so neither an older-generation candidate nor a SYSTEM fallback can false-pass.
- fn checksum_structure_findings(recipe: &ResolvedEasyconfig) -> Vec<String>¶
Structural findings for the
checksumslist or top-level dict.Real EasyBuild (
get_checksum_for+verify_checksum+ Cargo lifecycle):Cargo.``__init__`` appends each
cratestarball ontocfg['sources'].check_checksums_forthen builds filenames viachain(sources, data_sources)then patches, so the list order is declared sources, crate tarballs, data_sources, patches.A list of checksums is positional by index. A list entry may be a plain MD5/SHA256 string, a
(type, value)tuple, alternatives, or a dict looked up by artifact basename at that index; one dict may map several names for the same slot (multi-arch), not several sources.A top-level checksums dict is name-keyed and ignores index.
- fn easyblock_sites(src: &str) -> Result<(ResolvedEasyconfig, EasyblockSites), ParseError>¶
The recipe plus the easyblock each extension and component resolves to.
- fn easyblock_sites_for_arch(src: &str, arch: &str) -> Result<(ResolvedEasyconfig, EasyblockSites), ParseError>¶
As
easyblock_sites, with an explicit ARCH value.
- fn easyconfig_basename(name: &str, version: &str, tc: &Toolchain, versionsuffix: Option<&str>) -> String¶
Conventional basename:
Name-{versionprefix}{version}-Toolchain-tcver.eb(+ versionsuffix).versionis the bare recipe version; pass the candidate’sversionprefixseparately so callers never fold the install stem intoversion.
- fn easyconfig_basename_with_prefix(name: &str, versionprefix: &str, version: &str, tc: &Toolchain, versionsuffix: Option<&str>) -> String¶
Like
easyconfig_basename, with an EasyBuildversionprefix.Toolchain spelling matches EasyBuild 5.4.2 module/filename identity: only the exact name
systemomits the toolchain segment (dummy/Systemkeep theirs).
- fn easyconfig_letter_dir(name: &str) -> String¶
EasyBuild-style letter directory for
name(ExamplePkg→e).
- fn existing_versions(query: &ExistingVersionsQuery<'_>) -> Vec<String>¶
Versions of
query.namealready present atquery.generation, sorted newest first by the same comparison the miner uses.This is the producer for
crate::miner::is_backfill’sexisting_versions: without it a caller has to hand-build that list, which is why backfill detection had no production call site.
- fn filter_toolchain(cands: &[Candidate], tc: &Toolchain) -> Vec<Candidate>¶
Keep only candidates built at exactly this toolchain.
The narrow form, for callers that mean one level and nothing under it. A solve wants
filter_toolchain_hierarchyinstead, since a recipe’s own build dependencies routinely sit one level down.
- fn filter_toolchain_hierarchy(cands: &[Candidate], tc: &Toolchain, members: &[Toolchain]) -> Vec<Candidate>¶
Candidates built against exactly this toolchain. Keep the policy toolchain and everything below it in its hierarchy.
A stack is not built at one toolchain. A recipe at
GCC-15.2.0takes its CMake fromGCCcore-15.2.0and its licence file from SYSTEM, exactly as EasyBuild’s minimal-toolchain search does, so a universe filtered to the policy toolchain alone cannot satisfy the dependencies of its own members and reports them as missing packages.membersis the hierarchy for the policy toolchain, lowest first. Passing an empty slice reduces this tofilter_toolchain.
- fn lock_from_candidates(cands: &[Candidate], generation_label: Option<String>, engine: &str) -> StackLock¶
Build a lock from an already-selected candidate set.
The caller has done the choosing; this records it.
- fn merge_candidates_with_precedence(layers: &[Vec<Candidate>]) -> Vec<Candidate>¶
Merge candidate layers with later-layer precedence: when two candidates share the same name + version + versionprefix + toolchain + versionsuffix, the later layer wins (overlay). Distinct installable variants remain separate candidates.
Used for site overlays on top of an upstream easyconfigs tree.
- fn packaging_gate(recipe: &ResolvedEasyconfig, required_configopts: &[&str]) -> Result<(), Vec<String>>¶
Packaging gate: checksums present, moduleclass set, and optional required configopts substrings (e.g.
-Dwith_tests=false).A missing
easyblockis not an error: EasyBuild derives the easyblock from the software name when the recipe omits it (OpenMPI->EB_OpenMPI,GCC->EB_GCC, …), which is how the majority of upstream recipes are written. Only recipes whose name does not map to a software-specific easyblock need to declare one, and the recipe author — not this gate — makes that call.
- fn parse_easyconfig_file(path: &Path) -> Result<Candidate, ParseError>¶
Parse one
.ebfile into a solver-facingCandidate.
- fn parse_easyconfig_file_for_arch(path: &Path, arch: &str) -> Result<Candidate, ParseError>¶
As
parse_easyconfig_file, with an explicit ARCH value.
- fn parse_easyconfig_tree(root: &Path) -> Result<ParseTreeResult, ParseError>¶
Walk a directory tree for
*.eband parse all easyconfigs.Unparseable files are skipped (not fatal): they appear in
ParseTreeResult::skippedso callers can report coverage without aborting a real multi-thousand-file tree on the first bad recipe.Each real directory is read once, however many symlinks lead to it.
- fn parse_easyconfig_tree_for_arch(root: &Path, arch: &str) -> Result<ParseTreeResult, ParseError>¶
As
parse_easyconfig_tree, with an explicit ARCH value.
- fn parse_easyconfig_trees(roots: &[&Path]) -> Result<ParseTreeResult, ParseError>¶
Parse multiple easyconfig trees and merge with later-path precedence. Skipped paths from every tree are retained.
- fn parse_easyconfig_trees_for_arch(roots: &[&Path], arch: &str) -> Result<ParseTreeResult, ParseError>¶
As
parse_easyconfig_trees, with an explicit ARCH value.
- fn pr_title(ecs: &[ResolvedEasyconfig]) -> String¶
{class}[{toolchain}] Name vX, thenw/surrounded by spaces and the resolved versionsuffix.
- fn resolve_easyconfig_file(path: &Path) -> Result<ResolvedEasyconfig, ParseError>¶
Resolve one
.ebfile to fully expanded fields.
- fn resolve_easyconfig_file_for_arch(path: &Path, arch: &str) -> Result<ResolvedEasyconfig, ParseError>¶
As
resolve_easyconfig_file, with an explicit ARCH value.
- fn resolve_easyconfig_file_reporting(path: &Path) -> Result<(ResolvedEasyconfig, Vec<SkippedStatement>), ParseError>¶
As
resolve_easyconfig_file, also returning the skipped statements.
- fn resolve_easyconfig_file_reporting_for_arch(path: &Path, arch: &str) -> Result<(ResolvedEasyconfig, Vec<SkippedStatement>), ParseError>¶
As
resolve_easyconfig_file_reporting, with an explicit ARCH value.
- fn resolve_easyconfig_str(src: &str) -> Result<ResolvedEasyconfig, ParseError>¶
Resolve easyconfig source text to fully expanded fields (no filesystem path).
Tolerant: a statement the parser cannot model is skipped so the rest of the recipe still resolves. Use
resolve_easyconfig_str_reportingwhen you need to know what was skipped.
- fn resolve_easyconfig_str_for_arch(src: &str, arch: &str) -> Result<ResolvedEasyconfig, ParseError>¶
As
resolve_easyconfig_str, with an explicit ARCH value.
- fn resolve_easyconfig_str_reporting(src: &str) -> Result<(ResolvedEasyconfig, Vec<SkippedStatement>), ParseError>¶
Resolve easyconfig text, also returning the statements the parser skipped.
resolve_easyconfig_stris this without the second value; the parse is identical, so asking for the report costs nothing and changes nothing.
- fn resolve_easyconfig_str_reporting_for_arch(src: &str, arch: &str) -> Result<(ResolvedEasyconfig, Vec<SkippedStatement>), ParseError>¶
As
resolve_easyconfig_str_reporting, with an explicit ARCH value.
- fn resolve_local_files(src: &str) -> Result<(ResolvedEasyconfig, Vec<LocalFile>), ParseError>¶
The recipe and every file it expects EasyBuild to find rather than download: patches of the recipe, its extensions and its components, sources no URL fetches, and patch files named only in
checksums.An extension’s or component’s options are templated with that entry’s own name and version, which is what EasyBuild does, so
'%(name)s-%(version)s_fix.patch'inside anexts_listentry names the extension’s file and not the parent’s.
- fn validate_lock_deps(lock: &StackLock, cands: &[Candidate]) -> Result<(), String>¶
Check that every dependency of every locked package is itself locked.
A lock that omits a transitive dependency installs a stack that cannot build, so this is worth checking before one is trusted.
- fn version_field_to_req(version: &str) -> String¶
Map EasyBuild dependency version field to a solver requirement string.
Enums
- enum LocalFileKind¶
Which field named a
LocalFile.- Patch¶
A
patchesorpostinstallpatchesentry.
- Source¶
A
sourcesentry with no download URL and nogit_config.
- ChecksumPatch¶
A patch file named only as a
checksumskey, the way Python’spatch_ctypes_ld_library_pathis.
- enum LocalFileOwner¶
Whose options name a
LocalFile.- Recipe¶
The easyconfig’s own
patches,postinstallpatches,sources, orchecksums.
- enum ParseError¶
Why an easyconfig could not be read.
- Io(String, std::io::Error)¶
The file could not be read.
The operating-system error is part of the message. It is not also a source, or
{:#}prints that text a second time.
- Parse(String, String)¶
The file could not be understood. Carries the path and the reason, including a line number where the failure was positional.
Structs and Unions
- struct EasyblockSites¶
Easyblock choices a recipe makes beyond the top-level
easyblockline.Extensions and bundle components carry their own class, or inherit one from
exts_defaultclass,exts_classmap, ordefault_easyblock. The top-level class stays onResolvedEasyconfig::easyblock.- exts_defaultclass: Option<String>¶
exts_defaultclass, when the easyconfig sets one.
- default_easyblock: Option<String>¶
default_easyblockfor components that name no class of their own.
- exts_classmap: BTreeMap<String, String>¶
exts_classmap: extension name to easyblock class.
- extensions: Vec<NamedEasyblock>¶
One row per
exts_listentry that parsed.
- components: Vec<NamedEasyblock>¶
One row per
componentsentry that parsed.
- struct ExistingVersionsQuery<'a>¶
Which versions of one package a universe already carries at one toolchain generation.
Struct-shaped like the other request types in this crate so the three fields cannot be transposed at a call site.
- universe: &'a [Candidate]¶
Universe from
parse_easyconfig_treeorparse_easyconfig_trees, live checkout or historical merge-base tree alike.
- name: &'a str¶
Package name to match, exactly.
- struct LocalFile¶
A file EasyBuild has to find on disk because nothing downloads it.
- filename: String¶
File name after the owner’s templates are applied.
- kind: LocalFileKind¶
The field that named it.
- owner: LocalFileOwner¶
The recipe, extension, or component whose options name it.
- alt_location: Option<String>¶
alt_locationof a patch dict: EasyBuild searches under this name instead of the software name, and skips the easyconfig’s directory.
- struct MissingDep¶
One missing (or unmatched) dependency from a packaging/robot check.
- name: String¶
Dependency that could not be matched.
- version: String¶
Version the recipe asked for.
- versionsuffix: Option<String>¶
Versionsuffix required, when one was.
- role: String¶
Runtime vs build-time role in the recipe.
- reason: String¶
Why nothing matched: absent entirely, or present at the wrong version or toolchain.
- struct NamedEasyblock¶
An extension or bundle component, and the easyblock it names itself.
- name: String¶
Extension or component name.
- class_name: Option<String>¶
easyblockin the entry’s dict. Absent when the entry names none.
- struct ParseTreeResult¶
Result of walking an easyconfig tree: successes + skipped unparseable files.
- skipped: Vec<SkippedEasyconfig>¶
Everything that did not, kept so a tree is never silently partial.
Implementations
- impl ParseTreeResult¶
Functions
- fn coverage(&self) -> f64¶
Coverage fraction
parsed / (parsed + skipped); 1.0 when the tree is empty.
- fn merge_with_precedence(mut self, other: ParseTreeResult) -> ParseTreeResult¶
Merge another result (later candidates override on identity; skips append).
- fn parsed_count(&self) -> usize¶
How many parsed successfully.
- fn skip_count(&self) -> usize¶
How many easyconfigs failed to parse.
- struct RecipeDepCheck¶
Result of checking that a recipe’s deps exist somewhere in a robot universe.
- recipe: String¶
Easyconfig that was checked.
- name: String¶
Package it declares.
- version: String¶
Version it declares.
- easyblock: Option<String>¶
Easyblock it uses, when it sets one.
- configopts: Option<String>¶
Configure flags, after template expansion.
- moduleclass: Option<String>¶
Moduleclass, when it sets one.
- checksum_count: usize¶
How many checksum entries it carries.
- missing: Vec<MissingDep>¶
Dependencies the robot tree does not provide.
- found: Vec<String>¶
Dependencies that were matched.
- unverified_toolchain: Option<String>¶
Set when the recipe’s toolchain generation could not be resolved, so matching ran without a hierarchy to filter by.
Every entry in
foundis then a name-and-version match that ignores toolchain, which can pair a recipe with a dependency from an unrelated generation (theEB_MAINT_CROSS_GENshape).missingstaying empty is not evidence the recipe resolves in EasyBuild; callers must say so rather than report a clean result.
Implementations
- impl RecipeDepCheck¶
Functions
- fn ok(&self) -> bool¶
Whether every dependency was found.
- fn toolchain_verified(&self) -> bool¶
Whether the matches carry toolchain evidence, i.e. a hierarchy was known.
Distinct from
Self::okon purpose:okanswers “was anything left unmatched”, this answers “is a match worth anything”.
- struct ResolvedDep¶
One resolved dependency entry (2–4 element EasyBuild dependency tuple).
- name: String¶
Dependency name.
- version: String¶
Raw version field after template/local resolution (may be
1.2.3or >=1.2).
- versionsuffix: Option<String>¶
Versionsuffix required, when the entry names one.
- struct ResolvedEasyconfig¶
Fully resolved easyconfig fields (templates and locals applied).
Solver-facing co-selection uses
Self::to_candidate. Packaging / contribution checks also use the optional metadata fields below (easyblock,configopts,moduleclass, …).- name: String¶
Package name.
- version: String¶
Package version after template expansion.
- versionsuffix: Option<String>¶
Versionsuffix, when the easyconfig sets one.
- versionprefix: Option<String>¶
EasyBuild
versionprefixafter the common parse boundary (evaluated env value, then templates).Nonewhen unset or empty.
- dependencies: Vec<ResolvedDep>¶
Runtime dependencies.
- builddependencies: Vec<ResolvedDep>¶
Build-time-only dependencies.
- exts_list: Vec<ResolvedExt>¶
Bundled extensions.
- easyconfig_path: String¶
Path of the source
.ebwhen parsed from disk (empty for in-memory text).
- easyblock: Option<String>¶
EasyBuild easyblock class name (
MesonNinja,CMakeMake, …).
- configopts: Option<String>¶
Meson/CMake/configure flags string after template expansion.
- moduleclass: Option<String>¶
EasyBuild moduleclass (
chem,lib,tools, …).
- homepage: Option<String>¶
Homepage URL when set.
- checksums: Vec<String>¶
Source checksums list (strings), when present — used for packaging gates.
- sources_count: usize¶
Number of entries in
sources(0 when the field is absent).
- data_sources_count: usize¶
Number of entries in
data_sources(0 when absent).
- source_urls: Vec<String>¶
Expanded
source_urlsentries, used to classify which artifact the recipe actually downloads.
- source_files: Vec<SourceFile>¶
Resolved
sourcesentries as(filename, download_filename): the name EasyBuild saves the artifact under and the name it asks eachsource_urlsentry for, which differ when the entry is a dict with adownload_filename. Asource_urlskey inside the dict replaces the recipe-level list for that entry, as it does in EasyBuild.
- crate_source_names: Vec<String>¶
Filenames Cargo easyblocks append onto
sourcesfrom eachcratestuple (name-version.tar.gz, orname-version-revprefix.tar.xzfor git crates). Empty whencratesis absent.
- data_source_names: Vec<String>¶
Filenames from
data_sources(string or dictfilename/name).
- patch_names: Vec<String>¶
Patch file names from
patches(tuple/dict entries reduced to the name).
- checksum_entry_keys: Vec<Vec<String>>¶
Per-
checksums-LIST-ENTRY dict keys (empty inner vec for plain-string entries). Parallel to the checksums list entries, not the flattened values, so a multi-arch dict entry stays one entry. Empty whenchecksumsis a top-level filename dict rather than a list.
- checksums_by_filename: BTreeMap<String, String>¶
Checksums the recipe states by filename, for list entries written as
{'name': 'sha…'}and for a top-level checksums dict.
- checksums_top_level_dict: bool¶
checksumsis a single filename→digest dict (EasyBuild looks up by name and ignores list index).
- has_components: bool¶
The recipe sets
components: it bundles other packages, whatever its easyblock is called.
- component_checksums_complete: bool¶
Every parsed component source, patch, and postinstall patch carries a checksum.
Implementations
- impl ResolvedEasyconfig¶
Functions
- struct ResolvedExt¶
One
exts_listentry after resolution.- name: String¶
Extension name.
- version: String¶
Extension version.
- patches: Vec<String>¶
Patch files the entry’s options name (
'patches': [...]). They sit beside the easyconfig like its own patches and travel with it.
- struct SkippedEasyconfig¶
One easyconfig path that could not be parsed into a candidate.
- path: String¶
Easyconfig that was skipped.
- error: String¶
Why it could not be parsed.
- struct SkippedStatement¶
Resolve easyconfig source text to fully expanded fields (no filesystem path). One statement the tolerant parser could not model, and why.
The parser skips an unrepresentable statement so a single oddity does not cost the whole easyconfig. That is the right default, but it means a recipe can parse “successfully” while a line the author cared about was dropped. These records are what
--strictreports.- line: usize¶
1-based line the statement starts on.
- message: String¶
Why it could not be parsed, already carrying its own line prefix.
- text: String¶
The statement text as written, trimmed.
Implementations
- impl SkippedStatement¶
Functions
- fn reason(&self) -> &str¶
The reason with the parser’s own
line N:position prefix stripped, so a caller that prints its own position does not repeat it.
Traits implemented
- impl std::fmt::Display for SkippedStatement¶
- struct SourceFile¶
One
sourcesentry, resolved against the recipe’s templates.- filename: String¶
Name the artifact is saved under, which
checksumsis keyed by.
- download_filename: String¶
Name requested from each source URL; equals
filenameunless the entry setsdownload_filename.
- source_urls: Vec<String>¶
Per-entry
source_urls, when the dict entry carries its own.
- git: bool¶
The entry fetches a git checkout (
git_config), so no download URL names it and no tarball digest can be computed ahead of the build.