mod hierarchy

module hierarchy

Toolchain generation hierarchy: sub-toolchain membership for dependency resolve.

EasyBuild applications built with a composite toolchain (e.g. foss-2024a) pull dependencies from several sub-toolchain levels of the same generation (GCCcore, GCC, gfbf, gompi, …). Exact top-level toolchain string matching therefore finds none of them.

Hierarchy ground truth is EasyBuild’s get_toolchain_hierarchy (framework). Checked-in JSON fixtures under fixtures/toolchain_hierarchy/ capture that output so unit tests and runtime resolution do not require EasyBuild.

Functions

fn count_generation_dep_versions(name: &str, cands: &[Candidate], hierarchy: &ToolchainHierarchy) -> HashMap<String, usize>

Count exact version pins of dependency name among recipes whose own toolchain is in hierarchy (generation-scoped reverse-deps).

Both dependencies and builddependencies are counted. Only exact version pins are tallied (3.29.3 or solver form ==3.29.3); empty/range reqs skip.

Pure function for unit tests: pass a synthetic [Candidate] universe.

fn derive_hierarchy_from_candidates(parent: &Toolchain, cands: &[Candidate]) -> Option<ToolchainHierarchy>

The hierarchy for a toolchain, read from the tree that defines it.

Each family knows its own shape: the GCC composites reach GCCcore through their GCC pin, the NVIDIA ones walk their own chain, and a compiler-only toolchain states the GCCcore it was built on. Anything else is walked generically.

fn filter_candidates_in_hierarchy(cands: &[Candidate], hierarchy: &ToolchainHierarchy) -> Vec<Candidate>

Keep candidates whose toolchain is any member of hierarchy.

fn hierarchy_for(parent: &Toolchain, fixture_path: Option<&Path>) -> Result<ToolchainHierarchy, HierarchyError>

Resolve hierarchy for parent: optional fixture path, else built-in known map.

fn hierarchy_for_with_tree(parent: &Toolchain, fixture_path: Option<&Path>, cands: &[Candidate]) -> Result<ToolchainHierarchy, HierarchyError>

Like hierarchy_for, with the parsed easyconfig universe as a final fallback: fixture path, else built-in, else derived from the tree (derive_hierarchy_from_candidates).

fn hierarchy_member_rank(hierarchy: &ToolchainHierarchy, tc: &Toolchain) -> Option<usize>

Rank of tc in hierarchy (parent last = highest). Uses toolchains_match so SYSTEM empty/system labels compare equal. Out-of-hierarchy → None.

fn is_system_toolchain(tc: &Toolchain) -> bool

EasyBuild / parser SYSTEM toolchains: name system (any case), version empty or system.

fn known_hierarchy(parent: &Toolchain) -> Option<ToolchainHierarchy>

Built-in hierarchy fixtures embedded at compile time (no EasyBuild at test/runtime).

fn load_hierarchy_fixture(path: &Path) -> Result<ToolchainHierarchy, HierarchyError>

Load a hierarchy fixture JSON file.

fn nearest_candidates_hint(name: &str, cands: &[Candidate]) -> String

Short “what DOES exist” suffix for a missing dependency: up to four name-matching candidates across all generations, so the failure reads as a work item (bump/author from one of these) instead of a dead end. Returns an empty-or-parenthesised string safe to append to an error message.

fn pick_consensus_version(counts: &HashMap<String, usize>, eligible: &[String]) -> Option<String>

Pick a generation-consensus version of name among eligible install versions.

Clear majority (at least 80 percent of generation pins of name that land in eligible) yields that modal version. Else pick the newest among eligible versions that appear in the pin counts (weak signal / no unique consensus). If no pin counts hit eligible, pick the newest eligible (true no-signal fallback).

eligible must already be floor/suffix/hierarchy filtered; empty yields None. Callers that hold full Candidate values should first run prefer_non_system_candidates so SYSTEM does not beat GCCcore on version alone.

fn prefer_non_system_candidates<'a>(eligible: &[&'a Candidate]) -> Vec<&'a Candidate>

Prefer generation-compiler / composite toolchains over SYSTEM among already hierarchy-eligible install candidates.

SYSTEM remains a hierarchy member (for rare binary-only build tools), but when a GCCcore/GCC/gfbf/… candidate exists it must win empty-consensus “newest” fallbacks (e.g. CMake 3.29.3 @ GCCcore-13.3.0 over CMake 3.31.8 @ SYSTEM).

Returns eligible unchanged when every candidate is SYSTEM (or the list is empty).

fn resolve_dep_version_in_hierarchy(name: &str, cands: &[Candidate], hierarchy: &ToolchainHierarchy) -> Option<String>

Among hierarchy members for name, pick a safe version for the target generation.

Among hierarchy members for name, pick a safe version for the target generation.

Selection rules, in order: (1) candidate must be in the hierarchy and match name (strict name+version membership; out-of-generation GCCcore/GCC never qualify); (2) optional versionsuffix must match (empty/None matches candidates with no suffix); (3) optional floor_version never picks a version older than the floor; (4) prefer candidates on the hierarchy parent toolchain when present, otherwise the highest hierarchy member that has a candidate; (5) among the same hierarchy rank (or when use_consensus / floor is set), use generation consensus when available: if a version has a clear majority (at least 80 percent of pins) and is eligible, pick it, else the newest eligible version that has at least one generation pin (else newest eligible). Without floor and without consensus, pick the newest version by cmp_version. Returns None when no candidate satisfies the filters.

fn resolve_dep_version_in_hierarchy_opts(name: &str, cands: &[Candidate], hierarchy: &ToolchainHierarchy, opts: &ResolveDepOpts<'_>) -> Option<String>

Like resolve_dep_version_in_hierarchy with floor / versionsuffix filters.

Strict hierarchy: only candidates whose toolchain name and version are exact members of hierarchy (via ToolchainHierarchy::contains / toolchains_match) are eligible. A newer package on GCCcore-14.x is not valid for foss-2024a (GCCcore-13.3.0 only), even if the name matches.

fn resolve_dep_versions_for_specs(specs: &[SourceDepSpec], cands: &[Candidate], hierarchy: &ToolchainHierarchy, keep_old: bool) -> Result<(HashMap<String, String>, Vec<String>), HierarchyError>

Resolve many source dependency specifications with floor and versionsuffix safety.

• Deps with a non-empty versionsuffix are not bumped (caller keeps source).

• Deps with SYSTEM 4th-tuple toolchain are not bumped (keep source pin).

• Deps marked optional (# optional on that line) resolve/bump like any other dep when hierarchy candidates exist (# optional = optional to include, not frozen). If unresolved they soft-keep the source pin (no hard ERROR). The emitter preserves the comment text; only the version token changes.

• Resolved versions are never older than the source version.

• Missing non-optional candidates yield HierarchyError::MissingDep unless keep_old is true (then the source version is kept and listed in kept_old).

fn resolve_dep_versions_in_hierarchy(names: impl IntoIterator<Item = impl AsRef<str>>, cands: &[Candidate], hierarchy: &ToolchainHierarchy) -> HashMap<String, String>

Resolve versions for many dependency names. Names missing from the universe are omitted from the map (callers may leave source versions unchanged).

fn resolve_dep_versions_in_hierarchy_strict(names: impl IntoIterator<Item = impl AsRef<str>>, cands: &[Candidate], hierarchy: &ToolchainHierarchy) -> Result<HashMap<String, String>, HierarchyError>

Like resolve_dep_versions_in_hierarchy but errors if any name is unresolved.

fn toolchains_match(a: &Toolchain, b: &Toolchain) -> bool

Equality for hierarchy membership, with SYSTEM normalization.

Enums

enum HierarchyError

Why a toolchain hierarchy could not be resolved.

Io(String, std::io::Error)

A hierarchy file could not be read.

Parse(String, String)

A hierarchy file could not be understood.

UnknownToolchain(String, String)

A toolchain named in the hierarchy is not defined.

UndefinedToolchain(String, String)

The toolchain’s own defining recipe is absent from every tree given.

MissingDep(String, String, String, String)

A dependency could not be placed anywhere in the hierarchy.

Structs and Unions

struct ResolveDepOpts<'a>

Options controlling safe hierarchy dependency resolution.

floor_version: Option<&'a str>

Never return a version older than this floor (source recipe version).

versionsuffix: Option<&'a str>

When set, only candidates with this versionsuffix match. When the source pins a non-empty versionsuffix, callers should typically not bump the dep at all (see resolve_dep_versions_for_specs).

use_consensus: bool

When true (generation bump), prefer generation-consensus version selection among eligible install candidates (see resolve_dep_version_in_hierarchy_opts). Default false preserves prefer_newer among ranks when no floor is set.

struct SourceDepSpec

One dependency as scraped from a source recipe (name + version + optional suffix).

name: String

Dependency name.

version: String

Version required.

versionsuffix: Option<String>

Versionsuffix required, when the entry names one.

system_toolchain: bool

4th tuple element is EasyBuild SYSTEM (pseudo-external / binary pin).

optional: bool

Trailing same-line comment marks an optional extra (e.g. # optional). Soft-unresolved only — does not freeze the pin when candidates exist.

Implementations

impl SourceDepSpec

Functions

fn is_preserve_pin(&self) -> bool

Whether package bump planning must leave the source version untouched: SYSTEM 4th-tuple or non-empty versionsuffix.

A # optional comment marks the dependency optional to include, not frozen. Optional deps still bump when hierarchy candidates exist (soft-keep the source pin only if unresolved — no hard ERROR).

fn plain(name: impl Into<String>, version: impl Into<String>) -> Self

Convenience constructor for tests / simple name+version pins.

struct ToolchainHierarchy

Ordered sub-toolchain hierarchy for one parent toolchain generation.

Member order matches EasyBuild: most minimal first, parent last (e.g. system, GCCcore, GCC, gfbf, gompi, foss).

parent: Toolchain

The toolchain the members are subtoolchains of.

members: Vec<Toolchain>

Subtoolchains, which a dependency may be built against instead.

Implementations

impl ToolchainHierarchy

Functions

fn contains(&self, tc: &Toolchain) -> bool

Whether tc is a member of this generation’s hierarchy.

fn member_labels(&self) -> Vec<String>

Labels name-version for each member (system empty version → system).