mod shard_trees

module shard_trees

Easyconfig trees for build-list lines whose easyconfigs EasyBuild fetches from GitHub.

eb X.eb --from-commit=SHA downloads the easybuild-easyconfigs archive of that commit, reads X.eb from it, and appends its easybuild/easyconfigs to the robot path. eb X.eb --from-pr=N downloads the easyconfig files pull request N changes, at its head, and appends them the same way. A dependency comes from the robot path when it is there and from the fetched files otherwise. stack shard resolves such a line the same way.

A tree is found, in this order, in an explicit COMMIT=DIR or N=DIR entry, in the cache directory, or by downloading it into the cache. A commit is served from https://codeload.github.com/<repo>/tar.gz/<sha>; a commit on a fork that a pull request carries is served through the upstream repository too. Only the .eb files under easybuild/easyconfigs/ are extracted. A pull request’s head commit and changed files come from api.github.com and the file text from raw.githubusercontent.com; its cache entry is keyed by the head commit, so a pull request that moves is fetched again.

A tree is written under a temporary name and renamed into place, so a cache entry that exists is complete, and two planners filling one cache at once both end with the same entry.

Variables

const CACHE_DOWNLOAD_FAILED: &str

Prefix on a commit download that received no HTTP status.

run_stack_shard_with treats a message that starts with this as a failure and writes nothing. An HTTP status, including 404, stays a per-line warning: the server answered, and the commit is simply absent.

const EASYCONFIGS_REPO: &str

The repository --from-commit and --from-pr read.

const MANIFEST: &str

File in a downloaded commit tree listing sha256  relative/path.eb for every easyconfig in it, so a planner can skip the files the robot path already has without reading them.

Functions

fn extract_easyconfigs(archive: &[u8], dest: &Path) -> Result<usize, String>

Extract the .eb files under easybuild/easyconfigs/ of a GitHub archive (<repo>-<sha>/ first) into dest, at the same relative paths. Returns how many were written.

fn overlay_candidates(tree: &Path, base: &BTreeMap<String, String>) -> Result<(Vec<crate::domain::Candidate>, Vec<String>), String>

The easyconfigs of tree that differ from the robot path’s: a file whose name and content match the robot path’s effective easyconfig of that name resolves the same either way and is left out. base maps a file name to the SHA-256 of the robot path’s file. A tree’s MANIFEST, when present, stands in for reading its files. Returns the parsed candidates and the files that did not parse.

Structs and Unions

struct TreeSources<'a>

Where the trees of fetched lines come from.

commit_trees: BTreeMap<String, PathBuf>

--commit-tree COMMIT=DIR: a commit, full or abbreviated, to an easyconfigs directory.

pr_trees: BTreeMap<u64, PathBuf>

--pr-tree N=DIR: a pull request to a directory of its easyconfigs.

cache: Option<PathBuf>

--commit-cache DIR: trees found or downloaded there.

http: &'a dyn HttpGet

The client downloads go through.

repo: String

owner/name of the easyconfigs repository.

Implementations

impl TreeSources<'_>

Functions

fn any(&self) -> bool

Whether any source is configured.

fn commit_tree(&self, sha: &str) -> Result<PathBuf, String>

The easyconfigs directory of sha, or why there is none.

fn explicit_commit_tree(&self, sha: &str) -> Option<Result<PathBuf, String>>

The --commit-tree directory of sha, when one names it.

fn pr_tree(&self, number: u64) -> Result<PathBuf, String>

The directory of pull request number’s easyconfigs at its head, or why there is none.