Find shared builds across open pull requests

Two open pull requests on the European Environment for Scientific Software Installations (EESSI) software-layer often build the same dependency. If both are merged and built, the second build overwrites the first. eb-stack pr-overlap lists the modules each open pull request (PR) would still build, which of those modules more than one PR would build, and, with --merge-order, an order that builds each shared module once.

Use it before you merge a batch of open software-layer PRs that share a dependency. The command analyzes easyconfig and easystack text and writes report files. It does not run EasyBuild or modify a pull request.

Run the recorded fixture

This repository records two software-layer PRs under tests/pr_overlap_layer. Run this from the repository root:

eb-stack pr-overlap --pr tests/pr_overlap_layer/prs.json \
  --easyconfigs tests/pr_overlap_layer \
  --installed tests/pr_overlap_layer/installed.txt \
  --installed-label none \
  --generated-at 2026-10-24T00:00:00Z \
  --repo-url https://github.com/EESSI/software-layer \
  --eessi-version 2026.06 \
  --merge-order \
  --out-dir /tmp/pr-overlap-out

The command prints:

prs=2 records=2 shared_modules=91 skipped_easyconfigs=0 out=/tmp/pr-overlap-out

prs is how many pull requests were read. records is how many of them add at least one entry to an easystack for this version. A pull request with no such entry stays in prs and stays out of records. Each record has a to_build list, the build set: the modules that PR would still build. shared_modules counts modules in more than one build set. skipped_easyconfigs is how many recipe files the parser could not parse. out is the directory containing overlap.md, overlap.toml, one prs/<N>.toml file per record, and, with --merge-order, merge-order.toml.

Pull request 1585 adds SciPy-bundle/2026.05-gfbf-2026.1. Pull request 1728 adds PLUMED/2.10.0-foss-2026.1, and that recipe depends on SciPy-bundle 2026.05. The fixture’s installed.txt is a single comment line, so nothing counts as already built. The fixture is a replay: both PRs have since merged, and the titles in the report are the merge-commit subjects stored in prs.json. Both PRs are run against an empty installed list, which is why they share 91 modules, including compilers such as GCC/15.2.0.

Read the report

The top of overlap.md from this run is:

<!-- Generated by `eb-stack pr-overlap`. This branch is never merged. -->
# Shared builds between open pull requests (2026.06)

Computed 2026-10-24T00:00:00Z against the modules installed for `none`. 2 pull requests change an easystack for this version.

## Modules more than one pull request would build

| Module | Pull requests |
|---|---|

The line under the title names the installed list (none here). The table has one row per shared module, 91 rows in this run. SciPy-bundle is one of them:

| `SciPy-bundle/2026.05-gfbf-2026.1` | [#1585](https://github.com/EESSI/software-layer/pull/1585), [#1728](https://github.com/EESSI/software-layer/pull/1728) |

Read that row as: both pull requests would build this module, so one build has to finish before the other starts. Further down, the section for pull request 1585 gives a build count of 91 modules, and the section for pull request 1728 gives 109. All 91 of 1585’s modules are in 1728’s build set.

--merge-order also writes merge-order.toml. These lines are from that file. Each step’s builds line, and the modules list under steps.saves, is one long line of module names and is left out here.

[[steps]]
position = 1
pr = 1585
after = []
reason = "smallest residual build set (91)"

[[steps.saves]]
pr = 1728

[[steps]]
position = 2
pr = 1728
after = [1585]
saves = []
reason = "smallest residual build set (18)"

Merge in position order. after lists the PRs that must merge first. Pull request 1728 must wait for pull request 1585, because 1585 adds SciPy-bundle to the easystack by name, while 1728 only reaches it as a dependency of PLUMED. When two or more PRs are free to go next, the one with the fewest modules left to build goes first, and reason records that count. The word “residual” in reason means those modules left to build. In step 1, no earlier PR is required, and pull request 1585 has 91 modules left to build. The [[steps.saves]] entry says that building those modules for 1585 removes the same 91 modules from 1728’s later build. In step 2, pull request 1728 has after = [1585], and 18 modules remain, including PLUMED.

The same order is the Merge order table at the end of overlap.md. Merge and build pull request 1585 first, and pull request 1728 second.

prs/<N>.toml is that PR’s build set. overlap.toml is the same shared-module index as data. Field names are in output formats. A PR omitted from the current input has no file: each run rewrites prs/ to contain only records from that run. The live fetch supplies open PRs; static --pr files are not checked for open or closed status. How one easystack line becomes a build set, and how that order is chosen, is in from one easystack line to a build set.

Run it on a mounted stack

You need three things: the EESSI repository mounted under /cvmfs/software.eessi.io, a clone of EESSI/software-layer as the working directory, and a logged-in gh. The CPU target in this example is x86_64/amd/zen4. Source the EESSI init script and load the extend module before the block below:

source /cvmfs/software.eessi.io/versions/2026.06/init/lmod/bash
module load EESSI-extend

That sets EESSI_SOFTWARE_PATH (the install prefix for the CPU target you loaded) and EASYBUILD_FILTER_DEPS. Set EB_STACK_CHECKOUT to an eb-stack checkout. Set EB_EASYCONFIGS and EB_EASYBLOCKS to the easyconfig tree and the easybuild/easyblocks package for the EasyBuild version these easystacks name. scripts/fetch-upstream-trees.sh clones a pinned release, which can differ from that version. examples/eessi-pr-overlap/fetch-prs.sh reads every open PR and writes one JSON file per PR that changes an easystack for that EESSI version. It pages through the GitHub pulls API 100 at a time with no stop at 500, so a repository with more open PRs than one listing returns is still read in full. Changed files are read 100 per page through GitHub’s cap of 3000. A pull request whose file list stops there is still written, with file_list_truncated set, and the record notes truncated at GitHub's limit. It fetches each PR head into the clone as refs/remotes/pr/N and does not check it out. --eessi-version defaults to 2026.06.

: "${EESSI_SOFTWARE_PATH:?source the EESSI init script and module load EESSI-extend}"
: "${EASYBUILD_FILTER_DEPS:?source the EESSI init script and module load EESSI-extend}"
: "${EB_STACK_CHECKOUT:?set this to an eb-stack checkout}"
: "${EB_EASYCONFIGS:?set this to the easyconfig tree for the EasyBuild version these easystacks name}"
: "${EB_EASYBLOCKS:?set this to the easybuild/easyblocks package for that release}"

"$EB_STACK_CHECKOUT/examples/eessi-pr-overlap/fetch-prs.sh" \
  EESSI/software-layer 2026.06 /tmp/pr-overlap-in

find "$EESSI_SOFTWARE_PATH/software" -mindepth 2 -maxdepth 2 \
  -type d -printf '%P\n' | sort > /tmp/pr-overlap-installed.txt

shopt -s nullglob
args=()
for f in /tmp/pr-overlap-in/prs/*.json; do
  args+=(--pr "$f")
done
for d in /tmp/pr-overlap-in/commit-trees/*/; do
  args+=(--commit-tree "$(basename "$d")=$d")
done
for d in /tmp/pr-overlap-in/easyblocks/*/; do
  args+=(--easyblocks-commit "$(basename "$d")=$d")
done
shopt -u nullglob

eb-stack pr-overlap "${args[@]}" \
  --easyconfigs "$EB_EASYCONFIGS" \
  --easyblocks "$EB_EASYBLOCKS" \
  --installed /tmp/pr-overlap-installed.txt \
  --installed-modulerc "$EESSI_SOFTWARE_PATH/modules/all" \
  --filter-deps "$EASYBUILD_FILTER_DEPS" \
  --installed-label x86_64/amd/zen4 \
  --eessi-version 2026.06 \
  --exclude-path /accel/ \
  --exclude-path /additional_arch/ \
  --merge-order \
  --out-dir /tmp/pr-overlap-out

On EESSI/software-layer on 2026-10-31, fetch-prs.sh printed:

open pull requests: 52
pull requests with 2026.06 easystack changes: 26

The first line counts every open PR the API returned. The second line counts the files in prs/, which are the --pr inputs. A PR missing from prs/ changed no 2026.06 easystack.

nullglob keeps an empty commit-trees/ or easyblocks/ from being passed as a literal path. You get the same kinds of report files as in the recorded run. The current PRs and the installed prefix set the counts and module names. Installed modules drop out of each build set. shared_modules then counts the uninstalled modules in two or more build sets.

Flag

What to pass

--easyconfigs

The easyconfigs tree for the EasyBuild version the easystack names

--commit-tree C=DIR

The easyconfigs at one from-commit pin C

--easyblocks

That EasyBuild release’s easybuild/easyblocks package

--easyblocks-commit C=DIR

The easyblock files one include-easyblocks-from-commit touches

--installed-modulerc

The prefix’s modules/all, so Lmod aliases count as installed

--filter-deps

Dependencies EasyBuild filters. Defaults to $EASYBUILD_FILTER_DEPS

See what the bot builds on each architecture

The EESSI bot builds more than the lines a PR adds. The bot reads each changed easystack at the PR head. It then builds every entry not yet installed on that architecture, plus the missing dependencies of those entries. An entry can sit in the shared easystack for months and still be missing on one architecture. Every later PR that touches the file makes that bot try it again. Software-layer PR 1775 did not add GDRCopy. The riscv64 bot still failed on it.

Pass --arch-installed ARCH=FILE once per architecture to get that build set. The file is the same installed-module list as --installed, and the flag replaces --installed. --bot-build-set gives the same whole-file closure for a single --installed list. Without either flag the command closes only the added lines, as in the recorded run above.

This fixture has the shape of PR 1775. The shared easystack already lists GDRCopy. PR 1775 adds nvtop, and PR 1800 adds likwid. x8664has GDRCopy installed and riscv64 does not.

eb-stack pr-overlap \
  --pr tests/pr_overlap_arch/prs.json \
  --easyconfigs tests/pr_overlap_arch/release \
  --arch-installed x86_64=tests/pr_overlap_arch/x86_64.txt \
  --arch-installed riscv64=tests/pr_overlap_arch/riscv64.txt \
  --generated-at 2026-10-30T00:00:00Z \
  --repo-url https://github.com/EESSI/software-layer \
  --merge-order \
  --out-dir /tmp/pr-overlap-arch

The command prints:

prs=2 records=2 shared_modules=2 skipped_easyconfigs=0 out=/tmp/pr-overlap-arch

overlap.md now has one section per architecture. On x8664the shared table says None. and each PR builds only its own module. Here is the riscv64 section, trimmed:

| `GDRCopy/2.5.2-GCCcore-15.2.0` | [#1775](https://github.com/EESSI/software-layer/pull/1775), [#1800](https://github.com/EESSI/software-layer/pull/1800) |
| `Lib/1.0-GCCcore-15.2.0` | [#1775](https://github.com/EESSI/software-layer/pull/1775), [#1800](https://github.com/EESSI/software-layer/pull/1800) |

Already in the changed easystack and not installed: `GDRCopy/2.5.2-GCCcore-15.2.0`.

The “Already in the changed easystack” line means the PR did not add GDRCopy, and the riscv64 bot will still build it. Lib comes along because it is a dependency of GDRCopy. If that architecture cannot build the module at all, fix the easystack or mark it unsupported before you merge (see refuse a plan an architecture cannot build). The Merge order table is repeated per architecture. On riscv64, PR 1775 goes first and saves GDRCopy and Lib for PR 1800. In prs/1775.toml, entries is still only the nvtop line, and GDRCopy is listed under carried. The per-architecture fields are in output formats.

Wire it into CI

The draft workflow is examples/eessi-pr-overlap/pr_dependency_overlap.yml. It is not installed in the EESSI software-layer repository from this checkout.

On that repository:

  1. Exempt gh_action_pr_dependency_overlap from the branch push restriction, the same way gh_action_update_software_layer_commit_sha is exempt, so github-actions[bot] can push to that one branch. main stays protected.

  2. Copy pr_dependency_overlap.yml to .github/workflows/ and fetch-prs.sh to .github/workflows/scripts/fetch_pr_easystacks.sh.

  3. Set EB_STACK_VERSION and EB_STACK_SHA256 in the workflow to a release that carries pr-overlap.

The workflow runs on a schedule and on workflow_dispatch, from main. It mounts the stack, writes overlap.md, overlap.toml, and one prs/<N>.toml per recorded PR, then pushes that report to gh_action_pr_dependency_overlap without force. It opens no pull request and posts no comment. The publish job skips the commit when only the generated_at = ~ and ~Computed ~ lines change. A timestamp-only change is not committed. A title change is. ~--impact and --merged exit 2 and name the number when it is not in the input. Exclusion wins: a path that matches both --include-path and --exclude-path is skipped.

Easyconfigs pull requests

For open PRs on easybuild-easyconfigs, pass --kind easyconfigs. You do not need --installed: the build set is then the modules the base tree does not already have. The recorded open set is tests/pr_overlap_easyconfigs_open/.

eb-stack pr-overlap \
  --kind easyconfigs \
  --pr tests/pr_overlap_easyconfigs_open/prs.json \
  --easyconfigs tests/pr_overlap_easyconfigs_open/tree \
  --generated-at 2026-10-24T00:00:00Z \
  --repo-url https://github.com/easybuilders/easybuild-easyconfigs \
  --out-dir /tmp/ec-open-out

The command prints:

prs=4 records=4 shared_modules=1 skipped_easyconfigs=0 out=/tmp/ec-open-out

22823 and 24575 share Mathematica/14.2.1. That run, including the live fetch, is in find open easybuild-easyconfigs pull requests that add the same software.