This just moves the code over to ci/github-script to make it easy to
test and iterate on locally.
The name `prepare` is chosen, because the script will be extended with
the other steps from "PR / prepare" next.
(cherry picked from commit c787c66de6)
This only shows *some* of the additional hints, depending on what the
checks resulted in. Should hopefully reduce confusion a bit.
(cherry picked from commit 91fd9b10ac)
Splits the "occasionally" case into two, depending on whether the commit
has a diff or was not cherry-picked at all. Prepares the next commit,
where these are conditionally shown only.
(cherry picked from commit 2f8ed18c97)
This change allows giving a reason via footer of the commit message for
why this commit is not cherry-picked. This avoids having to "explain"
the automated review comment afterwards - instead, this explanation can
be given immediately when writing that commit.
For example, for an update of `xen` on the stable branch, this could be:
```
xen: 4.19.3-unstable-2025-07-09 -> 4.19.3
[... commit message ...]
Not-cherry-picked-because: unstable is on a different minor version
```
This would then be shown as part of the automated review. The severity
of this will be downgraded from "warning" to "important". We still treat
the review as "changes requested", because it would be very complicated
and noisy to handle two different categories of reviews, some with
requested changes and some with comments only.
An alternative would be to not show this review at all. However, given
that the reviewers expectation on backports should already be "if it's
not a clean backport, the automated review will tell me what to look
at", it seems better to show these and have the committer confirm by
dismissing the review. Otherwise we risk merging actually unreviewed
commits.
(cherry picked from commit bf3607aa87)
The filtered checkout we used before was a nice idea, but only worked
for small numbers of commits in a PR. It would fetch the whole history
initially, but only fetch objects on demand. This turns out to be much
too slow in a PR with 18 commits, regularly hitting the 10 minute
timeout, even when running it locally.
The new approach uses regular checkouts again. In contrast to the old
style, before we switched to the filtered checkout, this only fetches
exactly the commits we need - and all of them at once. This is
significantly faster than both other approaches, and scales much better.
A bigger number of commits doesn't have much of an effect, if any at
all.
(cherry picked from commit 0dff5ad6c5)
Nix 2.3 has been removed on unstable and is not used by CI anymore, thus
we don't need the custom config for it anymore.
(cherry picked from commit fa0cba1c39)
The `nixVersions.minimum` alias has been removed on unstable, so the
TODO is not needed anymore.
The lib tests can't be run with Nix 2.3 anymore, because this version is
not available on unstable anymore - and thus also not in the pinned
nixpkgs that CI is run with.
(cherry picked from commit cc3d2295b6)
With this change, we start running Eval on all available Lix and Nix
versions. Because this requires a lot of resources, this complete test
is only run when `ci/pinned.json` is updated.
The resulting outpaths are checked for consistency with the target
branch. A difference will cause the `report` job to fail, thus blocking
the merge, ensuring Eval consistency for Nixpkgs across different
versions.
This implements a kind of "ratchet style" check: Since we originally
confirmed that the versions currently in Nixpkgs at the time of this
commit match Eval behavior of Nix 2.3, we can ensure consistency with
Nix 2.3 down the road, even without testing for it explicitly.
There had been one regression in Eval consistency for Nix between 2.18
and 2.24 - two tests in `tests.devShellTools` produce different results
between Lix 2.91+ (which was forked from Nix 2.18) and Nix 2.24+. I
assume it's unlikely that such a change would be "fixed" by now, thus I
added an exception for these.
As a bonus, we also present the total time in seconds it takes for Eval
to complete for every tested version in a summary table. This allows us
to easily see performance improvements for Eval due to version updates.
At this stage, this time only includes the "outpaths" step of Eval, but
not the generation of attrpaths beforehand.
(cherry picked from commit b523f257ac)
Move "Packages" up, because it's much shorter and easier to scroll past.
This way both Packages and Performance are visible immediately.
(cherry picked from commit f05895fb3c)
This is slightly faster than downloading and extracting a tarball and
additionally allows a sparse checkout. No need to download docs or nixos
for our purpose.
The data is quite noisy, but suggests improvements from anywhere between
5-15 seconds for each job using the pinned nixpkgs.
(cherry picked from commit 8a9f0b8a47)
Using these defaults makes it possible to just run `nix-build ci -A
eval.singleSystem` without passing additional arguments and get a sane
result back. Especially helpful when testing or debugging.
A `chunkSize` of 5000 is conservative to be able to run on systems with
less memory as well. Run-time is not impacted by that, as recent
benchmarks show.
(cherry picked from commit 75f40a150c)
`procps` pulls in 180 MB of systemd, but busybox also provides `kill`.
`busybox` also ships `time`, so no need for that extra dependency.
Using `nativeBuildInputs` pulls in all the -dev outputs of the listed
packages - which we don't need. We only need to run these tools, thus
map to their bin outputs.
Brings down the closure size from 500+ MB to 193 MB for the Eval job.
This probably saves ~10 seconds for the job.
(cherry picked from commit 5c697b8aea)
No need to return bytes in these files. Also busybox has problems to
render `free -b` with sizes > 100 GB properly in the next commit,
leading to extraction errors with awk.
(cherry picked from commit cb527a04e0)
When a package's attrpath is renamed it is currently treated as a
rebuild, even though the outpath already exists and is already cached.
This also happens when adding new names for packagesets that already
exist, for example when starting to eval `perlPackages` in CI, which is
just the same as `perl540Packages` currently. It would also happen when
`perlPackages` is switched from `perl540Packages` to `perl999Packages`.
Assuming that `perl999Packages` had already been built before, this
doesn't really cause any rebuilds.
(cherry picked from commit c5c7cd5bc7)
This was run as a test in `doc/tests/check-nix-code-blocks.nix` before,
but its DX can be improved: By including it in `treefmt` we get better
error reporting and auto-fixing, as well as running it on *all* markdown
files (including READMEs etc.) for free.
(cherry picked from commit a7b3b2584a)
This gives us Nix 2.30 as `nixVersions.latest`, which enables it for
Eval in CI automatically.
It also gives us markdown-code-runner 0.2.0, which allows it to run with
treefmt.
From the nixpkgs-unstable channel:
https://hydra.nixos.org/eval/1817362#tabs-inputs
Changes for treefmt-nix:
421b56313c...58bd4da459
(cherry picked from commit 492225f4f7)
This reduces my codeownership to Firefox Rapid, ESR and the update
script. My hope is that will reduce my notification fatigue on Firefox
Beta/Devedition PRs that I needlessly get requested to.
(cherry picked from commit 4a59c529e5)
The builder has multiple consumers, so it is fine to separate it from
Firefox, but the main motivation is to separate code ownership between
the builder and individual firefox versions.
(cherry picked from commit 0026f50279)
This file takes attrpaths as input and returns chunks of derivations,
aka a subset of release-outpaths - while release-attrpaths-superset
returns only an attrpath, not any derivations.
Thus, as a specialization of release-outpaths, the name
release-outpaths-parallel matches much better.
(cherry picked from commit 4ad7d21c88)
This doesn't fail on *all* asserts, yet, because nix-env still ignores
these in the main eval step. But it already gives some converage during
the attrpath generation.
(cherry picked from commit 30f19cce69)
Nix 2.3 is marked as insecure and thus not cached anymore.
We'll either need to patch it and cache it again or drop it.
(cherry picked from commit ea970ff3be)
This adds a build job for the tarball, which might help uncover eval
issues on attributes not normally touched by Eval, aka those added in
`pkgs/top-level/packages-config.nix`.
(cherry picked from commit bfb20b9fea)
It has only happened once, but the GitHub API suddenly returned a
workflow run from a few weeks back. This lead to quickly hitting the
rate limit for search requests.
Prevent this from happening by going back a day max for the "recently
updated" case. After roughly a day, every PR will have been touched by
the regular batch processing anyway.
Also save a few API requests, by taking bigger chunks of the search
result.
(cherry picked from commit 7f9f7a20b0)
To avoid rendering issues when diffing a markdown file with these
markers in context, just increase the markers length.
(cherry picked from commit 524a47b2b4)
This broke when we moved the check-cherry-picks workflow into the bigger
PR workflow. At this time, the "workflow run" became the whole PR
workflow, which includes many more than just 1 job, thus the assumption
in `jobs[0]` doesn't hold anymore.
(cherry picked from commit c4496912fd)
Most of the checks we do for cherry-picks are dismissable warnings, with
one exception: When a commit hash has been found, but this hash is not
available in any of the pickable branches, we raise this with
severity=error. This should also *block* the merge and not be
dismissable. That's because this is a fixable issue in every case.
(cherry picked from commit 1fbcad0434)
Previously, when the diff contained a context line with ```, this would
end the code block and entirely break the markdown rendering.
Now we use the html code blocks provided by `core.summary` and properly
escape the content, so that it never escapes via html tags.
(cherry picked from commit b19798c8b0)
This reduces noise that the cherry-pick reviews produce when formatting
related conflicts had to be resolved. We only do this in the
length-limited review comment, though. All changes, including
whitespace, can still be double-checked in the job log if needed.
(cherry picked from commit 40dcbb1483)
This turns the check-cherry-pick script into a github-script based
JavaScript program. This makes it much easier to extend to check reverts
or merge commits later on.
(cherry picked from commit b46cb23251)
Since all github-scripts need to be written in commonjs, we now default
to it by not setting package.json. Support from editors for .js files is
slightly better than .cjs. To still allow using module imports in the
test runner script, we trick node into loading the script itself as a
module again via `--import ./run`.
(cherry picked from commit d11eba1e1d)
This just moves things around to use less specific naming - `labels` is
only *one* script that can potentially be run locally while still being
written in github-script. Later, we can add more.
(cherry picked from commit 6f6c625026)
This fixes labeling in the context of the `pull_request` trigger. Of
course, this was supposed to immediately label rebuilds after eval
finishes, but we somehow lost this along the way. Rebuilds are still
labeled fairly soon, because the scheduled trigger will pick up the same
PR within 10 minutes again and then apply the rebuild labels. But of
course, immediate is better.
The reason this happened is, that we're looking at `item.pull_request`
only. This is the correct distinction between "issue items" and "pull
request items", which we both get back from the /issues endpoint. But
the payload for the `pull_request*` event doesn't contain this, so
labeling in this case was treated like an issue and skipped the whole
pull request part.
(cherry picked from commit acf5bc2417)
