Commit Graph

1302 Commits

Author SHA1 Message Date
Wolfgang Walther
28c86b2359 workflows: support cachix in forks
This allows setting up a custom cachix cache in a contributor's fork.
That's most valuable when working on CI itself, testing and interating
in a fork. It's required when working on cachix related features
directly.

(cherry picked from commit 025424913f)
2025-08-20 16:54:39 +00:00
Wolfgang Walther
2c86ed707c workflows/reviewers: integrate codeowner reviews
These two jobs mostly used the same code anyway, so they can easily run
in the same job. This uses fewer resources and avoids running (and
failing) the codeowners review job on Pull Requests from development
branches like staging-next or haskell-updates.

It also opens the door for more improvements later on, for example it
will be much easier to implement a shared reviewer limit for both owners
and maintainers now.

(cherry picked from commit ddf81dead6)
2025-08-20 15:16:14 +00:00
Wolfgang Walther
2746652fdc workflows/reviewers: remove pull_request trigger
Changes to this file already trigger a run of this workflow via the PR
workflow, we don't need to run this *another* time.

(cherry picked from commit 7f4dd7e2ff)
2025-08-20 15:16:14 +00:00
Wolfgang Walther
aefe360c78 workflows/check: move owners check job from codeowners
This runs the "check owners" job as part of the main PR workflow, with
multiple advantages:
- The job doesn't run anymore when undrafting a PR, where it's not
required, because the OWNERS file's contents didn't change.
- A valid OWNERS file is now a requirement to merge a PR.
- The OWNERS file is always checked on the exact same test merge commit
that the remainder of the workflows are running on as well.

(cherry picked from commit 15be04f7a2)
2025-08-20 15:16:14 +00:00
Wolfgang Walther
f27f68194a workflows/codeowners: split comment for each job
Allows splitting the workflow file in the next step.

(cherry picked from commit 368a2d6254)
2025-08-20 15:16:14 +00:00
Wolfgang Walther
4daef5d847 workflows/codeowners: move global env into jobs
Preparation to split the whole file.

(cherry picked from commit c962461556)
2025-08-20 15:16:14 +00:00
Wolfgang Walther
b60acc12a9 ci/github-script/prepare: init from actions/get-merge-commit
This just moves the code over to ci/github-script to make it easy to
test and iterate on locally.

The name `prepare` is chosen, because the script will be extended with
the other steps from "PR / prepare" next.

(cherry picked from commit c787c66de6)
2025-08-20 13:22:29 +00:00
Wolfgang Walther
ea9f67944f actions/get-merge-commit: remove push branch
This action will not be called for push events anymore. For those, the
`push.yml` workflow is run instead, which hardcodes `mergedSha`.

(cherry picked from commit 2087f1be30)
2025-08-20 13:22:29 +00:00
Wolfgang Walther
19d5d69dc9 workflows/eval: run misc job with same merge commit
This was an oversight when we introduced the ability for
`get-merge-commit` to take the pre-computed sha as input.

(cherry picked from commit f5c5bc25ea)
2025-08-20 13:22:29 +00:00
Wolfgang Walther
fe1404b803 workflows/merge-group: init
Introduces a basic merge queue workflow to initially only run lints.
This will avoid accidentally merging changes which break nixfmt after
its recent update to 1.0.0.

(cherry picked from commit 04c039f993)
2025-08-14 07:28:44 +00:00
Wolfgang Walther
3ef8956f74 workflows/check: use regular checkout
The filtered checkout we used before was a nice idea, but only worked
for small numbers of commits in a PR. It would fetch the whole history
initially, but only fetch objects on demand. This turns out to be much
too slow in a PR with 18 commits, regularly hitting the 10 minute
timeout, even when running it locally.

The new approach uses regular checkouts again. In contrast to the old
style, before we switched to the filtered checkout, this only fetches
exactly the commits we need - and all of them at once. This is
significantly faster than both other approaches, and scales much better.
A bigger number of commits doesn't have much of an effect, if any at
all.

(cherry picked from commit 0dff5ad6c5)
2025-08-13 19:15:10 +00:00
Wolfgang Walther
521fd9bcd1 workflows/check: allow more time for check cherry picks job
This currently times out after 3 minutes. Give it a bit more time. 10
minutes might be excessive, but we only really want to guard against a
stuck job taking 6 hours.

(cherry picked from commit 17b1c6cb3d)
2025-08-12 17:15:00 +00:00
Wolfgang Walther
3bc8302a6a workflows/pr: fix condition for no-pr-failures job
The `cancelled()` condition seems to only apply when *the whole
workflow* was cancelled. This is not the case when a single job is
cancelled due to timeout.

We can replicate this by checking each needs.result manually.

(cherry picked from commit d1bb35448a)
2025-08-12 14:14:06 +00:00
Wolfgang Walther
eec72ea6e4 workflows/pr: block merging PRs when jobs have been cancelled
This currently happens, for still unknown reasons, for the "check cherry
picks" job. The job gets cancelled by GHA mid-way. This should be the
same as an error, because an important check didn't run: Merging should
be blocked and auto-merge should not succeed.

(cherry picked from commit 06f574addf)
2025-08-12 12:32:34 +00:00
Wolfgang Walther
a88329d954 [Backport release-25.05] workflows/eval: compare eval consistency and performance between Lix / Nix versions (#433033) 2025-08-12 09:32:24 +00:00
Wolfgang Walther
56022d779d workflows/eval: fix compare job not running
The conclusion of the `versions` job propagates from through `eval` to
`compare`, which meant the `compare` job was skipped. No rebuild labels,
no reviewer requests.

Also, we don't want to run eval when `versions` runs, but fails.

(cherry picked from commit 267d5cdf64)
2025-08-12 11:24:55 +02:00
Wolfgang Walther
835136c988 workflows/backport: fix token permissions
The additional `workflows` permissions are required to backport
Dependabot updates. The permissions had been added to the app a while
ago, but we forgot to actually use them.

(cherry picked from commit bb1529ef6a)
2025-08-12 09:18:40 +00:00
Wolfgang Walther
0262bcddf2 workflows/eval: test all available versions
With this change, we start running Eval on all available Lix and Nix
versions. Because this requires a lot of resources, this complete test
is only run when `ci/pinned.json` is updated.

The resulting outpaths are checked for consistency with the target
branch. A difference will cause the `report` job to fail, thus blocking
the merge, ensuring Eval consistency for Nixpkgs across different
versions.

This implements a kind of "ratchet style" check: Since we originally
confirmed that the versions currently in Nixpkgs at the time of this
commit match Eval behavior of Nix 2.3, we can ensure consistency with
Nix 2.3 down the road, even without testing for it explicitly.

There had been one regression in Eval consistency for Nix between 2.18
and 2.24 - two tests in `tests.devShellTools` produce different results
between Lix 2.91+ (which was forked from Nix 2.18) and Nix 2.24+. I
assume it's unlikely that such a change would be "fixed" by now, thus I
added an exception for these.

As a bonus, we also present the total time in seconds it takes for Eval
to complete for every tested version in a summary table. This allows us
to easily see performance improvements for Eval due to version updates.
At this stage, this time only includes the "outpaths" step of Eval, but
not the generation of attrpaths beforehand.

(cherry picked from commit b523f257ac)
2025-08-12 08:22:54 +00:00
Wolfgang Walther
7693833eeb workflows: fix actions/download-artifact hashes
The Dependabot update change the hashes to the latest main branch commit
instead of the v5.0.0 tag - also it didn't adjust the tags in the
comments accordingly. Last but not least, one of the references used a
`@v5` reference instead of the commit hash. The latter is probably what
Dependabot tripped on.

(cherry picked from commit 51e6b0e40b)
2025-08-12 08:22:54 +00:00
Wolfgang Walther
52a04aef17 workflows/build: build shells on a single darwin runner only
Because we have Rosetta, we can use a single darwin runner to build both
shells. Most time is spent for downloading Nix and checking out Nixpkgs
anyway and this can be saved if only run once.

Also, this prepares for the deprecation of the old macos-13 runners,
which were based on x86_64 CPUs in September 2025:
https://github.blog/changelog/2025-07-11-upcoming-changes-to-macos-hosted-runners-macos-latest-migration-and-xcode-support-policy-updates/

(cherry picked from commit 56d29bbae1)
2025-08-11 16:15:49 +00:00
Wolfgang Walther
fb86dcb57c workflows/build: avoid downloading from cachix
This avoids downloading results from cachix, when they don't need to be
rebuilt, which just wastes time and resources.

(cherry picked from commit f105ab12f6)
2025-08-11 16:15:49 +00:00
Wolfgang Walther
41315a2699 actions/get-merge-commit: bump actions/checkout to v5.0.0
A previous commit from Dependabot did this for all the workflows, but
Dependabot doesn't update the actions folder, apparently.

(cherry picked from commit 46d5594714)
2025-08-11 16:04:17 +00:00
Wolfgang Walther
06c8d5c918 workflows: checkout pinned nixpkgs explicitly
This is slightly faster than downloading and extracting a tarball and
additionally allows a sparse checkout. No need to download docs or nixos
for our purpose.

The data is quite noisy, but suggests improvements from anywhere between
5-15 seconds for each job using the pinned nixpkgs.

(cherry picked from commit 8a9f0b8a47)
2025-08-11 16:04:16 +00:00
Wolfgang Walther
15e17895b5 workflows: remove extra_nix_config.sandbox
This is true by default anyway.

(cherry picked from commit de7aa9f213)
2025-08-11 16:04:16 +00:00
dependabot[bot]
8a03e71552 build(deps): bump actions/create-github-app-token from 2.0.6 to 2.1.0
Bumps [actions/create-github-app-token](https://github.com/actions/create-github-app-token) from 2.0.6 to 2.1.0.
- [Release notes](https://github.com/actions/create-github-app-token/releases)
- [Commits](df432ceedc...0f859bf9e6)

---
updated-dependencies:
- dependency-name: actions/create-github-app-token
  dependency-version: 2.1.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
(cherry picked from commit d1d2650cba)
2025-08-11 17:45:44 +02:00
dependabot[bot]
c3969b49ee build(deps): bump actions/checkout from 4.2.2 to 5.0.0
Bumps [actions/checkout](https://github.com/actions/checkout) from 4.2.2 to 5.0.0.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](11bd71901b...08c6903cd8)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: 5.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
(cherry picked from commit 593aee095f)
2025-08-11 17:45:44 +02:00
dependabot[bot]
0962de4c4b build(deps): bump actions/download-artifact from 4 to 5
Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 4 to 5.
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](https://github.com/actions/download-artifact/compare/v4...v5)

---
updated-dependencies:
- dependency-name: actions/download-artifact
  dependency-version: '5'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
(cherry picked from commit c44f3e4699)
2025-08-11 17:45:43 +02:00
dependabot[bot]
8704c56787 build(deps): bump cachix/install-nix-action from 31.4.1 to 31.5.2
Bumps [cachix/install-nix-action](https://github.com/cachix/install-nix-action) from 31.4.1 to 31.5.2.
- [Release notes](https://github.com/cachix/install-nix-action/releases)
- [Changelog](https://github.com/cachix/install-nix-action/blob/master/RELEASE.md)
- [Commits](f0fe604f8a...fc6e360bed)

---
updated-dependencies:
- dependency-name: cachix/install-nix-action
  dependency-version: 31.5.2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
(cherry picked from commit b455a0a294)
2025-08-11 17:45:40 +02:00
Wolfgang Walther
6275050154 workflows/pr: run in pull_request context to test actions/get-merge-commit
This must run in the pull_request context, otherwise changes to the
actions are not tested.

(cherry picked from commit a2a9984ed9)
2025-08-10 14:29:42 +00:00
Wolfgang Walther
63301f9889 .github/workflows: set timeouts
None of our jobs is expected to run for 6 hours, the GitHub limit. These
limits are generous and take into accounts that some jobs need to wait
for others.

If jobs exceed these times, most likely something else is wrong and
needs investigation.

(cherry picked from commit 436d54174d)
2025-08-09 19:11:02 +00:00
Wolfgang Walther
e278610b7c Revert "workflows/eval: disable swap"
This reverts commit f2648b263b.

While the idea to never use swap was fine, in practice this meant that
when nix ran OOM, some other process was killed instead. This lead to
the job not being possible to be cancelled anymore and thus needing to
timeout, before subsequent jobs could be scheduled. This can take up to
6 hours for GitHub Actions by default.

Re-enabling the swap file to catch this case more gracefully. It's still
the goal to never actually *use* the swap file during Eval and just a
safeguard.

Keeping the changed chunkSize and not reverting it - this makes it
slightly less likely to hit the swap file when running with Lix.

(cherry picked from commit 9cde368b4c)
2025-08-09 19:11:02 +00:00
Wolfgang Walther
fe803fed24 workflows/eval: disable swap
Recent performance tests show that (a) swapping heavily slows down the
Eval job, while (b) lowering the chunkSize does not have an effect on
run-time. It does on memory usage, though - thus we can get rid of
swapping entirely by reducing chunkSize respectively.

(cherry picked from commit f2648b263b)
2025-08-06 16:33:02 +00:00
Wolfgang Walther
d3a5485375 workflows/build: build nixpkgs tarball
This adds a build job for the tarball, which might help uncover eval
issues on attributes not normally touched by Eval, aka those added in
`pkgs/top-level/packages-config.nix`.

(cherry picked from commit bfb20b9fea)
2025-07-24 10:34:29 +00:00
Wolfgang Walther
ca425fc98d ci/github-script/commits: block on errors
Most of the checks we do for cherry-picks are dismissable warnings, with
one exception: When a commit hash has been found, but this hash is not
available in any of the pickable branches, we raise this with
severity=error. This should also *block* the merge and not be
dismissable. That's because this is a fixable issue in every case.

(cherry picked from commit 1fbcad0434)
2025-07-17 10:12:42 +00:00
Wolfgang Walther
7f19a6e408 ci/github-script/commits: init from ci/check-cherry-picks
This turns the check-cherry-pick script into a github-script based
JavaScript program. This makes it much easier to extend to check reverts
or merge commits later on.

(cherry picked from commit b46cb23251)
2025-07-16 10:26:08 +00:00
Wolfgang Walther
6f32a5f04e ci/github-script: default to commonjs
Since all github-scripts need to be written in commonjs, we now default
to it by not setting package.json. Support from editors for .js files is
slightly better than .cjs. To still allow using module imports in the
test runner script, we trick node into loading the script itself as a
module again via `--import ./run`.

(cherry picked from commit d11eba1e1d)
2025-07-15 12:57:57 +00:00
Wolfgang Walther
d202219502 ci/github-script: move from ci/labels
This just moves things around to use less specific naming - `labels` is
only *one* script that can potentially be run locally while still being
written in github-script. Later, we can add more.

(cherry picked from commit 6f6c625026)
2025-07-15 12:57:57 +00:00
Wolfgang Walther
10d378b990 workflows/backport: fix concurrent jobs cancelling each other
When a PR is merged and labeled afterwards - with a non-backport label -
the following will happen:
- The first backport job is triggered on the merge.
- The second backport job is triggered on the label event.
- The second job will cancel the first one due to the concurrency group.
- The second job will cancel itself because the label event didn't
contain a backport label.

Both jobs end up cancelled and no backport happens.

We made the backport action idempotent upstream a while ago, so we don't
need to cancel those actions. Instead, we'll run all of them -
subsequent actions running through will just stay silent anyway.

(cherry picked from commit 58a3001a3a)
2025-07-12 16:37:15 +00:00
Michael Daniels
9ccb036e43 workflows/build: be clearer about what is being built
Committers could get the false impression from, e.g., `PR / Build / aarch64-linux` that this workflow builds the packages changed in the current PR. Such a misunderstanding could pair poorly with the "enable auto-merge" button, once that's enabled.

(cherry picked from commit 261bba1fcd)
2025-07-11 15:50:18 +00:00
Wolfgang Walther
d27bd954e6 ci/labels: run in dry mode locally
To avoid mistakes when developing and testing against the upstream repo.

(cherry picked from commit dd8357185a)
2025-07-09 07:07:33 +00:00
Wolfgang Walther
79af7b5419 ci/labels: init from workflows/labels
Moves the labels job into a separate ci/ subfolder to run it locally.
This eases debugging *a lot*.

(cherry picked from commit 89ee8975ab)
2025-07-09 07:07:29 +00:00
Wolfgang Walther
6f570b6fb8 workflows/labels: small refactor
To avoid having a diff when moving the file in the next commit.

(cherry picked from commit e90c62d5ab)
2025-07-09 07:07:29 +00:00
Wolfgang Walther
1e97b82fea workflows/labels: manage "needs: reviewer" label
This label allows finding pull requests which have no reviewer
requested, yet.

(cherry picked from commit 7900a1618f)
2025-07-04 12:35:01 +00:00
Wolfgang Walther
013a07e85c workflows/labels: paginate with cursor
Pagination via cursor is required above 10k items. To do so, we store
the current cursor as an artifact and read it back in in the next
scheduled run.

(cherry picked from commit 06a88df620)
2025-07-03 11:08:04 +00:00
Wolfgang Walther
fb77037d03 Reapply "workflows/labels: label stale issues"
This reverts commit c18e94361e.

(cherry picked from commit 23a32c9445)
2025-07-03 11:08:04 +00:00
Wolfgang Walther
e80b7744ac .github/workflows/README.md: one sentence per line
(cherry picked from commit 3d505c0361)
2025-06-29 21:35:59 +02:00
Wolfgang Walther
96ad003d2d workflows/labels: retry on transient API failures
Currently, the labels job fails a few times each day with network
failures. Retrying the requests should help.

(cherry picked from commit 1818027916)
2025-06-27 10:09:58 +00:00
Wolfgang Walther
4faa36a97d workflows/labels: label rebuilds on failed PR workflow
We already tried to fix this case earlier, but didn't account for all
cases: A scheduled workflow can also encounter a pull request with
failed PR workflow. This failure doesn't need to be in the Eval part, so
artifacts could *still* be available. To make sure PRs always get
rebuild labels, just ignore the status condition. Either the artifact is
there, or it is not.

(cherry picked from commit 3be9e2afc1)
2025-06-27 10:09:58 +00:00
Wolfgang Walther
ad1cfc34f3 workflows/labels: slightly improve logging
(cherry picked from commit 4e9df2fc31)
2025-06-27 10:09:58 +00:00
Wolfgang Walther
28ef24b4c2 workflows/labels: fix processing the 100 oldest PRs
The `page` number is 1-based, but the remainder might very well be 0.
This lead to not looking at the 100 oldest PRs, ever.

(cherry picked from commit 10c63e5117)
2025-06-27 10:09:58 +00:00