Instead of approximating how many requests we can still make and hoping
the best that concurrent jobs won't break the bank, we can just work
with the actual limits. By updating regularly, we make sure that
concurrent jobs are taken into account. We always keep a reserve of 1000
requests to make sure *any* non-labeling jobs using this app will always
succeed.
This will allow us to easily backfill labels across multiple days,
especially taking the increased rate limit for the app into account.
With this, we should get up to 11.5k requests per hour in.
(cherry picked from commit 24e7e47c91)
This gives us a much higher rate limit of 12.5k requests per hour. If
the app is not available, we fallback to the regular `github.token`.
This can happen when testing in forks without setting up an app.
(cherry picked from commit acc1c0ae59)
It makes no sense to check cherry picks or request reviewers on PRs from
staging-next to master, the same on backbranches or for haskell-updates
into staging.
(cherry picked from commit a3ce5970e0)
Some jobs purposefully only run on certain base or head branches. By
centralizing the logic, parts of it can easily be re-used later. Also,
this gives them an explicit name and thus makes them easier to
understand.
(cherry picked from commit 7763be5a80)
This will give us a better idea about:
- Which jobs use the most API calls and can possibly be made more
efficient.
- Which rate limits apply exactly to which tokens.
(cherry picked from commit 356bf98a32)
Instead of deleting each label separately and then making another call
to add new labels, this replaces all labels at once, thus saving API
calls in some cases. Also, the labels are now managed in object-style
compared to the array-style before. This allows putting all the
knowledge about each label into a single place instead of in multiple
places. For example, the rebuild labels had to be special cased in the
workflow before - and the nix code to compare had to match that. Also,
the approval labels had to be considered in the `before` and `after`
phases.
The next commit shows how easy it is to add a new label now.
(cherry picked from commit 5b5b18c875)
This can only happen if the target branch's eval run failed and we're
trying to fix it. In this case, Eval should not fail, even though it
might not be able to do a comparison and add the correct rebuild labels.
But if we failed here, we wouldn't be able to merge the fix once we have
required status checks.
(cherry picked from commit 83ce9e7b44)
When we made the switch from eval.yml to pr.yml we adjusted the labels
job as well - but didn't take into account that we also need to deal
with old PRs at the same time.
Here, we fallback to another API request to get a run for eval.yml when
we can't find one for pr.yml.
(cherry picked from commit d00d3190d6)
When labels.yml is changed the new code is not tested in the PR
directly, yet, because we forgot to add labels.yml to the list of
pull_request files in pr.yml. This lead to one syntax error merged
already.
(cherry picked from commit 155ea15a38)
First data shows, that we're unlikely to need more than 250 within an
hour of regular activity. Once this is empty, we'll need to wait until
the next hourly refill - thus, we'll rather set this a bit higher to be
on the safe side.
The hourly limit is at 5000 and we peaked around 3500, so far. We'll
certainly have to look into reducing API calls, but this should still
work out for now.
(cherry picked from commit 8e1f869261)
When we switched to a scheduled workflow, we also changed these lines to
take the labels directly from the pull request list we iterate over. At
the time it saved us an API request. Meanwhile, we have introduced
throttling to the workflow and this causes a potential race condition:
When the scheduled or manually triggered workflow is kicked off and
empties its reservoir of API requests it might be blocked to wait up
to.. an hour! If this happens, the labels taken from the pull request
list might already be outdated at the time the workflow continues. This
would cause some labels to be reset to their original state, which could
be wrong if, for example, another push has happened in the meantime.
This will have a much bigger impact after the next commit, where *all*
labels are set every time, thus the `before` part must be accurate.
Fetching the current labels right before managing them reduces this risk
significantly.
(cherry picked from commit 9581b0c55b)
This allows *not* depending on those two jobs with the required status
checks in the next commit, which wouldn't really make sense. If labeling
or pinging maintainers fails for obscure reasons or because the GitHub
API is down, a PR might still pass all other tests and be
merge-eligible.
(cherry picked from commit 9927d758e7)
This fixes a problem where each workflow would get their own merge
commit. This happens frequently when the target branch is merged into a
the same time, different workflows in the same run will run
get-merge-commit at different times and thus have different merge
commits.
Since the jobs don't really depend on each other, this doesn't cause
practical problems, yet. But it has already led to strange CI failures
in a still unmerged PR, which can be prevented from happening with this
clean approach.
And yes, this saves a few API calls on every run.
(cherry picked from commit 09ddb1a8a0)
This is only a refactor at this stage, but split into a separate commit
for better review. It's the base for the next two commits.
(cherry picked from commit 9422f30e47)
When the job is run with the pull_request trigger for validation of
changes to the workflow itself, we need to run everything that can be
run without privileges - but not more.
We tried to do so for the three actions/labeler steps, but failed to set
up the condition correctly. We also need to exit early for our
JavaScript based labeler, just before making the mutation requests.
(cherry picked from commit 8ab44fec37)
The overall idea is to use names short enough to fit into the status
checks list without shortening. This change mostly happened in the
commits before, here we just follow the same pattern for the remaining
workflows.
(cherry picked from commit c08b86e962)
We previously moved this out of the main eval workflow to avoid running
it on push and/or undrafting the PR. The latter has been removed in the
meantime and the former can be checked with a simple condition. Thus we
move it back in, to make it part of the 4 main workflows, which will be
required before merge eventually.
(cherry picked from commit 008527d7cd)
Those two workflows bundle all the main jobs in two event-specific
wrapper workflows. This enables us to do two things later on:
- Synchronize the merge commits between most of the jobs run in a PR.
- Create a single "required" job to be targeted by GitHub's "required
status checks to pass" feature.
(cherry picked from commit 959eed1f2a)
This should prevent us from hitting API rate limits, even when we run
huge manual jobs. It just takes a bit longer.
(cherry picked from commit 114b4fcf48)
This will give us much quicker approval labeling, but still need much
less resources than before, when we ran on every PR comment.
(cherry picked from commit 656b53b0dd)
This can still be manually dispatched for testing in forks, but it's
entirely useless to keep running it on schedule.
Also removing the "skip treewide" condition, which was a left-over and
removed everywhere else already. We don't want to skip any jobs,
especially not when considering required status checks.
(cherry picked from commit 075dc097a3)
Due to a type mismatch, maintainer approvals were never counted as such.
The API returns integers for the user IDs, but the JSON file has strings
as object keys.
(cherry picked from commit 25a0ee0817)
We don't need to handle the differently named artifacts in a special
way, because they have been expired anyway. But, we must handle the case
to not cause the job to fail.
(cherry picked from commit 5343c50acd)
Printing the URL to the PR in the logs allows clicking on it directly in
the GitHub runner logs for easy reference.
(cherry picked from commit 022bbfd663)
Instead of relying on workflow_run events, we now trigger the labeling
workflow by schedule. This avoids all permission/secrets problems of
running in the pull_request_review context - and also gets rid of the
"waiting for approval to run workflows" button for new contributors that
sometimes comes up right now.
Also, it's more efficient. Previously, the labeling workflow would run
on *any* pull_request_review event, which happens for all comments, too.
So quite a few runs.
This will cause a delay of up to 1 hour with the current settings until
approval labels are set. Depending on how long the job normally runs we
can adjust the frequency. The workflow is written in a way that will
work no matter what the frequency ends up being, even when it's
interrupted by transient GHA failures: It will always look at all PRs
which were updated since the last time the workflow ran successfully.
We also add the ability to run it manually via UI. This is useful:
- When fixing bugs in the labeler workflow to run it all the way back to
where the bug was introduced.
- When introducing new labels, to get a head start for a reasonable
amount of PRs immediately.
Of course, the workflow is also still run directly after Eval itself.
This is also the only case that the actions/labeler steps will run,
since they depend on the `pull_request` context.
(cherry picked from commit 6f12f662ae)
Most workflow files are already well formatted, but to make it easier to
keep it that way, we can add yamlfmt.
I personally have a preference for non-indented arrays for YAML, but
wanted to avoid bigger diffs here - the status-quo clearly are indented
arrays.
Some changes are made manually to the get-merge-commit action and the
issue templates. Those would otherwise make yamlfmt misbehave on those.
(cherry picked from commit 8ec23df6f6)
This currently fails with:
```
Method Set.prototype.has called on incompatible receiver undefined
```
Seems like my syntax test previously only hit the case without
maintainers, in which case it doesn't throw :/.
(cherry picked from commit 4b9fb45060)
This didn't work as intended. When a workflow is run with
`workflow_call`, it will have `github.workflow` set to the *parent*
workflow. So the `caller` input that we passed, resulted in this
concurrency key:
```
Eval-Eval-...
```
But that's bad, because the labels and reviewers workflows will cancel
each other!
What we actually want is this:
- Label and Reviewers workflow should have different groups.
- Reviewers called via Eval and called directly via undraft should have
*different* groups.
We can't use the default condition we use everywhere else, because
`github.workflow` is the same for Label and Reviewers. Thus, we hardcode
the workflow's name as well. This essentially means we have this as a
key:
```
<name-of-running-workflow>-<name-of-triggering-workflow>-<name-of-event>-<name-of-head-branch>
```
This should do what we want.
Since workflows can be made reusable workflows later on, we add those
hardcoded names to *all* concurrency groups. This avoids copy&paste
errors later on.
(cherry picked from commit 6793e238fa)
This can happen when two PRs run at the same time, which come from
different forks, but have the same head branch name.
github.head_ref is suggested by GitHub's docs, but.. that's not really
useful for cases with forks.
(cherry picked from commit 7ba7720b28)
The category 12 labels for number of approvals and approved by package
maintainer are now automatically managed by the labels workflow. The
logic is slightly different from the "by: package-maintainer" label. For
approval, it's enough if *any one* maintainer approves the PR to have
the label added, even if the PR touches multiple packages.
The workflow only counts approved reviews, no matter whether there had
been a push in the meantime or not. To achieve the currently used logic
of "expiring approvals after push", we will have to set up a branch
protection rule, which actually dismissed those reviews automatically.
(cherry picked from commit 5f09e16f00)
This moves the actual labeling from the eval workflow to the labels
workflow. At this stage, this only has a disadvantage: Adding the
topic-labels to the pull request will now only happen after eval has
finished, instead of instantly.
We will only benefit from this later, when we manage approval related
events. With this change, we will have the comparison results and thus
the package maintainer info available.
(cherry picked from commit 2d0bcd7165)
This brings back the "minimize CI reviews after dismissal" job that was
previously removed. The first time around, we had a single job triggered
by the `pull_request_review` event. This lacks permission to do
meaningful stuff, though.
This time, we trigger an empty no-op job on `pull_request_review` and
then run a second workflow on `workflow_run`. This can run with the
proper permissions.
(cherry picked from commit a34a22d8b9)
