If, for any reason (e.g. server crash), a task is recorded as done in the database but the logs are still in the database instead of being in storage, they need to be collected.
The log_in_storage field is only set to true after the logs have been transfered to storage and can be relied upon to reflect which tasks have lingering logs.
A cron job collects lingering logs every day, 3000 at a time, sleeping one second between them. In normal circumstances there will be only a few of them, even on a large instance, and there is no need to collect them as quickly as possible.
When there are a lot of them for some reason, garbage collection must happen at a rate that is not too hard on storage I/O.
Refs https://codeberg.org/forgejo/forgejo/issues/9999
---
Note on backports: the v11 backport is done manually because of minor conflicts. https://codeberg.org/forgejo/forgejo/pulls/10024
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests
- I added test coverage for Go changes...
- [x] in their respective `*_test.go` for unit tests.
- [x] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] I do not want this change to show in the release notes.
- [x] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
<!--start release-notes-assistant-->
## Release notes
<!--URL:https://codeberg.org/forgejo/forgejo-->
- Bug fixes
- [PR](https://codeberg.org/forgejo/forgejo/pulls/10009): <!--number 10009 --><!--line 0 --><!--description Z2FyYmFnZSBjb2xsZWN0IGxpbmdlcmluZyBhY3Rpb25zIGxvZ3M=-->garbage collect lingering actions logs<!--description-->
<!--end release-notes-assistant-->
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/10009
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.codeberg.org>
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: Earl Warren <contact@earl-warren.org>
Co-committed-by: Earl Warren <contact@earl-warren.org>
Although #9922 was deployed to Codeberg, it was reported on Matrix that a user observed a `-1` pull request count.
@Gusted checked and verified that the stats stored in redis appeared incorrect, and that no errors occurred on Codeberg that included the repo ID (eg. deadlocks, SQL queries).
```
127.0.0.1:6379> GET Repo:CountPulls:924266
"1"
127.0.0.1:6379> GET Repo:CountPullsClosed:924266
"2"
```
One possible cause is that when `UpdateRepoIssueNumbers` is invoked and invalidates the cache key for the repository, it is currently in a transaction; the next request for that cached count could be computed before the transaction is committed and the update is visible. It's been verified that `UpdateRepoIssueNumbers` is called within a transaction in most interactions (I put a panic in it if `db.InTransaction(ctx)`, and most related tests failed).
This PR fixes that hole by performing the cache invalidation in an `AfterTx()` hook which is invoked after the transaction is committed to the database.
(Another possible cause is documented in #10127)
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/10130
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
- Similair spirit of forgejo/forgejo!7453.
- Refactor the code in such a way that it always succeeds.
- To avoid doing mathematics if you use this function, define three security level (64, 128 and 256 bits) that correspond to a specific length which has that a security guarantee. I picked them as they fit the need for the existing usages of the code.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/10110
Reviewed-by: Michael Kriese <michael.kriese@gmx.de>
Reviewed-by: Lucas <sclu1034@noreply.codeberg.org>
Co-authored-by: Gusted <postmaster@gusted.xyz>
Co-committed-by: Gusted <postmaster@gusted.xyz>
Forgejo Actions allows variables in `jobs.<job_id>.runs-on`. However, the action list [skips checking whether a suitable runner](c3412d0579/routers/web/repo/actions/actions.go (L114-L148)) is available if an expression contains variables. That hampers a user's ability to figure out whether an expression was evaluated correctly and why a job might not be picked up by an available runner.
This PR adds the ability to surface more complex and additional diagnostic information on the action view screen. Previously, only a job's status (waiting, running, ...) was displayed. Now, extended messages like "Waiting for a runner with the following labels: docker, trixie" are displayed with the possibility to show multiple messages simultaneously.
How it looked before:
How it looks after updating Forgejo without reloading the window:
How it looks afterwards with a single label:
How it looks afterwards with multiple labels:
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests
- I added test coverage for Go changes...
- [x] in their respective `*_test.go` for unit tests.
- [x] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
- [x] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] I do not want this change to show in the release notes.
- [ ] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
<!--start release-notes-assistant-->
## Release notes
<!--URL:https://codeberg.org/forgejo/forgejo-->
- Features
- [PR](https://codeberg.org/forgejo/forgejo/pulls/9966): <!--number 9966 --><!--line 0 --><!--description ZGlzcGxheSBkZXRhaWxlZCBhY3Rpb24gcnVuIGRpYWdub3N0aWNz-->display detailed action run diagnostics<!--description-->
<!--end release-notes-assistant-->
Co-authored-by: Mathieu Fenniak <mfenniak@noreply.codeberg.org>
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/9966
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.codeberg.org>
Co-authored-by: Andreas Ahlenstorf <andreas@ahlenstorf.ch>
Co-committed-by: Andreas Ahlenstorf <andreas@ahlenstorf.ch>
Previously, the variable value was empty when asking for a list of all repository variables.
I have extended the test coverage of all related endpoints and made the existing tests independent of each other.
<!--start release-notes-assistant-->
## Release notes
<!--URL:https://codeberg.org/forgejo/forgejo-->
- Other changes without a feature or bug label
- [PR](https://codeberg.org/forgejo/forgejo/pulls/10036): <!--number 10036 --><!--line 0 --><!--description aW5jbHVkZSB2YXJpYWJsZSB2YWx1ZXMgaW4gL3JlcG9zLy4uLi9hY3Rpb25zL3ZhcmlhYmxlcyBBUEkgcmVzcG9uc2U=-->include variable values in /repos/.../actions/variables API response<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/10036
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.codeberg.org>
Co-authored-by: Andreas Ahlenstorf <andreas@ahlenstorf.ch>
Co-committed-by: Andreas Ahlenstorf <andreas@ahlenstorf.ch>
ActionUser is to keep track of pull requests posters that are
permanently trusted. It has a used field to track when it was last
used so records can be expired instead of accumulating forever.
ActionRun has new fields to make it possible to look them up given
either the pull request ID or the poster ID.
instead of code.gitea.io/actions-proto-go
It is a hard fork of code.gitea.io/actions-proto-go which has been used by the runner in the past few months.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/9981
Reviewed-by: Michael Kriese <michael.kriese@gmx.de>
Co-authored-by: Earl Warren <contact@earl-warren.org>
Co-committed-by: Earl Warren <contact@earl-warren.org>
When `concurrency.cancel-in-progress: false` is set on a Forgejo Action run, the run will be queued behind other runs of the same concurrency group. This requires a complex query to identify when a run can be dispatched to a runner. During a refactoring of this code in #9927, @earl-warren noted that the query was using a `status NOT IN (...)` clause, which we had discussed in another issue as a possible performance concern -- an index on `status` cannot be used to fulfill that query clause. This PR inverts the clause allowing an index to be used.
Covered by existing automated tests in `tests/integration/actions_concurrency_group_queue_test.go` -- a minor commented-out section was removed in these tests while reviewing for this change.
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests
- I added test coverage for Go changes...
- [ ] in their respective `*_test.go` for unit tests.
- [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [x] I do not want this change to show in the release notes.
- Minor implementation change for a feature that is new in the v14 release; not relevant for the release notes.
- [ ] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/9943
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
Fixes#9846. The number of open action runs on a repo is not precomputed and stored on the repo, but is computed as needed and cached. The computation is faster than the update because it only calculates the smaller set of which action runs are not completed (as opposed to counting all of them).
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests
- I added test coverage for Go changes...
- [x] in their respective `*_test.go` for unit tests.
- [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] I do not want this change to show in the release notes.
- [x] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/9927
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
Fixes#9755, performance regression from #9017.
Manually tested by application to a development database and verification that the index is used on `select * from action_task where job_id = 11 and attempt = 1`. I've made the index here cover just the `job_id` field, as the set of values for `attempt` is expected to be very small.
Will backport via just the `models/actions/task.go` change (allowing `SyncAllTables` to create it) to avoid a schema migration during the backport (complicated due to the new schema migration module in v14).
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests
- I added test coverage for Go changes...
- [ ] in their respective `*_test.go` for unit tests.
- [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] I do not want this change to show in the release notes.
- [x] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
<!--start release-notes-assistant-->
## Release notes
<!--URL:https://codeberg.org/forgejo/forgejo-->
- Bug fixes
- [PR](https://codeberg.org/forgejo/forgejo/pulls/9789): <!--number 9789 --><!--line 0 --><!--description Zml4KHBlcmYpOiBhZGQgbWlzc2luZyBpbmRleCBvbiBhY3Rpb25fdGFzayB0YWJsZQ==-->fix(perf): add missing index on action_task table<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/9789
Reviewed-by: Michael Kriese <michael.kriese@gmx.de>
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
This patch contains two fixes/enhancements to two functions that were updating all columns of the `access_token` and `repository` table when they were only updating a select few columns. Within Codeberg we saw these two queries quite often when something problematic with the database was going on, likely because of this all columns update pattern. `UpdateAccessToken` is removed and a new function `UpdateLastUsed` was added, for `updateRepoRunsNumbers` we can simply add which columns we want to have updated in that query.
It's likely there are more of such queries, but these were the ones being executed often.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/9572
Reviewed-by: Michael Kriese <michael.kriese@gmx.de>
Reviewed-by: Otto <otto@codeberg.org>
Co-authored-by: Gusted <postmaster@gusted.xyz>
Co-committed-by: Gusted <postmaster@gusted.xyz>
Currently references a pre-release version of `code.forgejo.org/forgejo/runner/v11`, pending release of https://code.forgejo.org/forgejo/runner/pulls/1026.
Fixes#5914.
This PR is quite large, but it can be reviewed commit-by-commit in relatively small, logical chunks.
Adds support for workflows with a `concurrency` block, and submembers `group` and `cancel-in-progress`. For example:
```
on:
workflow_dispatch:
jobs:
rust-checks:
runs-on: debian-latest
steps:
- run: sleep 300
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: false
```
The concurrency block effectively ends up with four supported behaviors that users will want to choose from:
- Backwards compatibility / default -- if omitted completely, the existing Forgejo behavior will be implemented. That behavior is that push and pull request synchronize events will cancel all previous runs on the same repository, branch, and workflow.
- Unlimited concurrency -- if the `cancel-in-progress` value is set to `false` and no `group` is provided, then the previously described Forgejo behavior will be disabled and an unlimited number of workflows can be executed simultaneously (to the maximum supported by the Forgejo Runner capacity).
- Queue-behind -- if a `group` is provided and `cancel-in-progress: false` is set, then every new action run with in the same repository with the same group value will be queued behind previous workflow runs, allowing only one workflow to execute at a time in the group, but allowing all workflows to finish naturally.
- Cancel-in-progress -- if a `group` is provided and `cancel-in-progress: true` is set, then every new action run with in the same repository with the same group value will cause previously queued or running runs to be cancelled, allowing only one workflow to execute at a time in the group, but preferring execution of the most recent workflow.
Both the `group` and `cancel-in-progress` values can access values from the `github`, `inputs` and `vars` context for dynamic behavior.
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests
- I added test coverage for Go changes...
- [x] in their respective `*_test.go` for unit tests.
- [x] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [x] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- https://codeberg.org/forgejo/docs/pulls/1513
- [ ] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] I do not want this change to show in the release notes.
- [x] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
<!--start release-notes-assistant-->
## Release notes
<!--URL:https://codeberg.org/forgejo/forgejo-->
- Features
- [PR](https://codeberg.org/forgejo/forgejo/pulls/9434): <!--number 9434 --><!--line 0 --><!--description aW1wbGVtZW50ICJjb25jdXJyZW5jeSIgYmxvY2sgaW4gRm9yZ2VqbyBBY3Rpb25zIGF0IHRoZSB3b3JrZmxvdyBsZXZlbA==-->implement "concurrency" block in Forgejo Actions at the workflow level<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/9434
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
Discovered a regression caused by #9017.
Steps to reproduce:
- Disable the forgejo-runner that will pick up a workflow
- Trigger any workflow to run
- Through the Actions list, click on the new workflow that is pending a runner to fetch it
- You'll be redirected to /user/repo/actions/runs/73/jobs/0/attempt/0 (attempt = 0)
- The UI will appear normal with the job "Waiting"...
- Startup the forgejo-runner to pick up the workflow
- The UI will begin to have errors:
- JavaScript promise rejection: JSON.parse: unexpected keyword at line 1 column 1 of the JSON data. Open browser console to see more details. (5)
The cause is that the redirect to `/attempt/0` occurs for a job that hasn't been started, but once the job is started attempt 0 is not a valid attempt and errors will occur when polling for data. This fix corrects the problem by redirecting to the attempt that will be present (attempt 1) when the job is fetched.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/9293
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
This PR contains the following updates:
| Package | Change | Age | Confidence |
|---|---|---|---|
| code.forgejo.org/forgejo/runner/v9 | `v9.1.1` -> `v11.0.0` | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) |
---
### Configuration
📅 **Schedule**: Branch creation - Between 12:00 AM and 03:59 AM ( * 0-3 * * * ) (UTC), Automerge - Between 12:00 AM and 03:59 AM ( * 0-3 * * * ) (UTC).
🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box
---
This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0MS43Ni4wIiwidXBkYXRlZEluVmVyIjoiNDEuNzYuMCIsInRhcmdldEJyYW5jaCI6ImZvcmdlam8iLCJsYWJlbHMiOlsiZGVwZW5kZW5jeS11cGdyYWRlIiwidGVzdC9ub3QtbmVlZGVkIl19-->
Co-authored-by: Earl Warren <contact@earl-warren.org>
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/9218
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Reviewed-by: Michael Kriese <michael.kriese@gmx.de>
Co-authored-by: Renovate Bot <forgejo-renovate-action@forgejo.org>
Co-committed-by: Renovate Bot <forgejo-renovate-action@forgejo.org>
Adds a new dropdown to the job logs, visible only when a job has been retried at least once:
When an older run attempt from the dropdown is selected, displays the older run's logs:
Context on implementation & design decisions:
- It is important that when a URL from an Action's log is shared, the person on the other side sees the exact same logs that were being viewed. For this reason, all log views are automatically redirected to a fully-qualified URL (including the *run*, *job*, and *attempt*), so that when they are shared there is a guarantee of stability in the viewed logs.
- Individual jobs can be rerun any number of times independent of other jobs in the same workflow. This means there isn't a "set" of related jobs that were executed at the same time, and this led me to remove the display of current status of jobs on the left-hand side of the view. There isn't a logical set of job statuses to display here.
Fixes#1043. Based upon @gmem's original work in #1416.
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests
- I added test coverage for Go changes...
- [x] in their respective `*_test.go` for unit tests.
- [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
- [x] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] I do not want this change to show in the release notes.
- [x] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
<!--start release-notes-assistant-->
## Release notes
<!--URL:https://codeberg.org/forgejo/forgejo-->
- User Interface features
- [PR](https://codeberg.org/forgejo/forgejo/pulls/9017): <!--number 9017 --><!--line 0 --><!--description YWJpbGl0eSB0byB2aWV3IHByZXZpb3VzIGxvZ3MgZm9yIEFjdGlvbnMgcnVucyB0aGF0IGhhdmUgYmVlbiByZXRyaWVk-->ability to view previous logs for Actions runs that have been retried<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/9017
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
- the run ID used to download artifacts is absolute (ID) instead of being
relative to the repository (Index) for compatibility with the url built
and returned as `artifact-url` by the the upload-artifact@v4 action.
- this is a breaking change because URLs to download artifacts
previous saved/bookmarked and not yet expired expired are no
longer working, they need to be looked up again by visiting the job
web page.
- add unit tests for getRunByID().
- RepoActionView.test.js verifies the download URL is built using the
run ID.
- lAdd integration tests to verify the RunID is set as expected in
the template used by RepoActionView.vue.
Refs https://code.forgejo.org/forgejo/runner/issues/187
The web endpoint
`/{owner}/{repo}/actions/runs/{run_id}/artifacts/{artifact_name_or_id}`
can be used with either the artifact name used when it is uploaded or the instance wide unique number of the artifact, if it is not found. For instance:
`/root/myrepo/actions/run/3/artifacts/my_artifact_name`
or
`/root/myrepo/actions/run/3/artifacts/42`
The `upload-artifact@v4` output value `artifact-url` is built in this way and is now a valid URL to access the artifact.
Refs https://codeberg.org/forgejo/forgejo/issues/6147
Refs https://code.forgejo.org/forgejo/runner/issues/187
Refs https://code.forgejo.org/forgejo/upload-artifact/src/tag/v4#outputs
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests
- I added test coverage for Go changes...
- [x] in their respective `*_test.go` for unit tests.
- [x] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] I do not want this change to show in the release notes.
- [x] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
<!--start release-notes-assistant-->
## Release notes
<!--URL:https://codeberg.org/forgejo/forgejo-->
- Bug fixes
- [PR](https://codeberg.org/forgejo/forgejo/pulls/8957): <!--number 8957 --><!--line 0 --><!--description YXJ0aWZhY3RzIGNhbiBiZSBkb3dubG9hZGVkIHVzaW5nIHRoZWlyIGlkIGluc3RlYWQgb2YgdGhlaXIgbmFtZQ==-->artifacts can be downloaded using their id instead of their name<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/8957
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.codeberg.org>
Co-authored-by: Earl Warren <contact@earl-warren.org>
Co-committed-by: Earl Warren <contact@earl-warren.org>
Should fix failures like: https://codeberg.org/forgejo/forgejo/actions/runs/94872/jobs/9
### Documentation
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [x] I do not want this change to show in the release notes.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/8847
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: oliverpool <git@olivier.pfad.fr>
Co-committed-by: oliverpool <git@olivier.pfad.fr>
- Make the migration to add an index a noop - do not remove it as it would break v12.next & v13.next
- Keep the logic that relies on finding the last run, only always fail to find which is the same as assuming each run is one of a kind
Refs forgejo/forgejo#8373
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/8374
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: Earl Warren <contact@earl-warren.org>
Co-committed-by: Earl Warren <contact@earl-warren.org>
* Add the `notify-email` column / NotifyEmail to ActionRun and set it:
* services/actions/workflows.go `Dispatch`
* services/actions/schedule_tasks.go `CreateScheduleTask`
* services/actions/notifier_helper.go `handleWorkflows`
* Only send an email if the workflow has `enable-email-notifications: true` by having `MailActionRun` return immediately if `NotifyEmail` is false.
* Ignore or silently fail on `enable-email-notifications: true` parsing errors. Reporting such errors belongs in workflow validation, not when it is evaluated for the notifications.
* Add unit and integration tests.
Refs: https://codeberg.org/forgejo/forgejo/issues/8187
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests
- I added test coverage for Go changes...
- [x] in their respective `*_test.go` for unit tests.
- [x] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
### Documentation
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [x] I do not want this change to show in the release notes.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/8242
Reviewed-by: Christopher Besch <mail@chris-besch.com>
Co-authored-by: Earl Warren <contact@earl-warren.org>
Co-committed-by: Earl Warren <contact@earl-warren.org>
Fixes#7646
Adds a cron job to cleanup action runners that have been offline or inactive for X amount of time.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/7803
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: Julian Schlarb <julian.schlarb@denktmit.de>
Co-committed-by: Julian Schlarb <julian.schlarb@denktmit.de>
Hello all, I am finally taking the time to follow-up on #6277 to get this implemented :)
## Checklist
### Tests
- I added test coverage for Go changes...
- [ ] in their respective `*_test.go` for unit tests.
- [x] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] I do not want this change to show in the release notes.
- [x] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
<!--start release-notes-assistant-->
## Release notes
<!--URL:https://codeberg.org/forgejo/forgejo-->
- Bug fixes
- [PR](https://codeberg.org/forgejo/forgejo/pulls/7930): <!--number 7930 --><!--line 0 --><!--description QWRkIGVycm9yIHJlcG9ydGluZyB0byBQUnMgd2l0aCBpbnZhbGlkIHdvcmtmbG93cw==-->Add error reporting to PRs with invalid workflows<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/7930
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Antonin Delpeuch <antonin@delpeuch.eu>
Co-committed-by: Antonin Delpeuch <antonin@delpeuch.eu>
This PR depends on https://codeberg.org/forgejo/forgejo/pulls/7510
This PR renames UpdateRunJob to UpdateRunJobWithoutNotification and UpdateRun to UpdateRunWithoutNotification and implements wrapper functions that also call the new ActionRunNowDone notification when needed.
This PR can be reviewed commit-by-commit.
# Things to Test
- [x] GetRunBefore
- [ ] integration test for sendActionRunNowDoneNotificationIfNeeded, UpdateRun and UpdateRunJob
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests
- I added test coverage for Go changes...
- [x] in their respective `*_test.go` for unit tests.
- [x] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] I do not want this change to show in the release notes.
- [ ] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
Co-authored-by: nobody <nobody@example.com>
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/7491
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: christopher-besch <mail@chris-besch.com>
Co-committed-by: christopher-besch <mail@chris-besch.com>
This enables all action run state changes (from a not done to a done
state) to also send a notification.
Moved these:
- models/actions/task.go|423 col 6| func StopTask(ctx context.Context, taskID int64, status Status) error {
- models/actions/run.go|190 col 6| func CancelPreviousJobs(ctx context.Context, repoID int64, ref, workflowID string, event webhook_module.HookEventType) error {
- models/actions/schedule.go|122 col 6| func CleanRepoScheduleTasks(ctx context.Context, repo *repo_model.Repository, cancelPreviousJobs bool) error {
- Unify the usage of [`crypto/rand.Read`](https://pkg.go.dev/crypto/rand#Read) to `util.CryptoRandomBytes`.
- Refactor `util.CryptoRandomBytes` to never return an error. It is documented by Go, https://go.dev/issue/66821, to always succeed. So if we still receive a error or if the returned bytes read is not equal to the expected bytes to be read we panic (just to be on the safe side).
- This simplifies a lot of code to no longer care about error handling.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/7453
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Gusted <postmaster@gusted.xyz>
Co-committed-by: Gusted <postmaster@gusted.xyz>
Fix the problem of untranslated text on the actions page
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
(cherry picked from commit ce65613690e4564d9961f847ebd6eb2137f0c885)
The web route to update and delete variables of runners did not check if
the ID that was given belonged to the context it was requested in, this
made it possible to update and delete every existing runner variable of
a instance for any authenticated user.
The code has been reworked to always take into account the context of
the request (owner and repository ID).
The web route to delete action runners did not check if the ID that was
given belonged to the context it was requested in, this made it possible
to delete every existing runner of a instance by a authenticated user.
The code was reworked to ensure that the caller of the delete
runner function retrieved the runner by ID and then checks if it belongs
to the context it was requested in, although this is not an optimal
solution it is consistent with the context checking of other code for
runners.
This PR wants to improve information of the tasks waiting to be executed on a global, organization, user and repository leve.
The main motivation is explained here https://codeberg.org/forgejo/discussions/issues/241
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests
- I added test coverage for Go changes...
- [x] in their respective `*_test.go` for unit tests.
- [x] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] I do not want this change to show in the release notes.
- [x] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
<!--start release-notes-assistant-->
## Release notes
<!--URL:https://codeberg.org/forgejo/forgejo-->
- Features
- [PR](https://codeberg.org/forgejo/forgejo/pulls/6300): <!--number 6300 --><!--line 0 --><!--description QWRkIHNlYXJjaCBhY3Rpb24gam9icyBmb3IgQVBJIHJvdXRlcywgcmVwbywgb3JnIGFuZCBnbG9iYWwgbGV2ZWw=-->Add search action jobs for API routes, repo, org and global level<!--description-->
<!--end release-notes-assistant-->
Co-authored-by: jaime merino <jaime.merino_mora@mail.schwarzª>
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/6300
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Jaime merino <cobak78@gmail.com>
Co-committed-by: Jaime merino <cobak78@gmail.com>
Make the result the same as GitHub:
* all skipped, then result is skipped
* any cancelled, then result cancelled
(cherry picked from commit 22c4599542ee3e10bcab4c9136467bbac8e90ba0)
fix#32857
(cherry picked from commit d28a4843b8de5d5e01ef3d7b2ad25f22853247ad)
Conflicts:
web_src/js/components/ActionRunStatus.vue
remove the refactoring, keep the additional cancelled status
Fix#32795
If a job uses a matrix, multiple `ActionRunJobs` may have the same
`JobID`. We need to merge the outputs of these jobs to make them
available to the jobs that need them.
(cherry picked from commit 7269130d2878d51dcdf11f7081a591f85bd493e8)
Conflicts:
models/fixtures/action_run.yml
models/fixtures/action_run_job.yml
trivial context conflicts
When a workflow has
on:
pull_request:
types:
- labeled
- unlabeled
The outcome of the workflow (success or failure) must be associated
with the head sha commit status. Otherwise it cannot be used as a
requirement for merging the pull request (branch protections).
Support compression for Actions logs to save storage space and
bandwidth. Inspired by
https://github.com/go-gitea/gitea/issues/24256#issuecomment-1521153015
The biggest challenge is that the compression format should support
[seekable](https://github.com/facebook/zstd/blob/dev/contrib/seekable_format/zstd_seekable_compression_format.md).
So when users are viewing a part of the log lines, Gitea doesn't need to
download the whole compressed file and decompress it.
That means gzip cannot help here. And I did research, there aren't too
many choices, like bgzip and xz, but I think zstd is the most popular
one. It has an implementation in Golang with
[zstd](https://github.com/klauspost/compress/tree/master/zstd) and
[zstd-seekable-format-go](https://github.com/SaveTheRbtz/zstd-seekable-format-go),
and what is better is that it has good compatibility: a seekable format
zstd file can be read by a regular zstd reader.
This PR introduces a new package `zstd` to combine and wrap the two
packages, to provide a unified and easy-to-use API.
And a new setting `LOG_COMPRESSION` is added to the config, although I
don't see any reason why not to use compression, I think's it's a good
idea to keep the default with `none` to be consistent with old versions.
`LOG_COMPRESSION` takes effect for only new log files, it adds `.zst` as
an extension to the file name, so Gitea can determine if it needs
decompression according to the file name when reading. Old files will
keep the format since it's not worth converting them, as they will be
cleared after #31735.
<img width="541" alt="image"
src="https://github.com/user-attachments/assets/e9598764-a4e0-4b68-8c2b-f769265183c9">
(cherry picked from commit 33cc5837a655ad544b936d4d040ca36d74092588)
Conflicts:
assets/go-licenses.json
go.mod
go.sum
resolved with make tidy
