jojo/services/actions
Earl Warren 238ecfdeb8 fix: garbage collect lingering actions logs (#10009)
If, for any reason (e.g. server crash), a task is recorded as done in the database but the logs are still in the database instead of being in storage, they need to be collected.

The log_in_storage field is only set to true after the logs have been transfered to storage and can be relied upon to reflect which tasks have lingering logs.

A cron job collects lingering logs every day, 3000 at a time, sleeping one second between them. In normal circumstances there will be only a few of them, even on a large instance, and there is no need to collect them as quickly as possible.

When there are a lot of them for some reason, garbage collection must happen at a rate that is not too hard on storage I/O.

Refs https://codeberg.org/forgejo/forgejo/issues/9999

---

Note on backports: the v11 backport is done manually because of minor conflicts. https://codeberg.org/forgejo/forgejo/pulls/10024

## Checklist

The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).

### Tests

- I added test coverage for Go changes...
  - [x] in their respective `*_test.go` for unit tests.
  - [x] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
  - [ ] in `web_src/js/*.test.js` if it can be unit tested.
  - [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).

### Documentation

- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.

### Release notes

- [ ] I do not want this change to show in the release notes.
- [x] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.

<!--start release-notes-assistant-->

## Release notes
<!--URL:https://codeberg.org/forgejo/forgejo-->
- Bug fixes
  - [PR](https://codeberg.org/forgejo/forgejo/pulls/10009): <!--number 10009 --><!--line 0 --><!--description Z2FyYmFnZSBjb2xsZWN0IGxpbmdlcmluZyBhY3Rpb25zIGxvZ3M=-->garbage collect lingering actions logs<!--description-->
<!--end release-notes-assistant-->

Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/10009
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.codeberg.org>
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: Earl Warren <contact@earl-warren.org>
Co-committed-by: Earl Warren <contact@earl-warren.org>
2025-11-18 18:59:01 +01:00
..
TestActions_CancelOrApproveRun feat: add the actions CancelRun and ApproveRun helpers 2025-11-06 11:07:38 +01:00
TestActionsNotifier_IsTrusted chore(refactor): replace ifNeedApproval with trust management 2025-11-06 11:07:39 +01:00
TestActionsTrust_GetPullRequestUserIsTrustedWithActions feat: trust management for runs created from a forked pull request 2025-11-06 11:07:38 +01:00
TestCancelPreviousJobs feat: implement "concurrency" block in Forgejo Actions at the workflow level (#9434) 2025-10-03 18:43:02 +02:00
TestCancelPreviousWithConcurrencyGroup feat: implement "concurrency" block in Forgejo Actions at the workflow level (#9434) 2025-10-03 18:43:02 +02:00
TestServiceActions_startTask fix: a corrupted Forgejo Actions scheduled workflow is disabled (#8942) 2025-08-18 22:45:10 +02:00
TestServicesActions_TransferLingeringLogs fix: garbage collect lingering actions logs (#10009) 2025-11-18 18:59:01 +01:00
auth.go chore(cleanup): replaces unnecessary calls to formatting functions by non-formatting equivalents (#7994) 2025-05-29 17:34:29 +02:00
auth_test.go Update module github.com/golangci/golangci-lint/cmd/golangci-lint to v2 (forgejo) (#7367) 2025-03-28 22:22:21 +00:00
cleanup.go feat: auto cleanup of offline runners (#7803) 2025-06-08 00:13:37 +02:00
cleanup_test.go Update module github.com/golangci/golangci-lint/cmd/golangci-lint to v2 (forgejo) (#7367) 2025-03-28 22:22:21 +00:00
clear_tasks.go Actions Done Notification (#7491) 2025-04-24 15:15:24 +00:00
commit_status.go Update module code.forgejo.org/forgejo/runner/v9 to v11 (forgejo) (#9218) 2025-09-10 22:02:55 +02:00
context.go feat: implement "concurrency" block in Forgejo Actions at the workflow level (#9434) 2025-10-03 18:43:02 +02:00
context_test.go feat: implement "concurrency" block in Forgejo Actions at the workflow level (#9434) 2025-10-03 18:43:02 +02:00
init.go chore: branding import path (#7337) 2025-03-27 19:40:14 +00:00
interface.go feat: add support for ephemeral runners compatible with autoscaling tools (#9409) 2025-10-01 00:38:35 +02:00
job_emitter.go Update module code.forgejo.org/forgejo/runner/v9 to v11 (forgejo) (#9218) 2025-09-10 22:02:55 +02:00
job_emitter_test.go chore: branding import path (#7337) 2025-03-27 19:40:14 +00:00
log.go fix: garbage collect lingering actions logs (#10009) 2025-11-18 18:59:01 +01:00
log_test.go fix: garbage collect lingering actions logs (#10009) 2025-11-18 18:59:01 +01:00
main_test.go chore: branding import path (#7337) 2025-03-27 19:40:14 +00:00
notifier.go feat: github compatability for removing label from issue API (#8831) 2025-08-30 03:29:23 +02:00
notifier_helper.go chore(refactor): replace ifNeedApproval with trust management 2025-11-06 11:07:39 +01:00
notifier_helper_test.go chore(refactor): replace ifNeedApproval with trust management 2025-11-06 11:07:39 +01:00
rerun.go chore: branding import path (#7337) 2025-03-27 19:40:14 +00:00
rerun_test.go chore: branding import path (#7337) 2025-03-27 19:40:14 +00:00
run.go feat: add the actions CancelRun and ApproveRun helpers 2025-11-06 11:07:38 +01:00
run_test.go feat: add the actions CancelRun and ApproveRun helpers 2025-11-06 11:07:38 +01:00
schedule_tasks.go chore(refactor): move actions_service.jobParser to actions_module.jobParser 2025-11-06 07:28:16 +01:00
schedule_tasks_test.go feat: implement "concurrency" block in Forgejo Actions at the workflow level (#9434) 2025-10-03 18:43:02 +02:00
task.go chore: use code.forgejo.org/forgejo/actions-proto (#9981) 2025-11-05 16:10:52 +01:00
trust.go feat: trust management for runs created from a forked pull request 2025-11-06 11:07:38 +01:00
trust_test.go feat: trust management for runs created from a forked pull request 2025-11-06 11:07:38 +01:00
variables.go fix: allow Forgejo Actions environment variables starting with CI (#8850) 2025-08-10 22:56:16 +02:00
variables_test.go fix: allow Forgejo Actions environment variables starting with CI (#8850) 2025-08-10 22:56:16 +02:00
workflows.go fix: workflow dispatch shouldn't include empty fields in inputs (#10123) 2025-11-15 12:58:48 +01:00
workflows_test.go fix: workflow dispatch shouldn't include empty fields in inputs (#10123) 2025-11-15 12:58:48 +01:00