User interfaces for authorized integrations will benefit from having a name field, to allow a list of authorized integrations to have an identifiable user-entered label.
I've also added a "description" column which is a `LONGTEXT` field. My thought for this field is that if I were creating authorized integrations, I'd like to be able to write down where they're used, what they're used for, and how the remote system is configured. For example, if it was an authorized integration to allow AWS -> Forgejo integration, the AWS side can be complicated -- IAM roles which are assumed, resources like EC2 instances or Lambdas that can access the roles -- and this would provide a natural place to make some notes to help me remember how the remote is configured. I expect to represent this as a `<textarea>` in the Authorized Integration, optional, possibly markdown-formatted to allow links & bullet-points.
Manually tested migration with PG backend, and manually tested creation of authorized integrations with the CLI updates.
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. All work and communication must conform to Forgejo's [AI Agreement](https://codeberg.org/forgejo/governance/src/branch/main/AIAgreement.md). There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests for Go changes
- I added test coverage for Go changes...
- [ ] in their respective `*_test.go` for unit tests.
- [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I ran...
- [x] `make pr-go` before pushing
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] This change will be noticed by a Forgejo user or admin (feature, bug fix, performance, etc.). I suggest to include a release note for this change.
- [x] This change is not visible to a Forgejo user or admin (refactor, dependency upgrade, etc.). I think there is no need to add a release note for this change.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12413
Reviewed-by: Andreas Ahlenstorf <aahlenst@noreply.codeberg.org>
Protect OIDC tokens generated by Forgejo Actions from threats arising when users or repositories are renamed or deleted, freeing their names up for reuse by another user. In this threat environment, relying on the name of users and repositories in validating JWT claims is unsafe because they can change.
Adds three new claims to Actions' OIDC tokens:
- `actor_id` -- the immutable identifier of the actor who triggered an Action run
- `repository_id` -- the immutable identifier of the repository on which the Action is running
- `repository_owner_id` -- the immutable identifier of the owner of the repository on which the Action is running
Repositories will change their subject (`sub`) OIDC claims to include these immutable identifiers. Existing repositories will not change, in order to maintain compatibility with existing JWT usage. The new format will be applied to new repositories, or can be applied by disabling and enabling the Actions unit. The new format embeds the identifiers:
- **Existing repos:** `repo:my-org/my-repo:ref:refs/heads/main`
- **New repos:** `repo:my-org-123456/my-repo-456789:ref:refs/heads/main`
Fixes#12244.
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. All work and communication must conform to Forgejo's [AI Agreement](https://codeberg.org/forgejo/governance/src/branch/main/AIAgreement.md). There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests for Go changes
- I added test coverage for Go changes...
- [x] in their respective `*_test.go` for unit tests.
- [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I ran...
- [x] `make pr-go` before pushing
### Documentation
- [x] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- New fields will be added to documentation soon.
- [ ] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [x] This change will be noticed by a Forgejo user or admin (feature, bug fix, performance, etc.). I suggest to include a release note for this change.
- [ ] This change is not visible to a Forgejo user or admin (refactor, dependency upgrade, etc.). I think there is no need to add a release note for this change.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12355
Reviewed-by: Andreas Ahlenstorf <aahlenst@noreply.codeberg.org>
Authorized Integrations is a new feature to allow users to define external systems which can generate JSON Web Tokens (JWTs) that Forgejo will trust in order to perform API access on behalf of that user. This is an authentication mechanism that requires zero preconfiguration of shared secrets, and instead establishes trust through short-lived secrets (JWTs) that are signed by the issuer, signatures are validated by comparison with published public keys, and a public-keys retrieved through well-known HTTP endpoints secured with TLS verification.
The primary goal of Authorized Integrations is to support a mechanism for Forgejo Actions to receive elevated, but controlled, additional access to Forgejo. More details as to what the end result will look like are available in the [design proposal](https://codeberg.org/forgejo/forgejo/issues/3571#issuecomment-13268004) on #3571.
This PR adds the core database storage and authentication verification for Authorized Integrations, with these capabilities:
- An Authorized Integration is resolved by a unique key of an "issuer" and an "audience". The value of "issuer" is defined by the remote integration, and the value of "audience" will incorporate a unique identifier generated by Forgejo.
- Example issuer: `https://token.actions.githubusercontent.com/` is the issuer for GitHub JWTs
- Example audience: `https://forgejo.example.org/-/mfenniak/authorized-integration/6cc55ba0` is the expected format for a random audience field that Forgejo will generate.
- JWTs can contain any number of claims, which are represented as a JSON object; Forgejo can validate these with a flexible policy.
- eg. a claim may be `{"sub": "repo:coolguy/forgejo-runner-testrepo:pull_request"}` indicating that an OIDC token was received from an Actions execution in a specific repo on a specific event.
- Authorized Integrations support a `ClaimRules` system which allows claim equal, glob, and nested object inspection.
- `{"claim":"sub","comparison":"eq","value":"repo:mfenniak/forgejo-runner-testrepo:pull_request"}` -- would validate that `sub` exactly equals the specific value
- `{"claim":"sub","comparison":"glob","value":"repo:mfenniak/forgejo-runner-testrepo:*"}` -- would validate that `sub` matches the given string prefix but allow any event
- When a JWT is received on an incoming API call, Forgejo retrieves the Authorized Integration from the DB (if present), validates the token signature against a remote JWKS, validates the claims, and grants API access as the user with a permission scope defined on the Authorized Integration.
In addition to the unit testing provided here, this PR has been manually integration tested against three JWT issuing systems: Forgejo Actions, GitHub Actions, and AWS STS GetWebIdentityToken.
Careful consideration has been made of these security concerns:
- SSRF attacks against Forgejo are prevented by:
- having a blocklist on remote HTTP validation requests which prevent access to internal network resources,
- ensuring that authorized integrations are created by users with matching issuers, before attempting to validate tokens
- Resource utilization attacks against Forgejo are reduced by limiting the possible size of external metadata requests; when fetching `/.well-known/openid-configuration` and `jkws_uri`'s from remote, untrusted servers, a maximum response size of 16 kB is enforced
- Only well-known secure assymmetric JWT signing algorithms are supported -- in particular, the sketchy `none` JWT algorithm isn't supported.
- JWT validation is covered by extensive unit tests, covering validation of all JWT timestamps, validation of the issuers, validation of the issuer's documented supported signing algorithms.
This PR serves as a core, and many enhancements are required for this to be a usable system for users.
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. All work and communication must conform to Forgejo's [AI Agreement](https://codeberg.org/forgejo/governance/src/branch/main/AIAgreement.md). There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests for Go changes
- I added test coverage for Go changes...
- [x] in their respective `*_test.go` for unit tests.
- [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I ran...
- [ ] `make pr-go` before pushing
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- Documentation updates for new config entries will be authored.
- [ ] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] This change will be noticed by a Forgejo user or admin (feature, bug fix, performance, etc.). I suggest to include a release note for this change.
- [x] This change is not visible to a Forgejo user or admin (refactor, dependency upgrade, etc.). I think there is no need to add a release note for this change.
- Marking not visible as there's no mechanism to interact with this backend yet.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12261
Reviewed-by: Andreas Ahlenstorf <aahlenst@noreply.codeberg.org>
GitHub recently added the ability to [specify a time zone for scheduled workflows](https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-syntax#onschedule), thereby making it possible to run scheduled workflows at a certain local time, no matter whether daylight saving time (DST) is currently active or not. Example copied from GitHub's documentation:
```yaml
on:
schedule:
- cron: '30 5 * * 1-5'
timezone: "America/New_York"
```
The workflow would run at 05:30 each morning in the America/New_York timezone every Monday through Friday. `timezone` accepts IANA time zone names. If `timezone` is absent, `Etc/UTC` is used. GitHub runs workflows that were scheduled during DST jumps forward, for example, between 2 o'clock and 3 o'clock, directly after the clock jumped forward. In this case, that would be 3 o'clock.
Forgejo already supports time zones by prepending cron schedules with `TZ=<zone-id>` or `CRON_TZ=<zone-id>`:
```yaml
on:
schedule:
- cron: 'CRON_TZ=America/New_York 30 5 * * 1-5'
```
However, that capability is not documented. Workflows that are scheduled to run during DST changes are skipped when the clock jumps forward and run twice when it jumps backward.
This two-part PR adds support for `timezone` to improve compatibility with GitHub. `TZ` and `CRON_TZ` continue working. When both `timezone` and `TZ` or `CRON_TZ` are present, `timezone` takes precedence. When neither `timezone` nor `TZ` nor `CRON_TZ` are present, `Etc/UTC` is used as before. Because `TZ` and `CRON_TZ` were already supported by Forgejo before GitHub introduced `timezone`, `timezone` behaves during DST changes as previous versions of Forgejo, thereby deviating from GitHub. That means that workflows that are scheduled to run during DST changes are skipped when the clock jumps forward. And they run twice when it jumps backwards. However, it is generally recommended not to schedule workflows during the time of day when DST changes occur.
This part of the PR integrates the [workflow validation and parsing of the `timezone` field](https://code.forgejo.org/forgejo/runner/pulls/1454) supplied by Forgejo Runner.
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. All work and communication must conform to Forgejo's [AI Agreement](https://codeberg.org/forgejo/governance/src/branch/main/AIAgreement.md). There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests for Go changes
(can be removed for JavaScript changes)
- I added test coverage for Go changes...
- [x] in their respective `*_test.go` for unit tests.
- [x] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I ran...
- [x] `make pr-go` before pushing
### Tests for JavaScript changes
(can be removed for Go changes)
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [x] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- https://codeberg.org/forgejo/docs/pulls/1853
- [ ] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [x] This change will be noticed by a Forgejo user or admin (feature, bug fix, performance, etc.). I suggest to include a release note for this change.
- [ ] This change is not visible to a Forgejo user or admin (refactor, dependency upgrade, etc.). I think there is no need to add a release note for this change.
*The decision if the pull request will be shown in the release notes is up to the mergers / release team.*
The content of the `release-notes/<pull request number>.md` file will serve as the basis for the release notes. If the file does not exist, the title of the pull request will be used instead.
<!--start release-notes-assistant-->
## Release notes
<!--URL:https://codeberg.org/forgejo/forgejo-->
- Features
- [PR](https://codeberg.org/forgejo/forgejo/pulls/11851): <!--number 11851 --><!--line 0 --><!--description c3VwcG9ydCBgdGltZXpvbmVgIGluIHNjaGVkdWxlZCB3b3JrZmxvd3M=-->support `timezone` in scheduled workflows<!--description-->
<!--end release-notes-assistant-->
Co-authored-by: Renovate Bot <bot@kriese.eu>
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/11851
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.codeberg.org>
Co-authored-by: Andreas Ahlenstorf <andreas@ahlenstorf.ch>
Co-committed-by: Andreas Ahlenstorf <andreas@ahlenstorf.ch>
Fixes#9629.
New pull mirrors have credentials stored encrypted in the database, the same as push mirrors, rather than in the repository's `config` file. `git fetch` on the pull mirror is updated to use the credential store. Pull mirrors will have their credentials migrated to the encrypted storage in the database as they're synced or otherwise accessed via the web UI.
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. All work and communication must conform to Forgejo's [AI Agreement](https://codeberg.org/forgejo/governance/src/branch/main/AIAgreement.md). There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests for Go changes
- I added test coverage for Go changes...
- [ ] in their respective `*_test.go` for unit tests.
- [x] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I ran...
- [x] `make pr-go` before pushing
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [x] This change will be noticed by a Forgejo user or admin (feature, bug fix, performance, etc.). I suggest to include a release note for this change.
- [ ] This change is not visible to a Forgejo user or admin (refactor, dependency upgrade, etc.). I think there is no need to add a release note for this change.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/11909
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Reviewed-by: Andreas Ahlenstorf <aahlenst@noreply.codeberg.org>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
Prevent access to "current" application models and services from migrations via `golangci` config:
eg:
```
models/forgejo_migrations/v14a_ap-change-fedi-handle-structure.go:18:2: import 'forgejo.org/models/user' is not allowed from list 'migration-isolation': Migrations must not import application models. Application models will be the most recent schema for Forgejo, while migrations will be operating against the database schema that existed when they were authored. (depguard)
user_model "forgejo.org/models/user"
^
models/forgejo_migrations/v14a_ap-change-fedi-handle-structure.go:21:2: import 'forgejo.org/services/user' is not allowed from list 'migration-isolation': Migrations must not import application services. Application services will reference application models which will use the most recent schema for Forgejo, while migrations will be operating against the database schema that existed when they were authored. (depguard)
user_service "forgejo.org/services/user"
```
Fixes an existing migration issue where it isn't possible to add a new column to the `User` table ([test errors that occur](https://codeberg.org/forgejo/forgejo/actions/runs/148633/jobs/10/attempt/1#jobstep-5-323)), but also guarantees that future migrations don't stumble into the same issue by inadvertently referencing live application code from historical migrations.
Originally identified and draft fix by @codecat w/ proposed fix in #11870.
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. All work and communication must conform to Forgejo's [AI Agreement](https://codeberg.org/forgejo/governance/src/branch/main/AIAgreement.md). There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests for Go changes
- I added test coverage for Go changes...
- [ ] in their respective `*_test.go` for unit tests.
- [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I ran...
- [x] `make pr-go` before pushing
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] This change will be noticed by a Forgejo user or admin (feature, bug fix, performance, etc.). I suggest to include a release note for this change.
- [x] This change is not visible to a Forgejo user or admin (refactor, dependency upgrade, etc.). I think there is no need to add a release note for this change.
Co-authored-by: Melissa Geels <melissa@nimble.tools>
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/11872
Reviewed-by: Andreas Ahlenstorf <aahlenst@noreply.codeberg.org>
Reviewed-by: Michael Kriese <michael.kriese@gmx.de>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
Forgejo Runner can optionally ask for a particular job. Example: `forgejo-runner one-job --handle 9d52c7d8-aebe-426b-b015-dd453aacaada`. This change adds the necessary job filtering to Forgejo.
See https://code.forgejo.org/forgejo/forgejo-actions-feature-requests/issues/76 for the motivation and design considerations.
PR for the extension of the runner protocol: https://code.forgejo.org/forgejo/actions-proto/pulls/18
Related change in Forgejo Runner with usage example: https://code.forgejo.org/forgejo/runner/pulls/1443
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests for Go changes
(can be removed for JavaScript changes)
- I added test coverage for Go changes...
- [x] in their respective `*_test.go` for unit tests.
- [x] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I ran...
- [x] `make pr-go` before pushing
### Tests for JavaScript changes
(can be removed for Go changes)
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [x] This change will be noticed by a Forgejo user or admin (feature, bug fix, performance, etc.). I suggest to include a release note for this change.
- [ ] This change is not visible to a Forgejo user or admin (refactor, dependency upgrade, etc.). I think there is no need to add a release note for this change.
*The decision if the pull request will be shown in the release notes is up to the mergers / release team.*
The content of the `release-notes/<pull request number>.md` file will serve as the basis for the release notes. If the file does not exist, the title of the pull request will be used instead.
<!--start release-notes-assistant-->
## Release notes
<!--URL:https://codeberg.org/forgejo/forgejo-->
- Features
- [PR](https://codeberg.org/forgejo/forgejo/pulls/11676): <!--number 11676 --><!--line 0 --><!--description YWxsb3cgcnVubmVycyB0byByZXF1ZXN0IGEgcGFydGljdWxhciBqb2I=-->allow runners to request a particular job<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/11676
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.codeberg.org>
Co-authored-by: Andreas Ahlenstorf <andreas@ahlenstorf.ch>
Co-committed-by: Andreas Ahlenstorf <andreas@ahlenstorf.ch>
When merging #9254 I didn't make the mental note that v14 is already out and the migration would now apply to v15. Document this mistake.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/11144
Reviewed-by: Michael Kriese <michael.kriese@gmx.de>
Reviewed-by: 0ko <0ko@noreply.codeberg.org>
Reviewed-by: Panagiotis "Ivory" Vasilopoulos <git@n0toose.net>
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.codeberg.org>
Co-authored-by: Gusted <postmaster@gusted.xyz>
Co-committed-by: Gusted <postmaster@gusted.xyz>
Project columns and cards use a sorting field for ordering, but nothing prevents duplicate values from being inserted. This causes unpredictable ordering and makes swap-based reordering unsafe. Additionally, the same issue can be added to a project multiple times, which shouldn't be possible.
This PR adds a migration to clean up existing data and enforce three unique constraints:
- (project_id, sorting) on project_board — one sorting value per column per project
- (project_id, issue_id) on project_issue — one card per issue per project
- (project_board_id, sorting) on project_issue — one sorting value per card per column
The migration deduplicates existing rows and reassigns sequential sorting values before adding the constraints.
Changes
- Migration: fix duplicate sorting values in project_board and project_issue, remove duplicate (project_id, issue_id) rows, add three unique constraints
- MoveColumnsOnProject: two-phase swap (negate then set) to avoid constraint collisions
- MoveIssuesOnProjectColumn: three-phase approach with duplicate validation and sorted lock ordering
- UpdateColumn: always persist sorting field (allows setting to 0)
- GetDefaultColumn: query max sorting before auto-creating
- createDefaultColumnsForProject: explicit sequential sorting
- changeProjectStatus: only set ClosedDateUnix when closing
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests
- I added test coverage for Go changes...
- [x] in their respective `*_test.go` for unit tests.
- [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] This change will be noticed by a Forgejo user or admin (feature, bug fix, performance, etc.). I suggest to include a release note for this change.
- [x] This change is not visible to a Forgejo user or admin (refactor, dependency upgrade, etc.). I think there is no need to add a release note for this change.
*The decision if the pull request will be shown in the release notes is up to the mergers / release team.*
The content of the `release-notes/<pull request number>.md` file will serve as the basis for the release notes. If the file does not exist, the title of the pull request will be used instead.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/11334
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.codeberg.org>
Co-authored-by: Myers Carpenter <myers@maski.org>
Co-committed-by: Myers Carpenter <myers@maski.org>
Probably fixes (or improves, at least) https://code.forgejo.org/forgejo/runner/issues/1391, paired with the runner implementation https://code.forgejo.org/forgejo/runner/pulls/1393.
When the FetchTask() API is invoked to create a task, unpreventable environmental errors may occur; for example, network disconnects and timeouts. It's possible that these errors occur after the server-side has assigned a task to the runner during the API call, in which case the error would cause that task to be lost between the two systems -- the server will think it's assigned to the runner, and the runner never received it. This can cause jobs to appear stuck at "Set up job".
The solution implemented here is idempotency in the FetchTask() API call, which means that the "same" FetchTask() API call is expected to return the same values. Specifically, the runner creates a unique identifier which is transmitted to the server as a header `x-runner-request-key` with each FetchTask() invocation which defines the sameness of the call, and the runner retains the value until the API call receives a successful response. The server implementation returns the same tasks back if a second (or Nth) call is received with the same `x-runner-request-key` header. In order to accomplish this is records the `x-runner-request-key` value that is used with each request that assigns tasks.
As a complication, the Forgejo server is unable to return the same `${{ secrets.forgejo_token }}` for the task because the server stores that value in a one-way hash in the database. To resolve this, the server regenerates the token when retrieving tasks for a second time.
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests for Go changes
(can be removed for JavaScript changes)
- I added test coverage for Go changes...
- [x] in their respective `*_test.go` for unit tests.
- [x] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I ran...
- [x] `make pr-go` before pushing
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [x] This change will be noticed by a Forgejo user or admin (feature, bug fix, performance, etc.). I suggest to include a release note for this change.
- [ ] This change is not visible to a Forgejo user or admin (refactor, dependency upgrade, etc.). I think there is no need to add a release note for this change.
*The decision if the pull request will be shown in the release notes is up to the mergers / release team.*
The content of the `release-notes/<pull request number>.md` file will serve as the basis for the release notes. If the file does not exist, the title of the pull request will be used instead.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/11401
Reviewed-by: Andreas Ahlenstorf <aahlenst@noreply.codeberg.org>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
As described in [this comment](https://gitea.com/gitea/act_runner/issues/19#issuecomment-739221) one-job runners are not secure when running in host mode. We implemented a routine preventing runner tokens from receiving a second job in order to render a potentially compromised token useless. Also we implemented a routine that removes finished runners as soon as possible.
Big thanks to [ChristopherHX](https://github.com/ChristopherHX) who did all the work for gitea!
Rel: #9407
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests
- I added test coverage for Go changes...
- [ ] in their respective `*_test.go` for unit tests.
- [x] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [ ] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] I do not want this change to show in the release notes.
- [ ] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/9962
Reviewed-by: Andreas Ahlenstorf <aahlenst@noreply.codeberg.org>
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.codeberg.org>
Co-authored-by: Manuel Ganter <manuel.ganter@think-ahead.tech>
Co-committed-by: Manuel Ganter <manuel.ganter@think-ahead.tech>
Adds a CI check which detects any usage of xorm's `Sync` method that doesn't include `IgnoreDropIndices: true`, and causes an error.
`semgrep` is a semantic grep tool that allows for the relatively easy authoring of linting tools that are customized to a project's specific needs, rather than generic like `golangci` and related tools. Although `semgrep` offers a suite of out-of-the-box rules (and a paid set of rules), neither of those are used here -- only one Forgejo-specific rule is added in `.semgrep/xorm.yaml`.
My intent with this change is to introduce the idea and infrastructure of `semgrep` with a single minimal rule. Once in-place, this will become a tool that we can use when we recognize bad coding patterns and wish to correct them permanently, rather than relying on human code review. While generic linting tools do this well for general patterns, this will allow Forgejo to apply domain-specific checks. For example, in #11112, an error indicates that it might be appropriate for us to always use `.StorageEngine("InnoDB")` when using an xorm engine -- if we made that determination, it could be cemented in-place with a `semgrep` rule relatively easily.
This specific rule looks for any access for xorm's `Sync` or `SyncWithOptions` methods on the `*xorm.Engine` or `*xorm.Session`. They are then considered errors if they don't include `IgnoreDropIndices: true`. This is *typically* correct and safe, but can also be ignored when specifically needed. In the `.semgrep/tests` folder, test code is added which validates that the `semgrep` rule matches the expected patterns; this self-test is run before `semgrep` runs on the PR in CI.
As a demonstration, when `IgnoreDropIndices` is removed from a migration, here's an error: https://codeberg.org/forgejo/forgejo/actions/runs/135750/jobs/12/attempt/1
```
models/forgejo_migrations/v14b_add-action_run-preexecutionerrorcode.go
❯❯❱ semgrep.xorm-sync-missing-ignore-drop-indices
xorm Sync operation may drop indices if used on an incomplete bean definition for an existing table.
Use SyncWithOptions with IgnoreDropIndices: true instead.
22┆ _, err := x.SyncWithOptions(xorm.SyncOptions{}, new(ActionRun))
```
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests
- I added test coverage for Go changes...
- [ ] in their respective `*_test.go` for unit tests.
- [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] This change will be noticed by a Forgejo user or admin (feature, bug fix, performance, etc.). I suggest to include a release note for this change.
- [x] This change is not visible to a Forgejo user or admin (refactor, dependency upgrade, etc.). I think there is no need to add a release note for this change.
*The decision if the pull request will be shown in the release notes is up to the mergers / release team.*
The content of the `release-notes/<pull request number>.md` file will serve as the basis for the release notes. If the file does not exist, the title of the pull request will be used instead.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/11142
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
This modifies usernames of ActivityPub accounts to use the @example@example.tld
format with an additional optional port component (e.g. @user@example.tld:42).
This allows accounts from ActivityPub servers with more relaxed username
requirements than those of Forgejo's to interact with Forgejo. Forgejo would
also follow a "de facto" standard of ActivityPub implementations.
By separating different information using @'s, we also gain future
opportunities to store more information about ActivityPub accounts internally,
so that we won't have to rely on e.g. the amount of dashes in a username as
my migration currently does.
Continuation of Aravinth's work: https://codeberg.org/forgejo/forgejo/pulls/4778
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/9254
Reviewed-by: jerger <jerger@noreply.codeberg.org>
Reviewed-by: Ellen Εμιλία Άννα Zscheile <fogti@noreply.codeberg.org>
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: Panagiotis "Ivory" Vasilopoulos <git@n0toose.net>
Co-committed-by: Panagiotis "Ivory" Vasilopoulos <git@n0toose.net>
This PR fixes a number of typos throughout the entire repository. Running https://github.com/crate-ci/typos and then changing all occurrences that I naively deemed "safe enough".
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/10753
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: Christoph Mewes <christoph@kubermatic.com>
Co-committed-by: Christoph Mewes <christoph@kubermatic.com>
In support of adding foreign keys to the `action_runner_token` table, this PR also had to:
- Add detection and error if a table with "soft delete" is used with a foreign key, because it causes a tricky to track down foreign key violation
- Remove unused xorm "soft delete" capability on the `action_runner_token` table
- Change the `RepoID` and `OwnerID` fields to use the value `NULL` to indicate that this scope wasn't valid for the token, rather than the value `0`
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests
- I added test coverage for Go changes...
- [x] in their respective `*_test.go` for unit tests.
- [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] I do not want this change to show in the release notes.
- [x] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/10756
Reviewed-by: Andreas Ahlenstorf <aahlenst@noreply.codeberg.org>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
Changes foreign key database inconsistency handling so that inconsistent records are automatically deleted with an administrator warning during migration. As noted in discussion: https://codeberg.org/forgejo/discussions/issues/385#issuecomment-9175566
Intended for backport to v14 to eliminate pain during upgrades.
Because these migrations are now deleting data, rather than allowing the administrator to do it, all migrations have been covered with an integration test that verifies expected data is deleted. This is particularly interesting with nullable fields.
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests
- I added test coverage for Go changes...
- [x] in their respective `*_test.go` for unit tests.
- [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] I do not want this change to show in the release notes.
- [x] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/10568
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
- Follow up of forgejo/forgejo!5041, forgejo/forgejo!6074, forgejo/forgejo!8692, forgejo/forgejo!9923
- The `webhook` table contains a encrypted header authorization.
- Use `keying` to safely store this secret and bound them to the table, column and row id
- The migration isn't spectacular but does closely follow what we learned in the previous three migrations: use a transaction and delete records when you can't decrypt them.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/10059
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.codeberg.org>
Reviewed-by: oliverpool <oliverpool@noreply.codeberg.org>
Co-authored-by: Gusted <postmaster@gusted.xyz>
Co-committed-by: Gusted <postmaster@gusted.xyz>
Make the variable `GITHUB_WORKFLOW_REF` available in Forgejo Action workflows. It is the ref path to the workflow and looks like `testowner/testrepo/.forgejo/workflows/test-workflow.yaml@refs/heads/main` ([GitHub documentation](https://docs.github.com/en/actions/reference/workflows-and-actions/variables)). GitHub Actions like [gh-action-pypi-publish](https://github.com/pypa/gh-action-pypi-publish) rely on its presence. See https://code.forgejo.org/forgejo/forgejo-actions-feature-requests/issues/56 for additional details.
`GITHUB_WORKFLOW_REF` cannot be generated easily during an action run. Either the path to workflow file has to be hardcoded or inferred by replicating the logic Forgejo uses to determine it. That is further complicated by the fact that Forgejo supports multiple search paths, namely `.forgejo/workflows`, `.gitea/workflows`, and `.github/workflows`. It is also the reason that the workflow directory is now stored in the database alongside the name of the workflow file.
Partial implementation is required in Forgejo Runner, see https://code.forgejo.org/forgejo/runner/pulls/1197.
Example workflow:
```yaml
on:
push:
workflow_dispatch:
schedule:
- cron: "* * * * *"
jobs:
test:
runs-on: ubuntu-latest
steps:
- run: |
echo "FORGEJO_WORKFLOW_REF=$FORGEJO_WORKFLOW_REF"
echo "GITHUB_WORKFLOW_REF=$GITHUB_WORKFLOW_REF"
echo "forgejo.workflow_ref=${{ forgejo.workflow_ref }}"
echo "github.workflow_ref=${{ github.workflow_ref }}"
```
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests
- I added test coverage for Go changes...
- [x] in their respective `*_test.go` for unit tests.
- [x] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [ ] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] I do not want this change to show in the release notes.
- [ ] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/10276
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.codeberg.org>
Co-authored-by: Andreas Ahlenstorf <andreas@ahlenstorf.ch>
Co-committed-by: Andreas Ahlenstorf <andreas@ahlenstorf.ch>
* sets all cached ActivityPub users to ProhibitLogin
* creates a new UserType to uniquely identify users from ActivityPub
This has the side effect that ActivityPub users will no longer be listed in the admin view.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/10434
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: famfo <famfo@famfo.xyz>
Co-committed-by: famfo <famfo@famfo.xyz>
To make sure that the code stays maintainable, I added the `importas` linter to ensure that the imports for models and services stay consistent.
I realised that this might be needed after finding some discrepancies between singular/plural naming, and, especially in the case of the `forgejo.org/services/context` package, multiple different aliases like `gitea_ctx`, `app_context` and `forgejo_context`. I decided for `app_context`, as that seems to be the most commonly used naming.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/10253
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: nachtjasmin <nachtjasmin@posteo.de>
Co-committed-by: nachtjasmin <nachtjasmin@posteo.de>
Identified in code review https://codeberg.org/forgejo/forgejo/pulls/10244#issuecomment-8576643, the `PreExecutionError` field in `ActionRun` isn't well implemented as it translates the error at action runtime rather than later when the action is viewed in the UI. This PR adds an error code and error details column that can be more correctly translated.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/10267
Reviewed-by: 0ko <0ko@noreply.codeberg.org>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
This change is motivated by 5e300a2a87
- Drop the `updated_by` and `commit_id` column, they are unused and have a index for no reason.
- Drop the index on `status` and `created_unix` and make a index on `(user_id, status)`.
## Test
1. Run migration.
2. Confirm the migration succeeds.
3. Check that `notification` table has the correct indexes.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/9926
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.codeberg.org>
Co-authored-by: Gusted <postmaster@gusted.xyz>
Co-committed-by: Gusted <postmaster@gusted.xyz>
Fixes#9963. This realigns all the indexes on the `action` table to best match their intended usages.
New:
- `IDX_action_created_unix (created_unix)`
- Intended for usage in `DeleteOldActions`.
- `IDX_action_repo_id_created_unix (repo_id, created_unix)`
- Intended for usage when fetching action feeds for a repo and a team, with the same logic as that described below in `IDX_action_user_id_created_unix`.
- `IDX_action_repo_id_op_type (repo_id, op_type)`
- Intended for `DeleteIssueActions` when it searches for CreateIssue & CreatePullRequest actions for cleanup. Could be optimized further with a denormalization of the issue identifier into a field, but there's no current evidence that this is required.
Replaced:
- `IDX_action_c_u (created_unix, user_id)`
- Replaced with `IDX_action_user_id_created_unix (user_id, created_unix)`. When action feeds are created w/ `ORDER BY created_unix DESC LIMIT 20`, an index beginning with `created_unix` will have to index scan until it can satisfy 20 records; the `user_id` portion of the index is effectively useless until two records appear at the same time. By inverting the order, the database will be able to identify the records created by a user and then pop the most recent 20 from the index order.
- At the scale of database I have access to, the performance difference is unmeasurable. This change is supported by theoretical grounds and the findings of #9963, but no experimental evidence.
Removed:
- `IDX_action_user_id (user_id)`
- Redundant with the new `IDX_action_user_id_created_unix`.
- `IDX_action_r_u (repo_id, user_id)`
- No clear consumer for this index.
Retained with no modification:
- `IDX_action_comment_id (comment_id)`
- Used in `DeleteIssueActions`.
- `IDX_action_au_r_c_u (act_user_id, repo_id, created_unix, user_id)`
- Heat map generation.
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests
- I added test coverage for Go changes...
- [ ] in their respective `*_test.go` for unit tests.
- [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] I do not want this change to show in the release notes.
- [x] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/10040
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
Currently `DeriveKey` is called every time that a secret must be encoded/decoded. Since this function is deterministic, its result can be cached to allow a 250x speedup (the original took less than half a microsecond, so this more of a micro-optimization...).
```
go test -bench=.
goos: linux
goarch: amd64
pkg: forgejo.org/modules/keying
cpu: Intel(R) Core(TM) Ultra 5 125H
BenchmarkExpandPRK-18 2071627 564.2 ns/op
BenchmarkExpandPRKOnce-18 541438192 2.206 ns/op
PASS
ok forgejo.org/modules/keying 2.369s
```
## Other changes
- Since the keys can be constructed once, it simplifies a bit the callsites (`keying.TOTP.Encrypt(...)` instead of `keying.DeriveKey(keying.ContextTOTP).Encrypt(...)`)
- All `Encrypt`/`Decrypt` calls will panic forever if called before `Init` has been called (current it panics as long as `Init` has not been called)
- Calling `Init` twice with different keys will trigger a panic (currently racy)
- Calling `Decrypt` with a short ciphertext does not panic anymore (like when calling with long-enough garbage)
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/10114
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: oliverpool <git@olivier.pfad.fr>
Co-committed-by: oliverpool <git@olivier.pfad.fr>
Fixes [#3525](https://codeberg.org/forgejo/forgejo/issues/3525) and supersedes [#9586](https://codeberg.org/forgejo/forgejo/pulls/9586)
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests
- I added test coverage for Go changes...
- [ ] in their respective `*_test.go` for unit tests.
- [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [ ] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] I do not want this change to show in the release notes.
- [x] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
<!--start release-notes-assistant-->
## Release notes
<!--URL:https://codeberg.org/forgejo/forgejo-->
- Features
- [PR](https://codeberg.org/forgejo/forgejo/pulls/9829): <!--number 9829 --><!--line 0 --><!--description Y2hvcmU6IFJlbW92ZSBJc0RlbGV0ZWQgZnJvbSBhY3Rpb24gKGFjdGl2aXR5KSB0YWJsZQ==-->chore: Remove IsDeleted from action (activity) table<!--description-->
<!--end release-notes-assistant-->
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/9829
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.codeberg.org>
Co-authored-by: Leni Kadali <lenikadali@noreply.codeberg.org>
Co-committed-by: Leni Kadali <lenikadali@noreply.codeberg.org>
ActionUser is to keep track of pull requests posters that are
permanently trusted. It has a used field to track when it was last
used so records can be expired instead of accumulating forever.
ActionRun has new fields to make it possible to look them up given
either the pull request ID or the poster ID.
- Follow up of forgejo/forgejo!5041, forgejo/forgejo!6074, forgejo/forgejo!8692
- The `task` table contains three secrets: clone address (with credentials), auth password and auth token. These secrets are stored for migrating repositories (also the only usage of this table, although it allows for more usages).
- Use `keying` to safely store these secrets and bound them to the table, column, row id and JSON field name.
- The migration isn't spectacular but does closely follow what we learned in the previous two migrations: use a transaction and delete records when you can't decrypt them. We also learned about `db.Iterate` not being happy when updating records but it has since been fixed.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/9923
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.codeberg.org>
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Gusted <postmaster@gusted.xyz>
Co-committed-by: Gusted <postmaster@gusted.xyz>
Fixes#9846. The number of open action runs on a repo is not precomputed and stored on the repo, but is computed as needed and cached. The computation is faster than the update because it only calculates the smaller set of which action runs are not completed (as opposed to counting all of them).
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests
- I added test coverage for Go changes...
- [x] in their respective `*_test.go` for unit tests.
- [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] I do not want this change to show in the release notes.
- [x] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/9927
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
The `repository` table has quite a few "count of related objects" fields on it, including the number of issues, closed issues, pull requests, and closed pull requests. These fields specifically will cause deadlocks during concurrent PR merges as documented in #9785.
These fields are not used in database queries. In order to eliminate the deadlock possibility on them, I've moved them to be calculated on-demand with caching, with the cache being invalidated in the same places that the recalc used to be triggered.
I've supplemented the already in-place automated testing with manual testing performing simple close & reopen of issues & PRs, and the counts which are used in the tabs at the top of the repo page are updated correctly as expected.
Near future work:
- Similar change can probably be performed to fix#9846
- Last known deadlock identified from #9785; I'm hoping to incorporate the synthetic deadlock test in a near future PR to prevent regressions
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests
- Tests were already in-place covering these fields; they've been adjusted from using the fields to the new accessor methods.
- I added test coverage for Go changes...
- [ ] in their respective `*_test.go` for unit tests.
- [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] I do not want this change to show in the release notes.
- [x] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
<!--start release-notes-assistant-->
## Release notes
<!--URL:https://codeberg.org/forgejo/forgejo-->
- Bug fixes
- [PR](https://codeberg.org/forgejo/forgejo/pulls/9922): <!--number 9922 --><!--line 0 --><!--description cmVkdWNlIGRlYWRsb2NrcyBtZXJnaW5nIFBScyBieSB1c2luZyBjYWNoaW5nIGZvciByZXBvIGlzc3VlIGNvdW50IHN0YXRz-->reduce deadlocks merging PRs by using caching for repo issue count stats<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/9922
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
Adds a foreign key to the table `forgejo_auth_token` (`AuthorizationToken` in go code). A review has shown that most capabilities around auth tokens seem to be well covered by test automation, but supplemented with manual testing around user deletion as well.
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests
- I added test coverage for Go changes...
- [ ] in their respective `*_test.go` for unit tests.
- [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] I do not want this change to show in the release notes.
- [x] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/9886
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
Adds foreign keys to the table pull_request which are covered by the doctor's db consistency check:
- issue_id -> issue
- base_repo_id -> repository
Note that other fields that look like references -- `head_repo_id` and `merger` -- are not covered by the db consistency check and therefore out-of-scope for the first phase of foreign keys. They're on my list for future more detailed evaluation.
In addition to running automated tests (and making a few tweaks to get them to pass), I performed manual testing of:
- Deleting the base repo of a pull request -- the repo is deleted without error and the pull request is deleted as well.
- Deleting the issue behind a PR via an issue delete API call -- this code-path handles deleting a PR correctly.
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests
- I added test coverage for Go changes...
- [ ] in their respective `*_test.go` for unit tests.
- [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] I do not want this change to show in the release notes.
- [x] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/9832
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
Fixes#9755, performance regression from #9017.
Manually tested by application to a development database and verification that the index is used on `select * from action_task where job_id = 11 and attempt = 1`. I've made the index here cover just the `job_id` field, as the set of values for `attempt` is expected to be very small.
Will backport via just the `models/actions/task.go` change (allowing `SyncAllTables` to create it) to avoid a schema migration during the backport (complicated due to the new schema migration module in v14).
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests
- I added test coverage for Go changes...
- [ ] in their respective `*_test.go` for unit tests.
- [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] I do not want this change to show in the release notes.
- [x] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
<!--start release-notes-assistant-->
## Release notes
<!--URL:https://codeberg.org/forgejo/forgejo-->
- Bug fixes
- [PR](https://codeberg.org/forgejo/forgejo/pulls/9789): <!--number 9789 --><!--line 0 --><!--description Zml4KHBlcmYpOiBhZGQgbWlzc2luZyBpbmRleCBvbiBhY3Rpb25fdGFzayB0YWJsZQ==-->fix(perf): add missing index on action_task table<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/9789
Reviewed-by: Michael Kriese <michael.kriese@gmx.de>
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
This appears to be as easy as a foreign-key addition possibly could be; no automated tests required adjustments.
Manual testing conducted:
- Added collaborator to a repo
- Removed collaborator from a repo
- Deleted a repo with a collaborator on it
- Deleted a user that is a collaborator
- Verified database definitions to ensure foreign keys are present.
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests
- I added test coverage for Go changes...
- [ ] in their respective `*_test.go` for unit tests.
- [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] I do not want this change to show in the release notes.
- [x] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
<!--start release-notes-assistant-->
## Release notes
<!--URL:https://codeberg.org/forgejo/forgejo-->
- Features
- [PR](https://codeberg.org/forgejo/forgejo/pulls/9724): <!--number 9724 --><!--line 0 --><!--description YWRkIGZvcmVpZ24ga2V5cyB0byB0YWJsZSBjb2xsYWJvcmF0aW9u-->add foreign keys to table collaboration<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/9724
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
Integration tests and new Forgejo installations work in a manner that I didn't expect, and didn't come up during testing of #9561.
- They run the migrations (gitea migrations -> forgejo migrations legacy -> forgejo migrations).
- The gitea & forgejo migrations detect if the `version` and `forgejo_version` table are **empty**, because they are on an empty DB and the tables were just created
- When they hit this case, they pretend all the migrations are applied by setting the version to the highest version number
- And then they rely on the DB engine executing `SyncAllTables` to actually create the schema.
The new `forgejo_migrations` module was not doing this. So, it would attempt to execute migrations and find that dependent database tables didn't exist, causing unexpected errors in running the integration tests (or starting forgejo on an empty database): https://codeberg.org/forgejo/forgejo/pulls/9397#issuecomment-7745969
This fixes the issue by following the same pattern in `forgejo_migrations`; relying on `SyncAllTables` for initial schema creation.
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests
- I added test coverage for Go changes...
- [x] in their respective `*_test.go` for unit tests.
- [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [x] I do not want this change to show in the release notes.
- [ ] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/9708
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
Adds two foreign keys:
- `access.user_id` -> `user`
- `access.repo_id` -> `repository`
Testing:
- Existing automated test suite
- Minor adjustments required to `reverseproxy_test.go`
- Test failures caused a reorganization of delete operations in `DeleteRepositoryDirectly`
- Manually tested
- On "user" and "repo" Collaborator page, added, removed, and changed the access-level of collaborators
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests
- I added test coverage for Go changes...
- [ ] in their respective `*_test.go` for unit tests.
- [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] I do not want this change to show in the release notes.
- [x] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
<!--start release-notes-assistant-->
## Release notes
<!--URL:https://codeberg.org/forgejo/forgejo-->
- Features
- [PR](https://codeberg.org/forgejo/forgejo/pulls/9557): <!--number 9557 --><!--line 0 --><!--description YWRkIGZvcmVpZ24ga2V5cyB0byB0aGUgYGFjY2Vzc2AgdGFibGU=-->add foreign keys to the `access` table<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/9557
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
Currently references a pre-release version of `code.forgejo.org/forgejo/runner/v11`, pending release of https://code.forgejo.org/forgejo/runner/pulls/1026.
Fixes#5914.
This PR is quite large, but it can be reviewed commit-by-commit in relatively small, logical chunks.
Adds support for workflows with a `concurrency` block, and submembers `group` and `cancel-in-progress`. For example:
```
on:
workflow_dispatch:
jobs:
rust-checks:
runs-on: debian-latest
steps:
- run: sleep 300
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: false
```
The concurrency block effectively ends up with four supported behaviors that users will want to choose from:
- Backwards compatibility / default -- if omitted completely, the existing Forgejo behavior will be implemented. That behavior is that push and pull request synchronize events will cancel all previous runs on the same repository, branch, and workflow.
- Unlimited concurrency -- if the `cancel-in-progress` value is set to `false` and no `group` is provided, then the previously described Forgejo behavior will be disabled and an unlimited number of workflows can be executed simultaneously (to the maximum supported by the Forgejo Runner capacity).
- Queue-behind -- if a `group` is provided and `cancel-in-progress: false` is set, then every new action run with in the same repository with the same group value will be queued behind previous workflow runs, allowing only one workflow to execute at a time in the group, but allowing all workflows to finish naturally.
- Cancel-in-progress -- if a `group` is provided and `cancel-in-progress: true` is set, then every new action run with in the same repository with the same group value will cause previously queued or running runs to be cancelled, allowing only one workflow to execute at a time in the group, but preferring execution of the most recent workflow.
Both the `group` and `cancel-in-progress` values can access values from the `github`, `inputs` and `vars` context for dynamic behavior.
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests
- I added test coverage for Go changes...
- [x] in their respective `*_test.go` for unit tests.
- [x] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [x] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- https://codeberg.org/forgejo/docs/pulls/1513
- [ ] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] I do not want this change to show in the release notes.
- [x] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
<!--start release-notes-assistant-->
## Release notes
<!--URL:https://codeberg.org/forgejo/forgejo-->
- Features
- [PR](https://codeberg.org/forgejo/forgejo/pulls/9434): <!--number 9434 --><!--line 0 --><!--description aW1wbGVtZW50ICJjb25jdXJyZW5jeSIgYmxvY2sgaW4gRm9yZ2VqbyBBY3Rpb25zIGF0IHRoZSB3b3JrZmxvdyBsZXZlbA==-->implement "concurrency" block in Forgejo Actions at the workflow level<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/9434
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
Adds four foreign keys:
- stopwatch -- issue_id -> issue, user_id -> user
- tracked_time -- issue_id -> issue, user_id -> user
The majority of work encompassed in this PR is updating testing and support infrastructure to support foreign keys:
- `models/db/foreign_keys.go` adds new capabilities to sort registered tables into the right insertion order to avoid violating foreign keys
- `RecreateTables`, used by migration testing and the `doctor recreate-table` CLI, has been updated to support tables with foreign keys; new restrictions require that FK-related tables be rebuilt at the same time
- test fixture data is inserted in foreign-key order, and deleted in the reverse
An upgrade to xorm v1.3.9-forgejo.2 is incorporated in this PR, as two unexpected behaviors in the foreign key schema management were discovered during development of the updated `RecreateTables` routine.
Work in this PR is laid out to be reviewed easier commit-by-commit.
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests
- I added test coverage for Go changes...
- [x] in their respective `*_test.go` for unit tests.
- [x] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] I do not want this change to show in the release notes.
- [ ] I want the title to show in the release notes with a link to this pull request.
- [x] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/9373
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
Mainly a port of https://github.com/go-gitea/gitea/pull/31082.
closes#3573
## Screenshots
---
## Checklist
### Tests
- I added test coverage for Go changes...
- [x] in their respective `*_test.go` for unit tests.
- [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] I do not want this change to show in the release notes.
- [x] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
<!--start release-notes-assistant-->
## Release notes
<!--URL:https://codeberg.org/forgejo/forgejo-->
- User Interface features
- [PR](https://codeberg.org/forgejo/forgejo/pulls/8759): <!--number 8759 --><!--line 0 --><!--description YWRkIHRhZyBsYWJlbCB0byBjb21taXQgbGlzdCB2aWV3-->add tag label to commit list view<!--description-->
<!--end release-notes-assistant-->
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/8759
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: pat-s <patrick.schratz@gmail.com>
Co-committed-by: pat-s <patrick.schratz@gmail.com>
- Use the keying module, that was introduced in forgejo/forgejo#5041, to store action secrets safely and securely in the database.
- Introduce a central function that sets the secret, `SetSecret` and let the caller do the update call. This is similar to how the twofactor (TOTP) models does it. Ref. https://codeberg.org/forgejo/forgejo/pulls/6074
- Add a relaxed migration, that is run inside a transaction. If it cannot decrypt a action secret, then it's deleted.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/8692
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Gusted <postmaster@gusted.xyz>
Co-committed-by: Gusted <postmaster@gusted.xyz>
- Some SQL queries were not being run in the transaction of v32, which could lead to the migration failing or hanging indefinitely.
- Use `db.WithTx` to get a `context.Context` that will make sure to run SQL queries in the transaction.
- Using `db.DefaultContext` is fine to be used as parent context for starting the transaction, in all cases of starting the migration `x` and `db.DefaultContext` will point to the same engine.
- Resolvesforgejo/forgejo#8580
## Testing
1. Have a v11 Forgejo database with a maven package.
2. Run this migration.
<!--start release-notes-assistant-->
## Release notes
<!--URL:https://codeberg.org/forgejo/forgejo-->
- Bug fixes
- [PR](https://codeberg.org/forgejo/forgejo/pulls/8609): <!--number 8609 --><!--line 0 --><!--description dXBncmFkZSBmYWlscyBvciBoYW5nIGF0IG1pZ3JhdGlvblszMl06IE1pZ3JhdGUgbWF2ZW4gcGFja2FnZSBuYW1lIGNvbmNhdGVuYXRpb24=-->upgrade fails or hang at migration[32]: Migrate maven package name concatenation<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/8609
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Reviewed-by: JSchlarb <jschlarb@noreply.codeberg.org>
Co-authored-by: Gusted <postmaster@gusted.xyz>
Co-committed-by: Gusted <postmaster@gusted.xyz>
Adds the ability to selectively choose which branches are pushed to a mirror.
This change adds an additional text box on the repository settings for each push mirror. Existing behavior is preserved when the field is left blank.
When the repository is being pushed, only branches matching the comma separated branch filter are pushed.
Resolvesforgejo/forgejo#7242
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/7823
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: Paul Campbell <pcampbell@kemitix.net>
Co-committed-by: Paul Campbell <pcampbell@kemitix.net>
Closes#8119, follow-up of #8234#8234 "only" fixed the case when a new wiki setting was applied.
However it lacked 2 aspects:
- fixing the already corrupted unit permission in the database (required a migration)
- fixing the API route
Both aspects should now be covered.
Additionally, I commented out the unused `UnitAccessMode` and indicated that they are only used for the wiki-unit (hopefully saving some time to future code-readers).
### Testing
- go to a commit before #8234 (e.g. 285f66b782)
- create 3 repositories
- save the wiki settings of the second, without any change
- set the wiki of the third to be globally writable
- verify the `default_permissions` column in the database, of the unit `5`: 0, 2, 3
- stop Forgejo, switch to this PR and start Forgejo
- verify that the logs writes `Migration[35]: Fix wiki unit default permission`
- verify the `default_permissions` column in the database, of the unit `5`: 0, 0, 3
Before:
After:
- [x] I did not document these changes and I do not expect someone else to do it.
- [x] I do not want this change to show in the release notes.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/8258
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: oliverpool <git@olivier.pfad.fr>
Co-committed-by: oliverpool <git@olivier.pfad.fr>
