jojo/services
Nils Goroll fb2f666535 fix: Wait & retry when primary rate limit are hit for Github migration (#10846)
This is a successor to #10805, which simply did not work. It is also much simpler and basically a one line change to enable an existing feature in [go-github](https://github.com/google/go-github).

Fixes #10845

With this fix and #10798 in place, a migration of a repo with ~3K issues and ~1.3k pull requests finally completed successfully.

## Patch

We use SleepUntilPrimaryRateLimitResetWhenRateLimited to instruct the go-github code to wait until the retry time and retry the request when the primary rate limit gets hit.

## Test case

TestGitHubDownloadRepo() has been modified such that 403 rate limit errors are injected every 7 requests with a retry time of one second, resulting in the rate limit condition being hit twice with the current tests. The test case confirms that the migration code itself is in fact unaffected by the rate limit being hit.

## Scope

This change does not affect secondary rate limits.

If the server is restarted during the wait for the rate limit refresh, the migration likely still fails when retried, because inserts for already present database objects will be attempted.

This approach effectively puts the task's goroutine to sleep until the retry time, which implies that the respective resources stay allocated.

A better approach might be to add the necessary infrastructure to support restarts of migration tasks at a later time, but this is much more involved, because the migration state would need to be saved and/or re-created based on already pulled data. This would also require adding support for database upserts.

Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/10846
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: Nils Goroll <nils.goroll@uplex.de>
Co-committed-by: Nils Goroll <nils.goroll@uplex.de>
2026-01-26 14:47:08 +01:00
..
actions fix: make concurrency group job cancellation effect runs that are failed (#10863) 2026-01-16 10:54:01 +01:00
agit fix: correctly mark reviews as stale for AGit PRs (#8450) 2025-07-09 07:38:00 +02:00
asymkey chore: do not clobber ~/.ssh/authorized_keys in certain tests (#10163) 2025-11-19 16:14:16 +01:00
attachment chore: move all test blank imports in a single package (#10662) 2026-01-02 05:32:32 +01:00
auth feat: add OIDC workload identity federation support (#10481) 2026-01-15 03:39:00 +01:00
automerge fix: check PR reference on base repository (#8431) 2025-07-06 14:23:47 +02:00
context Move web app manifest to a own cache-able route and add a setting to set "display": "standalone"; Closes #2638 (#5384) 2026-01-09 17:49:29 +01:00
contexttest Replace the 'relative-time' element scripting with custom, translatable rewrite (#6154) 2025-05-03 14:11:01 +00:00
convert fix: hide user profile anonymous options on public repo APIs 2026-01-06 09:58:20 -07:00
cron fix: garbage collect lingering actions logs (#10009) 2025-11-18 18:59:01 +01:00
doctor feat: ensure only expected ssh public keys are in authorized_keys file (#10010) 2025-11-09 01:06:04 +01:00
externalaccount chore(cleanup): replaces unnecessary calls to formatting functions by non-formatting equivalents (#7994) 2025-05-29 17:34:29 +02:00
f3 chore: update gof3/v3 v3.11.15 (#10673) 2026-01-13 16:59:56 +01:00
federation fix(user): set ActivityPub users to ProhibitLogin (#10434) 2025-12-17 15:38:32 +01:00
feed fix: load reviewer for pull review dismiss action notifier 2026-01-06 09:58:20 -07:00
forgejo chore: move all test blank imports in a single package (#10662) 2026-01-02 05:32:32 +01:00
forms feat: allow sync quota groups with oauth2 auth source (#8554) 2025-12-01 14:12:00 +01:00
gitdiff chore: move all test blank imports in a single package (#10662) 2026-01-02 05:32:32 +01:00
indexer chore: branding import path (#7337) 2025-03-27 19:40:14 +00:00
issue chore: move all test blank imports in a single package (#10662) 2026-01-02 05:32:32 +01:00
lfs fix: return on error if an LFS token cannot be parsed 2025-10-25 10:41:49 -06:00
mailer fix: don't display pending reviews as participants (#10528) 2026-01-06 10:47:21 +01:00
markup chore: remove branding from context imports (#9628) 2025-10-11 01:52:51 +02:00
migrations fix: Wait & retry when primary rate limit are hit for Github migration (#10846) 2026-01-26 14:47:08 +01:00
mirror fix: don't push LFS when using SSH authentication (#10475) 2025-12-18 23:23:07 +01:00
moderation chore: move all test blank imports in a single package (#10662) 2026-01-02 05:32:32 +01:00
notify chore: Dead Code: DeleteIssue Notify Topic (#10046) 2025-11-17 02:13:38 +01:00
org feat: add foreign keys to the action_runner_token table (#10756) 2026-01-12 21:59:40 +01:00
packages fix: reduce runtime of container cleanup by relying on mass digest cleanup (#10297) 2025-12-05 15:45:47 +01:00
pull fix: correctly compute required commit status (#10747) 2026-01-12 10:53:40 +01:00
redirect chore: move all test blank imports in a single package (#10662) 2026-01-02 05:32:32 +01:00
release chore: move all test blank imports in a single package (#10662) 2026-01-02 05:32:32 +01:00
remote chore: tune down remote user promotion debug message shown as error (#7687) 2025-04-27 20:50:48 +00:00
repository chore: remove dead code in WatchIfAuto and general model documentation (#10880) 2026-01-26 12:03:08 +01:00
secrets fix: actions variable and secret names validation (#10682) 2026-01-14 04:19:21 +01:00
shared/automerge fix: suppress false-positive error log when PR is already in the automerge queue (#9784) 2025-10-21 08:19:33 +02:00
stats fix: possible cause of invalid issue counts; cache invalidation occurs before a active transaction is committed (#10130) 2025-11-17 01:07:29 +01:00
task feat: cache derived keys for faster keying (#10114) 2025-11-16 14:29:14 +01:00
uinotification chore: branding import path (#7337) 2025-03-27 19:40:14 +00:00
user feat: add foreign keys to the action_runner_token table (#10756) 2026-01-12 21:59:40 +01:00
webhook chore: move all test blank imports in a single package (#10662) 2026-01-02 05:32:32 +01:00
wiki chore: move all test blank imports in a single package (#10662) 2026-01-02 05:32:32 +01:00