Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 21 additions & 18 deletions docs/releases/v2.3.3.md
Original file line number Diff line number Diff line change
@@ -1,44 +1,47 @@
## Rotation / Recovery / Refresh
## Runtime Rotation / Recovery

### Bugfixes

- Clamped the rate-limit / `retry-after` window to `MAX_RATE_LIMIT_DELAY_MS` (7 days). A hostile or buggy upstream value (`retry-after-ms`, `retry-after`, or `x-codex-*-reset-*` headers — e.g. a seconds-vs-milliseconds confusion or an anti-abuse misfire) was passed through unbounded, and the setter only lower-clamped while growing `resetAt` monotonically, so a single bad response could wedge an account unavailable for ~31 years with no self-heal. The clamp is now applied centrally in `markRateLimitedWithReason` and at source in `getQuotaNearExhaustionWaitMs`, matching the long-standing clamp in the legacy reactive path (H1).
- Added a per-owner nonce to the refresh lease. `release()` previously called `safeUnlink(lockPath)` unconditionally; if an owner's lease expired (a refresh taking ~as long as the TTL) and another process stole the lock, the slow owner would delete the new owner's lock on release, leaving two concurrent refreshers — which, because the refresh token is rotated per refresh, could submit an already-consumed token and log the account out. `release()` now reads the lock back and only unlinks when the on-disk nonce still matches; a lock written before this change (no nonce) keeps the prior best-effort behavior (H2).
- Stopped a cross-process token clobber on routine saves. `saveToDisk` discarded the disk-loaded state and re-serialized the entire in-memory pool, so if a second process refreshed an account and wrote a rotated single-use refresh token, a routine save (cooldown, rate-limit, refund) by a long-lived proxy could last-writer-win and revert it, permanently breaking that account's next refresh. The save now reconciles per-account token material from disk under the storage lock, adopting a strictly-newer on-disk token; the refresh-commit path still wins with its own fresh token (H3).
- Fixed an unbounded rate-limit window that could wedge an account unavailable for years. A `429` (or a `2xx` carrying `x-codex-*-reset-*` headers) was honored using the upstream `retry-after`/reset value with no upper clamp, and the setter only lower-clamped while growing `resetAt` monotonically — so a single hostile or buggy response (a seconds-vs-milliseconds confusion, an anti-abuse misfire, a `retry-after-ms` of `999999999999`) marked the account rate-limited for ~31 years, persisted that to disk, and never self-healed. Because the lockout is per-account the pool was never fully exhausted, so the stale-recovery guard never fired either. Retry and quota windows are now clamped to `MAX_RATE_LIMIT_DELAY_MS` (7 days), applied centrally in `markRateLimitedWithReason` and again at source in `getQuotaNearExhaustionWaitMs`, matching the clamp the legacy reactive fetch path already enforced (#617).
- Fixed the refresh lease deleting a lock it no longer owned. The lease wrote a `pid`/`acquiredAt` payload but never read it back at release, and `release()` called `safeUnlink(lockPath)` unconditionally. If an owner's refresh ran about as long as the lease TTL, its lease expired, a second process stole the lock, and the slow owner then deleted the new owner's lock on completion — leaving two concurrent refreshers. Because the OAuth refresh token rotates per refresh, the losing process could submit an already-consumed token and log the account out until re-login. The lock now carries a per-owner nonce and `release()` only unlinks when the on-disk nonce still matches; a lock written before this change (no nonce) keeps the prior best-effort behavior. This is scoped to deployments running more than one CLI/proxy instance against a shared auth directory (#617).
- Fixed a cross-process clobber of freshly-rotated refresh tokens. `saveToDisk` discarded the disk-loaded state and re-serialized the entire in-memory pool, so when a second process refreshed an account and wrote a rotated single-use token, a routine save from a long-lived proxy (cooldown, rate-limit, near-quota refund) could last-writer-win and revert it — permanently breaking that account's next refresh. The save now reconciles per-account token material from disk under the storage lock, adopting a strictly-newer on-disk token; the refresh-commit path still wins with its own fresher token (#617).

## Quota / Forecast

### Bugfixes

- A transient 429 no longer benches an account for the full 2h deferral cap. `markRateLimited` folded the existing weekly secondary reset (~7d out on a healthy 200 snapshot) into the primary window via `max(...)`, so a 30–60s `Retry-After` produced a multi-hour deferral. The 429 path now treats only genuinely-exhausted windows (used ≥ 100% or no usage gauge) as rate-limit windows, in both `markRateLimited` and `getDeferral`; the documented "longest active reset window" behavior for real rate-limit windows is preserved (H4).
- Fixed the live-quota forecast overstating the wait and inverting the recommendation. `getLiveQuotaWaitMs` took a blind `max` of both quota windows, so a healthy weekly secondary (~7d) dominated a binding 5h primary that frees in seconds, and `recommendForecastAccount` then preferred a strictly-worse account. Under usage pressure it now filters to exhausted windows (mirroring the quota-cache path); a 429 still honors every active window (H5).
- Fixed a transient `429` benching an account for the full deferral cap. `markRateLimited` folded the existing weekly secondary reset (normally ~7 days out on a healthy `200` snapshot) into the primary window via `max(...)`, so a blip with a 30–60s `Retry-After` produced a multi-hour deferral and the account was rotated away for the full 2h cap. The `429` path now treats only genuinely-exhausted windows (used ≥ 100%, or a window with no usage gauge) as rate-limit windows, in both `markRateLimited` and `getDeferral`; the documented "longest active reset window" behavior for real rate-limit windows is unchanged (#617).
- Fixed the live-quota forecast overstating the wait and inverting the recommendation. `getLiveQuotaWaitMs` took a blind `max` of both quota windows, so a healthy weekly secondary (~7 days) dominated a binding 5h primary that frees in seconds, and `recommendForecastAccount` — which sorts ascending by wait — then preferred a strictly-worse account and displayed a wait wrong by orders of magnitude. Under usage pressure it now filters to exhausted windows, mirroring the quota-cache path; a `429` still honors every active window (#617).

## Request / SSE Data Path

### Bugfixes

- Mid-stream SSE failures are no longer reported as success. A `{"type":"error"}` event, or a terminal `response.failed` event, caused `convertSseToJson` to return the raw SSE text at HTTP 200; downstream this ran the success path, `JSON.parse` threw on the SSE body and was swallowed, and the account was recorded as a success with rotation/retry suppressed. These now resolve to a synthesized non-2xx so the caller routes to failure (H6/H7).
- The SSE parser accepted only `data: ` with a trailing space; a spec-valid `data:value` line parsed as zero events, degrading a response to "no final response". It now accepts `data:` with optional whitespace (M1).
- `response.incomplete` (hitting `max_output_tokens` or a content filter) is handled as a normal early stop: it carries a final response object whose partial output is the answer, so it is delivered at HTTP 200 and counts as a healthy account — distinct from the `response.failed` failure path above.
- Fixed upstream SSE failures being reported to the client as success. A mid-stream `{"type":"error"}` event, or a terminal `response.failed` event, left `convertSseToJson` returning the raw SSE text at HTTP `200`; downstream this ran the success path, the empty-response guard's `JSON.parse` threw on the SSE body and was swallowed, and the account was recorded as a success with rotation and retry suppressed. A stream that opens `200` but ends without a successful final response now resolves to a synthesized non-`2xx` so the caller routes to failure, while a stream that simply yields no events without an error is still passed through so the empty-response retry path is preserved (#617, #618).
- Fixed the SSE parser requiring a trailing space after `data:`. A spec-valid `data:value` line (no space) parsed as zero events, silently degrading the response to "no final response" on any upstream or proxy formatting change. The parser now accepts `data:` with optional whitespace (#617).

### Behavior

- `response.incomplete` (hitting `max_output_tokens` or a content filter) is treated as a normal early stop, not a failure. It carries a final response object whose partial output is the answer, so it is delivered at HTTP `200` and counts as a healthy account — distinct from the `response.failed` path above, which routes to failure (#618).

## Storage / Auth / Logging

### Bugfixes

- V1→V3 storage migration no longer discards the migrated account bodies. `normalizeAccountStorage` rebuilt the account list from the raw V1 objects, dropping `migrateV1ToV3`'s scalar `rateLimitResetTime` → map `rateLimitResetTimes` conversion, so a rate-limited account upgrading from V1 was treated as immediately available and could burst 429s. The account list is now built from the migrated storage (M3).
- The local-client-token store fsyncs the temp file before rename. The previous `writeFile` + `rename` left a window where a crash or power-loss after the rename could truncate the store; the temp file is now flushed with `fsync` before the rename, matching the durable-write pattern already used by the app-bind and first-run writers (L3).
- OAuth `expires_in` (and the internal `expires`) now require a positive integer. A zero or negative value previously minted an already-expired token, which drove a tight refresh loop that consumed the single-use refresh token. A bogus value now fails schema validation instead (I1).
- The free-text log scrubber now masks this package's own local bearer tokens (`cma_local_…`) in addition to the existing JWT, long-hex, `sk-`, and `Bearer` patterns. Structured logging already masked by key and the OAuth path is scrubbed separately; this closes the last-line-of-defense gap for the project's own token shape (I2).
- Fixed the V1→V3 storage migration discarding the migrated account bodies. `normalizeAccountStorage` rebuilt the account list from the raw V1 objects rather than the migration output, dropping `migrateV1ToV3`'s scalar `rateLimitResetTime` → map `rateLimitResetTimes` conversion — so a rate-limited account upgrading from V1 was read with no reset times, treated as immediately available, and could burst `429`s. The account list is now built from the migrated storage (#619).
- Fixed a durability gap in the local-client-token store. The store wrote a temp file and renamed it with no `fsync` in between, so a crash or power-loss after the rename could leave it truncated. The temp file is now flushed with `fsync` before the rename, matching the durable-write pattern already used by the app-bind and first-run writers (#619).
- Fixed OAuth `expires_in` (and the internal `expires`) accepting any number. A zero or negative value minted an already-expired token, which drove a tight refresh loop that consumed the single-use refresh token; the value now must be a positive integer or it fails schema validation (#619).
- Hardened the free-text log scrubber to mask this package's own local bearer tokens (`cma_local_…`) alongside the existing JWT, long-hex, `sk-`, and `Bearer` patterns. Structured logging already masks by key and the OAuth path is scrubbed separately; this closes the last-line-of-defense gap for the project's own token shape (#619).

## Testing

### Improvements

- Added regression coverage for every fix: the 7-day retry-after clamp (including self-heal after the window) and the at-source quota clamp; the lease ownership nonce (a slow owner not deleting a stolen lock); the cross-process token-clobber reconciliation; the transient-429 deferral bound and the preserved "longest active reset window" semantics; the forecast exhausted-window filter and non-inverted recommendation; SSE `error` / `response.failed` → non-2xx, `response.incomplete` → 200 with partial output, and `data:` without a trailing space; the V1→V3 `rateLimitResetTimes` preservation; the OAuth `expires_in` positive-integer bound; and the `cma_local` log-masking pattern.
- Updated four existing tests that asserted the prior buggy behavior (SSE error events returning a raw HTTP 200) to assert the corrected failure routing.
- Added regression coverage for every fix: the 7-day retry-after clamp (including self-heal once the window elapses) and the at-source quota clamp; the lease ownership nonce, proving a slow owner does not delete a stolen lock; the cross-process token-clobber reconciliation; the transient-`429` deferral bound alongside the preserved "longest active reset window" semantics; the forecast exhausted-window filter and the no-longer-inverted recommendation; SSE `error`/`response.failed` → non-`2xx`, `response.incomplete` → `200` with partial output, and `data:` parsing without a trailing space; the V1→V3 `rateLimitResetTimes` preservation; the OAuth `expires_in` positive-integer bound; and the `cma_local` log-masking pattern.
- Updated four existing tests that asserted the prior behavior (SSE error events returning a raw HTTP `200`) to assert the corrected failure routing.

## Notes

- Patch release; intended for the `latest` dist-tag (`npm i -g codex-multi-auth`).
- No runtime-rotation routing, account-selection, storage layout, or normal auth-flow behavior changed. The fixes harden failure, concurrency, and edge-case paths.
- The multi-process fixes (H2 lease ownership, H3 token reconciliation) matter most for deployments that run more than one CLI/proxy instance against a shared auth directory; single-process usage is unaffected by those races.
- Patch release published under the `latest` dist-tag (`npm i -g codex-multi-auth`).
- No runtime-rotation routing, account-selection, storage layout, or normal auth-flow behavior changed; the fixes harden failure, concurrency, and edge-case paths.
- The multi-process fixes (lease ownership, token reconciliation) matter most when more than one CLI/proxy instance shares an auth directory; single-process usage is unaffected by those races.