Repository navigation
fix(token-budget): acquire _lock in _apply, before_agent, _clear_run_state, and _drain_pending_warnings - #3714
Conversation
…pending_warnings _lock was defined in __init__ and used only in reset(), leaving _apply(), before_agent(), _clear_run_state(), and _drain_pending_warnings() unprotected. Concurrent after_model callbacks for the same run_id could race on usage_accum.input += diff_input, causing lost-update on token counters. Also fix stray bare 'a' character in frontend/src/content/zh/introduction/index.mdx that rendered as visible text on the documentation page.
|
@kapil971390, thanks for your contribution. Please click the CLA assistant to sign the CLA first, before we star to review this PR. |
|
@WillemJiang Hi! I've tried signing the CLA multiple times — the cla-assistant page shows "You have agreed to the CLA for multiple repositories or organizations" under my account ( I also tried the recheck link (https://cla-assistant.io/check/bytedance/deer-flow?pullRequest=3714) but it hasn't updated. Could you help clarify if there's an additional step I'm missing, or manually trigger a recheck on your end? Happy to do whatever is needed to get this unblocked. Thanks! |
|
@WillemJiang Update: the CLA status has turned green — |
…pending_warnings (bytedance#3714) _lock was defined in __init__ and used only in reset(), leaving _apply(), before_agent(), _clear_run_state(), and _drain_pending_warnings() unprotected. Concurrent after_model callbacks for the same run_id could race on usage_accum.input += diff_input, causing lost-update on token counters. Also fix stray bare 'a' character in frontend/src/content/zh/introduction/index.mdx that rendered as visible text on the documentation page. Co-authored-by: kapilvus <kapilvus@gmail.com>
…pending_warnings (bytedance#3714) _lock was defined in __init__ and used only in reset(), leaving _apply(), before_agent(), _clear_run_state(), and _drain_pending_warnings() unprotected. Concurrent after_model callbacks for the same run_id could race on usage_accum.input += diff_input, causing lost-update on token counters. Also fix stray bare 'a' character in frontend/src/content/zh/introduction/index.mdx that rendered as visible text on the documentation page. Co-authored-by: kapilvus <kapilvus@gmail.com>
Why
TokenBudgetMiddlewaredefinesself._lock = threading.Lock()in__init__and acquires it only inreset(). The four methods that mutate per-run state —before_agent,_apply,_clear_run_state, and_drain_pending_warnings— all access the sameBoundedDictstructures (_seen_messages,_cumulative_usage,_warned,_pending_warnings) without holding the lock.LangGraph can call
after_modelconcurrently for parallel agent nodes that share the same run. That means two_apply()calls can interleave on the sameusage_accumobject, producing a lost-update race onusage_accum.input += diff_input. The budget counter silently under-counts, so hard-stop and warn thresholds may never fire.The second change removes a stray bare
acharacter infrontend/src/content/zh/introduction/index.mdx(line 5) that renders as visible literal text on the Chinese introduction documentation page.What changed
before_agent: shared-state mutations (setdefault, loop writes toseen) now run underwith self._lock:_apply: the entire per-run accumulation + threshold evaluation block now runs underwith self._lock:_clear_run_state: the fourpopcalls now run underwith self._lock:(was unprotected even thoughafter_agentcalls it while_applymay still be running)_drain_pending_warnings: thepopfrom_pending_warningsnow runs underwith self._lock:(prevents a TOCTOU race with_apply'sappend)previous runss→previous runsfrontend/src/content/zh/introduction/index.mdx: removed strayabetween front matter---and the firstimportstatementSurface area
frontend/backend/applanggraph.json, or prompt changedocker/or sandboxed executionskills/backend/pyproject.tomlorfrontend/package.json(say what it buys us)Screenshots / Recording
N/A — backend threading fix and a one-line doc file correction.
Bug fix verification
A minimal reproducer would spin up two threads calling
_apply()concurrently with the samerun_idand assert the finalusage_accum.totalequals the sum of both threads' token deltas (currently it under-counts non-deterministically). Writing that test requires internal access to aRuntimestub; I can add it if the team provides a test harness fixture forRuntime. In the meantime:usage_accum.input += diff_inputis a read-modify-write on a plain Pythonintattribute, which CPython's GIL does not protect across object attribute access in LangGraph's async task scheduling.with self._lock:around all four unprotected mutation sites, matching the pattern already used inreset().Validation
Full
make lint && make testrequiresuvand the full dependency graph which I could not install locally. The change is purely additive (with self._lock:wrapping) with no logic changes — the sameif/for/+=expressions run, just under the mutex that was already present forreset().AI assistance
Tool(s) used: Claude Code
How you used it: Used Claude Code to analyze the middleware source, identify the unprotected mutation sites by comparing
reset()(which uses the lock) against_apply()andbefore_agent()(which do not), and generate thewith self._lock:wrapping. I read every line of the diff and verified the lock scope covers all four mutation paths.