Skip to content

fix(token-budget): acquire _lock in _apply, before_agent, _clear_run_state, and _drain_pending_warnings - #3714

Merged
WillemJiang merged 1 commit into
bytedance:mainfrom
kapil971390:fix/token-budget-middleware-lock-race
Jun 22, 2026
Merged

WillemJiang merged 1 commit into
bytedance:mainfrom
kapil971390:fix/token-budget-middleware-lock-race

Conversation

@kapil971390

Copy link
Copy Markdown
Contributor

Why

TokenBudgetMiddleware defines self._lock = threading.Lock() in __init__ and acquires it only in reset(). The four methods that mutate per-run state — before_agent, _apply, _clear_run_state, and _drain_pending_warnings — all access the same BoundedDict structures (_seen_messages, _cumulative_usage, _warned, _pending_warnings) without holding the lock.

LangGraph can call after_model concurrently for parallel agent nodes that share the same run. That means two _apply() calls can interleave on the same usage_accum object, producing a lost-update race on usage_accum.input += diff_input. The budget counter silently under-counts, so hard-stop and warn thresholds may never fire.

The second change removes a stray bare a character in frontend/src/content/zh/introduction/index.mdx (line 5) that renders as visible literal text on the Chinese introduction documentation page.

What changed

  • before_agent: shared-state mutations (setdefault, loop writes to seen) now run under with self._lock:
  • _apply: the entire per-run accumulation + threshold evaluation block now runs under with self._lock:
  • _clear_run_state: the four pop calls now run under with self._lock: (was unprotected even though after_agent calls it while _apply may still be running)
  • _drain_pending_warnings: the pop from _pending_warnings now runs under with self._lock: (prevents a TOCTOU race with _apply's append)
  • Also fixed a typo in an inline comment: previous runss → previous runs
  • frontend/src/content/zh/introduction/index.mdx: removed stray a between front matter --- and the first import statement

Surface area

  • Frontend UI — page / component / setting / interaction under frontend/
  • Backend API — endpoint / SSE event / request-response shape under backend/app
  • Agents / LangGraph — agent node, graph wiring, langgraph.json, or prompt change
  • Sandbox — docker/ or sandboxed execution
  • Skills — change under skills/
  • Dependencies — new/upgraded entry in backend/pyproject.toml or frontend/package.json (say what it buys us)
  • Default behavior change — changes existing behavior without the user opting in (default model, default setting, data shape)
  • Docs / tests / CI only — no runtime behavior change

Screenshots / Recording

N/A — backend threading fix and a one-line doc file correction.

Bug fix verification

A minimal reproducer would spin up two threads calling _apply() concurrently with the same run_id and assert the final usage_accum.total equals the sum of both threads' token deltas (currently it under-counts non-deterministically). Writing that test requires internal access to a Runtime stub; I can add it if the team provides a test harness fixture for Runtime. In the meantime:

  • The race is deterministic by inspection: usage_accum.input += diff_input is a read-modify-write on a plain Python int attribute, which CPython's GIL does not protect across object attribute access in LangGraph's async task scheduling.
  • The fix is mechanical: add with self._lock: around all four unprotected mutation sites, matching the pattern already used in reset().

Validation

python3 -m ruff check backend/packages/harness/deerflow/agents/middlewares/token_budget_middleware.py
# All checks passed!

python3 -m ruff format --check backend/packages/harness/deerflow/agents/middlewares/token_budget_middleware.py
# 1 file already formatted

Full make lint && make test requires uv and the full dependency graph which I could not install locally. The change is purely additive (with self._lock: wrapping) with no logic changes — the same if/for/+= expressions run, just under the mutex that was already present for reset().

AI assistance

Tool(s) used: Claude Code

How you used it: Used Claude Code to analyze the middleware source, identify the unprotected mutation sites by comparing reset() (which uses the lock) against _apply() and before_agent() (which do not), and generate the with self._lock: wrapping. I read every line of the diff and verified the lock scope covers all four mutation paths.

  • I've read and understand every line of this change and take responsibility for it — it's not unreviewed AI output.

…pending_warnings

_lock was defined in __init__ and used only in reset(), leaving _apply(),
before_agent(), _clear_run_state(), and _drain_pending_warnings() unprotected.
Concurrent after_model callbacks for the same run_id could race on
usage_accum.input += diff_input, causing lost-update on token counters.

Also fix stray bare 'a' character in frontend/src/content/zh/introduction/index.mdx
that rendered as visible text on the documentation page.
@CLAassistant

CLAassistant commented Jun 22, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@github-actions github-actions Bot added area:agents Agents, subagents, graph wiring, prompts, langgraph.json area:docs Documentation and Markdown only area:frontend Next.js frontend under frontend/ needs-validation Touches front/back contract surface; needs real-path validation risk:high High risk: backend API, agents, sandbox, auth, deps, CI size/M PR changes 100-300 lines labels Jun 22, 2026
@WillemJiang

Copy link
Copy Markdown
Collaborator

@kapil971390, thanks for your contribution. Please click the CLA assistant to sign the CLA first, before we star to review this PR.

@kapil971390

Copy link
Copy Markdown
Contributor Author

@WillemJiang Hi! I've tried signing the CLA multiple times — the cla-assistant page shows "You have agreed to the CLA for multiple repositories or organizations" under my account (kapil971390), but the license/cla status on this PR is still showing as PENDING on GitHub.

I also tried the recheck link (https://cla-assistant.io/check/bytedance/deer-flow?pullRequest=3714) but it hasn't updated. Could you help clarify if there's an additional step I'm missing, or manually trigger a recheck on your end? Happy to do whatever is needed to get this unblocked. Thanks!

@kapil971390

Copy link
Copy Markdown
Contributor Author

@WillemJiang Update: the CLA status has turned green — license/cla → SUCCESS. All CI checks are passing as well. The PR is ready for review whenever you get a chance. Thanks!

@WillemJiang
WillemJiang merged commit d050fba into bytedance:main Jun 22, 2026
12 checks passed
@WillemJiang WillemJiang added this to the 2.1.0 milestone Jun 22, 2026
marvin9551 pushed a commit to marvin9551/deer-flow that referenced this pull request Aug 21, 2026
…pending_warnings (bytedance#3714)

_lock was defined in __init__ and used only in reset(), leaving _apply(),
before_agent(), _clear_run_state(), and _drain_pending_warnings() unprotected.
Concurrent after_model callbacks for the same run_id could race on
usage_accum.input += diff_input, causing lost-update on token counters.

Also fix stray bare 'a' character in frontend/src/content/zh/introduction/index.mdx
that rendered as visible text on the documentation page.

Co-authored-by: kapilvus <kapilvus@gmail.com>
jihtsan pushed a commit to jihtsan/dnx-deer-flow that referenced this pull request Aug 29, 2026
…pending_warnings (bytedance#3714)

_lock was defined in __init__ and used only in reset(), leaving _apply(),
before_agent(), _clear_run_state(), and _drain_pending_warnings() unprotected.
Concurrent after_model callbacks for the same run_id could race on
usage_accum.input += diff_input, causing lost-update on token counters.

Also fix stray bare 'a' character in frontend/src/content/zh/introduction/index.mdx
that rendered as visible text on the documentation page.

Co-authored-by: kapilvus <kapilvus@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:agents Agents, subagents, graph wiring, prompts, langgraph.json area:docs Documentation and Markdown only area:frontend Next.js frontend under frontend/ needs-validation Touches front/back contract surface; needs real-path validation risk:high High risk: backend API, agents, sandbox, auth, deps, CI size/M PR changes 100-300 lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants