Skip to content

fix(computer-use): add control action to restart the native helper - #7614

Open
chenzier wants to merge 1 commit into
agentscope-ai:mainfrom
chenzier:fix/computer-use-helper-tcc-restart
Open

chenzier wants to merge 1 commit into
agentscope-ai:mainfrom
chenzier:fix/computer-use-helper-tcc-restart

Conversation

@chenzier

@chenzier chenzier commented Sep 7, 2026

Copy link
Copy Markdown

Description

On macOS, TCC decisions (Screen Recording, Accessibility) are cached per process. A Computer Use helper that was started before the user granted those permissions keeps being denied even after the grant lands in System Settings — the running helper simply never sees it. Today the only way to make the authorization take effect is restarting the whole desktop app.

This PR adds a restart action to the computer-use control protocol:

  • Desktop host (computer_use_runtime.rs): on restart, stop the live helper and start a fresh one, answering with the new capability. Reuses the existing stop_helper / ensure / runtime_capability primitives; follows the same response shape as acquire.
  • Backend (app/computer_use/runtime.py): expose HostRuntimeProvider.restart_helper(). It retires the cached capability and the environment bootstrap (both name the old helper's endpoint) whether or not the replacement comes up, and caches the new capability when it does — so a failed restart degrades to the existing acquire-again recovery path instead of handing out a dead endpoint. The response parsing is extracted into _capability_from_response() and shared with _request_capability().
  • Tests: three new cases in test_capability_recovery.py covering the replace-success, failed-restart-stops-answering-with-the-old-endpoint, and no-control-endpoint paths.

The intended call site is any consumer that observes a permission-denied result from the helper: record that a TCC grant is pending, wait for the user to authorize, then call restart_helper() before the next operation instead of asking the user to relaunch the app.

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation
  • Refactoring

Component(s) Affected

  • Core / Backend (app, agents, config, providers, utils, local_models)
  • Console (frontend web UI)
  • Channels (DingTalk, Lark, QQ, Discord, iMessage, etc.)
  • Skills
  • CLI
  • Documentation (website)
  • Tests
  • CI/CD
  • Scripts / Deploy

Checklist

  • I ran pre-commit run --all-files locally and it passes
  • If pre-commit auto-fixed files, I committed those changes and rerun checks
  • I ran tests locally (pytest or as relevant) and they pass
  • Documentation updated (not needed for this fix)
  • Ready for review

Testing

Reproducing the original problem on macOS:

  1. Launch the desktop app with Computer Use available, but deny Screen Recording / Accessibility when prompted, so the helper starts without the grant.
  2. Grant the permissions in System Settings → Privacy & Security.
  3. Any further computer-use operation from the same session still fails as if unpermitted — the running helper cannot observe the new TCC decision; only a full app restart clears it.

With this change, calling HostRuntimeProvider.restart_helper() after the grant replaces the stale helper and returns a working capability.

Verified locally:

$ python -m pytest tests/unit/plugins/computer_use/test_capability_recovery.py -q
8 passed in 0.16s

$ cd console/src-tauri && cargo test --lib computer_use_runtime
test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured

$ cargo check --lib
Finished `dev` profile [unoptimized + debuginfo] target(s) in 1m 01s

Evidence

  • tests/unit/plugins/computer_use/test_capability_recovery.py gains three regression tests for the restart path (capability replaced, stale endpoint retired on failure, no-control-endpoint guard).
  • Behavior on the Rust side mirrors the existing acquire branch (stop → ensure → runtime_capability), and the existing computer_use_runtime unit tests still pass after the new match arm.

Security Considerations

The new control action is authenticated by the same control-token check as every other action and only restarts a helper process that this host already manages; no new listening surface or privilege is introduced.

macOS caches TCC decisions per process. A Computer Use helper that was
started before the user granted Screen Recording or Accessibility keeps
being denied even after the grant, so the only way to pick the
authorization up today is restarting the whole desktop app.

Add a "restart" action to the control protocol: the desktop host stops
the live helper and starts a fresh one, answering with its capability.
Expose it to the backend as HostRuntimeProvider.restart_helper(), which
also retires the cached capability and the environment bootstrap (both
name the old helper's endpoint) whether or not the replacement comes
up, and caches the new capability when it does.
@github-project-automation github-project-automation Bot moved this to Todo in QwenPaw Sep 7, 2026
@github-actions github-actions Bot added the first-time-contributor PR created by a first time contributor label Sep 7, 2026
@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown

Welcome to QwenPaw! 🐾

Hi @chenzier, thank you for your first Pull Request! 🎉

🙌 Join Developer Community

Thanks so much for your contribution! We'd love to invite you to join the official QwenPaw developer group! You can find the Discord and DingTalk group links under the "Developer Community" section on our docs page:
https://qwenpaw.agentscope.io/docs/community

We truly appreciate your enthusiasm—and look forward to your future contributions! 😊

We'll review your PR soon.


Tip

⭐ If you find QwenPaw useful, please give us a Star!

Star QwenPaw

Staying ahead

Star QwenPaw on GitHub and be instantly notified of new releases.

Your star helps more developers discover this project! 🐾

This branch had an error being deployed

2 failed deployments
maintainer-approved — a8308860 Deployed Sep 10, 2026 by chenzier via Maintainer Approval #9120
ai-review-approved — a8308860 Deployed Sep 7, 2026 by chenzier via AI Review Approval #3635
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

first-time-contributor PR created by a first time contributor

Projects

Status: Todo

Development

Successfully merging this pull request may close these issues.

2 participants