Skip to content

feat: executor tool invocation for js-exec#171

Closed
cramforce wants to merge 13 commits into
mainfrom
executor
Closed

feat: executor tool invocation for js-exec#171
cramforce wants to merge 13 commits into
mainfrom
executor

Conversation

@cramforce
Copy link
Copy Markdown
Contributor

@cramforce cramforce commented Mar 27, 2026

feat: experimental executor tool invocation for js-exec

Summary

Add experimental_executor option to BashOptions that gives JavaScript code
running in js-exec access to a tools proxy for calling external tools, and
auto-generates bash CLI commands
from those tools so they work directly from
the shell.

Tool calls are synchronous from the QuickJS sandbox's perspective — they block
via SharedArrayBuffer/Atomics while the host resolves them asynchronously.

Integrates with @executor-js/sdk and the official plugins (declared as
optional peer dependencies — see Risk-mitigation choices
below):

  • @executor-js/plugin-graphql — auto-discovers tools from GraphQL schemas via introspection
  • @executor-js/plugin-openapi — auto-discovers tools from OpenAPI specs
  • @executor-js/plugin-mcp — connects to MCP servers and discovers tools

Full docs: packages/just-bash/docs/EXECUTOR.md


Risk-mitigation choices

This is a sizeable feature surface and we want to land it with minimal blast
radius. Three deliberate decisions to make that easier:

  1. Option name is experimental_executor (not executor). The shape is
    subject to change; the prefix makes that obvious at every call site and
    gives us room to iterate on the API without a breaking-change cycle.
  2. @executor-js/* packages are optional peer dependencies, not direct
    dependencies.
    This keeps the just-bash install footprint unchanged for
    users who don't use the feature, and it means SDK version bumps don't
    force-update every just-bash consumer. Inline tools work without any peer
    deps installed; setup (SDK-driven discovery) requires consumers to install
    the relevant packages themselves.
  3. Dedicated docs file at
    packages/just-bash/docs/EXECUTOR.md,
    linked from the main package README and from src/commands/js-exec/README.md.
    Keeps the experimental surface visible in one place and out of the
    stable-feature docs.

A .npmrc is added at the repo root with
minimum-release-age-exclude for the four @executor-js/* packages so
contributors with a global age constraint don't need a per-command flag.


Auto-generated CLI commands

Every tool automatically becomes a bash command using the namespace +
subcommand pattern (like git commit, gh pr create):

# Inline tools
math add a=1 b=2                          # → {"sum":3}
math add --a 1 --b 2                      # → {"sum":3}
math multiply --a 3 --b 4                 # → {"product":12}

# GraphQL-discovered tools
countries country --code JP               # → {"name":"Japan",...}
countries continents                       # → [{"name":"Africa",...},...]

# OpenAPI-discovered tools (camelCase → kebab-case)
petstore list-pets --status available     # → [{"id":1,"name":"Fido"},...]
petstore get-pet --pet-id 123             # → {"id":123,"name":"Fido"}

# Piping and composition
echo '{"code":"JP"}' | countries country  # piped JSON input
math add a=1 b=2 | jq -r .sum             # → 3
countries countries | jq '.[].name' -r | head -5

# Bash loops
for code in JP US BR; do
  countries country --code "$code" | jq -r .name
done

# Store in variables
result=$(math add a=10 b=20)
echo "$result" | jq .sum  # → 30

# Redirect to files
countries countries | jq -r '.[] | .code + "," + .name' > /tmp/countries.csv

Input modes (Speakeasy-style three-tier precedence)

# 1. Flags (highest priority)
math add --a 1 --b 2
math add --a=1 --b=2
math add a=1 b=2

# 2. --json flag
math add --json '{"a": 1, "b": 2}'

# 3. Piped stdin (lowest priority, overridden by flags)
echo '{"a": 1}' | math add b=2    # → {"sum":3}  (stdin + explicit flag)
cat payload.json | petstore create-pet

Help output (gh CLI style)

$ math --help
Executor tools: math

USAGE
  math <command> [flags]

COMMANDS
  add             Add two numbers
  multiply        Multiply two numbers

EXAMPLES
  math add key=value
  math multiply --key value

LEARN MORE
  math <command> --help
$ math add --help
Add two numbers

USAGE
  math add [key=value ...]
  math add [--key value ...]
  math add --json '{...}'
  <stdin> | math add

FLAGS
  --json string    Pass all arguments as a JSON object
  --help           Show this help

EXAMPLES
  math add key=value
  math add --key value
  math add --json '{"key":"value"}'
  echo '{"key":"value"}' | math add
  math add key=value | jq -r .field

camelCase → kebab-case with aliases

petstore list-pets --status available  # kebab-case (primary)
petstore listPets --status available   # original camelCase also works

Opt-out

const bash = new Bash({
  experimental_executor: {
    tools: { ... },
    exposeToolsAsCommands: false,  // tools only available via js-exec
  },
});

js-exec tool proxy

Tools are also available from JavaScript code via the tools proxy:

const bash = new Bash({
  experimental_executor: {
    tools: {
      "math.add": {
        description: "Add two numbers",
        execute: (args) => ({ sum: args.a + args.b }),
      },
    },
  },
});

await bash.exec(`js-exec -c '
  const r = await tools.math.add({ a: 3, b: 4 });
  console.log(r.sum);  // 7
'`);

Passing experimental_executor implicitly enables javascript: true, so you
don't need both.


GraphQL tool discovery

Requires @executor-js/sdk and @executor-js/plugin-graphql peer
dependencies installed by the consumer.

const bash = new Bash({
  experimental_executor: {
    setup: async (sdk) => {
      await sdk.sources.add({
        kind: "graphql",
        endpoint: "https://countries.trevorblades.com/graphql",
        name: "countries",
      });
    },
  },
});

// Auto-discovered: countries country, countries countries,
// countries continent, countries continents, countries language, countries languages

// Works from bash:
await bash.exec('countries country --code JP | jq -r .data.name');

// Works from js-exec:
await bash.exec(`js-exec -c '
  const jp = await tools.countries.country({ code: "JP" });
  console.log(jp.data.name);
'`);

Offline introspection (no network call during setup):

await sdk.sources.add({
  kind: "graphql",
  endpoint: "https://countries.trevorblades.com/graphql",
  name: "countries",
  introspectionJson: fs.readFileSync("schema.json", "utf8"),
});

OpenAPI tool discovery

Requires @executor-js/sdk and @executor-js/plugin-openapi peer
dependencies installed by the consumer.

const bash = new Bash({
  experimental_executor: {
    setup: async (sdk) => {
      await sdk.sources.add({
        kind: "openapi",
        spec: fs.readFileSync("petstore.json", "utf8"),
        endpoint: "https://petstore3.swagger.io/api/v3",
        name: "petstore",
      });
    },
  },
});

// Auto-discovered: petstore list-pets, petstore create-pet, petstore get-pet, ...
await bash.exec('petstore list-pets --status available | jq ".[0].name"');

Tool approval and elicitation

const bash = new Bash({
  experimental_executor: {
    setup: async (sdk) => { /* ... */ },

    // Control which tools can run
    onToolApproval: async (request) => {
      if (request.operationKind === "read") return { approved: true };
      return { approved: false, reason: "only reads allowed" };
    },

    // Handle tool requests for user input (default: decline all)
    onElicitation: async (ctx) => {
      if (ctx.request._tag === "UrlElicitation") {
        await openBrowser(ctx.request.url);
        return { action: "accept" };
      }
      return { action: "decline" };
    },
  },
});

Credentials (never exposed to sandbox)

const bash = new Bash({
  experimental_executor: {
    setup: async (sdk) => {
      await sdk.sources.add({
        kind: "openapi",
        spec: mySpec,
        endpoint: "https://api.example.com",
        name: "myapi",
        headers: {
          Authorization: `Bearer ${process.env.API_TOKEN}`,
        },
      });
    },
  },
});
// Headers live in host closure → plugin internals → outbound HTTP only.
// Sandboxed code cannot inspect or exfiltrate credentials.

ExecutorConfig API

Option Type Default Description
tools Record<string, { description?; execute }> Inline tool definitions
setup (sdk) => Promise<void> SDK setup: add graphql/openapi/custom sources
plugins any[] Additional SDK plugins
onToolApproval "allow-all" | "deny-all" | callback "allow-all" Controls which tools can run
onElicitation handler | "accept-all" decline all Handles tool input requests
exposeToolsAsCommands boolean true Register tools as bash commands

Supported source kinds

Kind Plugin Config Description
"graphql" @executor-js/plugin-graphql endpoint, name, introspectionJson?, headers? Introspects schema, registers query/mutation tools
"openapi" @executor-js/plugin-openapi spec, endpoint, name, headers? Parses spec, registers operation tools
"mcp" @executor-js/plugin-mcp transport, endpoint or command, name Connects to MCP server (remote SSE or stdio)
"custom" built-in name, tools: { [name]: { execute } } Direct tool registration

Packaging

// packages/just-bash/package.json
{
  "peerDependencies": {
    "@executor-js/sdk":            "0.0.1-beta.5",
    "@executor-js/plugin-graphql": "0.0.1-beta.5",
    "@executor-js/plugin-mcp":     "0.0.1-beta.5",
    "@executor-js/plugin-openapi": "0.0.1-beta.5"
  },
  "peerDependenciesMeta": {
    "@executor-js/sdk":            { "optional": true },
    "@executor-js/plugin-graphql": { "optional": true },
    "@executor-js/plugin-mcp":     { "optional": true },
    "@executor-js/plugin-openapi": { "optional": true }
  }
}
  • Pinned to 0.0.1-beta.5 because the published 0.0.1 ("latest" tag) and
    0.0.1-beta.6 both have a broken transitive dep (@executor/config 404s on
    the registry).
  • All four are external in the esbuild bundle steps (build:lib,
    build:lib:cjs, build:cli), so the published bundle never inlines them.

Implementation details

CLI command generation:

  • src/commands/tool-command.ts — argument parser, help formatter, namespace command factory
  • Three-tier input: flags > --json > stdin (modeled after Speakeasy CLI)
  • gh CLI-style help output (USAGE/COMMANDS/FLAGS/EXAMPLES)
  • camelToKebab() for subcommand names, with original camelCase aliases
  • buildNamespaceCommands() groups tools by first dot-segment
  • Inline tools registered synchronously in constructor; SDK tools after ensureExecutorReady()

Tool invocation bridge:

  • New INVOKE_TOOL (400) opcode in the SharedArrayBuffer bridge protocol
  • SyncBackend.invokeTool() — worker-side sync call via Atomics.wait
  • Worker registers __invokeTool native function + tools Proxy when hasExecutorTools is set

SDK integration:

  • experimental_executor.setup lazily initializes @executor-js/sdk on first
    exec() (dynamic import — no SDK code touched until you actually use it)
  • sources.add() dispatches by kind: "graphql" → plugin-graphql,
    "openapi" → plugin-openapi, "mcp" → plugin-mcp, "custom"
    discovery plugin
  • GraphQL plugin supports offline introspectionJson
  • Init promise rejects are cleared for retry on next exec()

Security:

  • Inline tool maps use null-prototype objects to prevent prototype pollution
  • Malformed tool arguments throw instead of silently defaulting to undefined
  • onElicitation defaults to declining all requests
  • Credentials in headers config never reach the sandbox
  • Tool-author error messages are forwarded with a @banned-pattern-ignore
    annotation explaining they are author-supplied, not host data

Test plan

  • CLI commands — key=value, --flag, --json, stdin, piping, help, errors, aliases, opt-out (39 tests)
  • Inline tools via js-exec — call, chain, error handling, structured results (13 tests)
  • Custom source discovery — registration, listing, filtering, approval (9 tests)
  • GraphQL discovery — offline introspection JSON, multi-tool discovery (2 tests)
  • OpenAPI discovery — static spec parsing, namespace, multi-operation (2 tests)
  • pnpm test:run — all 13,244 unit tests pass (389 test files)
  • pnpm test:wasm — all 596 wasm tests pass (executor + js-exec integration)
  • pnpm test:dist — bundled output smoke tests pass
  • pnpm typecheck, pnpm lint, pnpm knip clean
  • pnpm install succeeds without flags after the .npmrc exemption

Migration notes

The option is new (experimental_executor), so there is nothing to migrate
from. Anyone who used the option name executor from earlier iterations of
this branch needs to rename it.
No other API surface changes.

@vercel
Copy link
Copy Markdown

vercel Bot commented Mar 27, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
just-bash-website Error Error Apr 29, 2026 8:47pm
1 Skipped Deployment
Project Deployment Actions Updated (UTC)
just-bash Ignored Ignored Apr 29, 2026 8:47pm

Comment thread src/Bash.ts Outdated
@socket-security
Copy link
Copy Markdown

socket-security Bot commented Apr 10, 2026

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addednpm/​@​executor-js/​plugin-graphql@​0.0.1-beta.5751009292100
Addednpm/​@​executor-js/​plugin-mcp@​0.0.1-beta.5761009088100
Addednpm/​@​executor-js/​plugin-openapi@​0.0.1-beta.5761009592100
Addednpm/​@​executor-js/​sdk@​0.0.1-beta.57710010092100

View full report

cramforce and others added 13 commits April 29, 2026 13:46
On Windows, path.join() / path.resolve() produce paths with backslash
separators. isPathWithinRoot() hardcoded '/' in its startsWith check,
causing every stat/read/write on OverlayFs to ENOENT on Windows.

- Accept both '/' and '\' as valid separators after the root prefix
- Normalize backslash to '/' in sanitizeSymlinkTarget relativePath
- Add Windows backslash test cases for isPathWithinRoot
* Fix #194

* deflake-test

* Disable strip types in node22
* Refactor just-bash into monorepo layout

* Fix website content generator

* Move website into workspace
* Introduce changesets and prep OIDC publishing

* pnpm-i
* Enable changset/oidc publishing

* Run all checks in release task
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
The awk lexer emitted a NEWLINE token unconditionally when it saw `\n`,
even when the previous token was one of the continuation-allowing tokens
POSIX awk specifies (`,`, `{`, `&&`, `||`, `?`, `:`, `do`, `else`, `if`,
`while`). Common multi-line idioms like

    printf "%s=%d\n",
      $1, $2

(comma at end-of-line followed by indented args on the next line) parsed
as two separate statements with a stray NEWLINE in the middle, surfacing
as "Unexpected token: NEWLINE" — even though gawk, mawk, and the BSD
one-true-awk all accept this form.

The lexer already tracks `lastTokenType` as an instance property, so the
fix is a small inject in `nextToken`: when the next character is `\n`
and `lastTokenType` is in `CONTINUES_ACROSS_NEWLINE`, swallow the
newline and recurse to the next real token instead of emitting one.

Adds eight regression tests covering each continuation-allowing token
plus the TSV → SQL INSERT printf idiom that motivated the fix.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants