Support plugin-tee in the web host

[topheman]

Support plugin-tee in the web host

This PR adds full support for the plugin-tee plugin in the web host, fixing the issues described in #12.

Example of use: tee command writes the content of $0 (last command output) to the file specified in the argument.

> echo Some Content
Some Content
> tee output.txt
Some Content
> cat output.txt
Some Content
> echo More Content
More Content
> tee -a output.txt
More Content
> cat output.txt
Some Content
More Content

The Challenge

The plugin-tee plugin allows users to write content to files, making it the first plugin that performs WRITE operations in the REPL. While it worked correctly in the CLI version, the web host (relying on a virtual filesystem for the browser) encountered several critical errors:

• tee over existing files: RangeError: offset is out of bounds followed by RuntimeError: unreachable
• tee -a (append mode): TypeError: Resource error: Not a valid "OutputStream" resource
• File overwriting behavior: Incorrect concatenation instead of proper file replacement

The Solution: Customizing the Filesystem Shim

1. Setting up a Local Fork

To fix these filesystem issues, a local fork of @bytecodealliance/preview2-shim was set up in packages/web-host/overrides. This allows customization of the filesystem implementation without waiting for upstream changes.

Files copied from the original: 0906dd3

packages/web-host/overrides/@bytecodealliance/preview2-shim/
├── lib/
│   └── browser/
│       ├─── filesystem.js  # Customized filesystem implementation
│       └─── ...            # Other files
├── types/                 # TypeScript definitions
├── package.json
└── README.md

Configuration Updates: ce56bee

• Vite config: Added alias for the local shim
• TypeScript config: Included overrides directory in compilation

2. Fix #1: Correcting File Overwriting Behavior 01a56a9

The first issue was in the Descriptor#writeViaStream() method, which incorrectly tried to preserve existing file content when writing. This broke the tee command's ability to overwrite files.

Changes to packages/web-host/overrides/@bytecodealliance/preview2-shim/lib/browser/filesystem.js:

  writeViaStream(_offset) {
    const entry = this.#entry;
    let offset = Number(_offset);
    return new OutputStream({
      write (buf) {
-       const newSource = new Uint8Array(buf.byteLength + entry.source.byteLength);
-       newSource.set(entry.source, 0);
-       newSource.set(buf, offset);
+       const newSource = new Uint8Array(buf.byteLength);
+       newSource.set(buf, 0);
        offset += buf.byteLength;
        entry.source = newSource;
        return buf.byteLength;
      }
    });
  }

What This Fix Does:

• Removes buffer concatenation: No more trying to preserve existing content
• Implements direct buffer replacement: File content is properly overwritten
• Fixes tee command behavior: Both new and existing files work correctly

3. Fix #2: Implementing appendViaStream 58bf163

The second issue was in the Descriptor#appendViaStream() method, which was just a stub that logged to console.

Changes to packages/web-host/overrides/@bytecodealliance/preview2-shim/lib/browser/filesystem.js:

Before (Broken Implementation):

appendViaStream() {
  console.log(`[filesystem] APPEND STREAM`);
}

After (Working Implementation):

appendViaStream() {
  const entry = this.#entry;
  return new OutputStream({
    write (buf) {
      const entrySource = getSource(entry);
      const newSource = new Uint8Array(buf.byteLength + entrySource.byteLength);
      newSource.set(entrySource, 0);
      newSource.set(buf, entrySource.byteLength);
      entry.source = newSource;
      return buf.byteLength;
    }
  });
}

What This Fix Does:

• Creates a proper OutputStream: Returns a valid WASI OutputStream resource
• Handles append logic: Concatenates new data to existing file content
• Maintains file integrity: Preserves existing content while adding new data
• Returns proper byte count: Satisfies the WASI interface contract

4. Why This Approach?

The @bytecodealliance/preview2-shim is the bridge between WASI system calls and JavaScript implementations. By customizing it locally:

• Immediate fixes: Issues can be resolved without waiting for upstream
• Full control: WASI interfaces can be implemented exactly as needed
• Consistency: The implementation matches the expected WASI behavior
• Maintainability: Changes are isolated and easy to track

Forking the shim also means that it can't be updated automatically anymore. This is a necessary trade-off. Maybe the modifications could be upstreamed to the original repo.

Results

After implementing both fixes:

✅ tee creates new files - Works perfectly
✅ tee overwrites existing files - No more range errors or incorrect concatenation
✅ tee -a appends to files - No more resource errors
✅ File truncation works correctly - Proper file management
✅ All e2e tests pass - Including comprehensive tee scenarios

Technical Impact

This implementation demonstrates how to:

• Extend WASI shims for custom filesystem behavior
• Handle file streams in browser environments
• Maintain WASI compliance while adding custom functionality
• Debug complex WASI integration issues
• Fix multiple related filesystem problems systematically

Additional Changes

• Comprehensive e2e tests for all tee scenarios (95af7b0, e02db13)
• Fix flaky tests (76d16d1)
• Improved test utilities for better validation (d21b6c4)
• Updated help and command listings (92e3d49)
• Split filesystem template preparation (3ccf867)

---

Closes #12

Now the web-host provides to the browser a virtual filesystem that supports WRITE operations.