Stow S3

Go + WebAssemblyApache 2.0

An object store
you can embed.

Stow runs an object engine in Go or WebAssembly, with a native S3 endpoint for existing SDKs. Use it to checkpoint an agent's files, keep browser edits offline, isolate test data, or cache objects beside your application.

01 / Why Stow

Choose storage for the job you're running.

An agent's working files need to survive a handoff. A browser editor needs to save without a connection. Test data usually needs to disappear when the test ends. Stow supports each of these through a different API.

  • Agent workspaces

    Checkpoint a run before handing it off.

    Prepare declared inputs, checkpoint outputs, compare changes, and hand verified working files to the next host.

    Workspace lifecycle →
  • Offline browser apps

    Edit a file without a connection.

    Save edits and undo history in IndexedDB. Sync a selected file stage through your app's S3 transport when the connection returns.

    Browser file history →
  • Tests & CI

    Start with a fresh object store.

    Use an in-process object runtime for direct tests or a disposable S3 endpoint for SDK integration tests. No cloud account required.

    In-process test example →
  • Edge storage & caching

    Cache objects beside application code.

    Embed the WASM object engine beside your application, or put Stow's native read-through cache in front of an S3-compatible upstream.

    WASM hosts and S3 caching →
02 / Offline browser apps

Undo still works offline.

Stow's WebAssembly engine stores browser file edits in IndexedDB. It saves each stage with the history cursor, so undo and redo still work after a reload. Saving a local stage makes no network request.

Your appEdit, save, undo, redo
WASM + IndexedDBLocal bytes and retained stages
Explicit syncConditional PUT to your remote

Sync publishes the selected file stage. If a remote writer changed the object, Stow reports a conflict and retains your local history. Your application supplies credentials and networking through an S3 SDK transport or an authenticated proxy.

Run the real browser demo

WASM, IndexedDB, offline reload, undo/redo, explicit sync, and remote conflicts. A private local S3 fixture supplies the remote.

make browser-editor
Demo setup & walkthrough →

Each retained stage stores the full file bytes, so allow room for history in your byte budget. Browsers can evict local storage. Sync detects remote conflicts; your app decides how to resolve them. File history limits and recovery.

03 / AI agent workspaces

Hand off the files, with a record of what changed.

Give each run a real working directory prepared from declared inputs. Checkpoint it before a handoff, inspect what changed, and adopt verified files on another host. Workspace identity outlives the process that opened it.

From task inputs to a portable checkpoint
# Declare local inputs and a workspace root in task.json
stow-s3 workspace prepare --manifest task.json
# Run your agent, then quiesce writers before capturing the files
stow-s3 workspace checkpoint --id WORKSPACE_ID
stow-s3 workspace diff --from BASE_CHECKPOINT --to NEW_CHECKPOINT
stow-s3 workspace diff what the agent changed
$ stow-s3 workspace diff --from cp_5fac28a6… --to cp_b19c02e4… --registry-dir ./registry
{
  "changes": [
    {
      "path": "report.md", "kind": "changed",
      "from": { "size": 28, "mode": 420,
                "sha256": "8568539c13e1dd98…" },
      "to":   { "size": 45, "mode": 420,
                "sha256": "23270d02488298ea…" }
    }
  ]
}

Included in a checkpoint

  • Files with modes and SHA-256 hashes
  • Logical buckets and object bytes
  • Content type, user metadata, checksums
  • Declared Git source and commit
  • Relative working directory

Outside the checkpoint

  • Private .stow state or credentials
  • Your .git or process memory
  • Anything outside the workspace root
  • Effects outside the filesystem
  • Original object version IDs are not guaranteed

Your host owns agent execution and sandbox enforcement. Stow owns working storage and verified checkpoints; it does not isolate host processes or network access. Direct filesystem writes are not hard-limited by object API quotas. Read the portable workspace guide.

04 / Tests & CI

Open an object store inside your test.

For direct object tests, open the engine in your process without an HTTP listener or cloud credentials. For SDK integration tests, use a scoped session with its own S3 endpoint and automatic cleanup.

In-process Node + WASM · from a source checkout
import { EmbeddedStow } from "@chester-hill-solutions/stow-s3/embedded";
import { loadNodeWasmHost } from "@chester-hill-solutions/stow-s3/node-wasm";

const host = await loadNodeWasmHost();
const store = EmbeddedStow.open(host);
try {
  store.createBucket("fixtures");
  store.putObject("fixtures", "hello.txt", new TextEncoder().encode("hello"));
  // Run assertions against a fresh, private object store.
} finally {
  store.close();
  await host.close();
}

Use the direct object API for embedded tests. Use the HTTP facade when the test needs standard S3 SDK requests and SigV4. See the scoped-session example.

05 / Storage at the edge

Cache S3 reads near your application.

The object engine compiles to WebAssembly, so a compatible host can run it alongside application code. The host supplies persistence and network access. Edge platform integrations need testing on the host you intend to use.

For an S3 read-through cache, use native run-through mode. Stow fetches missing objects from your upstream and can revalidate cached reads. If the upstream has a transient failure, it can serve a stale cached copy. Upstream data never replaces a local write.

Read throughFetch missing objects from your upstream
PrewarmFetch named keys before going offline
Bound the cacheByte limits, object limits, and TTL

Select run-through explicitly. Offline mode makes no upstream calls; reads use local or cached bytes. Live upstream writes require separate consent. Read the cache and run-through guide.

06 / APIs & compatibility

Call the engine directly or use an S3 SDK.

Use Go directly, embed the WASM engine, or point an existing SDK at the native S3 endpoint. The browser API stores objects locally and uses your app's transport for remote sync. It does not run an HTTP S3 server.

  • TypeScript

    source checkout TypeScript client

    Use the AWS SDK with disposable sessions or a persistent native server. Workspace helpers add preparation, checkpoints, diff, and portable transfer.

    await withStow(cb)
    Stow.start(opts)
    Stow.connect(endpoint)
  • Python

    source checkout Python client

    Run boto3 against a scoped session with byte and object limits and automatic cleanup. Prepare, resume, checkpoint, and restore workspaces through the Python helpers.

    with with_session() as s:
        s.s3_client()
        s.new_bucket_name()
  • Go

    go get github.com/chester-hill-solutions/stow-s3/pkg/stow

    Call bucket and object operations in-process. Set quotas, check write conditions, retain metadata and checksums, or connect a host store with streaming reads and writes.

    rt, _ := stow.Open(
      stow.Options{
        Backend: stow.BackendMemory,
      },
    )
    // no listener, no creds
  • WebAssembly

    6.1 MB runtime

    Embed the engine in Node or run it in a browser worker. Browser hosts support IndexedDB persistence, object streams, bounded caches, and limits on queued requests.

    const host = await
      loadNodeWasmHost();
    const s = EmbeddedStow.open(host);
    s.putObject(b, k, data)
    // node and browser

Select memory for throwaway work or filesystem when data has to survive a restart. Opt into run-through mode to put a native local S3 cache in front of an upstream bucket. Defaults are 64 MiB and 10,000 objects. Read the returned capability block for the limits your instance actually uses.

Portable workspace checkpoints
Capture file bytes, modes, and SHA-256 hashes. Compare checkpoints, restore files, or export verified data for another host. Checkpoint and transfer APIs
Browser project history
Commit several files as one revision and undo or redo the project. Revisions share unchanged content chunks; reads load payloads on demand. Project storage and sync
Persistent workers and streams
Read ranges or stream larger objects through a WASM worker. Inspect interrupted uploads, release abandoned stages, and collect unused content. Worker storage and recovery
Durable upstream write outbox
Native run-through saves locally before sending a write upstream. With live writes enabled, a durable outbox preserves the exact version and retries transient failures with bounded backoff. Changed remote objects stop propagation and leave your local data available. Recovery checks the remote ETag, content type, metadata, and requested checksum before acknowledging a write whose reply was lost. Outbox guarantees and recovery limits

Native S3 support

  • Buckets: create, list, inspect, delete
  • Objects: put, get, head, copy, delete
  • List with prefixes and pagination
  • Range reads
  • Conditional reads and writes
  • MD5, CRC32, CRC32C, CRC64NVME, SHA-1, SHA-256
  • Multipart uploads
  • Presigned requests
  • SigV4 authentication
  • Object content types and user metadata
  • Health, inspection, and metrics endpoints

Not implemented

  • Object versioning, and versionId is refused
  • ACLs and bucket policies
  • Lifecycle rules and replication
  • Event notifications, tagging, object lock
  • S3 Select
  • KMS-backed encryption

The native service is for development and controlled private deployments. Stow is not production object storage.

Build Stow from source.

Stow is free and open source. The Go module is available now; TypeScript, Python, and the browser demos run from a source checkout. The 0.3.0 candidate is not yet published to npm or PyPI.

git clone https://github.com/chester-hill-solutions/stow-s3