# Unbrowse

Unbrowse is an open-source action layer for AI agents. The public surface is a single contract-shaped hole: the agent supplies intent, optional URL/params/approval, and Unbrowse chooses the cheapest capable layer behind it.

Most web agents still pay the browser tax by default: open the page, wait for it to load, inspect the UI, click, and re-read state. Unbrowse learns the structured request path behind a site once, then reuses it, so the agent can act through the site's real APIs when that route exists. When a site genuinely needs a real browser session (cookies, sign-in, redirect handling), Unbrowse keeps that browser context in the loop.

It is also a **fair-compensation engine**: routes are a shared, maintained asset, and the people who index and keep them fresh are fairly compensated when those routes run. Unbrowse is open source, runs locally, and is a member of the [NVIDIA Inception program](https://www.nvidia.com/en-us/startups/).

This documentation is organised by who is reading it.

* **Start Here** explains what Unbrowse is in plain language, no background assumed.
* **For Agents** is the operating model for an AI agent filling the Unbrowse hole.
* **For Developers** is how to integrate it in code.
* **Concepts** is the conceptual model behind the system, drawn from the published papers.
* **For Investors** is the wedge, the moat, and where to read the research.

## Research

Unbrowse is built on a published research trilogy. They are the canonical source for the concepts in these docs:

* [**Internal APIs Are All You Need**](https://arxiv.org/abs/2604.00694) — the first-party routes already powering modern websites are the machine-native interface agents should try before they drive a browser. This is the route-discovery layer, not the whole current product surface.
* **Crypto Was All You Needed** — one signing discipline across every layer an agent touches (screen, browser, CLI, OS), with credentials bound to the agent's key and results sealed.
* **Unbrowse Maintenance Network** — a shared route graph only stays useful if freshness is maintained, witnessed, and economically accountable.

The full index and PDFs live at [unbrowse.ai/papers](https://www.unbrowse.ai/papers).

Source and licensing scope is described in the [Open Source Notice](/reference/open-source-notice).


# Where This Goes

The wedge is shipped and provable today. The vision is what a shared route graph becomes once many agents depend on it. This page is deliberately explicit about both, and about the line between them.

## The wedge (shipped, stands alone)

Shared route lookup beats browser rediscovery on cost, latency, and reliability. A rational agent prefers shared execution whenever the route fee stays below the expected cost of rediscovery. That inequality is enough to make the graph create real surplus, and it needs no vision to be true. Discovery is free; you only pay when you execute a paid route, and payment settles over x402. Everything below is layered on a thing that already works.

## The direction (where a maintained graph leads)

As agent traffic concentrates on the graph, two things compound:

1. **Coverage and freshness.** Every reused route makes the next agent's task cheaper; every piece of feedback makes the graph more trustworthy. Usage and quality reinforce each other.
2. **Accountable maintenance.** A graph carrying meaningful traffic needs accountable maintainers, challengeable claims, and trust tiers, not just access payments: open routes for low-risk traffic, higher-trust routes for authenticated and high-value paths, ranking grounded in route quality rather than capital.

## The discipline

The sequencing is the point: prove the wedge, then strengthen maintenance. The vision does not get to skip the wedge, and the user never has to think in anything but the task. Discovery stays free, and an agent only ever pays for the paid routes it actually executes — settled fairly over x402. The product proves itself directly in the market before any of the higher-trust coordination machinery is asked to exist.

The honest split, for anyone reading this for diligence: the wedge is measurable now; the richer accountability layer is documented direction, not claims of current revenue mechanics. The verification posture behind trust claims is described in [Verification and Proofs](/concepts/verification-and-proofs).


# Catalogue — all docs & code

> The single navigable index for the repo. Part A maps the documentation tree; Part B maps the code (`src/`, `backend/`, `frontend/`, `packages/`) to a one-line responsibility and an entry path. Use it to find the right doc, or the right module, in one hop.

> Generated 2026-06-17 against build v9.4.12 (`src/build-info.generated.ts`, git `ae37bf4a`). Every path here is real; the four architecture deep-dives it points to cite their own anchors. For prose start at [README.md](/); for the system map start at [architecture/OVERVIEW.md](/architecture/overview).

***

## Part A — Documentation map

### Read-by-audience (the published spine — see [SUMMARY.md](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/SUMMARY.md))

| Path                     | For whom       | Subject                                                                                   |
| ------------------------ | -------------- | ----------------------------------------------------------------------------------------- |
| `start-here/` (4)        | newcomers      | What Unbrowse is, the browser-discovery tax, the shared route graph, plain English        |
| `for-agents/` (6)        | agent builders | How an agent fills the hole, resolve/execute, MCP, when it browses, search, wallets       |
| `for-developers/` (6)    | integrators    | Integration surfaces, route lifecycle, SDK quickstart, drop-in/python/agent-SDK adapters  |
| `concepts/` (7)          | everyone       | Shadow APIs, route-graph-as-asset, trust, evaluation, verification, fare splits, claiming |
| `for-investors/` (2)     | investors      | The wedge, market framing                                                                 |
| `sdk/` (5)               | builders       | Build archetypes, recipes, onboarding users/validators, rewards & economics               |
| `whitepaper/` (14)       | research       | Companion text to the published paper trilogy                                             |
| `built-on-unbrowse/` (2) | builders       | Reference consumers (Aiko)                                                                |
| `guides/quickstart.md`   | new users      | Install + first run                                                                       |

### Architecture (`architecture/` — code-grounded, cite real paths)

| Path                                                                                                                              | Subject                                                                                   |
| --------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- |
| [architecture/OVERVIEW.md](/architecture/overview)                                                                                | Whole-system map: three surfaces, identity, four money rails, deploy topology             |
| [architecture/CLI.md](/architecture/cli)                                                                                          | Local engine + command/tool inventory                                                     |
| [architecture/BACKEND.md](/architecture/backend)                                                                                  | Cloudflare Worker routes, auth/keys, billing, marketplace, data model                     |
| [architecture/FRONTEND.md](/architecture/frontend)                                                                                | Next.js pages, session handling, billing & wallet UI                                      |
| [architecture/SECURITY.md](/architecture/security)                                                                                | **(new)** Anti-tamper, anti-bot handling, trust graph, proof layer, x402 gate             |
| [architecture/PRIVACY.md](/architecture/privacy)                                                                                  | **(new)** Thin client, obfuscation + audit gate, commitments, sealed storage              |
| [architecture/AUTH.md](/architecture/auth)                                                                                        | **(new)** Identity, key model, auth gates, token resolution, wallet precedence            |
| [architecture/PERFORMANCE.md](/architecture/performance)                                                                          | **(new)** Replay-over-re-drive, pointer cache, egress tiering, fast paths, honest numbers |
| [architecture/ACCEPTANCE-CRITERIA.md](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/architecture/ACCEPTANCE-CRITERIA.md) | Given/When/Then per subsystem                                                             |
| [architecture/TEST-SPECS.md](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/architecture/TEST-SPECS.md)                   | Required unit tests + coverage                                                            |

### Operations, economics & integration (root + folders)

| Path                                                                                                                                                                                                                                                | Subject                                                                       |
| --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------- |
| [SECURITY.md](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/SECURITY.md)                                                                                                                                                                   | Honest threat model (package binding, anti-tamper)                            |
| [caching.md](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/caching.md)                                                                                                                                                                     | Pointer-reactive cache design                                                 |
| [benchmarks.md](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/benchmarks.md), [benchmarks-history.md](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/benchmarks-history.md)                                                        | Coverage methodology + append-only log                                        |
| [HOW\_UNBROWSE\_PAYS.md](/research/how_unbrowse_pays), [THE\_FDRY\_ECONOMY.md](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/THE_FDRY_ECONOMY.md)                                                                                          | Money model + token economy                                                   |
| [CLAIM\_YOUR\_DOMAIN.md](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/CLAIM_YOUR_DOMAIN.md), [EARN\_AS\_INDEXER.md](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/EARN_AS_INDEXER.md)                                            | Domain claim + indexer economics                                              |
| [wallets.md](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/wallets.md), `ows.md`, `pay-sh-integration.md`, [public/lobster-cash-integration.md](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/public/lobster-cash-integration.md) | Wallet & payment integrations                                                 |
| `mcp-workflow-guide.md`                                                                                                                                                                                                                             | MCP workflow reference                                                        |
| [public/primitives/README.md](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/public/primitives/README.md) (15)                                                                                                                              | Auditable transparency primitives                                             |
| [OPEN-SOURCE-NOTICE.md](/reference/open-source-notice)                                                                                                                                                                                              | Open-source scope / moat boundary of record                                   |
| `design/` (4)                                                                                                                                                                                                                                       | Design explorations (per-contract VM, runpod-bound VM, openai tools, windows) |
| `decisions/faremeter-evaluation.md`                                                                                                                                                                                                                 | ADOPT decision record                                                         |

### Not published (kept for reference)

| Path              | Why                                                             |
| ----------------- | --------------------------------------------------------------- |
| `internal/` (3)   | Internal bench / acceptance / week-review — **never published** |
| `archive/` (6)    | Frozen historical snapshots & post-mortems                      |
| `issues-to-rach/` | Partner-facing bug logs — internal                              |

***

## Part B — Code map

### `src/` — local engine, CLI & MCP (Bun single binary)

**Entry points & dispatch**

| Module          | Responsibility                                    | Entry                                            |
| --------------- | ------------------------------------------------- | ------------------------------------------------ |
| CLI             | Command entry, dispatch, local-server lifecycle   | `src/cli.ts`                                     |
| MCP             | stdio JSON-RPC server, in-process API, \~45 tools | `src/mcp.ts`                                     |
| v7 dispatch     | build/act/eval op routing                         | `src/cli-v7/`                                    |
| Server / router | HTTP app + route composition (compat facade)      | `src/server.ts`, `src/router.ts`, `src/index.ts` |
| Intent match    | Form detection + API-type inference               | `src/intent-match.ts`                            |

**Capture → infer → publish (the product loop)**

| Module             | Responsibility                                                                    | Entry                                                      |
| ------------------ | --------------------------------------------------------------------------------- | ---------------------------------------------------------- |
| capture/           | Record real interactions; obfuscate; server-first reverse-engineering; fast paths | `src/capture/index.ts`                                     |
| extraction/        | HTML/JSON structure extraction, SPA detection, readability                        | `src/extraction/index.ts`                                  |
| transform/         | Schema transform + drift detection/recovery                                       | `src/transform/index.ts`                                   |
| lib/graph-core     | Operation graph: schema inference, endpoint planner, hole bindings                | `src/lib/graph-core/index.ts`                              |
| lib/indexer-core   | Background indexing, capture spool, queue                                         | `src/lib/indexer-core/index.ts`                            |
| publish/, foundry/ | Manifest validation, sanitization, publish bundle                                 | `src/publish/sanitize.ts`, `src/foundry/publish-bundle.ts` |
| publish-admission  | Publish admission gates                                                           | `src/publish-admission.ts`                                 |

**Resolve & execute**

| Module                     | Responsibility                                                                                              | Entry                                          |
| -------------------------- | ----------------------------------------------------------------------------------------------------------- | ---------------------------------------------- |
| orchestrator/              | DAG planning, execution flow, hole-producer selection                                                       | `src/orchestrator/index.ts`                    |
| execution/                 | Endpoint execution (GraphQL/JSON-RPC/gRPC/form/page), retry, proxy, **anti-bot handlers**, **egress chain** | `src/execution/index.ts`                       |
| ranking/, lib/ranking-core | Endpoint ranking signals                                                                                    | `src/ranking/`, `src/lib/ranking-core/`        |
| site-policy, ratelimit/    | Session-bound-param detection, per-route limits                                                             | `src/site-policy.ts`, `src/ratelimit/index.ts` |

**Browser layer**

| Module         | Responsibility                                              | Entry                                       |
| -------------- | ----------------------------------------------------------- | ------------------------------------------- |
| browser/, cdp/ | Chrome tabs, network proxy, spoof; Chrome DevTools Protocol | `src/browser/index.ts`, `src/cdp/chrome.ts` |
| kuri/          | Stateless browser spawn/FFI wrapper                         | `src/kuri/client.ts`                        |
| sandbox/       | Bundle-replay client                                        | `src/sandbox/bundle-replay-client.ts`       |

**Identity, security, money** *(deep-dives:* [*SECURITY*](/architecture/security) *·* [*PRIVACY*](/architecture/privacy) *·* [*AUTH*](/architecture/auth)*)*

| Module          | Responsibility                                                                     | Entry                                       |
| --------------- | ---------------------------------------------------------------------------------- | ------------------------------------------- |
| auth/           | Pre-resolve gate, runtime auth, stale-endpoint feedback, browser cookies/history   | `src/auth/index.ts`                         |
| verification/   | Auth-gate, candidate selection, integration matrix                                 | `src/verification/index.ts`                 |
| payments/       | x402 fetch, EVM/Base signer, wallet resolution, OWS, lobster/privy/pay.sh adapters | `src/payments/index.ts`                     |
| vault/, values/ | Credential storage, wallet sealing, keychain, signer, storage holes                | `src/vault/index.ts`, `src/values/index.ts` |
| proof/          | Response commitment, input censoring, notary client                                | `src/proof/index.ts`                        |
| trust/          | Proof-of-indexing, bond-challenge, ledger checkpoint, sealed cache, refresh job    | `src/trust/mount.ts`                        |

**Client, config, telemetry & misc**

| Module                                                               | Responsibility                                                                          | Entry                                                               |
| -------------------------------------------------------------------- | --------------------------------------------------------------------------------------- | ------------------------------------------------------------------- |
| client/                                                              | Config/key load, telemetry, agent API, skill lookup                                     | `src/client/index.ts`                                               |
| config/, settings.ts, compat/                                        | Contribution mode, payment-provider, capture-pipeline settings                          | `src/config/`, `src/settings.ts`                                    |
| telemetry/, telemetry.ts, routing-telemetry.ts                       | Session logging, route traces, routing events, issue reporting                          | `src/telemetry/index.ts`                                            |
| sdk/                                                                 | Public TypeScript SDK + wallet adapters                                                 | `src/sdk/index.ts`                                                  |
| setup/, cli-setup.ts                                                 | MCP registration, contract resolver, skill installer, setup wizard                      | `src/setup/claude-mcp-register.ts`                                  |
| runtime/, single-binary.ts                                           | Local server, browser lifecycle, principal scope; binary packaging                      | `src/runtime/local-server.ts`                                       |
| interop/, bridges/, contract-shape/, contract-\*.ts                  | Agent primitives, MCP/CLI/impl contract transport                                       | `src/interop/agent-primitives.ts`, `src/contract-shape/registry.ts` |
| types/                                                               | Core shared TypeScript types (SkillManifest, EndpointDescriptor, ExecutionTrace, Proof) | `src/types/index.ts` (`skill.ts`, `proof.ts`)                       |
| protobuf/, domain.ts, template-params.ts, skillmd.ts, version.ts     | Wire serialization + small shared utilities                                             | `src/protobuf/wire.ts`, `src/version.ts`                            |
| stale-cleanup\*.ts, session-logs.ts, impact-log.ts, agent-outcome.ts | Cleanup scheduler, session logs, impact/earnings log, outcome hints                     | `src/stale-cleanup.ts`                                              |

> **Internal method surface (out of public scope).** A small set of internal planning / accountability-ledger modules under `src/` are part of the private build method, not the public product surface. They are intentionally omitted here and held out of published docs by the repo's public-artifact gate.

### `backend/` — Cloudflare Worker (Hono) at `beta-api.unbrowse.ai`

See [architecture/BACKEND.md](/architecture/backend) for the full route map.

| Group       | Notable members                                                                                                                                                   |
| ----------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| routes/     | `skills`, `search`, `reveng`, `auth`, `account`, `billing`, `claim`, `credits`, `llm`, `proxy`, `solve`, `dashboard`, `webhooks`, `trace`, `audit`, …             |
| services/   | `keys`, `marketplace`, `stripe`, `crypto-sub`, `flex`, `splits`, `sponsor-pool`, `economics`, `pricing`, `rank`, `scoring`, `settlement`, `domain-claim`, `kv`, … |
| middleware/ | `auth`, `sponsor`, `x402-gate`, `rate-limit`, `exec-token`, flex-onboarding gates                                                                                 |

### `frontend/` — Next.js on Cloudflare at `unbrowse.ai`

See [architecture/FRONTEND.md](/architecture/frontend).

| Area  | Notable                                                                                                                                                                                              |
| ----- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Pages | landing (per-domain), `dashboard`, `skill/[id]`, `search`, `login`, `account`, `billing`, `playground`, `agents`, `docs/*`, `pricing`, plus `mcp.json` / `llms.txt` / `skill.md` discovery endpoints |
| lib/  | `api.ts`, `auth-context.tsx`, `privy-provider.tsx`, `account-client.ts`, `claim-client.ts`, `web-telemetry.ts`, `landing-experiment.ts`                                                              |

### `packages/` — published npm

| Package            | Name               | Status                             |
| ------------------ | ------------------ | ---------------------------------- |
| `packages/skill/`  | `unbrowse`         | Main CLI + single binary (v9.4.12) |
| `packages/sdk/`    | `@unbrowse/sdk`    | Legacy (binary-spawn) — deprecated |
| `packages/sdk-v2/` | `@unbrowse/client` | HTTP-first, zero-dep SDK           |

***

## How to keep this honest

* Every `src/...`/`backend/...` path above is verifiable with the path-anchor check (`scripts/docs-anchor-check.sh`).
* Published docs must pass the repo's public-artifact gate (no economic constants, no server-side engine internals, no internal method vocabulary).
* When a subsystem is added or renamed, update its row here and in [SUMMARY.md](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/SUMMARY.md).


# What Is Unbrowse

Unbrowse is an action layer for AI agents on the web. Instead of driving a browser by default, the agent tries to act through the site's real first-party APIs when those routes are available.

When an AI assistant books a flight, pulls a report, or posts an update, it usually controls a real browser: it opens the site, waits for the page, finds buttons, clicks, and re-reads the screen after every step. Every one of those steps can fail, and every one costs time and money. Unbrowse learns the request the browser was going to make underneath all that clicking, and makes that request directly the next time the same task comes up.

The reason this is faster is that the slow part of web automation is not the network, it is the interface dance: rendering pages, locating elements, recovering from layout changes, and asking a language model what to click next. Skipping that dance when it is not needed turns a multi-second, failure-prone sequence into a single call. When the site truly needs a browser, for example to sign in or carry a session cookie, Unbrowse still uses one; it just stops paying that cost on every routine run.

That is the whole idea: do the expensive discovery once, reuse the result, and keep a browser for the cases where a browser is actually required.


# The Browser-Discovery Tax

Every time an agent re-drives a website's interface, it pays a tax that produces nothing reusable.

A browser-first agent that checks the same dashboard a hundred times performs the same DOM parsing, the same element lookups, the same retries, and the same language-model reasoning a hundred times. The published research calls this the browser-discovery tax: the recurring cost of rediscovering a workflow that has not changed. None of that work is saved for the next run, so the hundred-and-first visit costs exactly as much as the first.

This matters because agent workloads are repetitive by nature. The same small set of tasks (read this inbox, list these events, fetch this price) runs over and over across many agents, and a browser-first design pays full price each time. Removing the tax does not require a smarter model; it requires not redoing solved work.

Unbrowse exists to collect that solved work once and hand it back cheaply, which is the subject of the next page.


# The Shared Route Graph

Unbrowse turns one agent's successful web task into a reusable route that other agents can call.

When an agent completes a task through Unbrowse, the structured request path behind it (the route) is recorded with the information needed to run it again: the request shape, what it needs as input, what it returns, and how reliable it has been. That route goes into a shared graph, so the next agent with the same intent can look it up instead of rediscovering it from the visible page.

The payoff compounds because web tasks are shared, not unique to one user. A route learned by the first agent that needed it saves every later agent the full discovery cost, the way a map drawn once spares everyone after from re-surveying the road. The graph also tracks which routes still work, so stale ones fall out of the way rather than misleading callers.

A shared graph that many agents rely on raises its own question, which is how route quality is kept honest over time; that is covered in Concepts.


# In Plain English

Think of a website as having two layers: the front-of-house that humans see, and the request layer the browser quietly uses underneath.

Traditional automation stays stuck in the front-of-house, clicking through the human interface. Unbrowse learns the request layer and reuses it, the way a regular at a restaurant skips the menu and orders the dish by name. Nothing about your permissions changes; the assistant still only does what you could already do yourself.

This is usually faster and more reliable because the human interface is the brittle part: it changes layout, shows popups, and slows down, while the underlying request is comparatively stable. Reusing the stable layer means fewer surprises and far less waiting.

A few things Unbrowse is not, to avoid a common misread:

* It is not a permission bypass. It cannot reach anything you could not already reach yourself.
* It does not publish your credentials. Sign-in stays on your machine.
* It does not replace the browser everywhere. It uses one whenever a site genuinely needs it.

If you want the operating model an AI agent actually follows, continue to For Agents.


# How an Agent Uses Unbrowse

This page is the operating model for an AI agent that has Unbrowse available.

The current mental model is one step: **get the result**. The agent describes the internet result it needs — intent, optional URL, optional params, and explicit approval for writes — and Unbrowse decides how to satisfy it. That may mean a direct document fetch, a shared contract in the route graph, a standard adapter, a local-auth browser capture, HAR inspection, or indexing a newly discovered route for the next call.

The agent should not choose between `resolve`, `execute`, `go`, `snap`, `fetch`, HAR, or cookies for ordinary work. Those are implementation layers under the typed hole.

The public contract is inspectable:

```bash
unbrowse contract surface
```

It exposes five client-fillable holes:

* `intent`
* `wallet_proof`
* `approval`
* `local_capability_result`
* `typed_pointer`

In shell or code, the same surface is:

```bash
unbrowse "top stories with point counts"
unbrowse "top stories with point counts" --url "https://news.ycombinator.com"
```

```ts
import { createHole } from "unbrowse/sdk";

const hole = createHole();
const result = await hole.fill({
  intent: "top stories on Hacker News with point counts",
  url: "https://news.ycombinator.com",
});
```

Use the old route view only when debugging or when a host cannot call the hole directly. In that compatibility path, resolve gathers candidates and execute replays one chosen route. It is not the preferred surface for new agents.


# Hole Contract and Legacy Route View

The current Unbrowse contract is one typed-hole request. The caller supplies an intent and optional context; the runtime descends through the graph, adapters, browser capture, cookies, HAR, and indexing as needed.

```ts
import { createHole } from "unbrowse/sdk";

const hole = createHole();
const result = await hole.fill({
  intent: "latest releases from this repository",
  url: "https://github.com/unbrowse-ai/unbrowse",
});
```

From a shell, call the same contract as:

```bash
unbrowse "latest releases from unbrowse-ai/unbrowse"
unbrowse "latest releases from this repository" --url "https://github.com/unbrowse-ai/unbrowse"
```

The machine-readable shape is:

```bash
unbrowse contract surface
```

## Why the old route view still exists

`resolve` and `execute` are the compatibility decomposition of the same contract. They are useful when you are inspecting the route graph, debugging a bad endpoint, or integrating with an older MCP host that cannot call the hole directly.

In that view:

* `resolve` searches local/server contracts and returns candidate endpoints with evidence.
* The agent or debugger judges which candidate matches the intent.
* `execute` runs the selected endpoint with params and projection.
* `feedback` records whether the selected route satisfied the intent.

This is deliberately no longer the default training path for agents. The dogfood failure mode was obvious: agents guessed CLI verbs and flags instead of submitting one intent-shaped gap. New integrations should call the hole and let the runtime pick the descent.

## When to use it

Use the route view for:

* endpoint inspection
* regression diagnosis
* manual replay of a known contract
* old MCP/tool hosts

Do not use it as the default user-task loop. For user tasks, use bare `unbrowse "task"` or the SDK hole.


# MCP Integration

The Agent Skill plus SDK hole is the primary way for an agent host to use Unbrowse. MCP remains a compatibility surface for hosts that cannot load the skill or call the SDK directly.

Add Unbrowse as an MCP server in the host config (Claude, Cursor, Codex, or any MCP-compatible client):

```json
{
  "mcpServers": {
    "unbrowse": {
      "command": "npx",
      "args": ["-y", "unbrowse", "mcp"]
    }
  }
}
```

Then run setup once on the host machine with MCP enabled:

```bash
npx unbrowse setup --mcp
```

Setup bootstraps the local runtime, accepts terms, registers an agent identity, and pairs a wallet for payment where relevant.

The MCP server exposes the legacy route-inspection tools (`resolve`, `execute`, `search`, plus browser-session tools). They are the compatibility decomposition of the one-hole contract, not the preferred mental model for new agents. New agents should use the installed Skill or SDK `createHole().fill(...)` surface when possible.

If you are integrating from code rather than an agent host, see For Developers.


# When It Uses a Browser

Unbrowse keeps a real browser in the loop only when the site genuinely depends on browser-bound state, and treats opening one as a cost to avoid, not a feature.

A browser session is used when the task needs things a bare request cannot carry:

* sign-in and session cookies
* cross-site request tokens
* redirect chains
* stricter authenticated single-page behaviour

For everything else, the reused route is cheaper and faster, so that path is preferred.

When live capture is unavoidable, Unbrowse opens a browse session, the agent drives it (navigate, snapshot, click, fill, submit), and the traffic is indexed passively so the next agent with the same intent does not have to repeat the session. The result is that browser use trends toward zero as the shared graph fills, rather than being paid on every run.

The operating principle: a browser open during normal operation is a multi-step event to be designed out, not relied on.

## Truth and capability boundaries

Browser driving supports Chrome/Chromium. Importing cookies from another browser does not mean that browser family can be driven. Results expose `static_unevaluated` or `javascript_evaluated`; static content cannot satisfy a task that requires evaluated JavaScript. Stealth is best-effort and `stealth_guaranteed` is always false.

Cookie presentation and successful navigation do not prove login. Auth-bearing flows report `auth_outcome` (`not_presented`, `presented_accepted`, `presented_rejected`, or `presented_unknown`) and may terminate with `auth_required` or `session_expired`.

Capability failures are explicit: `chromium_unavailable`, `chromium_cdp_unavailable`, and `kuri_sandbox_unavailable`. Cloudflare, PerimeterX, and `empty_capture` blockers are reported separately. Follow a `next_step` only when its required runtime is available; Unbrowse does not claim to bypass every challenge.


# Search on Top

`unbrowse_search` (MCP) / `unbrowse search` (CLI) / `client.search()` (SDK) is the single discovery surface on top of Unbrowse. Give it an intent; it finds the best **route/skill** in the shared route graph, and when no indexed route fits it falls back to a **live web search**. One call, ranked results — each hit carries `skill_id` + `endpoint_id` where applicable so you can chain straight into `unbrowse_execute`.

```ts
import { Unbrowse } from "unbrowse/sdk";
const unbrowse = new Unbrowse({ apiKey: process.env.UNBROWSE_API_KEY });

const hits = await unbrowse.search({ intent: "best machine learning frameworks" });
```

## Route discovery is free — web search and execution are priced

**Route discovery is free.** `/v1/search` searches the route graph (the index of captured routes) and never charges per query.

**Web search (`/v1/search/web`) is an Exa-backed lookup for finding answers and sources beyond the route graph.** It runs on the operator's own Exa key, so it is **free for the operator**; every other caller pays a small per-query fee over [x402](https://www.x402.org) before the lookup runs (the lookup has a real upstream cost), settled through the same Flex split as execution.

You also pay when you **execute** a returned route that is priced — `unbrowse_execute` on a paid endpoint settles **per-request over x402**:

1. The execute request returns `402 Payment Required` with the price (USDC on Solana).
2. Your agent's **wallet** signs the payment and the request is retried — the client never sees or handles private keys.
3. On settlement you get the results plus a receipt.

**Bring your own wallet.** Payment execution, approval, and final status are handled by the agent wallet, not by this skill. If a wallet step is required and wallet context is missing, complete your wallet setup first. Any Solana wallet that settles USDC over x402 works; agent wallets such as **lobster.cash** are compatible and tested. The skill prepares the payment *requirements* (amount, currency, reason) and delegates execution to the wallet.

### The fee split (on execution)

When a priced route executes, the fee is split **50 / 35 / 15** — platform / indexer pool / route owner — so everyone who created the value gets their share. The split is settled on-chain by the payment layer (Faremeter/Flex); the CLI/SDK shows the same breakdown in the receipt (`computeSplit`, `src/payments/split-constants.ts`). Web search via Exa is funded by the platform (Exa is called with an API key), so discovery stays free to the caller.

### Payment states the surface handles (on a priced execute)

* **Wallet not configured** — prompts you to set up a wallet before executing.
* **Not enough balance** — reports the required amount; fund the wallet and retry.
* **Payment failure** — surfaces a clear error; the call is retryable.
* **Awaiting confirmation** — waits for the wallet to report final status before returning results.

## Route graph only (skip web enrichment)

Set `web: false` to search the route graph alone, without the Exa enrichment.

```ts
const routes = await unbrowse.search({ intent: "list GitHub repos", web: false });
```

> Implementation note: the surface (CLI/MCP/SDK) lives in this repo and calls the priced `/v1/search` route; the provider call and on-chain split are performed by the Unbrowse backend.


# Wallets and Payments

Unbrowse priced calls (search-on-top, route execution) settle **per request via** [**x402**](https://www.x402.org) in USDC on Solana. Unbrowse never holds your keys: it prepares the payment *intent* (amount, currency, reason) and **delegates execution to your wallet**. Transaction signing, approval, broadcast, and final status are owned by the wallet.

## Native wallet support: the Wallet Standard

Unbrowse speaks the [Wallet Standard](https://github.com/wallet-standard/wallet-standard) (open-wallet-standard), so **any** standard Solana wallet works — no per-wallet integration. Discover wallets the app already exposes and turn one into a payment handler:

```ts
import { Unbrowse } from "unbrowse/sdk";
import { walletStandardPay, pickSolanaWallets } from "unbrowse/sdk/wallet-standard";
import { getWallets } from "@wallet-standard/app"; // your app supplies this

const wallet = pickSolanaWallets(getWallets().get())[0];
const unbrowse = new Unbrowse({ apiKey: process.env.UNBROWSE_API_KEY, pay: walletStandardPay(wallet) });
```

The bridge in `unbrowse/sdk/wallet-standard` is **zero-dependency** — it consumes the Wallet Standard shape structurally, so the SDK stays light and browser-safe.

### Compatible & tested wallets

* **lobster.cash** — compatible and tested. An agent wallet for the web; it owns provisioning, authentication, signing, and the final transaction state. If a payment step is required and your wallet context is missing, complete your wallet setup first.
* Any Wallet Standard Solana wallet (Phantom, Solflare, Backpack, a Privy embedded Solana wallet, …).

The skill describes *what* to pay and *why*; the wallet decides *how*. We do not prescribe a currency, token, or method beyond the x402 requirement, and we do not call wallet operations by name — execution is the wallet's.

## Two ways to pay

A priced call can be satisfied in **either** of two ways — both resolve to the same question (*who is the payer-of-record*) through one admission boundary. See the [x402 Payment API](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/api/x402.md) for the full contract.

**(a) Wallet signature** — sign the x402 authorization (Solana USDC) and present it in the `X-PAYMENT` header on the retry. The signer is the payer-of-record. This is the path the Wallet Standard examples above walk, and it works today.

**(b) A bound API key** — an API key is a web2 wrapper around a wallet. Bind a key to a wallet and the key *authenticates* the request while the bound wallet is recognized as the payer-of-record:

```ts
import { Unbrowse } from "unbrowse/sdk";

const unbrowse = new Unbrowse({ apiKey: process.env.UNBROWSE_API_KEY });
```

> Status: two ways an API key pays from a wallet, both real:
>
> * **Prepaid (shipped).** Bind the key to a wallet, deposit USDC once (`POST /v1/account/keys/:id/deposit` — a single signature), and the key then pays per call from that prepaid balance with **no per-call signature**. The platform custodies the *deposited balance* (not the wallet key); the unspent remainder is an IOU.
> * **Non-custodial delegated (built, activating).** The wallet keeps its funds in its *own* on-chain escrow and grants a **cap-bounded, expiring, revocable** session key; the key draws per call within the cap, the funds never leaving your custody. Built and tested; it activates once the operator configures the delegation key and your escrow + session-key registration are on-chain — until then, pay via mode (a) or the prepaid lane.
>
> See [x402 Payment API](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/api/x402.md) for the precise per-lane scope.

## x402 facilitator

| Property  | Value                       |
| --------- | --------------------------- |
| Chain     | Solana                      |
| Settle in | USDC                        |
| Wallets   | PDA smart-wallets supported |

This matches the lobster.cash integration requirements (Solana, USDC, PDA).

## Payment states the surface handles

* **Wallet not configured** — prompts you to set up a wallet before paying.
* **Not enough balance** — reports the required amount; fund the wallet and retry.
* **Payment failure** — surfaces a clear error; the call is retryable.
* **Awaiting confirmation** — waits for the wallet to report final status before continuing.

> The fee on a priced call is split among the parties who created the value (the platform / indexer / route-owner split); the wallet just authorizes the payment. See [search-on-top](/for-agents/search-on-top) for the search surface.


# Integration Surfaces

There are three ways to call Unbrowse from your own software. They are the same contract behind different front doors.

| Surface         | Use it when                                                  | Entry point                                 |
| --------------- | ------------------------------------------------------------ | ------------------------------------------- |
| **Agent Skill** | You are wiring a skill-aware agent host                      | `unbrowse setup`                            |
| **SDK hole**    | You are writing browser, edge, or Node TypeScript/JavaScript | `import { createHole } from "unbrowse/sdk"` |
| **CLI**         | Shell scripts, CI, one-off use, contract inspection          | `unbrowse contract surface`                 |
| **MCP server**  | Legacy host compatibility                                    | `unbrowse setup --mcp` / `npx unbrowse mcp` |

The preferred contract is the same everywhere: fill one hole. The caller supplies intent plus optional URL/params/approval; the runtime chooses whether the right descent is a direct document fetch, shared route graph hit, standard adapter, local auth/cookies, browser capture, HAR inspection, or newly indexed contract.

```ts
import { createHole } from "unbrowse/sdk";

const hole = createHole();
const r = await hole.fill({
  intent: "latest issues in this repository",
  url: "https://github.com/unbrowse-ai/unbrowse/issues",
});
```

## Already Using Another Library?

If your code already calls `axios`, `got`, `ky`, `undici`, `superagent`, `wretch`, `node-fetch`, `cross-fetch`, `playwright`, `puppeteer`, `selenium-webdriver`, `@browserbasehq/stagehand`, `@mendable/firecrawl-js`, `exa-js`, or `@tavily/core`, you do not need to rewrite it. Swap one import for the matching Unbrowse drop-in. See [Drop-in Adapters](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/for-developers/drop-in-adapters.md).

## Building an Agent?

Start with the installed Agent Skill or the SDK hole. Older framework adapters and MCP tools may expose `resolve`/`execute`; those are compatibility route-inspection tools, not the default mental model for new agents.

See [Agent SDK Adapters](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/for-developers/agent-sdk-adapters.md).

## Writing Python?

The same drop-in story holds for the Python layer: `requests`, `httpx`, `aiohttp`, and `urllib3` HTTP clients, plus `crewai` and `pydantic-ai` agent tools. See [Python Adapters](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/for-developers/python-adapters.md).


# The Route Lifecycle

This page describes, at a conceptual level, what happens to a route from first discovery to retirement. It is the model, not the implementation.

A route moves through a small set of states:

1. **Discovered.** An agent completed a task live and the structured request path behind it was recorded with what it needs and what it returns.
2. **Published.** The route entered the shared graph so other agents with the same intent can find it.
3. **Reused.** Resolve ranked it into a shortlist, an agent executed it, and the outcome fed back.
4. **Scored.** Reliability, freshness, and verification state move the route up or down in future shortlists.
5. **Drifted or retired.** When a site changes and a route stops returning what it used to, it is detected, demoted, and eventually deprecated so it stops misleading callers.

The practical consequence for an integrator: a route is not a frozen cURL string. It carries enough state for the system to keep good ones hot and route around dead ones, which is why reuse stays reliable as sites change. The deeper mechanics of discovery, scoring, and verification are described conceptually in the published paper and are out of scope here.

For what is and is not open, see the [Open Source Notice](/reference/open-source-notice).


# SDK Quickstart

`unbrowse/sdk` is the TypeScript client for the current Unbrowse contract: one typed-hole request from intent plus optional URL/params/approval. It runs in browsers, edge runtimes, and Node.

```bash
npm i unbrowse
```

## Web3-native auth (preferred)

The credential root is a wallet signature. Pass a `walletSigner` callback that returns the three web3 auth headers (`X-Unbrowse-Wallet`, `X-Unbrowse-Auth-Ts`, `X-Unbrowse-Signature`) and the backend authenticates the caller as `wallet:<pk>` — a full principal, never key-gated. The unbrowse CLI ships a ready signer at `src/lib/wallet-auth-headers.ts:mergedAuthHeaders` that reads the local wallet at `~/.unbrowse/wallet.json`; external consumers wire their own ed25519 signer (any lib that produces the headers will do).

```ts
import { createHole, mergedAuthHeaders } from "unbrowse/sdk";

const hole = createHole({
  client: { walletSigner: mergedAuthHeaders },
});

const result = await hole.fill({
  intent: "list tomorrow's events",
  url: "https://calendar.google.com",
});
```

## Deprecated web2 wrapper (account-bound flows only)

If you still need payouts accrual / dashboard sync / ToS surface tied to an email account, layer a `ubr_` api-key over the wallet. A wallet-only caller is already a full principal — the key is ONLY for account-bound continuity and will be retired.

```ts
import { createHole, mergedAuthHeaders } from "unbrowse/sdk";

const hole = createHole({
  client: { walletSigner: mergedAuthHeaders, apiKey: process.env.UNBROWSE_API_KEY },
});
```

The shell equivalent is:

```bash
unbrowse "list tomorrow's events"
unbrowse "list tomorrow's events" --url "https://calendar.google.com"
```

Need to inspect route selection? The legacy `Unbrowse` client still exposes `resolve`/`execute` for debugging and compatibility, but new agents should start from `createHole().fill(...)`.

Reused routes can be priced. A paid call returns an HTTP 402 that the SDK raises as a typed error you can catch and retry after settling payment; brand-new agents get a sponsored allowance first. The same wallet that authenticates the request signs the x402 payment envelope — "who you are" and "who pays" are the same handle.

The open/closed source split is described in the [Open Source Notice](/reference/open-source-notice).


# Architecture — Start Here

> **At a glance** — documents describing how Unbrowse actually works at v9.4.12, generated from the code with every claim citing a real file path. Four describe the system surfaces (overview + CLI/backend/frontend), four are cross-cutting deep-dives (security, privacy, auth, performance), and two define what "correct" means (acceptance criteria + test specs). For the full repo index see [../CATALOGUE.md](/catalogue).

## Pick your reading path

| You want to…                                                     | Read                                                                                                                 |
| ---------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- |
| Get the whole system in 5 minutes                                | [OVERVIEW.md](/architecture/overview)                                                                                |
| Find any doc or any code subsystem                               | [../CATALOGUE.md](/catalogue)                                                                                        |
| Work on the CLI, MCP server, SDK, or local capture/replay engine | [CLI.md](/architecture/cli)                                                                                          |
| Work on the API: routes, auth, keys, billing, marketplace        | [BACKEND.md](/architecture/backend)                                                                                  |
| Work on the web UI or the public metrics dashboard               | [FRONTEND.md](/architecture/frontend)                                                                                |
| Understand anti-tamper, anti-bot, the trust graph, the x402 gate | [SECURITY.md](/architecture/security)                                                                                |
| Understand secret handling and the thin-client guarantee         | [PRIVACY.md](/architecture/privacy)                                                                                  |
| Understand identity, keys, auth gating, wallet resolution        | [AUTH.md](/architecture/auth)                                                                                        |
| Understand why it's fast: caching, egress tiering, fast paths    | [PERFORMANCE.md](/architecture/performance)                                                                          |
| Know what a subsystem must do before changing it                 | [ACCEPTANCE-CRITERIA.md](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/architecture/ACCEPTANCE-CRITERIA.md) |
| Write or find tests; see coverage and gaps                       | [TEST-SPECS.md](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/architecture/TEST-SPECS.md)                   |

## The 12 subsystems (one index for both quality docs)

Acceptance criteria and test specs share the same 12 numbered sections, so §N in one maps to §N in the other:

| §  | Subsystem                              | Criteria tags |
| -- | -------------------------------------- | ------------- |
| 1  | Authentication (magic link)            | AC-AUTH       |
| 2  | API keys                               | AC-KEY        |
| 3  | Key funding — API key wraps the wallet | AC-FUND       |
| 4  | Stripe subscriptions                   | AC-STR        |
| 5  | Crypto (USDC) subscriptions            | AC-CSUB       |
| 6  | Per-request x402 payments              | AC-X402       |
| 7  | Sponsored free tier                    | AC-SPON       |
| 8  | Wallets & OWS                          | AC-WAL        |
| 9  | Marketplace: publish / verify / claim  | AC-MKT        |
| 10 | Earnings & discovery attribution       | AC-EARN       |
| 11 | CLI / MCP core loop                    | AC-CLI        |
| 12 | Frontend (product UI)                  | AC-FE         |

## Conventions

* **Citations**: every factual claim names the implementing file (`path/to/file.ts`). If a citation has gone stale, fix the doc.
* **Honesty**: known gaps and unimplemented features are stated as such — these docs describe what exists, not what is planned.
* **Mirror**: this set is mirrored to the team wiki (Architecture — Unbrowse Ecosystem collection); the repo copy is canonical.


# System Overview

> **At a glance** — Unbrowse turns captured website interactions into reusable API routes ("skills"). Three product surfaces (CLI/MCP binary, Cloudflare-Workers backend, Next.js frontend) share one identity system (email magic link → `ubr_` API key) and four money rails (Stripe, USDC subscription, per-request x402, platform-sponsored). An API key can be bound to a wallet or credit budget — the key fronts the money. Earnings from paid executions are split deterministically among platform, site owner, contributors, and first discoverer.

> Reviewed 2026-06-17 against build v9.4.12 (`src/build-info.generated.ts`). Every claim cites a real file path. Start at [README.md](/architecture/architecture) for reading paths, or [../CATALOGUE.md](/catalogue) for the full repo index. Cross-cutting detail lives in the deep-dives: [SECURITY](/architecture/security) · [PRIVACY](/architecture/privacy) · [AUTH](/architecture/auth) · [PERFORMANCE](/architecture/performance).

## What Unbrowse is

Unbrowse captures website interactions once and replays them as reusable API routes ("skills") for agents. The system has three product surfaces plus a shared cloud backend:

| Surface            | Where                                                     | Tech                                                                                                         | Serves                                                                                                           |
| ------------------ | --------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------- |
| CLI / local engine | `src/`, distributed via `packages/skill` (npm `unbrowse`) | Bun-compiled single binary (`scripts/build-binaries.sh`, `src/single-binary.ts`)                             | Agents and developers on their own machines                                                                      |
| MCP server         | `src/mcp.ts`                                              | JSON-RPC 2.0 over stdio, \~45 tools                                                                          | MCP-compatible agent harnesses                                                                                   |
| Backend API        | `backend/`                                                | Cloudflare Workers + Hono (`backend/src/index.ts`), Neon Postgres, 7 KV namespaces (`backend/wrangler.toml`) | `https://beta-api.unbrowse.ai`                                                                                   |
| Web frontend       | `frontend/`                                               | Next.js 16 App Router on Cloudflare via open-next (`frontend/wrangler.jsonc`)                                | `https://unbrowse.ai`                                                                                            |
| Metrics dashboard  | `../unbrowse-dashboard` (separate repo)                   | Next.js on Cloudflare Pages                                                                                  | `launch.unbrowse.ai` — public read-only adoption metrics from Unkey/GitHub/npm; does **not** talk to the backend |

## System map

```
┌─────────────────────────────┐
│ Agent harness (Claude, etc.)│
└──────┬──────────────┬───────┘
       │ MCP stdio    │ shell
┌──────▼──────┐ ┌─────▼─────┐     ┌────────────────────────┐
│ src/mcp.ts  │ │ src/cli.ts│     │ frontend/ (unbrowse.ai)│
│ 45 tools    │ │ 60+ cmds  │     │ registry, account,     │
└──────┬──────┘ └─────┬─────┘     │ wallet, billing UI     │
       │  in-process Fastify app  └──────────┬─────────────┘
┌──────▼──────────────▼─────────┐            │ fetch
│ Local engine                  │            │
│ capture/ → execution/ →       │   ┌────────▼─────────────┐
│ indexer/ → graph/ →           │   │ backend/ (CF Worker) │
│ intent-match.ts               ├──►│ beta-api.unbrowse.ai │
│ payments/ (x402 client rails) │   │ auth, keys, skills,  │
└───────────────────────────────┘   │ billing, x402, splits│
                                    └──┬────────┬──────────┘
                              Neon PG ◄┘        └► 7× CF KV
                              (accounts,           (keys, stripe cache,
                               telemetry)           sponsor ledger, skills,
                                                    sessions, traces, audit)
```

## Core data flows

### 1. Capture → publish → replay (the product loop)

1. **Capture** — `src/capture/index.ts` records a real browser interaction (with secret obfuscation in `src/capture/obfuscate.ts`, template holes in `src/capture/hole-template.ts`, credential binding in `src/capture/zk-bound-hole.ts` / `src/capture/wallet-bind.ts`).
2. **Infer (server-side, secret-stripped)** — the client is **thin**: it does not carry the route-inference intelligence. `src/capture/obfuscate.ts` strips every secret/PII *value* locally and replaces it with a one-way, wallet-bound commitment, then `src/capture/reveng-server-first.ts` POSTs only the **structure** (method / URL shape / param keys / schema) to `POST /v1/reveng`. The reverse-engineering / indexing / ranking engine runs **server-side only**; the client sees the inferred endpoints, never the inference IP. "Credentials never leave the machine" holds by construction — the server sees shape, never a secret. (`scripts/thin-client-gate.sh` = 0 enforces that no moat module is reachable from the public client closure.)
3. **Publish / contribute** — `unbrowse publish` posts a skill manifest to `POST /v1/skills` (`backend/src/routes/skills.ts`), which validates, sanitizes residual secrets (`backend/src/services/marketplace.ts`), and indexes endpoints for search. A contributed route is a **content-addressed, wallet-sealed, signed delta** (`src/values/content-address.ts`, `src/values/sealed-ledger.ts`, `src/values/signed-descent.ts`): the value is sealed to the contributor's wallet and only its content hash enters the append-only, hash-chained shared graph — tamper-evident end to end.
4. **Resolve & execute** — any agent resolves an intent (`src/intent-match.ts`, backend `/v1/search`) and replays the route (`src/execution/index.ts`), with anti-bot challenge handlers and proxy fallback (`src/execution/proxy-fetch.ts`, `src/execution/server-proxy-fallback.ts`).

> **Contribution to the shared graph (target architecture).** The write path is moving from "publish a sanitized manifest" to a **verified delta contribution**: a remote skill execution yields a route-delta that is admitted into the shared graph only behind a contribution-validity proof and an execution attestation bound to the contributor's wallet — the delta is proven well-formed and produced against the real origin **without revealing the captured traffic**. Discovery and routing stay free; paid execution settles fairly over x402 across the parties who created the value. The cryptographic construction is detailed in the forthcoming whitepaper.

### 2. Identity & auth

* Users sign in with **email magic links** (`backend/src/routes/auth.ts`, frontend `frontend/src/app/login/page.tsx`); there are no passwords.
* Auth artifacts are **API keys** (`ubr_<48-hex>`), SHA-256-hashed in KV (`backend/src/services/keys.ts`), validated by `backend/src/middleware/auth.ts` with timing-safe comparison, a Terms-of- Service version gate, and a global kill switch (`ALL_KEYS_REVOKED`).
* The CLI stores its key in `~/.unbrowse/config.json` (`src/client/index.ts`); the frontend stores it in `localStorage` (`frontend/src/lib/auth-context.tsx`).
* The client also **gates before it spends**: `src/auth/pre-resolve-gate.ts` blocks resolve for a personal/auth-shaped intent on a known login-walled host with no fresh cookie, and `src/auth/stale-endpoints.ts` removes endpoints that just returned 401/403 from future resolves. Full detail in [AUTH.md](/architecture/auth).

### 3. Money (four rails, one ledger)

* **Stripe subscriptions** — checkout/portal/webhooks + usage metering and tier detection (`backend/src/services/stripe.ts`, `backend/src/routes/billing.ts`).
* **Crypto subscriptions** — monthly USDC plans through a short-lived intent record, activated by an x402 payment, cached under the same KV shape as Stripe so the read side treats both identically (`backend/src/services/crypto-sub.ts`).
* **Per-request x402** — HTTP 402 responses carry signed payment terms (USDC on Solana mainnet, plus Base); the client signs and retries (`src/payments/x402-fetch.ts`, server gate `backend/src/middleware/x402-gate.ts`, settlement splits `backend/src/services/flex.ts`).
* **Sponsored (free tier)** — the platform fronts the cost up to daily caps (`backend/src/middleware/sponsor.ts`), partly refilled from a fixed fraction of Stripe revenue (`backend/src/services/sponsor-pool.ts`).

**API key wraps the wallet**: a key can be bound to a funding source — either an external wallet address or a prepaid credit budget — via `POST /v1/account/keys/:keyId/funding` (`backend/src/routes/account.ts`). Contributors who published before attaching a wallet are paid retroactively when the binding appears (`backend/src/services/splits.ts`).

**Earnings**: each paid execution is split among roles — infrastructure (platform), site owner (opt-in via DNS-verified domain claim), contributors (delta-weighted), optional maintainer/treasury — summing to exactly 100% (`backend/src/services/flex.ts`). A first-discoverer ledger additionally rewards whoever first captured a route (the toll ledger/emit pair in `src/` — fire-and-forget, never blocks the request path).

### 4. Wallets (pluggable, resolution order)

Client wallet resolution (`src/payments/x402-fetch.ts`, `src/cli-wallet.ts`):

1. **OWS (Open Wallet Standard)** vault at `~/.ows/wallets/*.json` — CAIP-2/ CAIP-10 identifiers and a declarative policy engine (`src/payments/ows.ts`).
2. `LOBSTER_WALLET_ADDRESS` / `~/.lobster/agents.json` — lobster.cash CLI delegation (`src/payments/lobster-pay.ts`).
3. `AGENT_WALLET_ADDRESS` (+ provider) — bring-your-own Solana signer.
4. Privy embedded wallet (web sign-in; backend-side signing endpoint is declared but not yet live — see `src/payments/x402-fetch.ts`).
5. None → sponsored free tier or honest `x402_no_wallet` failure.

## Deploy topology

* **Backend**: Cloudflare Worker, envs production/staging/experiments/ gate-staging (`backend/wrangler.toml`); cron every 6h for notification flush and buyback evaluation; Neon Postgres via `DATABASE_URL`.
* **Frontend**: Cloudflare Worker via open-next, zones `unbrowse.ai` and `www.unbrowse.ai` (`frontend/wrangler.jsonc`), R2 incremental cache.
* **CLI**: npm package `unbrowse` (`packages/skill/package.json`) and prebuilt binaries for darwin-arm64/x64, linux-arm64/x64, win-x64 (`scripts/build-binaries.sh`).

## Where to go next

* Command/tool inventory and local engine internals → [CLI.md](/architecture/cli)
* Route map, auth/key internals, billing internals, marketplace, data model → [BACKEND.md](/architecture/backend)
* Pages, auth/session handling, billing & wallet UI → [FRONTEND.md](/architecture/frontend)
* What "done" means per subsystem → [ACCEPTANCE-CRITERIA.md](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/architecture/ACCEPTANCE-CRITERIA.md)
* Required unit tests and current coverage → [TEST-SPECS.md](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/architecture/TEST-SPECS.md)


# CLI & Local Engine

> **At a glance** — one Bun-compiled binary exposes the same engine three ways: 60+ CLI commands, \~45 MCP tools over stdio, and an embedded SDK. The engine pipeline is capture → index → rank → execute, with secrets kept as vault pointers (never plaintext in artifacts). Client payment rails (x402 envelope signing, OWS vault, lobster.cash delegation) resolve a wallet at request time and always report honest outcomes.

> Surface: everything that runs on the user's machine. Source of truth: `src/` and `packages/` at v8.3.0-preview\.2.

## 1. CLI

### Entry & dispatch

* Entry point: `src/cli.ts` (single bundled entry, 60+ subcommands dispatched via switch).
* Newer verb-based layer: `src/cli-v7/` — 37 operations dispatched through a kind map (`src/cli-v7/dispatch/index.ts`), grouped under three verbs (build / act / inspect style groupings).
* Run from source: `bun src/cli.ts` (`package.json` script `cli`).

### Command inventory (by area)

| Area           | Commands                                                                                                                                                             |
| -------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Lifecycle      | `setup`, `login`, `register`, `account`, `status`, `restart`, `stop`, `health`, `mcp`                                                                                |
| Core loop      | `index`, `resolve`, `run`, `execute`/`exec`, `search`, `explain`, `publish`, `review`, `feedback`, `annotate`                                                        |
| Browser verbs  | `go`, `click`, `fill`, `type`, `press`, `select`, `scroll`, `screenshot`, `snap`, `text`, `markdown`, `back`, `forward`, `submit`, `eval`, `close`, `connect-chrome` |
| Auth & secrets | `auth`, `auth-capture`, `cookies`, `browse-cookies`                                                                                                                  |
| Money          | `wallet`, `payment-provider`, `billing`, `earnings`, `flywheel`                                                                                                      |
| Skills         | `skills`, `skill`, `capture`, `contract`                                                                                                                             |
| Diagnostics    | `stats`, `sessions`, `inspect`, `dashboard`, `corpus-test`, `corpus-run`, `note`, `fetch`, `sync`, `mode`, `plan`                                                    |

### Distribution

* npm package `unbrowse` (`packages/skill/package.json`), wrapper `packages/skill/bin/unbrowse-wrapper.mjs`.
* Single binaries compiled with `bun build --compile` for five platforms (`scripts/build-binaries.sh`); binary entry `src/single-binary.ts`.
* Release attestation headers `X-Unbrowse-Release-Manifest` / `X-Unbrowse-Release-Signature` are checked client-side (`src/client/index.ts`).

## 2. MCP server

* Implementation: `src/mcp.ts` — JSON-RPC 2.0 over **stdio**; supported protocol versions 2024-11-05 through 2025-11-25.
* Launches the same in-process HTTP app the CLI uses (`src/runtime/in-process-app.ts`), so MCP and CLI share one engine.
* \~45 tools, `unbrowse_*`-prefixed. Core set: `unbrowse_resolve`, `unbrowse_execute`, `unbrowse_run`, `unbrowse_search`, `unbrowse_search_endpoints`, `unbrowse_index`, `unbrowse_publish`, `unbrowse_skill`/`unbrowse_skills`, `unbrowse_auth_capture`, `unbrowse_auth_inventory`, `unbrowse_cookies`, `unbrowse_sessions`, `unbrowse_earnings`, `unbrowse_settings`, `unbrowse_health`, `unbrowse_stats`, `unbrowse_feedback`, `unbrowse_review`, `unbrowse_annotate`, `unbrowse_diagnose`, `unbrowse_trace`, `unbrowse_validate`, `unbrowse_spec`, plus the browser verbs (`unbrowse_go`, `unbrowse_click`, `unbrowse_fill`, `unbrowse_type`, `unbrowse_press`, `unbrowse_select`, `unbrowse_scroll`, `unbrowse_screenshot`, `unbrowse_snap`, `unbrowse_text`, `unbrowse_markdown`, `unbrowse_submit`, `unbrowse_eval`, `unbrowse_close`, `unbrowse_fetch`, `unbrowse_sync`).
* Optional env `UNBROWSE_MCP_V7_DISPATCH` routes tool calls through the v7 kind-map dispatch (`src/mcp.ts`).
* Resources: cookies, history, vault exposed as MCP resources (`src/mcp.ts` textResource helpers). MCP prompts are declared in types but handlers are not fully wired (known gap).

## 3. SDK & shim packages (`packages/`)

* `@unbrowse/sdk` — **deprecated**; points to the HTTP-first client. The maintained SDK ships inside the main package as `unbrowse/sdk` (and `unbrowse/sdk/wallet-standard`) from `packages/skill/dist-sdk/`.
* Drop-in shims that route existing libraries through the Unbrowse cache:
  * Browser automation: `playwright-shim`, `stagehand-shim`
  * Scraping: `firecrawl-shim`
  * HTTP clients: `axios-shim`, `got-shim`, `ky-shim`, `node-fetch-shim`, `cross-fetch-shim`, `undici-shim`, `wretch-shim`
  * Agent frameworks: `langchain-js`, `llamaindex`, `mastra`, `openai-agents`, `superagent-shim`
  * Search providers: `exa-shim`, `tavily-shim`
  * Python bridges: `py-requests`, `py-httpx`, `py-aiohttp`, `py-urllib3`, `py-browser-use`, `py-crewai`, `py-exa`, `py-pydantic-ai`

## 4. Local engine

| Stage     | Module                               | Responsibility                                                                                                                                                                                                                                                                                                              |
| --------- | ------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Capture   | `src/capture/index.ts`               | Record an interaction for a URL+intent; secret obfuscation (`obfuscate.ts`), template holes (`hole-template.ts`), proof-bound credential holes (`zk-bound-hole.ts`), wallet signature binding (`wallet-bind.ts`), SSR fast path, HTTP fallback (`curl-impersonate-fallback.ts`), escalation on miss (`escalate-on-miss.ts`) |
| Execution | `src/execution/index.ts`             | Replay with resolved pointers; anti-bot challenge handlers (`cf-challenge.ts`, `px-challenge.ts`, `akamai-challenge.ts`, `kasada-challenge.ts`); token resolution (`token-resolver.ts`); proxy + server-proxy fallback; drift recovery (`drift-page-recovery.ts`)                                                           |
| Indexing  | `src/indexer/`                       | Background queue (`capture-spool.ts`, `queue-store.ts`, `worker.ts`)                                                                                                                                                                                                                                                        |
| Ranking   | `src/graph/` + `src/intent-match.ts` | Route cache, endpoint ranking, planner, session tracking, decision trace store                                                                                                                                                                                                                                              |

## 5. Auth from the client side

* First run: `unbrowse setup` (`src/cli.ts` `cmdSetup`) → `ensureRegistered()` (`src/client/index.ts`) prompts for email, exchanges it for an API key against the backend, prompts contribution mode (`src/cli-setup.ts`) and optional wallet setup.
* Credentials live in `~/.unbrowse/config.json` (fields: `api_key`, `agent_id`, `agent_name`, `email`, `user_id`, ToS acceptance), with multi-profile support via `UNBROWSE_PROFILE` → `~/.unbrowse/profiles/<name>/config.json` (`src/client/index.ts`).
* Site credentials are never stored as plaintext values in routes: cookies cache per-domain (`src/auth/browser-cookies.ts`); vault adapters (1Password, Bitwarden, keychain — `src/values/adapters/`) resolve pointers lazily at execution time.
* `unbrowse auth-capture <url>` opens an interactive login and binds credential pointers for later replay.

## 6. Configuration

* Env loaded at startup from `.env` / `.env.runtime` (`src/cli.ts`).
* Key variables (see `src/config/`, `src/env/`):
  * URLs: `UNBROWSE_URL` (local daemon, default `http://localhost:6969`), `UNBROWSE_BACKEND_URL`, `UNBROWSE_FRONTEND_URL`
  * Identity: `UNBROWSE_API_KEY`, `UNBROWSE_PROFILE`, `UNBROWSE_CONFIG_DIR`
  * Wallet/payments: `UNBROWSE_WALLET_ADAPTER`, `UNBROWSE_WALLET_KEY`, `UNBROWSE_WALLET_SECRET`, `UNBROWSE_DISABLE_LOCAL_WALLET`, `UNBROWSE_X402_MAX_COST_USD` (default $1.00), `UNBROWSE_X402_SIGNER` (extensible signer hook), `OWS_WALLET_ADDRESS`, `LOBSTER_WALLET_ADDRESS`, `AGENT_WALLET_ADDRESS`, `FLEX_ESCROW_ADDRESS`, `FLEX_SESSION_KEY_ADDRESS`
  * Proxy/egress: `UNBROWSE_PROXY_URL`, `UNBROWSE_DIRECT_EGRESS`
  * Telemetry/tracing: `UNBROWSE_TELEMETRY`, `UNBROWSE_TRACE`, `UNBROWSE_TRACE_DIR`
  * Test/dev: `UNBROWSE_NON_INTERACTIVE`, `UNBROWSE_LOCAL_ONLY`, `UNBROWSE_MCP_V7_DISPATCH`

## 7. Client payment rails (detail)

* **x402 wrapper** `src/payments/x402-fetch.ts`: intercepts HTTP 402/407, reads the `accepts[]` payment-terms envelope, resolves a wallet adapter, enforces the per-request cost ceiling, signs, retries once, and records an honest outcome state (`x402_signed`, `x402_no_wallet`, `x402_signer_error`, `x402_cost_exceeded`, `x402_retry_blocked`, `x402_passthrough`). It never fabricates success.
* **Flex settlement** `src/payments/flex-pay.ts`: pays server-frozen splits verbatim (never recomputed client-side); signing is delegated to the session-key SDK; returns `{data, settled, authorization}` or throws.
* **lobster.cash bridge** `src/payments/lobster-pay.ts`: shells out to the `lobstercash` CLI for sign/broadcast; availability check is the presence of `~/.lobster/agents.json`.
* **OWS provider** `src/payments/ows.ts`: Open Wallet Standard v1.3 vault (`~/.ows/wallets/<uuid>.json`), CAIP-2/CAIP-10 account identifiers, and a declarative allow/deny/warn policy engine (`allowed_chains`, `expires_at`). Preferred provider when present.
* **Wallet status** `src/cli-wallet.ts`: read-only reconciliation of local wallet config vs the server-side agent profile (`/v1/agents/me`); warns on mismatch.
* **Provider chooser** `src/cli-payment-setup.ts`: pay.sh / lobster.cash / external Solana / Privy / skip(free tier); persisted locally and synced to the backend.
* **Discovery toll ledger** — the `*-toll-ledger.ts` / `*-toll-emit.ts` pair in `src/`: immutable first-discoverer binding per route; per-charge metering splits operator / discoverer / site-owner with exact conservation; emission is fire-and-forget and never breaks the request path.


# Backend

> **At a glance** — a Hono app on Cloudflare Workers (Neon Postgres + 7 KV namespaces). Auth is email magic link → SHA-256-hashed API keys with a ToS gate and global kill switch. Billing admits a request via any of four rails — Stripe sub, USDC sub, per-request x402, or platform sponsorship — all converging on one subscription-cache shape and one settlement-split function. Marketplace publishing sanitizes secrets and verifies domain ownership via `.well-known` or dual-provider DNS TXT.

> Source of truth: `backend/` at v8.3.0-preview\.2. Public base URL: `https://beta-api.unbrowse.ai`.

## 1. Runtime & topology

* **Framework**: Hono on Cloudflare Workers; entry `backend/src/index.ts`, routes registered there from `backend/src/routes/` (\~48 modules).
* **Environments**: production / staging / experiments / gate-staging (`backend/wrangler.toml`).
* **Postgres (Neon)** via `DATABASE_URL`: accounts, telemetry (`backend/schema/telemetry-sessions.sql`). Note: only the telemetry DDL is checked in; accounts/usage tables are managed by service code.
* **KV namespaces** (`backend/wrangler.toml`):
  * `STATS_KV` — analytics, search index, sponsor ledger, skill manifests
  * `AUDIT_LOG` — pointer-only Ed25519-signed receipts
  * `RESPONSE_CACHE` — response cache (optional binding, graceful miss)
  * `SESSION_STATE` — persisted session pointers (per-wallet prefixes)
  * `TRACE_STATE` — decision traces (TTL 7d)
  * `SETTINGS_STATE` — durable per-wallet preferences
  * `SCREENSHOT_BLOB` — content-addressed PNGs (TTL 30d)
* **Cron**: `17 */6 * * *` — flush queued GitHub notifications + evaluate buyback trigger (`backend/src/index.ts`).

## 2. Route map

### Public (no auth)

| Route                                                                                                   | Purpose                             |
| ------------------------------------------------------------------------------------------------------- | ----------------------------------- |
| `GET /v1/health`                                                                                        | Health check                        |
| `GET /v1/skills`                                                                                        | Skill list (card view, edge-cached) |
| `GET /v1/skills/popular`                                                                                | Trending skills                     |
| `GET /v1/skills/:id/card`                                                                               | Trimmed skill card                  |
| `GET /v1/skills/by-domain/:domain/skill.md`                                                             | Rendered skill doc for a domain     |
| `GET /v1/skills/:id/endpoints/:eid/schema`                                                              | Endpoint response schema            |
| `GET /v1/search`                                                                                        | Search (BM25 + semantic)            |
| `GET /v1/stats/traction/:domain` · `GET /v1/stats/by-wallet/:wallet` · `GET /v1/stats/validate/:intent` | Public stats                        |
| `GET /v1/agents/:id`                                                                                    | Public agent profile                |
| `GET /v1/claim/status` · `GET /v1/claim/takedown/status`                                                | Domain claim/opt-out status         |
| `GET /v1/dashboard/trends` · `GET /v1/miners/demand` · `GET /v1/issues/:id`                             | Misc public reads                   |
| `POST /v1/auth/email/start` · `/v1/auth/email/verify/:token`                                            | Magic-link flow (public by nature)  |

### Bearer auth (API key — `backend/src/middleware/auth.ts`)

| Route                                                                                        | Purpose                                                                     |
| -------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------- |
| `GET/POST /v1/account/keys`, `DELETE /v1/account/keys/:keyId`                                | API key list / create / revoke                                              |
| `POST /v1/account/keys/:keyId/funding`                                                       | Bind key funding: wallet or credit budget (`backend/src/routes/account.ts`) |
| `GET /v1/account/me` · `GET/POST /v1/account/preferences` · `GET /v1/account/sponsor-status` | Account profile, preferences, sponsor balance                               |
| `POST /v1/skills` · `PATCH /v1/skills/:id` · `PUT /v1/skills/:id/endpoints/:eid/schema`      | Publish / update skills                                                     |
| `POST /v1/skills/by-domain/:domain/verify/{challenge,probe}`                                 | Domain verification (`.well-known`)                                         |
| `POST /v1/claim/{challenge,verify}` · `POST /v1/claim/takedown/{challenge,verify}`           | DNS-TXT domain↔wallet claim and owner opt-out                               |
| `GET /v1/billing/me` · `POST /v1/billing/checkout` · `POST /v1/billing/portal`               | Stripe state / checkout / portal                                            |
| `POST /v1/billing/crypto-sub/intent` · `POST /v1/billing/crypto-sub/activate/:intentId`      | USDC subscription (activation x402-gated)                                   |
| `GET /v1/dashboard/me`                                                                       | Spend/earn dashboard data                                                   |
| `POST /v1/stats`                                                                             | Record custom stats                                                         |

### x402 payment-gated

| Route                             | Purpose                                                                                                            |
| --------------------------------- | ------------------------------------------------------------------------------------------------------------------ |
| `POST /v1/skills/:id/execute`     | Execute a published skill (per-manifest payment terms)                                                             |
| `POST /v1/search`                 | Paid semantic search lane                                                                                          |
| `POST /v1/llm/:provider/messages` | Universal LLM proxy with markup, upstream xgate.run (`backend/src/routes/llm.ts`, `backend/src/services/xgate.ts`) |

### Admin / internal

| Route                                       | Purpose             |
| ------------------------------------------- | ------------------- |
| `GET /v1/admin/sponsor-ledger` (ADMIN\_KEY) | Read sponsor ledger |
| `POST /v1/ops/reindex`                      | Force reindex       |

### State & audit surface (v7)

A unified state-append route (audit/session/trace/settings writes share one entry point), `GET /v1/audit/verify/:key`, `POST /v1/session/park`, `GET /v1/session/restore/:id`, `POST /v1/trace/append`, `GET /v1/trace/by-receipt/:cacheKey`, `GET /v1/trace/by-wallet`, `POST /v1/settings/set`, `GET /v1/settings/get/:keyHash`, `POST /v1/screenshot/store`, `GET /v1/screenshot/by-sigkey/:sigKey`.

## 3. Auth & API keys

* **Magic link**: `POST /v1/auth/email/start` validates the address and sends a one-time link (Resend); `verify/:token` (30-min TTL) upserts the user in Postgres and mints an API key (`backend/src/routes/auth.ts`).
* **Key format**: `ubr_` + 48 hex chars; `keyId` = first 32 chars of the hex body (`backend/src/services/keys.ts`).
* **Storage**: only SHA-256 hashes — KV `keyhash:<sha256>` → `{keyId, name, created_at, revoked_at}` plus reverse index `keyid:<keyId>`. Plaintext is shown once at creation and never stored.
* **Verification**: hash the presented key, KV lookup, timing-safe compare (`backend/src/middleware/auth.ts`); revocation flips `revoked_at` on both records idempotently.
* **Gates**: ToS version check (403 on stale acceptance); global kill switch `ALL_KEYS_REVOKED` (401 + rotation pointer); staging accepts any bearer for dev convenience — production always verifies.
* **Key funding binding**: `keyfund:<keyId>` ties a key to a wallet or a prepaid credit budget — this is the "API key wraps the wallet" mechanism (`backend/src/routes/account.ts`). Agent registration auto-binds a wallet delivered at sign-in (`backend/src/routes/agents.ts`).

## 4. Billing & payments

### Stripe (card rail) — `backend/src/services/stripe.ts`, `backend/src/routes/billing.ts`

* Customer per user (`getOrCreateCustomer`), cached: KV `stripe:user:<userId>` → customerId (1y TTL); `stripe:customer:<customerId>` → subscription cache JSON `{status, current_period_*, priceId, productId, brand, last4, paymentMethod}` (90d TTL).
* Webhooks: 19 allow-listed event types (checkout, subscription, invoice, payment-intent) → `processBillingEvent`.
* Usage metering: monotonic KV counter `billing:usage:<userId>:<YYYY-MM>`; tier inferred from priceId (Base / Pro / Enterprise); auto-refill charge on overage; `subscriptionAdmits()` **fails closed** when Stripe is unconfigured or the sub is inactive.

### Crypto subscription (USDC rail) — `backend/src/services/crypto-sub.ts`

* Plans: base ($19, 200k quota) and pro ($59, 1M quota), env-tunable.
* Flow: mint a 10-minute intent (`/crypto-sub/intent`) → pay via x402 → `activate/:intentId` writes the same subscription-cache shape Stripe uses (customerId `crypto-<userId>`), so downstream admission code is rail-agnostic. Stripe↔crypto double-subscription is rejected (`assertNoStripeConflict`).

### Per-request x402 (pay-as-you-go rail)

* Gate: `backend/src/middleware/x402-gate.ts` returns HTTP 402 with payment terms: scheme (`exact` or session-key escrow), network (Solana mainnet / Base / devnets), asset (USDC mint/contract), amount, recipient, and frozen split metadata.
* Settlement splits (`backend/src/services/flex.ts`): five roles summing to exactly 10000 bps — infrastructure (platform, default 50% `PLATFORM_BPS`/`FLEX_PLATFORM_BPS`), site\_owner (only when the domain owner opted in with a verified wallet), contributors (delta-weighted, up to 5), maintainer and treasury (env-gated, default 0). Markup clamped to 500–8000 bps.
* Contributor wallet back-fill from key-funding bindings (`backend/src/services/splits.ts`): publish first, attach a wallet later, earn retroactively.
* Payment-term selection per skill manifest: `direct`, `subscription`, `flex`, `auction`, `sponsored`.

### Sponsored tier (platform-funded) — `backend/src/middleware/sponsor.ts`

* `maybeSponsor()` → `sponsored` (with ledger id) / `exhausted` (agent\_cap | global\_cap | no\_wallet) / `opted_out`.
* Caps in micro-cents: per-agent `SPONSOR_CAP_DAILY_USD` (default $1/day), global `SPONSOR_GLOBAL_DAILY_USD` (default $50/day); `SPONSOR_FREE_MODE` lifts the per-agent cap to the global cap.
* KV: `sponsor:agent:<id>:<date>`, `sponsor:global:<date>`, `sponsor:ledger:<ledgerId>`.
* Funding flywheel (`backend/src/services/sponsor-pool.ts`): a configured fraction of Stripe revenue (default 10%, `PLATFORM_REVENUE_TO_POOL_BPS`) is carved into the sponsor pool (`sponsor:pool:balance:uc`), idempotent on event id.
* Settlement (`backend/src/services/settlement.ts`): batches unsettled ledger rows by skill → recipient wallets; zeroes the owner lane for opted-out domains; supports dry-run; on-chain submission via the facilitator (`backend/src/services/sponsor-flex.ts`) using a dedicated platform escrow + short-lived session key; settlement runs after the response (`waitUntil`), never blocking.

### LLM proxy

`POST /v1/llm/:provider/messages` proxies to upstream providers via xgate.run with a markup; payable either by subscription credit (bearer) or x402 (`backend/src/routes/llm.ts`, `backend/src/services/xgate.ts`).

## 5. Marketplace & publishing

* **Manifest** (`backend/src/types.ts`): skill\_id, version, name, intent\_signature, domain, endpoints\[], contributors\[], owner wallet (USDC ATA), compensation opt-in, markup\_bps, payment\_term, lifecycle.
* **Publish** `POST /v1/skills` (`backend/src/routes/skills.ts` → `backend/src/services/marketplace.ts`): schema validation → secret-leak sanitization pass (including an AI scrub step) → search indexing → KV store (`skill:<id>`) → graph edges (requires/yields) → cache invalidation. Updates are version bumps, not in-place edits.
* **Domain verification** (`backend/src/services/domain-verifier.ts`): challenge token placed at `https://<domain>/.well-known/<token>`; probe enforces HTTPS, 5s timeout, 4KB cap, no redirects, and SSRF guards (private/link-local IP bans). Production enforcement is flag-gated (`REQUIRE_DOMAIN_VERIFICATION`).
* **Domain claim** (`backend/src/services/domain-claim.ts`, `backend/src/routes/claim.ts`): DNS TXT record `_unbrowse.<domain>` binding domain → Solana wallet, verified against **two independent DoH providers** (Cloudflare + Quad9); apex domains only; 10 challenges/hr/ domain. Takedown flow lets a verified owner opt out — settlement then zeroes that domain's owner lane. Bindings live in KV (`domain-binding:<domain>`, `domain-optout:<domain>`).

## 6. Data model (summary)

| Entity                               | Store    | Key/table                                                                            |
| ------------------------------------ | -------- | ------------------------------------------------------------------------------------ |
| Account (email, ToS)                 | Postgres | accounts (via service code)                                                          |
| Telemetry sessions/clusters          | Postgres | `telemetry_sessions`, `telemetry_clusters` (`backend/schema/telemetry-sessions.sql`) |
| API keys                             | KV       | `keyhash:<sha256>`, `keyid:<keyId>`                                                  |
| Key funding                          | KV       | `keyfund:<keyId>`                                                                    |
| Stripe/crypto sub cache              | KV       | `stripe:user:<userId>`, `stripe:customer:<customerId>`                               |
| Crypto intents                       | KV       | `crypto:intent:<intentId>`                                                           |
| Usage counters                       | KV       | `billing:usage:<userId>:<YYYY-MM>`                                                   |
| Skills                               | KV       | `skill:<skillId>` (+ search index)                                                   |
| Sponsor spend/ledger/pool            | KV       | `sponsor:agent:*`, `sponsor:global:*`, `sponsor:ledger:*`, `sponsor:pool:*`          |
| Domain bindings/opt-outs             | KV       | `domain-binding:<domain>`, `domain-optout:<domain>`                                  |
| Sessions/traces/settings/screenshots | KV       | wallet-prefixed namespaces                                                           |

## 7. Known gaps / uncertainties

* Full Postgres schema is not checked in (only telemetry DDL); accounts and usage tables are defined implicitly by service code.
* Live on-chain settlement depends on the external facilitator SDK; the repo tests it via dry-runs.
* Privy-backed server-side x402 signing endpoint is referenced by the client but not yet implemented.


# Frontend

> **At a glance** — two unrelated Next.js apps. The product UI (`unbrowse.ai`) does registry browsing, magic-link sign-in with localStorage sessions, dashboards, wallet pairing, and sponsored-tier billing display. The metrics dashboard (`launch.unbrowse.ai`) is a public, no-auth showcase fed by Unkey/GitHub/npm — it never calls the Unbrowse backend. Known gaps: no key-management UI, no Stripe pricing page, client-side-only auth gating.

> Two apps. `frontend/` (in this monorepo) is the canonical product UI at `unbrowse.ai`. `unbrowse-dashboard` (sibling repo) is a public, read-only metrics page at `launch.unbrowse.ai` and does not talk to the backend.

## A. `frontend/` — product UI (unbrowse.ai)

### Stack & deploy

* Next.js 16 App Router, React 19, TypeScript (`frontend/package.json`).
* Cloudflare Workers via open-next (`frontend/wrangler.jsonc`), zones `unbrowse.ai` and `www.unbrowse.ai`, staging + experiments envs, R2 incremental cache.
* Backend base URL `https://beta-api.unbrowse.ai`, overridable with `NEXT_PUBLIC_API_URL` (`frontend/src/lib/api-base.ts`).

### Route map (`frontend/src/app/`)

| Route                                                                      | Purpose                                                                    |
| -------------------------------------------------------------------------- | -------------------------------------------------------------------------- |
| `/`                                                                        | Skill registry: search, popular skills, marketing sections (`page.tsx`)    |
| `/search`                                                                  | Intent-based skill discovery                                               |
| `/skill/:id`                                                               | Skill detail                                                               |
| `/aiko`                                                                    | Conversational chat that executes skills                                   |
| `/login`                                                                   | Magic-link email sign-in (`login/page.tsx`)                                |
| `/account`                                                                 | Account hub & onboarding wizard                                            |
| `/account/wallet`                                                          | Solana wallet pairing for x402 settlement (`account/wallet/page.tsx`)      |
| `/account/session-key`                                                     | Session & API key display                                                  |
| `/account/escrow`, `/account/cookies`                                      | Advanced settings                                                          |
| `/dashboard`                                                               | Authed: agent stats, execution history, preferences (`dashboard/page.tsx`) |
| `/dashboard/:wallet`                                                       | Public per-wallet earnings/ledger view                                     |
| `/billing`                                                                 | Sponsored-tier status and pay-per-request explanation (`billing/page.tsx`) |
| `/docs`, `/faq`, `/contact`, `/privacy`, `/terms`, `/security`, `/classic` | Docs/legal/marketing                                                       |
| `/compare/:slug`, `/vs/:slug`                                              | SEO comparison pages                                                       |
| `/ops`                                                                     | Internal ops view (auth required)                                          |
| `/[domain]`                                                                | Dynamic per-domain proxy/capture pages                                     |

### Auth

* Magic-link only (no passwords): email → `POST /v1/agents/login` → token polling → `POST /v1/agents/token/consume` returns `{api_key, agent_id, user_id, email}` (`frontend/src/app/login/page.tsx`, `frontend/src/lib/auth-context.tsx`).
* Session = `localStorage["unbrowse_auth"]` holding the API key and agent identity; all authed fetches send `Authorization: Bearer <api_key>`. There is no server session cookie and no Next.js middleware gate — auth checks are client-side via `useAuth()`.
* CLI↔web pairing: `GET /v1/local/pair?token=…`.
* Optional Privy embedded-wallet provider is dynamically imported and feature-gated (`frontend/src/lib/privy-provider.tsx`).

### Billing & wallet UI

* `/billing` shows the sponsored allowance (calls `GET /v1/account/sponsor-status`) and explains the per-request USDC settlement model; the card-subscription checkout flow is backend-driven (`/v1/billing/checkout`) and not currently surfaced as a Stripe pricing page in this UI.
* `/account/wallet` pairs an external Solana wallet (manual address entry or Privy modal). Transaction signing/broadcast is **not** done in the frontend — it happens CLI-side or backend-side.
* API keys: created implicitly at registration; displayed at `/account/session-key`. There is **no create/revoke key management UI** yet (the backend endpoints exist — see gap list).

### Backend contract used by the UI (`frontend/src/lib/api.ts`)

* Auth/profile: `POST /v1/agents/register`, `POST /v1/agents/login`, `POST /v1/agents/token/consume`, `GET /v1/agents/me`, `GET /v1/agents/:id`
* Skills: `GET /v1/skills` (+card view), `GET /v1/skills/popular`, `GET /v1/skills/:id`, `POST /v1/search`, `POST /v1/search/domain`
* Stats/dashboard: `GET /v1/stats/summary`, `GET /v1/dashboard/me`
* Account: `GET /v1/account/me`, `GET/POST /v1/account/preferences`, `GET /v1/account/sponsor-status`
* Misc: `GET /v1/tos/current`, `GET /v1/ops`

## B. `unbrowse-dashboard` — public metrics (launch.unbrowse.ai)

* Next.js 16 on Cloudflare Pages (`unbrowse-dashboard/wrangler.toml`); single page (`src/app/page.tsx`) auto-refreshing every 60s from its own edge route `GET /api/metrics` (`src/app/api/metrics/route.ts`).
* **No auth, no billing, no wallet** — read-only public showcase.
* Data sources (server-side only; secrets never reach the client — `src/lib/api.ts`):
  * Unkey API: key list + verification analytics (DAU/WAU, retention, outcomes)
  * GitHub API: stars/forks/watchers for the public repo
  * npm API: package download counts (CLI + integration plugin)
  * Cloudflare Analytics: env vars wired but not yet queried
* Known placeholders are listed in `unbrowse-dashboard/MISSING_DATA.md` (geo distribution, endpoint breakdown, latency percentiles, etc.).
* Relationship: complementary, zero coupling — different domain, different data sources, no calls to `beta-api.unbrowse.ai`.

## Gaps observed (frontend)

1. No API-key management UI (create/rename/revoke) despite backend support.
2. No Stripe pricing/checkout page in the UI; subscription purchase relies on backend endpoints being called from elsewhere (CLI/dashboard link).
3. Client-side-only auth gating (`localStorage`) — acceptable for an API-key product but means authed pages render a shell before redirect.
4. Privy wallet path feature-gated and incomplete (matching the backend's unimplemented signing endpoint).


# Security

> **At a glance** — Unbrowse's client is a thin, readable transport; the value lives on the server. Security therefore rests on four pillars: (1) a tampered or republished build cannot authenticate as official (a replayed signature only works on the attacker's own copy, which gains nothing — §1), (2) anti-bot challenges are handled through detect-then-replay handlers, (3) the shared route graph is tamper-evident and economically accountable, and (4) paid execution is gated and settled server-side. Every claim below cites a real file path.

> Reviewed 2026-06-17 against build v9.4.12 (`src/build-info.generated.ts`). This document is the code-grounded companion to the honest threat model in [../SECURITY.md](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/SECURITY.md). Start at [OVERVIEW.md](/architecture/overview) for the whole-system map.

## The honest premise

The CLI is JavaScript and ships readable in an npm tarball. Anyone who installs it can read the source — obfuscation is a tax on the reader, not a wall. The design goal is **not** "the code is unreadable." It is: *a modified build is useless because it cannot authenticate to the unbrowse index*, so it loses the marketplace, the route graph, ranking, recipes, and the x402 economics. The value lives on the servers; the client is transport. See [../SECURITY.md](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/SECURITY.md) for the full threat model.

## 1. Anti-tamper / official-package binding

Three independent layers make a tampered or republished client worthless:

| Layer                       | Where                                                                                | What it enforces                                                                                                                                                                                                                                                                                           |
| --------------------------- | ------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Release-manifest HMAC**   | `scripts/build-release-manifest.ts`, `src/build-info.generated.ts`, `src/version.ts` | At CI build time a manifest `{release_version, git_sha, code_hash, issued_at}` is signed with HMAC-SHA256 (`UNBROWSE_RELEASE_MANIFEST_SIGNING_SECRET`, CI-only) and baked into every binary. The backend HMAC-verifies it on marketplace calls; the secret never ships, so the signature cannot be forged. |
| **npm provenance**          | `.github/workflows/release.yml`                                                      | Publishes with `npm publish --provenance` (Sigstore attestation), binding the tarball to the exact GitHub Actions run that built it. Republishing a modified clone under the official `unbrowse` name is cryptographically blocked.                                                                        |
| **Server-bound exec-token** | minted at `POST /v1/session/exec-token` (backend), client carries it                 | Per-session HMAC bound to `{agent_id, build_sha, deployed_at, exp}`. Currently observe-mode; `EXEC_TOKEN_ENFORCE=1` flips it to hard 401 rejection.                                                                                                                                                        |

The thin-client boundary itself is a runnable gate: `scripts/thin-client-gate.sh` must exit 0 — it proves no server-side "moat" module is reachable from the public client closure.

**Residual gap (stated honestly).** An attacker can extract the manifest + signature from an official tarball and replay it against their own locally modified copy — the signature signs the manifest, not the running code. This is the DRM impossibility. It only affects *their own* copy (provenance blocks redistribution) and every paid action settles server-side, so the modified client gains nothing.

## 2. Anti-bot / challenge handling

When a replayed route or a fetch hits a bot-management wall, Unbrowse detects the specific system and runs a matched handler rather than failing blindly. Each handler follows the same shape: *extract the challenge bundle → replay it in the Kuri sandbox → harvest the clearance cookie → retry the original request.* Handlers degrade honestly (return null / a typed sub-state) rather than fabricating a fake clearance.

| Anti-bot system                                                       | Handler                                              | Clearance signal               | Status                                                                                                                                                                                                                                                    |
| --------------------------------------------------------------------- | ---------------------------------------------------- | ------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Cloudflare (JS challenge)                                             | `src/execution/cf-challenge.ts`, `capzy-cf-solve.ts` | `cf_clearance`                 | Capzy solver wired (`AntiCloudflareTask`, **proxy-required**, IP+UA-bound) + in-house bundle-replay fallback. API contract live-witnessed; a successful clearance depends on the target's solvability (challenges can return `ERROR_CAPTCHA_UNSOLVABLE`). |
| PerimeterX                                                            | `src/execution/px-challenge.ts`                      | `_pxhd` + `_px3`               | bundle extract + replay                                                                                                                                                                                                                                   |
| Akamai Bot Manager                                                    | `src/execution/akamai-challenge.ts`                  | `_abck`                        | detection live; solver pending                                                                                                                                                                                                                            |
| Kasada                                                                | `src/execution/kasada-challenge.ts`                  | `x-kpsdk-cd` + `x-kpsdk-ct`    | detection only (needs live DOM/crypto, slated for browser-eval path)                                                                                                                                                                                      |
| Tencent Cloud WAF (TCaptcha)                                          | `src/execution/tencent-waf-solve.ts`                 | `/WafCaptcha` clearance cookie | solved via Capzy (`UNBROWSE_CAPZY_KEY`)                                                                                                                                                                                                                   |
| Generic captcha (reCAPTCHA, hCaptcha, Turnstile, FunCaptcha, GeeTest) | `src/execution/captcha-solve.ts`, `captcha-clear.ts` | injected token                 | Capzy first, x402-paid solver fallback                                                                                                                                                                                                                    |

> **Status legend.** *live* = shipping; *wired* = a real solve path exists (managed solver or replay) but live verification needs a key + a gated target; *detection only* = the blocker is detected but not yet solved. Cloudflare is *wired* via Capzy (`src/execution/capzy-cf-solve.ts`, proxy-required). Akamai and Kasada remain *detection only*: Capzy offers **no** task type for them (live-witnessed `ERROR_TASK_NOT_SUPPORTED`), so they await a different solver — their `solve*AndRetry` bodies stay stubs rather than fake a path that cannot exist.

Cost guardrails live in `src/execution/captcha-solve.ts`: a per-probe budget (default $0.01, prevents double-solve) and a per-day budget (default $1.00, env-configurable). The backend holds the solver key — the client never does (`captcha-clear.ts`). Honest degrade emits a typed sub-state (`no_sitekey`, `no_payment`, `solver_error`) instead of a fake token.

## 3. Trust layer — a tamper-evident, accountable route graph

The shared graph only stays useful if freshness is maintained and contributions are accountable. The trust layer (`src/trust/`) provides this:

* **Proof-of-indexing** (`src/trust/proof-of-indexing.ts`) — a maintainer re-fetches a route's live source, computes an order-independent, value-independent **schema fingerprint** (`schemaDescriptor` → `schemaHash` = `sha256:<hex>`), and emits a signed, content-addressed attestation hash-chained to the prior proof. `proofDiverged` makes it falsifiable: if the live schema no longer matches the committed hash, the proof is provably stale.
* **Bond-challenge** (`src/trust/bond-challenge.ts`) — a maintainer bonds collateral to become *eligible* to publish proofs (eligibility is boolean, never a score). `resolveChallenge` re-indexes on challenge; if the proof diverged, the bond is slashed. The economic policy constants are **injected**, never hard-coded in the module.
* **Ledger-checkpoint** (`src/trust/ledger-checkpoint.ts`) — batches signed ledger records into Merkle roots (`merkleRoot` / `merkleProof`); checkpoints hash-chain so history cannot be silently rewritten, and any record's membership is provable with a log-sized proof.
* **Refresh-job + scheduler** (`src/trust/refresh-job.ts`, `scheduler.ts`, `mount.ts`) — a 6-hour, **read-only** freshness pass. By law it re-issues only `idempotency === "safe"` endpoints over GET/HEAD, never a mutation. Opt-in via `UNBROWSE_TRUST_REFRESH=1`.

## 4. Proof layer — commitments without disclosure

`src/proof/` lets Unbrowse prove *what happened* without persisting secrets:

* **Commitment** (`src/proof/commitment.ts`) — `createCommitment` binds a captured request to `sha256` of its response body plus non-sensitive metadata (domain, url\_template, method, status, captured\_at). It never includes auth headers, cookies, or PII. `verifyCommitmentAgainstResponse` re-checks later.
* **Input-censor** (`src/proof/input-censor.ts`) — before any request shape is persisted or published, sensitive leaf values are replaced with `sha256:<hex>` commitments (`censorInputBody`). The reusable route shape survives; the secret never crosses the persistence/publish boundary. See [PRIVACY.md](/architecture/privacy).
* **Notary** (`src/proof/notary.ts`) — a TLS-transcript notarization client. The shipped path is the commitment-only proof above; richer transcript notarization is gated behind `UNBROWSE_NOTARY_URL` and is forthcoming.

## 5. Payment-security gate (client side)

`src/payments/x402-fetch.ts` is a drop-in `fetch` wrapper that intercepts HTTP 402 (and proxy 407), parses the signed payment terms, **enforces a cost ceiling** (`UNBROWSE_X402_MAX_COST_USD`, default $1.00), signs via the resolved wallet adapter, and retries once. EVM (Base) signing via EIP-3009 lives in `src/payments/base-x402-signer.ts`. Failure is honest: with no wallet it surfaces the 402 unchanged with sub-state `x402_no_wallet` — never a fake success. The server-side gate and settlement are in `backend/src/middleware/x402-gate.ts` and `backend/src/services/flex.ts`. See [AUTH.md](/architecture/auth) for wallet resolution and [../HOW\_UNBROWSE\_PAYS.md](/research/how_unbrowse_pays) for the money model.

## 6. Site policy & rate limiting

* **Site policy** (`src/site-policy.ts`) — detects session-bound parameters that cannot be safely replayed (`detectSessionBoundParams`) and flags mutating endpoints that require third-party-terms confirmation (`getEndpointPolicy`). Policies apply only to non-safe (mutating) methods.
* **Rate limiting** (`src/ratelimit/index.ts`) — `ROUTE_LIMITS` defines per-route ceilings (resolve, execute, publish, login, feedback). Disabled in the single-user local runtime; the config is the source of truth for the server tier.

## What "secure" means here, in one line

A tampered client can't authenticate, so it can't reach the value; secrets are committed not stored; the graph is hash-chained and slashable; and every paid action is ceiling-checked client-side and settled server-side.

## See also

* Honest threat model → [../SECURITY.md](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/SECURITY.md)
* Data handling & secrets → [PRIVACY.md](/architecture/privacy)
* Identity, keys, wallets → [AUTH.md](/architecture/auth)
* Public transparency primitives → [../public/primitives/README.md](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/public/primitives/README.md)


# Privacy & Data Handling

> **At a glance** — "Credentials never leave the machine" is a *construction*, not a promise. The client strips every secret value locally before anything crosses the network, replaces it with a one-way commitment, and a separate audit pass refuses to send if any known secret survived. The server sees request *structure* (method, URL shape, param keys, schema), never values.

> Reviewed 2026-06-17 against build v9.4.12. Companion to [SECURITY.md](/architecture/security) and the public primitive [../public/primitives/05-user-response-never-contains.md](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/public/primitives/05-user-response-never-contains.md).

## The boundary, stated precisely

There are three boundaries a secret could cross, and the design closes each:

1. **The network boundary** (client → unbrowse server). Closed by obfuscation + an audit gate (§1, §2).
2. **The persistence/publish boundary** (local disk, shared marketplace). Closed by input-censoring to commitments (§4).
3. **The at-rest boundary** (local vault on disk). Closed by encryption, with an optional wallet-seal so even a stolen vault file is unreadable (§5).

## 1. Thin client: only structure crosses the wire

The reverse-engineering / route-inference engine runs **server-side only**. The client does not carry it. What the client sends to `POST /v1/reveng` is *structure*, obfuscated first:

* `src/capture/reveng-server-first.ts` — `revengServerFirst()` obfuscates the capture (`obfuscateCaptureForReveng`) **before** the POST. If the server is unreachable (offline, no key, non-2xx) it returns an empty endpoint list — there is deliberately **no local inference fallback**, so raw traffic can never be a fallback's input. `revengEgressPayload()` exposes the exact bytes on the wire for audit testing.
* `src/capture/backend-reveng-endpoint.ts` — the client-side wiring to the server engine.

The result: the server sees method / URL shape / param keys / response schema — the inference IP stays server-side, the secrets stay client-side.

## 2. Secret/PII obfuscation + the audit gate

`src/capture/obfuscate.ts` redacts in two layers, then `obfuscate-audit.ts` verifies the redaction worked:

* **Heuristic redaction** — sensitive field names (token, secret, credential, auth, cookie, sid, …) and sensitive headers (Authorization, Cookie, X-CSRF-Token, X-API-Key, …) are always redacted; values that *look* like secrets are caught by shape.
* **Known-secret scrub** — the caller passes the local vault secrets (`opts.secrets`); `scrubKnownSecrets` does an exact-match sweep (longest-first) so no vault value slips through a heuristic gap.
* **Audit gate** (`obfuscateAuditedCapture` in `obfuscate-audit.ts`) — scans the *outgoing* payload against the vault. If even one secret survives, it throws `ObfuscationLeakError` and the send is refused. This is the open-source belt-and-suspenders: the engine redacts; the audit verifies it against the known vault secrets (it cannot detect a secret the vault has never seen — the heuristic layer is the only guard there).

## 3. Wallet-bound commitments

When a wallet public key is available, a redacted secret is replaced with a deterministic, one-way commitment instead of a bare `[REDACTED]` (`src/capture/wallet-bind.ts`): `bindSecretToWallet` returns `sha256(walletPubkey ‖ domain-separator ‖ secret)`, embedded as a short `bound:<hex>` tag. Properties:

* **One-way** — the tag is a digest; the value is not recoverable from it.
* **Wallet-scoped** — the same secret under a different wallet yields a different tag, so commitments are not correlatable across owners.
* **Holder-verifiable** — only the holder, who has the local secret, can re-derive and verify the tag.

Stated honestly: a simple commitment is computationally hiding for high-entropy secrets (tokens, session IDs, keys). A *low-entropy* secret (e.g. a 4-digit PIN) is brute-forceable from its commitment — a documented limitation of the shipped commitment scheme (`src/capture/wallet-bind.ts`).

## 4. Censoring at the persistence/publish boundary

The live request still sends the real value to the target, but any **persisted or published** copy (local skill cache, shared marketplace manifest) carries a commitment, never the cleartext:

* `src/proof/input-censor.ts` — `censorInputBody` deep-walks a request body, detects sensitive leaves by field name and by vault-pointer form (`op://`, `keychain://`, …), and replaces each with `sha256:<hex>`. `censorSkillForPersistence` applies this to skill manifests, censoring only WRITE-endpoint bodies (GET/HEAD carry no sensitive input).
* `src/capture/bundle-scanner.ts` — when mining routes out of JS bundles, the scanner extracts endpoint shapes but **skips sensitive query params** (api\_key, access\_token, secret, password, session\_id, …), so the harvested skeleton has no secret params.

## 5. Local storage & at-rest protection

| Store                 | Path                                | Protection                                                                                                                                                                                                                                                                                                                                                      |
| --------------------- | ----------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Credential vault      | `~/.unbrowse/vault/credentials.enc` | AES-256 encrypted under a local key (`.key`, mode 0o600). With `UNBROWSE_WALLET_SECRET` set, each value is further sealed (AES-256-GCM under a wallet-derived key) and the plaintext is removed — only the wallet holder can open it. macOS keychain is tried first with a safe fallback to the file vault. (`src/vault/index.ts`, `src/vault/wallet-vault.ts`) |
| Sealed fills          | in-memory / sealed blob             | `src/capture/sealed-fill.ts` seals fill values to the wallet and reveals them **locally** at execute time; the filled concrete request is built locally and never sent plaintext to the server.                                                                                                                                                                 |
| Config                | `~/.unbrowse/config.json`           | mode 0o600; settings only, no secrets expected. (`src/client/index.ts`)                                                                                                                                                                                                                                                                                         |
| Session logs / traces | `~/.unbrowse/traces/`               | Metadata only — `src/telemetry.ts` never stores raw cookies, tokens, or bodies; URLs are stripped of query/fragment (`anonymizeUrl`), bodies are hashed (`hashResponseBody`), binding *names* are kept but sensitive keys excluded (`safeBindingNames`). Opt-out: `UNBROWSE_DISABLE_TRACES=1`.                                                                  |

## Does the guarantee hold? — honest verdict

**At the network boundary: yes, by construction.** Plaintext secrets are stripped before the POST; the audit gate refuses any send where a known secret survived; the server has no path to a raw value.

**Caveats, stated plainly:**

* Low-entropy secrets are brute-forceable from a simple commitment (§3).
* The route-inference engine runs server-side; the *client inputs* remain obfuscated, but a compromised server could mis-handle inferred structure.
* A compromised local machine can read the vault key unless `UNBROWSE_WALLET_SECRET` is set (then a stolen vault file is unopenable without the wallet).

## See also

* Threat model & anti-tamper → [SECURITY.md](/architecture/security)
* What a user response may never contain → [../public/primitives/05-user-response-never-contains.md](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/public/primitives/05-user-response-never-contains.md)
* Verification & proofs (concept) → [../concepts/verification-and-proofs.md](/concepts/verification-and-proofs)


# Identity, Auth & Wallets

> **At a glance (web3-native, 2026-06-25 rip)** — One identity (the local self-custody **ed25519 wallet pubkey**) fronts every backend call. The client signs a fresh domain-separated challenge and the backend verifies it as the SOLE REQUIRED credential, authenticating the caller as `wallet:<pk>` BEFORE any bearer path. A legacy `ubr_` api-key, if present, is an OPTIONAL **web2 wrapper** layered over the wallet for account-bound continuity (payouts accrual, dashboard sync); a wallet-only caller is a full principal. Unbrowse is a **child environment executor** under the **contract (aiko) parent platform**, which signs the on-chain parent root; the wallet is the only identity the runtime ever persists.

> Reviewed 2026-06-25 against build v9.4.12+web3-rip. Companion to [SECURITY.md](/architecture/security), [PRIVACY.md](/architecture/privacy), and the money model in [../HOW\_UNBROWSE\_PAYS.md](/research/how_unbrowse_pays).

## 1. Identity & the wallet signature

* **Identity root = the wallet pubkey (ed25519)**, persisted at `~/.unbrowse/wallet.json` (mode 0o600) + OS keychain seal. Every install creates one on first run (`src/values/signer.ts` → `ensureLocalWalletAddress()`); there is no email/password gate to identity.
* **The auth credential** is a capability signature minted per-request by `src/lib/wallet-auth-headers.ts` (`mergedAuthHeaders()`). The signature is over `AUTH_DOMAIN ":" pubkeyHex ":" ts` (60s TTL, `AUTH_DOMAIN = "unbrowse-auth:v1"`), sent as the three headers `X-Unbrowse-Wallet`, `X-Unbrowse-Auth-Ts`, `X-Unbrowse-Signature`. The backend (`backend/src/services/auth-signature.ts` → `authBySignature`) re-derives the challenge, verifies the ed25519 sig against the pubkey, and resolves to the agent\_id `wallet:<pk>` (or its bound account, if any).
* **The wallet sig IS the principal — never key-gated.** Verified by `backend/test/wallet-principal-never-keygated.test.ts`: a wallet-only caller (no api-key bound) authenticates and reads `/v1/agents/wallet`, `/v1/agents/accept-tos`, `/v1/agents/me`, `/v1/account/me`, `/v1/account/credits` regardless of key state. The three auth middlewares (`bearerAuth`, `optionalAuth`, `bearerAuthNoTos`) all verify the sig FIRST and short-circuit to `wallet:<pk>` before any Bearer 401 path.
* **`ubr_` api-key = DEPRECATED web2 wrapper**, layered over the wallet when present (`Authorization: Bearer ubr_…`). Used only to bind account-bound flows (payouts to the linked email, dashboard sync, ToS surface). The wrapper will be retired; new code MUST NOT gate on it. Client-side `getApiKey()` (`src/client/index.ts`) is now opt-in: `ensureUsableKey()` returns `{key: ""}` on the resolve hot path when a wallet is present, and only attempts a key-mint when `opts.allowMint` is set.
* **Parent/child platform model.** The wallet at `~/.unbrowse/wallet.json` is unbrowse's identity. The contract platform (aiko) is the **parent** — its deployer keypair at `~/.aiko/keys/deployer.key` signs the parent root on-chain, and unbrowse is its **child environment executor** with access only to web primitives. The bridge lives in `src/bridges/contract-mcp-bridge.ts` + `src/lib/contract-thin-client.ts` (HTTP thin client over `/v1/contract/*`); the child never oversteps into the parent's signing scope.
* **Client storage** (`src/client/index.ts`): `~/.unbrowse/config.json` (mode 0o600) carries `agent_id`, `email`, `user_id`, `wallet_address`, `wallet_provider`, ToS acceptance (`UnbrowseConfig`). `getApiKey()` reads `UNBROWSE_API_KEY` first then config; `validateApiKey()` HEADs `/v1/agents/me` (note: when wallet-only, the route accepts the sig directly — the key is not required to be present).

## 2. Pre-resolve auth gate (don't spend effort you'll lose)

`src/auth/pre-resolve-gate.ts` blocks resolve *before* the costly routing race when **all three** hold:

1. the intent is personal/auth-shaped (a personal pronoun, or a keyword like `login` / `account` / `auth` / `credentials`), **and**
2. the host is in `AUTH_GATED_HOSTS` (a fixed list of known login-walled hosts), **and**
3. there is no fresh local cookie for that host (`scripts/check_cookie_freshness.py`, lock-safe).

If the cookie DB is locked or errors, it passes (uncertain → attempt). The decision returns `gate: "auth_required"` with the host and reason, so the agent can prompt for sign-in instead of failing mid-route.

## 3. Runtime auth state & the post-execute feedback loop

* **Runtime** (`src/auth/runtime.ts`) — the in-process `LocalAuthRuntime` (`authRuntime`) resolves auth in order: cached session (memory TTL) → vault cookies → browser extraction fallback. `UNBROWSE_DISABLE_AUTH_FALLBACK=1` forces "unauthenticated" for tests. Cookie extraction supports Chrome / Firefox / Brave / Arc / Edge (`src/auth/browser-cookies.ts`); history is surfaced as eTLD+1 domains only, redacted (`src/auth/browser-history.ts`).
* **Stale endpoints** (`src/auth/stale-endpoints.ts`) — the post-execute feedback loop. A 401/403 marks `(domain, endpoint_id, status, cookie_source, reason)` stale for 30 min in `~/.unbrowse/stale-endpoints.json`; `isEndpointStale` then keeps resolve from returning that endpoint, and `buildAuthHint` surfaces the login URL + refresh surfaces (keychain → local browser → agent browser). `markCookieExpiry` pre-marks endpoints whose cookies have already expired, before an execute is even attempted.

## 4. Auth-bearing execution & token resolution

* **Auth-bearing classifier** (`src/execution/auth-bearing.ts`) — a pure, I/O-free predicate (`isAuthBearing`) that returns true if a request carries a credential a terminating server tier could read in the clear (any non-benign header, an `Authorization`-scheme value, or a locally-dereferenced sealed/storage-bound fill). The egress router uses it to keep credentialed requests **off** the server proxy tier. See [PERFORMANCE.md](/architecture/performance#3-egress-tiering).
* **Token resolver** (`src/execution/token-resolver.ts`) — resolves an endpoint's `auth_tokens` bindings at execute time: immediate cookie lookup (no network) → plain HTTP fetch (8s) extracting from HTML/meta/inline-script → Kuri browser fallback (12s) only when a binding is HTML-resolvable. Adds the `Bearer` prefix when needed.

## 5. Verification of auth state

`src/verification/` decides what can be auto-verified:

* `auth-gate.ts` — `isAuthGatedEndpoint` returns true if the skill has an `auth_profile_ref` or the endpoint declares `auth_required`; such endpoints are excluded from the periodic (6h) auto-verification and only verified manually.
* `candidates.ts` — `selectVerificationCandidates` picks GET-only endpoints (never mutations), optionally only the stale ones (disabled, failed, low reliability, or not verified in 24h).
* `matrix.ts` / `index.ts` — integration-coverage matrix and the `verifyEndpoint` / `verifySkill` / `schedulePeriodicVerification` orchestration.

## 6. Wallet resolution order (the wallet is the identity; wrappers layer on top)

There are two distinct resolutions — **which wallet address** the agent has, and **which signer adapter** pays a 402. Keep them separate.

**Wallet address** — `src/payments/wallet.ts` (`getWalletContext()`), first match wins:

1. **OWS (Open Wallet Standard)** — env `OWS_WALLET_ADDRESS` or the `~/.ows` vault, with a declarative policy engine (`src/payments/ows.ts`); the vault probe is gated by `UNBROWSE_DISABLE_LOCAL_WALLET=1`.
2. **lobster.cash (env)** — `LOBSTER_WALLET_ADDRESS`.
3. **Generic env wallet** — `AGENT_WALLET_ADDRESS` (+ optional `AGENT_WALLET_PROVIDER`).
4. **lobster.cash (local config)** — `~/.lobster/config.json` (gated by the same flag).
5. **Unbrowse-local native wallet** — `~/.unbrowse/wallet.json` + OS keychain (gated). Every install gets a real self-custody wallet with zero setup. **This entry is the IDENTITY ROOT** — when this is the resolved address, the same keypair signs every auth capability via `mergedAuthHeaders()`. The wallet is the principal; the key (if bound) is the wrapper.
6. **None** → sponsored free tier, or an honest `x402_no_wallet` failure.

**Signer adapter** — at payment time `src/payments/x402-fetch.ts` (`resolveWalletConfig`) picks how to sign: explicit `UNBROWSE_WALLET_ADAPTER` → `~/.lobster` ⇒ lobster → `~/.privy` ⇒ **privy** → `UNBROWSE_WALLET_KEY` ⇒ generic → none (pay.sh is explicit-only). Note Privy is an *adapter* here, not a `getWalletContext` address source. The adapter enforces the cost ceiling, signs the x402 envelope, and retries. Credentials are sealed to the wallet in `src/vault/wallet-vault.ts` (`sealToWallet` / `open`); `commitmentOf` exposes a host-independent commitment that reveals nothing about the secret.

**Funding binds the wallet to a key wrapper.** A `ubr_` api-key, when present, is bound to a funding source (external wallet address or prepaid credit budget) via `POST /v1/account/keys/:keyId/funding` (backend `account.ts`) — this is the optional account-bind that carries payouts accrual. Contributors who published before attaching a wallet are paid retroactively when the binding appears (backend `splits.ts`). Client-side, `src/cli-wallet.ts` reads and reconciles the local wallet vs the server-bound wallet for the `unbrowse wallet` command. **The wallet always remains the identity; the key is only a funded wrapper that the agent may or may not have.**

## 7. Parent/child platform (aiko → unbrowse)

The wallet at `~/.unbrowse/wallet.json` is unbrowse's identity, but unbrowse itself is a **child environment executor** under the **contract (aiko) parent platform**:

* **Parent (aiko/contract):** the on-chain truth-root. Deployer keypair lives at `~/.aiko/keys/deployer.key`; the contract binary's signed attestations are the witness ledger rows the rest of the system dereferences. The wallet signs FOR the Word declared on the parent — the wallet is rotatable, the Word is not.
* **Child (unbrowse):** the environment executor with access only to **web primitives** (fetch, scrape, browse, sign web challenges, x402 pay). It never oversteps into the parent's signing scope — it never signs a parent truth claim, only its own web-auth capability.
* **Bridge:** `src/bridges/contract-mcp-bridge.ts` + the HTTP thin client at `src/lib/contract-thin-client.ts` (`/v1/contract/*` server route) expose the parent's declarations to the child as MCP tool surface. The child reads parent-signed ledger rows as pointers (never inlines the payload), dereferences through the platform, and acts on the resolved truth.

## One-line model

The local ed25519 wallet is the single identity. It signs every request as a fresh capability, the backend verifies it as `wallet:<pk>` BEFORE any bearer path, and a `ubr_` key (if present) is a deprecated funded wrapper layered for account-bound continuity — under a contract (aiko) parent platform where the wallet signs FOR the Word, never in its place. "Who you are" (wallet pubkey) and "who pays" (the same wallet, optionally key-funded) are the same handle.

## See also

* Anti-tamper, anti-bot, trust graph → [SECURITY.md](/architecture/security)
* Secrets & data handling → [PRIVACY.md](/architecture/privacy)
* Wallets & payments (agent view) → [../for-agents/wallets-and-payments.md](/for-agents/wallets-and-payments)
* Money model → [../HOW\_UNBROWSE\_PAYS.md](/research/how_unbrowse_pays)


# Performance & Speed

> **At a glance** — Unbrowse is fast because it avoids the browser tax: it replays a learned request path instead of re-driving Chrome, serves repeats from a correctness-guaranteed cache, and escalates egress only as far as a block forces it. The headline speedups are peer-reviewed or witnessed by a reproducible bench; this doc says which is which.

> Reviewed 2026-06-17 against build v9.4.12. Companion to [../benchmarks.md](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/benchmarks.md), [../caching.md](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/caching.md), and [OVERVIEW.md](/architecture/overview).

## 1. The speed thesis: replay over re-drive

The resolve → execute pipeline picks the **cheapest capable layer** for an intent rather than always opening a browser:

| Layer                      | Typical cost       | When                                                     |
| -------------------------- | ------------------ | -------------------------------------------------------- |
| Route-cache (local)        | near-zero, instant | a previously captured endpoint with a still-valid recipe |
| Marketplace (shared graph) | low, server-side   | a route someone else already indexed                     |
| Live capture (browser tax) | high, seconds      | nothing indexed yet, or the recipe went stale            |

Intent → route binding is `src/intent-match.ts` (form detection + API-type inference); the execute pipeline is `src/execution/index.ts`. When a route is missing or stale, execution escalates through SSR fast-path → curl-impersonate → stealth browser → paid unblocker → full browser, stopping at the first rung that works (§3, §4). The whole decision is instrumented (§5).

## 2. Caching: correctness-guaranteed, pointer-reactive

Unbrowse's cache never silently serves stale data — freshness is **dependency driven**, not a guessed TTL (`docs/caching.md`, `src/values/pointer-cache.ts`):

* **Content addressing** (`src/values/content-address.ts`) — pointers are `sha256:<hex>` of bytes; a genesis hash anchors the chain. `valueSetPointer` gives an order-independent pointer for a set of resolved values; `intentKey` scopes an intent pointer.
* **Pointer-reactive invalidation** — each cache entry pins the addresses of its dependencies. A read is a HIT only if **every** dependency's current address still equals the pinned one; when any dependency's value changes its address changes, so dependents recompute automatically. Reads are O(1) (`recomputeCount` is observable).
* **Wallet-sealed entries** (`src/trust/sealed-cache.ts`) — sensitive cache values are encrypted to the wallet; they still respect pointer dependencies.

Short-lived operational TTLs that *are* time-based: residential sticky sessions \~25 min (under the proxy's own lifetime), x402 proxy-authorization \~5 min.

## 3. Egress tiering: cheapest IP first, escalate honestly

`src/execution/egress-chain.ts` walks a three-rung ladder and returns the best outcome; it never leaks credentials to a tier that could read them:

1. **LOCAL** — direct `fetch` from the client's own IP. Returned immediately unless the status is a block (0/401/403/429/5xx). Skipped for auth-bearing requests (see [AUTH.md](/architecture/auth#4-auth-bearing-execution--token-resolution)).
2. **SERVER clean IP** — `POST /v1/proxy` (`src/execution/server-proxy-fallback.ts`); the server tries its own datacenter IP first and escalates only if blocked. Also skipped for auth-bearing requests (the server tier terminates TLS).
3. **CLIENT residential proxy** — last resort (`src/execution/proxy-fetch.ts`); residential egress with a sticky session for IP-bound clearance, or a paid x402 unblocker chain.

`isBlock(status)` classifies blocks; `egressFetchWithBlockCheck` catches soft-blocks (a 2xx body that is actually an error/challenge page); the `authExcluded` flag is the honest "stayed local, never leaked the credential" result.

## 4. Fast paths

| Path                      | File                                       | What it saves                                                                                                                                                                   |
| ------------------------- | ------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| SSR fast-path             | `src/capture/ssr-fastpath.ts`              | On a bot-block, fetches the page via libcurl-impersonate (TLS fingerprint spoof) inside the Kuri sandbox — no browser spin-up. Returns null on non-2xx / tiny HTML (non-fatal). |
| Graph prefetch            | `src/capture/prefetch.ts`                  | Traverses parent→child operation edges and runs up to 3 satisfiable GET endpoints in parallel (2s timeout) so an agent gets list + detail in one round-trip.                    |
| Fetch ladder              | `src/capture/fetch-ladder.ts`              | Ordered anti-bot escalation: curl-impersonate direct (12s) → curl-impersonate via proxy (45s); advances only on a detected block phrase; refuses to cache error pages.          |
| curl-impersonate fallback | `src/capture/curl-impersonate-fallback.ts` | JA3/JA4 TLS spoof helper, stealth-browser fallback for JS challenges, and the x402 paid-unblocker chain with per-provider negative caching.                                     |
| Recipe replay hints       | `src/execution/recipe-replay-hints.ts`     | Reuses a captured recipe's known-good request shape to skip rediscovery.                                                                                                        |

## 5. Telemetry that measures the path

* `src/routing-telemetry.ts` — per-step routing events (`routing_session_started/_candidates_ranked/_step_executed/_completed`) with `execution_latency_ms`, candidate/binding counts, `source` (route-cache / marketplace / live-capture / dom-fallback / …), and a classified `failure_reason`. State is captured as `state_hash_before/after`, results as `response_hash` — never raw bodies.
* `src/telemetry.ts` — anonymized `RouteTraceArtifact`s under `~/.unbrowse/traces/` (see [PRIVACY.md](/architecture/privacy#5-local-storage--at-rest-protection)). Opt-out `UNBROWSE_DISABLE_TRACES=1`.

## 6. The numbers — substantiated vs marketing

**Substantiated (cite these):**

| Claim                                                                                         | Source                                                                  | Status                 |
| --------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------- | ---------------------- |
| **3.6× mean / 5.4× median** speedup over a browser across 94 live domains; \~40× fewer tokens | peer-reviewed paper *Internal APIs Are All You Need* (arXiv:2604.00694) | externally validated   |
| **\~30× faster, \~90× cheaper** than driving a browser                                        | same paper                                                              | externally validated   |
| **21.1s cold → 4.1s warm** (≈80% faster) on a fixed probe set as the route cache fills        | `docs/benchmarks.md`                                                    | reproducible witness   |
| Anti-bot: **9/9 vs naive 0/9** on a JS-challenge-gated platform                               | `docs/benchmarks.md`                                                    | ground-truth validated |

**Marketing simplification — do not cite as measured:** "sub-200ms cache hit" has no direct latency witness in code or bench. The cache *read* is O(1), but the network round-trip for the actual call dominates end-to-end; the substantiated figure is the cold→warm probe-set result above. Prefer the witnessed numbers.

## One-line model

Speed = replay instead of re-drive + a cache that recomputes only when a real dependency changed + egress that escalates no further than a block forces.

## See also

* Benchmark methodology & history → [../benchmarks.md](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/benchmarks.md), [../benchmarks-history.md](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/benchmarks-history.md)
* Caching design → [../caching.md](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/caching.md)
* Egress & auth interaction → [AUTH.md](/architecture/auth)


# Shadow APIs

Every website an agent visits is already an API; the browser is just the client.

When a page loads or a button is clicked, the browser issues structured requests to the site's own backend and renders the responses. Those requests are the shadow API: a real, working interface that exists whether or not the site documents one. A browser-first agent ignores this layer and re-derives the same outcome through the visible page every time.

The published paper's argument is that internal APIs are all you need: most agent web tasks reduce to a request the browser was already going to make, so an agent that learns that request can skip the interface. This reframes web automation from "control a browser" to "discover and reuse the request behind the browser."

Shadow APIs are per-site and undocumented, which is why they have to be discovered from real use rather than looked up. How discovered routes are stored and kept useful is the subject of the next pages. The deeper extraction mechanics are described at the paper's level of abstraction and not below it.


# The Route Graph as a Productive Asset

A route graph is not a static registry; it is a productive asset with upkeep costs.

The Maintenance Network paper makes this distinction directly: a registry is written once and assumed correct, while a route graph that carries real agent traffic produces ongoing value and therefore incurs ongoing cost. Routes fail, authentication flows change, schemas drift, and previously valid execution plans decay. The graph is worth something precisely because it is kept current, not because it was once complete.

This framing matters because it explains why discovery alone is insufficient. A graph that only accumulates routes and never maintains them degrades toward noise as the web changes underneath it. The value is in the freshness, not the count.

Treating the graph as an asset with upkeep is what makes the maintenance problem, and the accountability question that follows it, unavoidable rather than optional.


# Trust and Accountability

In a shared route graph, trust is a practical signal about whether a route still does what it claims, not a cryptographic guarantee.

The shipped model is reliability-oriented: routes carry success and failure behaviour, freshness, verification state, and folded-in feedback, and that composite signal moves good routes up and bad routes out of future shortlists. This is a continuous trust model in the sense the paper uses: quality is observed from real outcomes over time rather than asserted once at publish.

A maintained graph then asks how it should express trust and enforce accountability once it carries meaningful traffic. The answer is deliberately narrow: higher-trust route tiers, accountable maintainers, and challenge mechanisms are the coordination tools. This is a quieter accountability layer, not a redesign of the product.

What does not exist today, and is described as forward-looking rather than shipped, is a full validator market or cryptographic attestation. The honest reading: practical reliability and verification ship now; the richer accountability layer is research direction, not current behaviour. Throughout, discovery stays free — an agent only pays when it executes a paid route, settled over x402.

## Two currencies, two jobs — settlement vs trust

The agentic web needs two distinct units, and conflating them is the bug. Usage is **settled in USDC** — a paid `execute` clears in stable value over x402, so an agent (or the human behind it) pays a predictable price and never has to hold or understand a volatile asset. That is the only currency a caller ever touches.

The second unit is not for paying — it is for **trust**. The native token (FDRY) is the agentic web's *accountability currency*: the stake a maintainer or indexer bonds to stand behind a route. You do not spend it to use the network; you bond it to be *trusted by* the network. Keeping the two separate is deliberate — if the trust asset were also the payment rail, every use would be a forced sale, draining the very stake that is supposed to signal commitment. Settlement is stable; trust is staked; they do not mix. (This is the same separation the [contract platform](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/concepts/contract-platform.md#the-economy-rides-the-same-platform) keeps at the protocol layer.)

## How /contract and stFDRY carry it

A maintained route is not a free externality — under the [contract platform](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/concepts/contract-platform.md) it is a **signed, bondable asset**. The forward-looking accountability layer makes that concrete:

* **/contract** turns each route into a typed, wallet-signed claim with a verifiable freshness proof. A maintainer who bonds behind a route is making a checkable promise: *this still resolves, and its shape still matches.* A proof that fails to reproduce is challengeable, and a dishonest claim is slashable — so the bond is what gives a trust tier teeth.
* **stFDRY** is the *staking* form of the stake — the staked receipt held by those who stay committed through the cooldown rather than trading in and out. The design rewards **holding, not spending**: an staking staker earns a larger maintenance reward and a higher trust weight, weighted by *commitment* (how much of one's position is staked) rather than by raw size, so a proportionally-committed small staker can outweigh a nominally-committed whale. The reward is *earn-by-staking*, never a discount on usage — usage stays USDC, and the value flows back to committed stakers through the network's growth, not by making anyone pay in the trust asset.

So FDRY is "the currency of the agentic web" in exactly one sense: it is the unit of **trust and accountability** that makes a maintained, agent-usable web economically honest. It is bonded to be trusted, abided in (stFDRY) to earn, and never the thing an agent spends to act. This layer is **forward-looking** — practical reliability and USDC settlement ship today; the bonded proof-of-indexing accountability economy is the research direction described in the maintenance-network paper, not current behaviour.

Read [Where This Goes](/vision) for how this accountability layer sequences behind the wedge.


# How Quality Is Evaluated

Unbrowse is judged by whether an agent actually got the data the intent asked for, not by whether a call returned a 200.

The canonical check is an agent-experience harness: it runs real intents end to end, collects the artifacts (what was returned, from where, how long it took), and an agent judges whether the result satisfied the intent. There is no regex pass or fail, because a page can return HTTP 200 and still be a captcha, an empty array, or the wrong shape. Evidence is collected; judgement is made against the intent.

The paper separates two performance regimes, and it is worth keeping them distinct when reading any claim:

* **Warmed-cache** performance: the route already exists in the graph, so the cost is a lookup and a call.
* **Cold-start** performance: the route has to be learned live first, which is the expensive path the wedge is designed to amortise.

A claim about speed or cost only means something once you know which regime it describes. Aggregate cost and network-growth analysis live in the published paper; specific internal numbers move and are not pinned here. Current live metrics are exposed at the public stats endpoint rather than baked into this page, because numbers in prose go stale.


# Verification and Proofs

Unbrowse is deliberately honest about what its proofs do and do not establish today. The short version: proofs today are local commitments, not third-party-verifiable attestations, and the system refuses to claim otherwise.

## What ships today

Endpoint descriptors can carry proof metadata. The shipped proof type is a commitment: a SHA-256 commitment to a captured response body, signed locally by the publisher. The backend validates proof shape and rejects malformed proof objects.

Trust is exposed as a four-state channel, not a binary badge:

* **proven** reserved, never set today, it waits for real provenance
* **client commitment** a local commitment exists (shown as a cautious state, not a green check)
* **unverified proof** a proof is present but could not be verified
* **no proof** nothing attached

The marketplace's verified count deliberately excludes local commitments, so the strong signal stays dark until something stronger than self-attestation ships. A proof-required filter returns empty today, by design. This is the system telling the truth rather than lighting a badge it has not earned.

## What is not shipped, and not overclaimed

Third-party notarised proofs (MPC notary handshakes, WASM verifiers), selective disclosure, and full cryptographic route-verification are research direction, not current behaviour. Stronger provenance is part of an ongoing security and verification effort; specifics will be detailed in a forthcoming whitepaper. It is described here as direction, and the docs will not claim it as shipped until it is.

## The security work behind this

The proof-metadata model, the four-state trust channel, the trust-channel boundary for downstream consumers, and the discipline of not lighting a Verified badge until provenance is real are part of security and verification work led by Goh Ee Sheng. The conservative posture is the point: a trust system that overstates itself is worse than one that is honest about its current limits.

For how this connects to economic accountability, see [Trust and Accountability](/concepts/trust-and-accountability).


# Fare Splits

How a paid call to an Unbrowse skill divides into three on-chain shares.

## The three lanes

When an agent calls a paid skill via `unbrowse execute`, the platform's facilitator computes the split across up to three recipients. The math lives in `backend/src/services/flex.ts:computeFlexSplits`.

> **Settlement: atomic on devnet, custodial on mainnet today.** The trustless on-chain atomic split (one Faremeter Flex transaction across all three recipients) is live on **devnet only** — the split program is not yet deployed to Solana mainnet. On **mainnet today the platform settles these splits custodially**: it collects the payment and disburses each contributor's and owner's earned cut from the attribution ledger (`backend/src/services/disburse.ts`, default dry-run). The lane math below is identical either way; only the settlement venue differs.

| Lane                                 | Share                                   | When it fires                                                                                                                                                    |
| ------------------------------------ | --------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Platform** (`PLATFORM_BPS = 5000`) | 50%                                     | Always                                                                                                                                                           |
| **Site owner** (`OWNER_BPS = 1500`)  | 15%                                     | Only when the skill carries `owner_compensation_opt_in === true` and a verified `owner_wallet_usdc_ata` (see [Claiming a Website](/concepts/claiming-a-website)) |
| **Contributors** (the remainder)     | 35% when owner is active; 50% otherwise | When the skill has a non-empty `contributors[]` with at least one `wallet_address`                                                                               |

The three lanes always sum to exactly 10000 basis points (100%). The Flex on-chain program rejects authorizations that do not sum to 10000 (`FLEX_ERROR__SPLIT_SUM_INVALID`), so the computation is checked at compile-time-of-trust in the tests at `backend/tests/flex-owner-bps.test.ts` and `backend/tests/flex-owner-bps-edges.test.ts`.

## Why three lanes instead of two

The pre-claim shape was 50/50 (platform / contributor pool). When a site owner publishes their canonical API or proves ownership via DNS-TXT, they have done work that did not previously exist: surfacing a stable contract for agents to consume. The 15% lane lets that work get paid while preserving a 35% indexer share, the largest single slice for the people doing the discovery and maintenance work.

If the domain is never claimed, indexers absorb the would-be owner share automatically (the contributor pool grows to 50%). No domain has to pay for a missing owner; the math degrades gracefully.

## Contributor sub-splits

Within the contributor pool, individual recipients are weighted by `cumulative_delta` on each `SkillContributor` record:

```
contributor_share_i = max(cumulative_delta_i, 0.01) / sum(max(cumulative_delta_j, 0.01))
                    * contributor_pool_bps
```

The top `FLEX_MAX_SPLITS - 1` (or `- 2` when the owner lane is active) by `cumulative_delta` get a slot; the rest are dropped from this settlement (they accrue weight for future calls). The on-chain Flex program caps at 5 split recipients per authorization, so the caller of `unbrowse setup` cannot be the 99th contributor and silently lose all attribution.

## What happens when there are no eligible contributors

* **No payable contributors AND no active owner:** `computeFlexSplits` returns an empty array. The caller (`backend/src/services/flex-payment-terms.ts:99`) falls back to a single-recipient transfer to the platform. The price still settles; no contributor earnings, no owner earnings.
* **No payable contributors but owner IS active:** the helper folds the unallocated contributor pool back into the platform recipient so the on-chain authorization still sums to exactly 10000 bps. The owner still gets their 1500 bps.

These edge cases are explicitly pinned in `backend/tests/flex-owner-bps-edges.test.ts` so any future re-tune cannot silently leak basis points.

## When the lanes change

The constants `PLATFORM_BPS` and `OWNER_BPS` are policy. They live in `backend/src/services/flex.ts:39-49` and are the only source of truth for the on-chain settlement math. Doc, dashboard, and frontend copy should never hard-code numbers that disagree with these constants; if a re-tune lands, the docs move with the constants.

The current numbers (50/15/35 when claimed, 50/50 when not) are the live constants as of this writing. They have shipped on origin/main and any earlier prose that called 50/20/30 the live value is stale. If you see a number in production that disagrees with this doc, the constants are still the truth, not the prose.

## Settlement rail and timing

* **Rail:** x402 over Solana, USDC, using `@faremeter/flex`.
* **Cadence:** each paid execute signs a Flex authorization off-chain; the platform's facilitator holds the authorization in memory, then submits a batched on-chain settlement after the refund window closes. Distribution is atomic per the signed splits.
* **Visibility:** `unbrowse stats --earnings` (CLI), or `GET /v1/stats/indexer/:id/ledger`, `GET /v1/account`, `GET /v1/analytics/payments` (HTTP).

## Related

* [Claiming a Website](/concepts/claiming-a-website) — how a site owner verifies DNS ownership and earns the 15% lane
* [Rewards & Economics (SDK)](/sdk-reference/rewards-and-economics) — operator-facing pricing + payout details


# Claiming a Website

How a site operator proves they own a domain and starts earning the 15% owner lane on every paid call to Unbrowse skills that talk to it.

## Why claim

Unbrowse marketplace skills capture the public APIs your site already serves. When agents call those skills, the price settles on-chain as a three-way [fare split](/concepts/fare-splits). The 15% **owner lane** routes to a Solana wallet you control — but only after you've proven the domain is yours via a DNS TXT record. Until then, the 15% folds back into the indexer pool.

You claim a domain once. Future paid calls to any skill whose `domain` matches your verified apex carry your wallet as a recipient in the on-chain split, atomically, in the same transaction the platform and indexers get paid.

## What you need

* **A Solana wallet.** [lobster.cash](https://lobster.cash) is the recommended provisioner (the platform never holds private keys), but any signer that exposes a base58 pubkey and an SPL USDC ATA works. The verify step accepts the wallet pubkey; the USDC ATA derivation is deferred to a follow-up.
* **DNS edit access on the apex domain.** Cloudflare, Route53, Namecheap, Google Domains — anything that lets you publish a TXT record at `_unbrowse-claim.<your-apex>`.

## The flow

1. Visit `/claim` on the Unbrowse frontend.
2. Paste your apex domain (e.g. `example.com`, not `www.example.com`). Subdomains are deferred to v2.
3. Paste your Solana wallet address. The verify step refuses to bind the domain to any wallet other than the one in the request.
4. Click **Get challenge**. The backend mints a one-time challenge and shows you the TXT record to publish:
   * **Name:** `_unbrowse-claim.example.com`
   * **Value:** `unbrowse-claim=<32-byte hex>;wallet=<your wallet>`
5. Publish that TXT record at your DNS provider. Wait a minute or two for global propagation.
6. Click **Verify**. The backend calls Cloudflare's DNS-over-HTTPS resolver AND Google's DNS-over-HTTPS resolver in parallel. Both must independently return your TXT record, byte-for-byte, before the binding lands.
7. On success, the `domain-wallet:<your-domain>` KV row is written, and a post-verify stamping hook walks every published skill for your domain and stamps:
   * `owner_compensation_opt_in = true`
   * `owner_wallet_address = <your wallet>`
   * `owner_wallet_usdc_ata = <your wallet>` (USDC ATA derive deferred)
   * `owner_wallet_verified_at = <verify timestamp>`
8. The next paid call against any of those skills routes 15% (1500 bps) to your wallet via Faremeter Flex.

The whole flow is documented in code at `backend/src/routes/claim.ts` and `backend/src/services/domain-claim-effects.ts`.

## Anti-spoofing rules

* **The wallet is part of the TXT value.** A stolen or leaked TXT cannot be replayed against a different wallet — the value itself embeds the wallet. The verify endpoint reconstructs `txt_value` server-side from the stored challenge and compares byte-for-byte; it never trusts a client-supplied value.
* **Dual-DoH agreement.** Cloudflare's resolver is signed, but a hostile-network MITM could intercept. Cross-checking with Google makes that a two-target attack. Single-provider success returns `partial_propagation` (soft retry); both unreachable returns `doh_unreachable` (502).
* **Tuple-scoped challenge KV.** Two pending claims for the same domain by two different wallets coexist. Verifying one does NOT consume the other. Challenge key is `domain-claim-challenge:<domain>:<wallet>`, not domain-scoped.
* **Rate limit.** No more than 10 challenge mints per domain per hour. Prevents an attacker from churning challenges to spam your DNS UI.
* **Caller's agent wallet must match.** A third party cannot verify your published TXT and bind it to a wallet they own — the verify endpoint requires the calling agent's wallet to equal the `wallet_address` parameter.
* **Server-owned binding fields.** `owner_wallet_*` fields on `SkillManifest` are server-stamped only. `PATCH /v1/skills/:id` rejects any user-supplied value.

## What if you change wallets later

Re-verify with the new wallet. The binding is `domain → wallet`, not `domain → user`. The post-verify stamping overwrites the previous owner fields on every matching skill.

## What if you lose your domain

Bindings do not auto-expire — Unbrowse cannot detect a registrar transfer. The new owner mints a fresh challenge with their wallet and runs verify. This produces a `409 wallet_conflict` against the existing binding; the new owner contacts support to clear it.

## What you can also do via `/claim`

* **Opt out of the marketplace entirely.** Same DNS-TXT primitive with a different value (`unbrowse-takedown=<challenge>`). Verify flips `lifecycle: "disabled"` on every existing skill for your domain AND writes a persistent `domain-optout:<domain>` KV row that blocks future captures from ever publishing.
* **Submit your official API.** A form on `/claim` posts to `POST /v1/claim/submit-official` with your canonical x402-supported endpoints. The team triages and promotes approved submissions to `verification_status: "verified"` with an `owner_submitted: true` provenance flag so they rank above captured endpoints. There's also a `mailto:hello@unbrowse.ai` fallback if you prefer email.

## Related

* [Fare Splits](/concepts/fare-splits) — how the three on-chain lanes are computed
* `backend/src/services/domain-claim.ts` — the DoH verifier + KV key shapes
* `backend/src/services/domain-claim-effects.ts` — the post-verify owner-wallet stamping hook
* `backend/src/routes/claim.ts` — the HTTP surface


# The Wedge


# Market Framing


# Built on Unbrowse


# Aiko: The Reference Consumer Agent


# Build on Unbrowse

Unbrowse is not just a tool you call; it's a platform your product can stand on. This doc is the opinionated guide to *building* with it: which archetypes work, how to compose, when not to use it.

> If you only need to consume APIs from sites, see [developer-recipes.md](/sdk-reference/developer-recipes). If you want to *build a product or business* on the platform, this is the plan.

## What Unbrowse gives you as a primitive

Five capabilities you can compose, behind one hole:

1. **Intent → capability fill.** `createHole().fill({ intent, url })` returns the data/action result the caller asked for. A verified cache hit avoids browser startup; latency varies by route and environment. A miss may capture, map, and index a route.
2. **Marketplace of captured routes.** Every successful capture is reusable by any other agent. You inherit the routes other operators captured; they inherit yours.
3. **x402 micro-payments.** Per-execution USDC settlement on Solana. Earnings auto-attribute to your wallet; spending pulls from the same wallet.
4. **Local-first runtime.** Captures, auth, and replays stay on the operator's box by default. Marketplace publishes only the route shape, never response bodies.
5. **Headless Chrome using your existing browser session.** Auth is reused from the user's real Chrome/Firefox profile, so gated sites work without re-logging in.

Treat these as five independent legos. The interesting products combine 2-4 of them.

## Build archetypes

Pick one of these as your starting point. Each is a real category of product that fits Unbrowse's grain.

### Archetype A — Vertical AI agent on top of a specific domain

Examples: an AI recruiter that lives in LinkedIn, an AI procurement bot for industrial suppliers, an AI booking concierge.

You leverage:

* Unbrowse for *site coverage* (resolve handles the long-tail of vendor SaaS without you maintaining scrapers)
* Auth import for gated sites
* The marketplace to inherit routes already captured by other operators

Build cost: 1-2 weeks for the wrapper agent + domain prompt; Unbrowse covers most of what would otherwise be 6 months of scraper maintenance.

### Archetype B — Marketplace producer / data validator

Examples: a fleet of agents whose business is mining the marketplace; a paid "fresh routes" service.

You leverage:

* Hole fills as the work loop
* `feedback()` to push your validator reputation up
* x402 earnings as direct revenue

Build cost: 2-4 weeks for the orchestration layer. Revenue starts on day one (small) and compounds with reputation.

See [onboarding-validators.md](/sdk-reference/onboarding-validators) for the operational manual.

### Archetype C — Workflow automation tool with web steps

Examples: Zapier-for-AI-agents, an autonomous research analyst, an outbound-sales operator.

You leverage:

* Unbrowse as the "web step" primitive in a longer DAG
* Captured routes as repeatable workflow nodes
* `next_actions` handoffs for human-in-the-loop fallbacks

Build cost: 3-6 weeks. The win is *deterministic replay* — every web step you ran once is now a callable function for your DAG.

### Archetype D — MCP / OpenClaw server for an org

Examples: an internal MCP that gives your engineering team's Claude Code access to your private internal SaaS apps (Jira, Notion, Linear, your own dashboards).

You leverage:

* Local-only mode (no marketplace publishing for private domains)
* Auth import from the user's real browser
* The MCP protocol surface that ships with `unbrowse mcp`

Build cost: 1 week. The product *is* configuration plus access policy.

### Archetype E — Browser-augmented copilot

Examples: a side-panel browser extension that turns the page the user is on into a callable API for their AI; a Cursor-style IDE where every doc page becomes a tool.

You leverage:

* Hole fills with `url` set to the user's current tab
* Captured routes as on-the-fly tools
* `commitment_only` proofs for traceability

Build cost: 2-4 weeks. The novelty is the UX, not the platform.

## Composition patterns

Once you've picked an archetype, you'll hit one of these patterns. Each has a default answer.

### Pattern 1 — Fill one hole, inspect routes only when debugging

The current contract is `createHole().fill(...)`. Let the runtime choose the descent: route graph, adapter, local auth, browser capture, HAR, or newly indexed contract. Use `resolve`/`execute` only when you are debugging a route choice or supporting an older MCP host.

### Pattern 2 — One runtime per worker

For any fleet >5 concurrent calls, run multiple Unbrowse runtimes (distinct `UNBROWSE_PORT` + `UNBROWSE_HOME`). One shared runtime serializes capture and tanks throughput.

### Pattern 3 — Auth in, auth out

Either use `login()` (interactive) on dev boxes or `importAuth()` from a real Chrome profile on production boxes. Don't try to inject cookies you scraped from somewhere else; vendor security catches that fast.

### Pattern 4 — Feedback closes the loop

Every `execute` should be followed by `feedback({ outcome })`. This is the only signal the marketplace ranker has for "this skill is good vs. broken." Skipping feedback means your published skills will eventually be demoted and your validator reputation flatlines.

### Pattern 5 — Treat misses as explicit handoffs

When `available_endpoints` is empty, read `next_actions[0].command`. Don't loop `resolve` on the same URL. The runtime is telling you what to do next; trust it.

### Pattern 6 — Privacy boundary on internal domains

Use `unbrowse settings --publish-blacklist <domain>` for any internal/PII-bearing site. Captured patterns from those domains never reach the marketplace.

## Phased build plan

For builders going from zero to production, expect roughly:

### Week 1 — Local prototype

* `npm install -g unbrowse unbrowse/sdk`
* `unbrowse setup` + wallet
* Wire SDK into a single test agent
* 5-10 manual hole fills against your target sites
* Confirm cache hits on iteration 2+

Goal: prove the basic loop on YOUR domains.

### Weeks 2-3 — Integration with your product

* Move from manual calls to a worker pool (one runtime per worker)
* Add `feedback()` loop
* Add `clientId` per worker for payout attribution (if relevant)
* Wire `next_actions` handoffs into your error-recovery flow

Goal: 100s of real intents per day, observable failure modes.

### Weeks 4-6 — Hardening

* Set up monitoring: track resolve cache-hit rate, `next_actions` distribution, per-domain success rate
* Decide private vs. public per domain (publish-blacklist)
* Pick a wallet provider and load it (Crossmint Lobster recommended for headless)
* If gated sites: lock down `importAuth` source profiles

Goal: the system is observable and the failure modes are named.

### Months 2-3 — Scale

* Multi-machine fleet (one runtime per box, distinct `UNBROWSE_PORT`)
* Reputation accumulation: feedback hygiene matters now
* Paid x402 routes if your domain is in the marketplace's paid tier
* (Optional) MCP / OpenClaw exposure to make your wrapper consumable by other agents

Goal: the platform disappears into your product. Customers don't think about Unbrowse; they think about your agent.

## Extension points

Where you can plug in instead of waiting for the platform.

| Hook                                    | What it lets you do                                                                    | Surface                     |
| --------------------------------------- | -------------------------------------------------------------------------------------- | --------------------------- |
| Custom `fetch`                          | Route through a residential proxy, instrument every call, run in offline mode          | `new Unbrowse({ fetch })`   |
| Custom `headers`                        | Add tracing headers, mTLS client cert hooks, client-tier identification                | `new Unbrowse({ headers })` |
| `confirmThirdPartyTerms`                | Per-domain ToS gate; default-deny third-party-restricted captures                      | resolve/execute input       |
| `feedback({ diagnostics })`             | Feed your own quality signal (wrong-endpoint detection, latency anomalies) into ranker | feedback input              |
| `unbrowse settings --publish-blacklist` | Domain-level privacy policy enforced in the runtime                                    | CLI                         |
| Local-only mode                         | Run with no marketplace publish at all (set `UNBROWSE_PUBLISH=off`)                    | env                         |

What's NOT yet a documented extension point but is on the roadmap:

* Custom ranker plugins (local override of marketplace ranking)
* Custom proof systems beyond `commitment_only` (stronger provenance schemes are an active research direction; details will follow in a forthcoming whitepaper)
* Custom payment rails (non-x402 settlement)

If you need any of these, file an issue or email <security@unbrowse.ai> for an NDA discussion.

## When NOT to build on Unbrowse

Be honest about the misfit cases. Don't pick the platform if:

* **You only need one site.** If your entire product is "scrape exactly this one API," a hand-built scraper is faster than learning Unbrowse. Unbrowse pays off when you cover dozens to hundreds of sites.
* **You need guaranteed millisecond latency.** Unbrowse does not publish a fixed cache-hit latency guarantee. If you're building HFT-adjacent systems, this is the wrong layer.
* **You can't accept x402 / Solana / USDC settlement.** The earnings rail is fixed. You can run local-only with no earnings, but if your business model needs a different rail, that's a hard mismatch today.
* **Your target sites have iron-clad ToS prohibitions you must respect.** Unbrowse is a capability, not a license. `confirmThirdPartyTerms` is your gate, but the legal call is yours.
* **You need cryptographic origin proofs today.** `commitment_only` is tamper-evident, not cryptographic. Stronger provenance schemes are an active research direction; specifics will be detailed in a forthcoming whitepaper.

## Distribution channels for what you build

Once you've built something on Unbrowse, where does it ship?

1. **As an npm package** that depends on `unbrowse/sdk`. Standard.
2. **As an MCP server** so any Claude Code / Cursor / etc. user can mount your wrapper. The Unbrowse runtime already speaks MCP via `unbrowse mcp`; you can layer your own MCP server on top.
3. **As an OpenClaw plugin** so the agent's default browser auto-routes through your wrapper. See `openclaw` metadata in [the packaged skill manifest](https://github.com/unbrowse-ai/unbrowse/tree/main/SKILL.md).
4. **As a hosted endpoint** (your service runs Unbrowse, exposes a thin API to your customers). The SDK runs server-side fine.

## See also

* [SDK API reference](https://github.com/unbrowse-ai/unbrowse/tree/main/packages/sdk/docs/api-reference/README.md) — the actual call surface
* [Developer recipes](/sdk-reference/developer-recipes) — composition patterns at the call level
* [Onboarding validators](/sdk-reference/onboarding-validators) — operating a fleet
* [Rewards & economics](/sdk-reference/rewards-and-economics) — the payment layer
* [Open source notice](/reference/open-source-notice) — what's MIT vs. proprietary
* [Whitepaper: network layer](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/whitepaper/network-layer.md) — the long-form economic + architectural model


# Developer Recipes

Common SDK patterns. The current SDK surface is the hole: `createHole().fill(...)`. The older `resolve`/`execute` methods remain for route inspection and compatibility.

## Recipe 1: Fill One Hole

```bash
unbrowse "top stories with point counts"
unbrowse "top stories with point counts" --url "https://news.ycombinator.com"
```

```ts
import { createHole } from "unbrowse/sdk";

const hole = createHole({
  client: { apiKey: process.env.UNBROWSE_API_KEY },
});

const r = await hole.fill({
  intent: "top stories on Hacker News with point counts",
  url: "https://news.ycombinator.com",
});
```

This is the agent-facing contract. Internally the runtime may resolve, execute, open a browser, inspect HAR, reuse cookies, or index a newly discovered route.

## Recipe 2: Inspect a Route Manually

Use the legacy route view only when you need to debug endpoint selection:

```ts
const resolved = await u.resolve({ intent, url });
const pick = resolved.available_endpoints?.[0];
if (!pick) throw new Error("no route");

const r = await u.execute(pick.endpoint_id, {
  projection: { raw: true },
});
```

## Recipe 3: Auth Before a Fill

```ts
await u.login({ url: "https://linkedin.com" });

const r = await hole.fill({
  intent: "my recent messages",
  url: "https://www.linkedin.com/messaging/",
});
```

For headless workers, import cookies from a real Chrome/Firefox profile:

```ts
await u.importAuth({
  url: "https://linkedin.com",
  browser: "chrome",
  chromeProfile: "Default",
});
```

## Recipe 4: Long-running Worker Pool

```ts
import { createHole } from "unbrowse/sdk";
import pLimit from "p-limit";

const hole = createHole({ client: { clientId: `worker-${process.pid}` } });
const limit = pLimit(8);

async function processBatch(tasks: { intent: string; url: string }[]) {
  return Promise.all(tasks.map(t => limit(async () => {
    try {
      const r = await hole.fill(t);
      return { task: t, status: r.ok ? "ok" : "fail", data: r };
    } catch (e) {
      return { task: t, status: "error", error: String(e) };
    }
  })));
}
```

Key constraint: 8 concurrent calls against one runtime is roughly the safe ceiling. For more, run multiple runtimes.

## Recipe 5: Custom Fetch / Proxy

Useful for routing through a residential proxy or for instrumentation.

```ts
import { createHole } from "unbrowse/sdk";
import { ProxyAgent, fetch as undiciFetch } from "undici";

const dispatcher = new ProxyAgent("http://geo.iproyal.com:12321");
const hole = createHole({
  client: {
    fetch: (url, init) => undiciFetch(url, { ...init, dispatcher }),
  },
});
```

## Recipe 6: Decide Between SDK and CLI

| Need                               | Use                                 |
| ---------------------------------- | ----------------------------------- |
| In-process agent making many calls | SDK `createHole().fill(...)`        |
| One-off shell automation           | CLI `unbrowse "task" [--url <url>]` |
| Inspect current contract           | `unbrowse contract surface`         |
| Auth flow with user-facing browser | CLI (`unbrowse auth`)               |
| Route-selection debugging          | Legacy `resolve`/`execute`          |
| Wallet config                      | `unbrowse setup`                    |


# Onboarding Users

For an individual developer or operator who wants to use Unbrowse as their agent's browser **and** earn rewards from the routes their normal browsing creates.

## The 60-second path

```bash
npm install -g unbrowse
npx @crossmint/lobster-cli setup    # provision the wallet that will receive earnings
unbrowse setup                       # detects the wallet automatically
unbrowse account --register --email you@example.com
```

That's it. Every later `resolve` / `execute` you run mines the marketplace, credits your wallet, and is free for cache hits.

Next, see [unbrowse/sdk installation](/for-developers/sdk-quickstart) to wire the SDK into your code.

That's it. Every later `resolve` / `execute` you run mines the marketplace, credits your wallet, and is free for cache hits.

## What "mining" actually means

When you (or your agent) runs:

```bash
unbrowse eval resolve --intent "top stories" --url https://news.ycombinator.com
```

Unbrowse:

1. Checks the marketplace cache for a matching skill. If found: free, fast, you've consumed.
2. If not, captures network traffic from a headless browser session.
3. Reverse-engineers the captured traffic into a callable skill.
4. Publishes the admitted skill back to the marketplace under your wallet address.

Later, when another agent anywhere runs the same intent against the same domain and the marketplace serves your skill, **you get paid**. Payment is in USDC over x402, settled on Solana.

## Wallet setup

The simplest path:

```bash
npx @crossmint/lobster-cli setup
unbrowse setup    # detects the lobster config automatically
```

Lobster is a self-custodial wallet sized for agent micropayments.

If you already have a wallet:

```bash
export AGENT_WALLET_ADDRESS="<your solana address>"
unbrowse setup
```

## See your earnings

```bash
unbrowse stats --earnings
unbrowse stats --json
```

For the web dashboard, visit `https://www.unbrowse.ai/dashboard` once your account is paired.

From the SDK:

```ts
import { Unbrowse } from "unbrowse/sdk";
const u = new Unbrowse({ apiKey: process.env.UNBROWSE_API_KEY });
const me = await u.dashboard();
console.log(me.earnings?.total_usd, me.earnings?.unsettled_usd);
```

From the SDK:

```ts
await u.request("GET", "/v1/dashboard/me");
```

## Make your normal agent calls earn

The SDK is the same as the validator path:

```ts
import { Unbrowse } from "unbrowse/sdk";

const u = new Unbrowse();
const result = await u.resolve({
  intent: "my newsletter signups",
  url: "https://app.beehiiv.com/dashboard",
});
```

If your wallet is configured during `unbrowse setup`, every captured skill is auto-published under your address.

## Privacy boundary

* Captured skills are **patterns**, not data. The marketplace stores the request shape (URL template, headers, response schema), not your actual response bodies.
* Auth headers and cookies are stripped before publish.
* You can exclude domains from publish via `unbrowse settings --publish-blacklist <domain>`. Captures from those domains never reach the marketplace.

## When to upgrade to the validator path

If you start running 10+ parallel agents, see [onboarding-validators.md](/sdk-reference/onboarding-validators). The single-user path serializes captures through one runtime, which becomes the bottleneck above \~5 concurrent resolves.


# Onboarding Validators

For clients running a fleet of agents (10+ workers) that should contribute to the Unbrowse marketplace as **validators** and earn x402 rewards.

> A *validator* in Unbrowse is any agent that runs real intents through resolve/execute, generates traffic that becomes captured skills, and gets attribution on every later replay of those skills. Validators are paid in proportion to how often their captured routes are reused.
>
> **Term note:** the [whitepaper](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/whitepaper/network-layer.md) reserves "validator" for a future verification/staking role. Until that layer ships, "validator" in product docs means "earning agent in the contributor pool." If the whitepaper sense matters in your context, prefer "contributor" or "earning agent."

## Mental model

```
Client agent fleet  --->  Local Unbrowse runtime  --->  Marketplace
(Claude/Codex/etc.)       (one per worker box)         (publish + pay)
        |                          |                          |
        | resolve(intent, url)     | capture + index          | x402 settle
        v                          v                          v
   real work                  routes published          wallet credited
```

## Prerequisites

* A Solana wallet that can receive USDC. We recommend [Crossmint Lobster](https://lobster.cash) for fleet-level payouts (it supports headless agents).
* Per-worker isolation: each agent runs against its own local Unbrowse runtime so captures are not cross-contaminated.
* A registered agent account so payouts attribute to your org.

## Step 1 — Provision the operator wallet

```bash
npx @crossmint/lobster-cli setup
# captures wallet address into ~/.lobster/agents.json; unbrowse setup picks it up automatically.
```

Or set the env var directly:

```bash
export AGENT_WALLET_ADDRESS="<solana address>"
# or, equivalently, LOBSTER_WALLET_ADDRESS
```

Wallet resolution order at runtime (verified in `src/payments/wallet.ts:getWalletContext`): `LOBSTER_WALLET_ADDRESS`, then `AGENT_WALLET_ADDRESS`, then the local Lobster `agents.json`. Whichever wins becomes the address the marketplace credits.

## Step 2 — Boot the runtime per worker

Every worker box runs its own runtime so capture state stays clean.

```bash
npm install -g unbrowse
unbrowse setup
unbrowse account --register --email ops@yourco.com
```

The runtime auto-starts on demand on `http://localhost:6969` when an SDK or CLI call needs it. There's no separate `server start` command — just call resolve and the runtime spins up.

For multi-worker boxes, give each worker its own runtime by setting a distinct port and home dir per process and warming it once:

```bash
UNBROWSE_PORT=6970 UNBROWSE_HOME=/var/unbrowse/worker-1 unbrowse stats >/dev/null
```

Then point that worker's SDK at `http://localhost:6970`.

## Step 3 — Wire the SDK in your agent

```ts
import { Unbrowse } from "unbrowse/sdk";

const unbrowse = new Unbrowse({
  baseUrl: process.env.UNBROWSE_URL ?? "http://localhost:6969",
  apiKey: process.env.UNBROWSE_API_KEY,
  clientId: `worker-${process.env.WORKER_ID}`,
});

async function doTask(intent: string, url: string) {
  const resolved = await unbrowse.resolve({ intent, url });

  const pick = resolved.available_endpoints?.[0];
  if (!pick) {
    // Resolve miss: agent decides whether to follow next_actions or skip.
    return resolved.next_actions?.[0];
  }

  const r = await unbrowse.execute(pick.endpoint_id, {
    contextUrl: url,
    projection: { raw: true },
  });

  return r.trace.success ? r.result : r.trace.error;
}
```

`clientId` is important — it lets the marketplace track which worker captured which skill, so payouts attribute correctly even when several workers race the same domain.

## Step 4 — Confirm earnings flow

Every published skill is re-executable by other Unbrowse users. When that happens, your wallet gets credited.

```bash
unbrowse stats --earnings
unbrowse stats --json
```

From the SDK (typed in 6.9.69423+):

```ts
const me = await unbrowse.dashboard();                          // GET /v1/dashboard/me
const public_view = await unbrowse.dashboardByWallet(addr);     // public, no auth
const { ledger, transactions } = await unbrowse.creatorTransactions(agentId);
const attribution = await unbrowse.indexerAttribution(indexerId);
```

Funds settle on-chain via x402 once the unsettled balance crosses the platform threshold.

## Step 5 — Tune for high-volume validation

For swarms running 100+ resolve/sec across the fleet:

* **Pin one runtime per worker.** A shared runtime serializes capture and tanks throughput.
* **Leave `force_capture` false (default).** Cache hits are free for you and credit the original publisher (often you); that's the system working as intended.
* **Set `confirmThirdPartyTerms: true` only on domains you actually have permission to scrape.** This is a real legal gate, not a flag to flip blindly.
* **Use `feedback()` aggressively.** Every `outcome: "success" | "failure"` improves marketplace ranking and your validator reputation, which lifts payout weighting.
* **Use `unbrowse settings --publish-blacklist <domain>`** to keep sensitive internal domains out of the marketplace.

## Anti-patterns

* Running the same `clientId` on every worker — payout attribution collapses.
* Sharing one runtime across 50 workers — capture races, dropped skills, lost earnings.
* Forcing `force_capture: true` on every call — you'll pay browser-open cost without earning more, since cache-hit earnings already credit the original publisher.
* Pointing at the public OSS repo for a custom build — see [OPEN-SOURCE-NOTICE.md](/reference/open-source-notice). Use the npm CLI binary.

## See also

* [Rewards & economics](/sdk-reference/rewards-and-economics)
* [Developer recipes](/sdk-reference/developer-recipes)
* [SDK API reference](https://github.com/unbrowse-ai/unbrowse/tree/main/packages/sdk/docs/api-reference/README.md)


# Rewards and Economics

How Unbrowse pays contributors and operators. Read this before wiring a swarm.

## The flow

```
agent calls resolve()
        |
        v
cache hit? --yes--> Flex authorization settles payment --> return data
        |                          |
        |                          v
        |             contributor share + platform share
        |             distributed atomically per signed splits
        |
        v
live capture --> extract endpoints --> admit + publish
        |                                       |
        |                                       v
        |                          publisher = caller's wallet
        v
   return data
        |
        v
  (later) some other agent runs same intent
        |
        v
  payment authorized → splits include this skill's contributors
```

## Three roles paid by the protocol

> **Term note:** the [whitepaper](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/whitepaper/network-layer.md) uses "validator" for a future verification/staking role. In product docs (and below), **contributor** is the umbrella term for any agent that earns from captured routes — publisher, indexer, or attributed worker. "Validator-mode" agents (running intents at scale, [onboarding-validators.md](/sdk-reference/onboarding-validators)) are contributors at the call-volume end of the spectrum.

| Role            | What they do                                                                                                                                                | How they earn                                                                                                                                                      |
| --------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **Contributor** | Publisher of a skill, indexer of a captured route, or otherwise attributed for the work that produced a callable endpoint                                   | The remaining 35% (3500 bps) of each paid execute when the site owner has DNS-claimed the domain; 50% (5000 bps) when no owner has claimed                         |
| **Site owner**  | Verified operator of the domain the skill talks to (proven via DNS-TXT at `_unbrowse-claim.<apex>`, see [Claiming a Website](/concepts/claiming-a-website)) | 15% (1500 bps), routed via the on-chain split, only when both `owner_compensation_opt_in === true` and a verified `owner_wallet_usdc_ata` are stamped on the skill |
| **Platform**    | Runs marketplace, settles x402, maintains anti-fraud                                                                                                        | 50% (5000 bps)                                                                                                                                                     |

The three lanes are computed by `computeFlexSplits` in `backend/src/services/flex.ts` (see `PLATFORM_BPS = 5000` and `OWNER_BPS = 1500`). The site-owner lane stays dormant until a DNS claim verifies and the post-verify stamping hook lands `owner_wallet_usdc_ata` on the skill; up to that moment the indexer/contributor pool collects the full 50%.

Most agents who run validators are contributors on every successful capture. The split between sub-roles inside the contributor pool (publisher vs. indexer vs. reviewer) is governed by the attribution model and evolves — don't hardcode a sub-split into your tooling. Read your live ledger via the dashboard or `/v1/stats/indexer/:id/ledger` rather than assuming a fixed weight.

## Pricing model

* **Cache hit**: small per-execution micro-payment (USDC over x402). Exact amounts depend on skill rarity and the live rate card.
* **Live capture**: free for the caller. The captured skill becomes inventory.
* **Paid x402 routes**: skills marked `paid` cost more (per-skill pricing). The 50/15/35 split (platform/owner-when-claimed/contributors) still applies.
* **Attribution weighting**: contributors whose routes are uniquely useful earn larger shares than those whose routes have good alternatives. Stop adding marginal value and your share decays over subsequent executions.

Exact rate cards live at [unbrowse.ai/pricing](https://www.unbrowse.ai/pricing). The runtime never settles below the platform threshold to keep gas-equivalents tractable — Faremeter Flex batches authorizations and finalizes on-chain when the refund window closes.

## How payouts settle

* **Rail**: x402 over Solana, USDC, using the [Faremeter Flex](https://docs.faremeter.xyz/flex/overview) scheme (`@faremeter/flex`).
* **Cadence**: each paid execute signs a Flex authorization off-chain; the platform's facilitator holds the authorization in memory, then submits a batched on-chain settlement after the refund window. Distribution is atomic per the signed splits.
* **Wallet**: paired via `unbrowse setup` (lobster.cash recommended) or `--wallet-address <addr>`. Once paired, the SDK signs payment authorizations with a session key registered against your Flex escrow — see [docs/wallets.md](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/wallets.md) for the wallet → escrow → session-key onboarding sequence.
* **Visibility**: `unbrowse stats --earnings` (CLI), or `GET /v1/stats/indexer/:id/ledger`, `GET /v1/account`, `GET /v1/analytics/payments` (HTTP).
* **Ledger model**: the attribution and fee ledgers are **append-only event logs** — one immutable content-addressed row per execution — and your balance is a **projection** (fold) over those rows, not a mutated running total. Concurrent executions therefore never lose credits, replaying the same `execution_id` is idempotent, and every credit is individually auditable (the same integrity model as the route-attestation ledger). See `backend/LEDGER-UNIFICATION-PLAN.md`.

## Anti-fraud (current state and roadmap)

Marketplace ranking and payout weighting fold in the following signals. Each is at a different stage; treat the list as the design, with the current state called out so you don't over-rely on a guarantee that isn't there yet.

* **Outcome feedback (live)**: `feedback({ outcome })` calls flow into ranking. Skills that draw repeated `failure` from independent operators get demoted in resolve.
* **`commitment_only` proofs (live)**: every published skill carries a SHA-256 commitment over the captured response. This is **not** cryptographic origin proof — it's tamper-evident metadata for after-the-fact-edit detection. The four-state proof model and the boundary are documented in [docs/concepts/verification-and-proofs.md](/concepts/verification-and-proofs).
* **Admission filters (live)**: synthetic-capture / captcha-page / write-on-read / phantom-URL detectors live in the capture pipeline and reject obvious adversarial publishes before they reach the marketplace.
* **Replay verification (planned)**: independent re-execution of a captured skill before it accrues attribution weight. Not yet enforced backend-side. Don't depend on it being active today.
* **Reputation-weighted payouts (planned)**: operators with high reject rates accumulating negative reputation that reduces payouts on legitimate captures too. Roadmap, not enforced today.

If you are scoping an audit, take the **live** items as production behavior and the **planned** items as forward-looking. The 50/15/35 split, Flex settlement, and `feedback` ingestion are demonstrably wired today. (The site-owner lane stays dormant until DNS-claim verify; see the role table above and the [Claiming a Website](/concepts/claiming-a-website) doc.)

## When the system pays nothing

* Resolve miss with no admitted endpoint: no publish, no earnings.
* Capture admitted but never re-executed: stored as inventory, no income until first replay.
* Domains excluded via `unbrowse settings --publish-blacklist <domain>`: never publish, never earn.
* Skills published under unpaired wallets: balance accumulates server-side and is forfeited per platform policy. Pair a wallet via `unbrowse setup` to claim.

## See also

* [Onboarding validators](/sdk-reference/onboarding-validators)
* [Whitepaper: network layer](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/whitepaper/network-layer.md)
* [Wallets, escrow, session keys](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/wallets.md)
* [Fare splits & x402 payments](/concepts/fare-splits)
* [Open source notice](/reference/open-source-notice): why the engine that does this is closed-source


# Whitepaper: Internal APIs Are All You Need

Implementation-aware companion docs for the Unbrowse whitepaper.

* Authors: Lewis Tham, Nicholas Mac Gregor Garcia, Jungpil Hahn
* Canonical PDF: [unbrowse-whitepaper.pdf](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/whitepaper/unbrowse-whitepaper.pdf)
* Status: official paper plus implementation-aware companion docs
* Canonical paper draft synced here: April 1, 2026

> Important The PDF mixes shipped product behavior, research results, and forward-looking economic design. These companion docs separate those three things so readers can tell what exists in the codebase today, what already ships in the current x402/payment lane, and what is still coming soon.

## What This Companion Covers

* What Unbrowse ships today
* Which whitepaper claims map directly to the codebase
* Which paper sections are partial implementations
* Which paper sections are still `coming soon` beyond the shipped payment lane
* Which evaluation paths are current product truth versus paper benchmark context

## The Short Hook

The web contains a huge amount of usable value, but most of it is trapped behind interfaces built for humans.

Unbrowse is a way to unlock that layer for agents.

It learns the request paths underneath websites, turns successful routes into reusable skills, and makes later agents faster and less brittle because they do not have to rediscover the same workflows from scratch.

## Quick Navigation

* [Unbrowse In Plain English](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/whitepaper/plain-english.md)
* [For Technical Readers](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/whitepaper/for-technical-readers.md)
* [For Investors](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/whitepaper/for-investors.md)
* [Marketplace and Maintenance](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/whitepaper/network-layer.md)
* [What Is Unbrowse?](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/whitepaper/what-is-unbrowse.md)
* [The Problem](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/whitepaper/the-problem.md)
* [Mental Models](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/whitepaper/mental-models.md)
* [How It Works](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/whitepaper/how-it-works.md)
* [Key Concepts](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/whitepaper/key-concepts.md)
* [System Today](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/whitepaper/system-today.md)
* [Paper vs Product Status](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/whitepaper/paper-vs-product.md)
* [Evaluation and Benchmarks](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/whitepaper/evaluation.md)
* [Coming Soon](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/whitepaper/coming-soon.md)

## Recommended Reading Order

Start with [For Investors](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/whitepaper/for-investors.md) or [For Technical Readers](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/whitepaper/for-technical-readers.md) for the public/product truth.

Read [Marketplace and Maintenance](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/whitepaper/network-layer.md) for the public explanation of how shared route reuse creates freshness, validation, attribution, and maintenance requirements over time.

## What Ships Today

Unbrowse today is a local-first web capability layer for agents:

* local CLI plus local server
* browser capture through Kuri
* route discovery that maps a site's internal API endpoints into reusable routes
* marketplace-backed reuse of discovered skills
* route cache plus marketplace search plus live-capture fallback
* local credential storage and auth reuse
* MCP server mode plus host integrations for major agent environments
* reliability scoring, verification, and schema-drift-aware endpoint health
* x402-gated marketplace search/execution paths, HTTP 402 payment requirements, wallet-linked payment metadata, and current payout routing
* canonical product evals in this repo

## What To Read First

Read [Unbrowse In Plain English](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/whitepaper/plain-english.md) if you want the shortest narrative explainer in normal language.

Read [For Technical Readers](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/whitepaper/for-technical-readers.md) if you want the current architecture, eval truth, and paper-vs-product boundary in one place.

Read [For Investors](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/whitepaper/for-investors.md) if you want the market framing, compounding product loop, and the clean line between shipped product and roadmap.

Read [System Today](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/whitepaper/system-today.md) if you want the current product.

Read [What Is Unbrowse?](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/whitepaper/what-is-unbrowse.md) and [How It Works](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/whitepaper/how-it-works.md) if you want the narrative explainer layer that used to live in the old docs set, now rewritten against the current repo and whitepaper.

Read [Paper vs Product Status](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/whitepaper/paper-vs-product.md) if you want the strict audit: shipped, partial, or `coming soon`.

Read [Coming Soon](https://github.com/unbrowse-ai/unbrowse/tree/main/docs/whitepaper/coming-soon.md) if you care about the parts of the route economy that are still forward-looking: richer multi-party fee splits, fuller attribution, validator attestation, and the rest of the paper beyond the shipped payment lane.

## Citation

```bibtex
@misc{tham2026internal,
  title = {Internal APIs Are All You Need},
  author = {Lewis Tham and Nicholas Mac Gregor Garcia and Jungpil Hahn},
  year = {2026},
  note = {Official Unbrowse whitepaper with implementation-aware companion docs}
}
```


# How Unbrowse Pays

### What this document is

This page explains how money moves through Unbrowse on a paid call: what is free, what costs, how a charge is split, and who signs for the wallet. Every claim cites a file and line in the codebase so the behaviour can be verified, not just trusted.

### Free vs paid

Discovery and internal-API routing are free. When an agent asks Unbrowse to resolve an intent, search cached endpoints, or read a route graph, nothing is charged.

Paid execution is the only billed step. When a workflow runs a captured route on your behalf, that single call settles over x402 in USDC. There is no subscription and no monthly plan: you pay per request, and only for the requests that execute.

### The split

Every paid call is divided into a fixed split by Faremeter Flex. The settlement roles are defined in `backend/src/services/flex.ts:39`, and the split percentages are set by named basis-point constants:

* Platform: 50% (`PLATFORM_BPS = 5000`, see `backend/src/services/flex.ts:68`).
* Indexer / contributor pool: 35%, shared across the contributors who captured and maintain the route, weighted by their cumulative contribution.
* Domain owner: 15% (`OWNER_BPS = 1500`, see `backend/src/services/flex.ts:87`), carved off the top when a verified owner wallet is bound to the domain.

So a paid call with a bound domain owner settles as a **50/35/15** split: half to the platform that runs the infrastructure, just over a third to the indexers who discovered the route, and the rest to the website owner. When no owner wallet is bound, the owner share folds back into the contributor pool.

**Where it settles: atomic on devnet, custodial on mainnet today.** The trustless on-chain atomic split (one Flex transaction across all recipients) is live on **devnet only** — the split program is not yet on Solana mainnet. On **mainnet today the platform settles the split custodially**: it collects the payment and disburses each contributor's and owner's earned cut from the attribution ledger (`backend/src/services/disburse.ts`, default dry-run). The percentages above are identical either way; only the settlement venue differs.

### Brokered costs (fair compensation)

The split above taxes a route's **own price** — the opt-in lane. **Execution itself is free: unbrowse takes no cut on the commons.** When unbrowse fronts a paid upstream on your behalf — a web-unblocker for a hard-protected site, an LLM proxy, a paid third-party API, a facilitator or gas fee — it passes the raw upstream cost straight through to you **at cost**, adding nothing. You never pay unbrowse to execute; you only ever pay the genuine upstream, and only when one exists.

The broker markup is a single named constant — **0% by default** (`FAIR_COMPENSATION_BPS`, `backend/src/services/fair-compensation.ts`): a fronted upstream is pass-through-at-cost. Monetization is opt-in and lives at the edge — an endpoint owner prices their own route and the router tolls *that* (the Flex split above); a deployment can also opt into a broker markup via env. Either way, every brokered ledger row records the raw `upstream_cost_uc` next to `compensation_uc` (which is `0` unless someone has opted in), so the take-rate is auditable.

#### `POST /v1/unlock` — brokered web unblocking

The first agent-facing brokered surface. When a site is behind hard anti-bot protection that the local capture ladder can't clear, the agent hands the URL to unbrowse and pays once, in the same Solana USDC x402 it already uses for routes:

```
POST /v1/unlock     { "url": "https://…", "js_render": true }
  → 402  with the sponsor envelope, priced at the raw upstream cost (pass-through; no markup by default)
  → agent pays (Flex x402, single payee — no Base wallet, no vendor signup)
  → unbrowse fronts the upstream web-unblocker on Base x402 and returns the cleared HTML
```

The response carries `x-unbrowse-charge-usd`, `x-unbrowse-passthrough-usd`, and `x-unbrowse-compensation-bps` so the breakdown is visible on every call. The agent never holds a Base wallet, never registers with the unblocker vendor, and never sees the vendor's payment header — unbrowse holds one upstream account and brokers it for everyone (`backend/src/routes/unlock.ts`, paying via `backend/src/services/base-x402-pay.ts`).

### Who signs the wallet

Unbrowse owns the payment intent: what is being paid for, how much, and to which recipient token account. It does not own the wallet. Wallet ownership, session lifecycle, and the sign and broadcast pipeline are delegated to an agent wallet.

The compatible and tested agent wallet is `lobster.cash`. Fund a `lobster.cash` wallet once, and it pays each x402 challenge automatically. Unbrowse never creates wallets and never asks for private keys, seed phrases, or raw card details.

### Seeing what your wallet received

Because the owner share is paid on-chain, the authoritative record is the wallet itself: the USDC is already there. For a quick summary keyed by domain, a verified owner can read `GET /v1/claim/earnings?domain=<your-domain>`, which sums the owner-lane payouts across settled batches and returns the total earned, the count of payouts, and the most recent settlement transaction. There is no balance to release and no button to press: the read is a mirror of on-chain settlement, not a withdrawal.

### Verify it yourself

The split constants above are read straight from the code. Run `bash scripts/lobster-compat-gate.sh` to confirm Unbrowse's live x402 challenge meets the wallet requirements (Solana settlement, USDC currency). The gate exits 0 when the contract holds, which is the runnable proof behind this page.


# Open Source Notice

**The Unbrowse client boundary is open source and auditable** at [github.com/unbrowse-ai/unbrowse](https://github.com/unbrowse-ai/unbrowse). The local runtime, CLI bridge, SDK and drop-in adapters, and wallet/auth/signing layer are MIT and readable. The CLI ships **unsigned and readable by design**: an agent runs code on your machine and touches your credentials, so you should be able to read exactly what it does rather than trust an opaque binary. Trust comes from auditability, not from a signature.

The private product surface is the **backend** plus the **web app**. The backend owns the route graph, ranking, settlement, and recursive contract compilation. The public client boundary sees only typed holes, approvals, pointer-only receipts, wallet-sealed fills, and local capability dispatch. `GET /v1/contract/surface` is the machine-readable bridge contract for this split; its client-fillable holes are `intent`, `wallet_proof`, `approval`, `local_capability_result`, and `typed_pointer`, none of which carries a secret value.

The split:

| Surface                                                                                              | Where it lives                       | License / visibility                            |
| ---------------------------------------------------------------------------------------------------- | ------------------------------------ | ----------------------------------------------- |
| **Client boundary runtime** — local execution bridge, typed holes, approvals, wallet/auth/signing    | npm + public repo                    | **MIT, fully open & auditable**                 |
| **Client SDK + drop-in adapters** (`unbrowse/sdk`, every `@unbrowse/*` shim + agent-SDK adapter)     | npm + public repo                    | **MIT, fully open**                             |
| `unbrowse` CLI runtime                                                                               | npm `unbrowse`                       | readable, unsigned bridge bundle of this source |
| **Backend** (route graph, ranking, recursive contract compilation, marketplace, payouts, settlement) | **private repo**, Cloudflare Workers | proprietary (server-side)                       |
| **Web app** (unbrowse.ai)                                                                            | **private repo**                     | proprietary (product surface)                   |

The client carries no server secret: credentials stay local, the secret bytes never cross the wire, and integrity between client and the private backend is established by a hash-chained, auditable ledger (reference implementation under `paper/reference/`). You can read every line the client runs; the server you settle against is the only part you take on trust, and the ledger is how that trust is kept honest.

## What this means for you

* **Building on the SDK?** New code should use `unbrowse/sdk`. Existing local-runtime integrations can keep using `unbrowse/sdk` plus a running `unbrowse` runtime (`npx unbrowse setup`). Both SDKs are MIT.
* **Reading the repo for architecture?** It reflects the current client boundary — `src/` is the runtime and bridge the `unbrowse` npm bundle is built from. The `docs/` and the public [whitepaper](/research/whitepaper) describe the same behavior.
* **Filing a bug?** Use [github.com/unbrowse-ai/unbrowse/issues](https://github.com/unbrowse-ai/unbrowse/issues) for SDK/CLI issues. The published runtime tracks this source.
* **Want source access for security review?** Email <security@unbrowse.ai>. Code review under NDA is available for serious enterprise integrators.

## Why it's open

An agent that drives your browser, reads your sessions, and signs actions with a wallet is exactly the kind of software that should be readable. We publish the full client so you can verify those claims line by line — that credentials stay local, that secret bytes never cross the wire, that every action is signed — instead of trusting a binary. Auditability is the security model.

Two things we ask of anyone who builds on or forks this:

1. **Attribution and integrity.** The client interoperates with marketplace-publish, paid-routes, and ToS gates by design. Stripping those to turn a discovery tool into an unattributed scraping fleet is a misuse, not a fork we endorse — keep attribution and the integrity gates intact.
2. **Responsible disclosure.** If you find a way the client could be aimed at services that disallow automated access, or at accounts an operator does not own, email <security@unbrowse.ai> rather than weaponising it.

## Open standards we build on — and credit

We fault forks for *unattributed* rebranding; we will not do the same to the layers below us. Unbrowse's agent-interop surface (`src/interop/`) is a **drop-in for, and builds on, open standards authored by others**. We interoperate with them and credit them; we do not fork-and-rebrand them:

| Standard                             | Author / owner                                              | What we do with it                                                                      |
| ------------------------------------ | ----------------------------------------------------------- | --------------------------------------------------------------------------------------- |
| **Agent Skills** (`SKILL.md` format) | Anthropic — released as an open standard (agentskills.io)   | ingest + serve our routes as skills, to the published spec                              |
| **Model Context Protocol (MCP)**     | Anthropic — open spec (modelcontextprotocol.io)             | expose our surface as MCP tools; map every tool to the uniform route shape              |
| **x402** + **x402 Bazaar**           | Coinbase — open payment protocol + public discovery catalog | settle usage over x402; rank a site's already-listed Bazaar resources above any re-wrap |

A route is a drop-in *replacement* only in the sense of *interoperating with* these formats — never of replacing their authorship. Where we build on a cited source, we keep its `source_id` in the code and build **on top** of it, not over it.

## What we give first

The open part is given before anything is asked back. Freely available today, MIT: the `unbrowse/sdk` + `unbrowse/sdk` SDKs, and the standards-interop above — so any agent can use Unbrowse through the formats it already speaks, at no cost and with no lock-in (the browser fallback is always the exit).

The deeper layers open **as gifts over time, as they mature safely** — not hoarded, not sold as the point. The maintenance/trust economy (proof-of-indexing, bonded accountability) is staged for reveal, not extraction (see `RELEASE_STRATEGY.md`); USDC settles usage while the bond only secures trust (one master, never a money-first root). Give first, hidden from money-motive, planted in good soil — then it grows on its own.