Projets/llm-bridge
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

Add initial llm-bridge project
CI / fmt, clippy and tests (ubuntu-latest) (push) Successful in 21m0s
Details
CI / browser end-to-end (Chromium + fixture) (push) Skipped
Details
CI / fmt, clippy and tests (windows-latest) (push) Canceled after 0s
Details

Browse Source
This commit is contained in:
bruno
2026-09-18 18:01:07 -04:00
commit 7b5aa6fcc9
58 changed files with 19914 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/ci.yml
+98
View File
@@ -0,0 +1,98 @@
name: CI
# The gates this repository relies on: formatting, lints and the 190-odd tests
# that need neither a browser nor a network. The browser end-to-end suite is a
# separate, opt-in job: it drives a real Chromium against a local fixture, and a
# runner without a browser should not make the whole pipeline red.
on:
# A push to the default branch, and every pull request. Add the branch name
# here if this repository defaults to something else.
push:
branches: [main, master]
pull_request:
workflow_dispatch:
schedule:
# Every night at 04:17 UTC, so the browser suite still runs on its own.
- cron: "17 4 * * *"
permissions:
contents: read
concurrency:
group: ci-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
env:
CARGO_TERM_COLOR: always
RUST_BACKTRACE: 1
jobs:
check:
name: fmt, clippy and tests (${{ matrix.os }})
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest]
steps:
- uses: actions/checkout@v4
- name: Install the Rust toolchain
uses: dtolnay/rust-toolchain@stable
with:
components: rustfmt, clippy
- name: Cache the build
uses: Swatinem/rust-cache@v2
- name: Check formatting
run: cargo fmt --all --check
- name: Lint every target
run: cargo clippy --all-targets --locked -- -D warnings
# Unit tests, binary tests, HTTP integration tests against the mock
# backend, and doc tests. Everything here runs without Chrome.
- name: Run the test suite
run: cargo test --locked --all-targets
- name: Run the doc tests
run: cargo test --locked --doc
- name: Build the release binary
run: cargo build --locked --release
browser-e2e:
name: browser end-to-end (Chromium + fixture)
runs-on: ubuntu-latest
# On demand, or with the nightly run: these tests need a browser and are the
# ones most sensitive to the runner image.
if: github.event_name == 'workflow_dispatch' || github.event_name == 'schedule'
steps:
- uses: actions/checkout@v4
- name: Install the Rust toolchain
uses: dtolnay/rust-toolchain@stable
- name: Cache the build
uses: Swatinem/rust-cache@v2
# The runner image ships Chrome; the tests find it through CHROME, which
# is the same override a user can set on a machine with an unusual layout.
- name: Locate a Chromium-based browser
run: |
for path in /usr/bin/google-chrome /usr/bin/google-chrome-stable /usr/bin/chromium /usr/bin/chromium-browser; do
if [ -x "$path" ]; then
"$path" --version
echo "CHROME=$path" >> "$GITHUB_ENV"
exit 0
fi
done
echo "no Chromium-based browser found on this runner" >&2
exit 1
# The suite is #[ignore]d by default so that "cargo test" stays usable on
# a machine without a browser.
- name: Run the browser suite
run: cargo test --locked --test e2e_browser -- --ignored --test-threads=1 --nocapture
.gitignore
+3
View File
@@ -0,0 +1,3 @@
/target
*.log
.demo-state/
Cargo.lock
Generated
+2742
View File
File diff suppressed because it is too large Load Diff
Cargo.toml
+57
View File
@@ -0,0 +1,57 @@
[package]
name = "llm-bridge"
version = "0.1.0"
edition = "2021"
description = "OpenAI-compatible local API gateway that drives a real authenticated browser to talk to LLM web UIs"
license = "MIT"
default-run = "llm-gateway"
rust-version = "1.85"
[[bin]]
name = "llm-gateway"
path = "src/main.rs"
[lib]
name = "llm_bridge"
path = "src/lib.rs"
[dependencies]
tokio = { version = "1", features = ["full"] }
axum = "0.8"
tower = "0.5"
tower-http = { version = "0.6", features = ["cors", "trace"] }
hyper = "1"
serde = { version = "1", features = ["derive"] }
serde_json = "1"
chromiumoxide = "0.9"
futures = "0.3"
async-trait = "0.1"
tokio-stream = "0.1"
async-stream = "0.3"
clap = { version = "4", features = ["derive", "env", "wrap_help"] }
tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter", "json"] }
tracing-appender = "0.2"
anyhow = "1"
thiserror = "2"
toml = "0.8"
uuid = { version = "1", features = ["v4", "v7", "serde"] }
notify = "8"
arc-swap = "1"
dirs = "6"
base64 = "0.22"
regex = "1"
sha2 = "0.10"
mime_guess = "2"
tempfile = "3"
reqwest = { version = "0.12", default-features = false, features = ["json", "rustls-tls"] }
tiktoken-rs = "0.6"
[dev-dependencies]
tower = { version = "0.5", features = ["util"] }
http-body-util = "0.1"
[profile.release]
opt-level = 3
lto = "thin"
strip = true
README.md
+498
View File
@@ -0,0 +1,498 @@
# llm-bridge — llm-gateway
A local, OpenAI-compatible HTTP API that answers by driving a **real, signed-in
browser session** on an LLM web UI instead of a paid API key.
```
OpenAI SDK / curl / Continue / Cursor
|
| POST /v1/chat/completions (OpenAI format)
v
llm-gateway (Axum, Rust)
|
| Chrome DevTools Protocol
v
Chrome/Chromium, dedicated profile, signed in once
|
v
chatgpt.com (or any configured chat web UI)
```
The client never knows a browser is involved: it sends OpenAI requests and
receives OpenAI responses, including SSE streaming.
## What it does
- `POST /v1/chat/completions` and `POST /v1/completions` (legacy), with `stream: true`.
- `GET /v1/models`, `GET /health`, `GET /v1/providers/{provider}/status`,
`POST /v1/providers/{provider}/validate`, `POST /v1/admin/reload`. Both
provider routes accept the `provider@account` spelling, so one account of a
provider can be checked on its own.
- Multi-turn continuity: each OpenAI conversation is mapped to a web
conversation, so only the newest message is sent once the thread exists.
- **Markdown fidelity**: once the answer has settled the gateway clicks the
provider's own copy button and reads the clipboard over CDP, so tables, code
fences and lists arrive intact instead of being flattened by `innerText`.
- **Several providers**: ChatGPT, Claude and DeepSeek ship as configuration
files; adding another one is a TOML file, not a code change.
- **Several accounts per provider**: each declared account gets its own browser
profile, session and tab pool, and is selected with `model = "chatgpt@perso"`.
- **Several completions per request**: `n > 1` is honoured, and the variants run
in parallel when the account has more than one tab (`max_tabs`).
- Optional local dashboard on `/dashboard` with live logs and manual tests.
- Every selector lives in `~/.llm-gateway/providers/<provider>.toml`, reloaded
live: adapting to a UI change is a configuration edit, not a rebuild.
## Requirements
- Rust 1.85 or newer (tested with 1.98).
- Chrome, Chromium, Edge or Brave. Firefox is **not** supported: it does not
speak CDP.
## Install and first run
```bash
cargo build --release
./target/release/llm-gateway config init # creates ~/.llm-gateway
./target/release/llm-gateway login chatgpt # sign in once, in the window that opens
./target/release/llm-gateway test chatgpt # send one prompt and print the answer
./target/release/llm-gateway serve # starts the API on 127.0.0.1:8080
```
Three providers are installed on the first run: `chatgpt`, `claude` and
`deepseek`. Each needs its own sign-in:
```bash
./target/release/llm-gateway login claude
./target/release/llm-gateway selftest claude # do the selectors still match?
```
The Claude and DeepSeek selectors were collected from public sources and have not
been validated against a signed-in session here: run the selftest first, and see
`providers/selectors.md` for which key to fix if the UI moved.
Then, with any OpenAI client:
```bash
# bash / zsh
curl -s http://127.0.0.1:8080/v1/chat/completions \
-H 'content-type: application/json' \
-d '{"model": "chatgpt", "messages": [{"role": "user", "content": "Bonjour"}]}'
```
```powershell
# PowerShell: single quotes are literal, so do NOT escape the double quotes.
# Writing '{\"model\":...}' sends the backslashes and the body is rejected with
# "Failed to parse the request body as JSON".
curl.exe -s http://127.0.0.1:8080/v1/chat/completions -H "content-type: application/json" `
-d '{"model":"chatgpt","messages":[{"role":"user","content":"Bonjour"}]}'
# Or, without any quoting puzzle at all:
$body = @{ model = "chatgpt"; messages = @(@{ role = "user"; content = "Bonjour" }) } |
ConvertTo-Json -Depth 5
Invoke-RestMethod -Uri http://127.0.0.1:8080/v1/chat/completions -Method Post `
-Body $body -ContentType application/json
# ./demo.ps1 walks through the whole API (health, models, streaming, status,
# validation, reload) so you do not have to type any of this by hand.
```
```python
from openai import OpenAI
client = OpenAI(base_url="http://127.0.0.1:8080/v1", api_key="unused")
print(client.chat.completions.create(
model="chatgpt",
messages=[{"role": "user", "content": "Bonjour"}],
).choices[0].message.content)
```
Streaming works the same way with `stream=True`.
## Command line
```
llm-gateway [GLOBAL OPTIONS] [COMMAND]
serve [--backend browser|mock] start the API server (default command)
test <provider[@account]>|--all [--prompt TEXT] [--json]
selftest <provider[@account]>|--all [--json]
list [--json]
login <provider[@account]> [--account ID] open a window to sign in once
logout <provider[@account]> [--account ID] [--all] delete the stored profile(s)
conversations list|clear [--provider NAME[@ACCOUNT]] [--account ID]
reload ask a running server to reload its configuration
config path|init|show
```
Global options: `--host`, `--port`, `--browser <chrome|chromium|edge|brave>`,
`--browser-path`, `--profile-dir`, `--provider`, `--headless`, `--log-level`,
`--log-format pretty|json`, `--config`, `--debug`, `--max-tabs`, `--api-key`,
`--markdown auto|clipboard|dom|text`, `--yes`.
Exit codes: `0` success, `1` runtime error, `2` selftest failure.
## Model names
`model` selects the provider:
| Value | Meaning |
|---|---|
| `chatgpt` | the provider, its default model |
| `chatgpt/gpt-4o` | the provider and a model name recorded in the response |
| `gpt-4o` | accepted when exactly one provider is configured |
The model is **echoed back** but the web UI keeps whatever model it has
selected; a warning is added to `x_gateway_warnings` when you ask for another
one. Driving the UI model picker is not implemented yet.
When a provider declares several accounts, the account is part of the same
string, between the provider and the model name:
| Value | Meaning |
|---|---|
| `chatgpt` | the provider, its default account |
| `chatgpt@perso` | the account whose id (or label) is `perso` |
| `[email protected]@gmail.com` | the account with that address |
| `chatgpt@perso/gpt-4o` | that account, and a model name |
`GET /v1/models` lists one entry per provider and per declared account, so a
client that only reads that endpoint still discovers them.
The same spelling works outside `model`, wherever a provider is named:
| Command | Effect |
|---|---|
| `test chatgpt@perso` / `selftest chatgpt@perso` | checks that one account |
| `test --all` / `selftest --all` | checks **every account** of every provider |
| `GET /v1/providers/chatgpt@perso/status` | the state of that account |
| `POST /v1/providers/chatgpt@pro/validate` | a full report for that account |
A bare provider name means its default account, and the answer says which one was
actually probed (`"provider": "chatgpt@perso"`), so two accounts of the same
provider are never confused in a report or in the logs.
## Multi-turn conversations
A client that resends the whole history (every OpenAI SDK does) is mapped to one
web conversation:
1. The conversation id comes from the `X-Conversation-Id` request header when
present, otherwise from a fingerprint of the system prompt and the first user
message (`conversation.id_source = "fingerprint"` in the config).
2. The first turn replays the whole history into the page.
3. Later turns reopen the stored web conversation and send **only the newest
message**.
4. The response always carries `X-Conversation-Id`, so a client can pin a
thread explicitly.
5. If the web conversation is gone, the turn automatically falls back to
replaying the history in a new one.
`~/.llm-gateway/conversations.json` is plain JSON you can read and edit:
```json
{
"version": 1,
"threads": {
"fp-3f1a...": {
"provider": "chatgpt",
"account": "perso",
"web_url": "https://chatgpt.com/c/68f0...",
"web_id": "68f0...",
"created_at": 1760000000,
"last_used_at": 1760000100,
"turns": 3,
"messages_sent": 7,
"state": "ready"
}
}
}
```
`llm-gateway conversations list` and `clear` inspect and prune it. The listing
shows the account that served each thread, and both commands accept
`--provider chatgpt@perso` or `--account perso` to narrow the selection;
`--account default` selects the threads of a provider that declares no account.
A thread is marked `incomplete` when a client disconnects mid-generation; the
next turn then replays the history instead of trusting a half-written thread.
## OpenAI parameter support
| Parameter | Behaviour |
|---|---|
| `model` | selects the provider (see above) |
| `messages` | string content or an array of `text` / `image_url` parts |
| `stream` | SSE chunks, `data: [DONE]` termination |
| `stream_options.include_usage` | adds a final usage chunk |
| `temperature`, `top_p`, `max_tokens`, `stop`, `response_format`, `seed`, `user`, penalties, `logprobs` | ignored, logged, and listed in `x_gateway_warnings` |
| `tools`, `functions`, `tool_choice` | **not supported**: HTTP 400 `unsupported_parameter` |
| `n > 1` | supported: `n` completions, each in its own web conversation. Clamped to `capture.max_variants` (4 by default) with a warning. Variants run in parallel when the account allows more than one tab |
`usage` is estimated with `tiktoken` when its vocabulary can be loaded, and with
a characters/4 heuristic otherwise (offline machines).
`/v1/completions` accepts a string or an array prompt and answers with the
legacy `text_completion` shape. Streaming is not available on that endpoint.
## Signing in with Google (or any OAuth provider that blocks automation)
Symptom: after `llm-gateway login chatgpt`, the Google window answers
"**This browser or app may not be secure**".
Why it happens: Google refuses to sign a browser it identifies as automated or
outdated. Two things mattered here, and the first one was a bug in this project:
1. The stealth helper used to advertise an **outdated Chrome user agent**
(`Chrome/107` in 2026). That alone reads as "old browser = insecure", which is
exactly what the message means. It is fixed: the default is now
`[browser] stealth = "off"` and the user agent is never spoofed.
2. Google can also refuse a browser it knows is being driven over CDP. That part
cannot be argued with, so the gateway offers a sign-in path where no
automation is involved at all.
### Recommended: sign in from a browser you start yourself
```bash
llm-gateway login chatgpt --manual # prints the exact command, or add --launch
```
It tells you to run something like:
```
"C:\Program Files\Google\Chrome\Application\chrome.exe" \
--user-data-dir="C:\Users\you\.llm-gateway\profiles\chatgpt\user-data-dir" \
--no-first-run --no-default-browser-check https://chatgpt.com/
```
Sign in with Google in that window: it is an ordinary Chrome launched by you,
with no automation switch and no debugging port. Then close it and press Enter.
The gateway relaunches that same profile for later requests, so the session is
already there.
### Alternative: keep that window and let the gateway drive it
Start the same command with `--remote-debugging-port=9222` (the `--manual` output
prints it) and keep it open, then:
```bash
llm-gateway serve --attach --debug-port 9222
```
The gateway attaches to your browser, reuses a tab it can see (otherwise it opens
one in that same browser, with your session), and **never closes your browser**
when it shuts down. If Google still refuses the sign-in with the debugging port
open, use the recommended path above.
### Also worth knowing
- Email + password sign-in on the provider site is not blocked by any of this;
only Google's OAuth screen applies these checks.
- `--stealth minimal` adds a `navigator.webdriver` patch on top of the default;
`--stealth aggressive` restores the old user-agent spoofing and is documented
as harmful for sign-in.
- When a request hits a signed-out page, the API answers with
`401 requires_login` and the message now mentions the `--manual` path.
## Configuration
`~/.llm-gateway/config.toml` (created on first run, see
`config.toml.example` for the annotated version). Precedence:
**CLI flags > `LLM_GATEWAY_*` environment variables > the file > defaults**.
The settings that matter most:
```toml
[browser]
executable = "" # empty: auto-detect Chrome, Edge or Brave
headless = false # keep it false so you can sign in and solve captchas
max_tabs = 1 # concurrent turns per provider *account*, i.e. tabs kept open
busy_wait_s = 30 # how long a request waits for a free tab before a 429
[conversation]
strategy = "auto" # auto | reuse | replay
id_source = "fingerprint" # fingerprint | header | uuid
[capture]
poll_interval_ms = 400 # DOM polling while waiting for the answer
quiet_ms = 1500 # silence after which the answer is considered final
response_timeout_s = 180 # hard budget for one answer
markdown = "auto" # auto | clipboard | dom | text (see below)
max_variants = 4 # upper bound accepted for the "n" parameter
```
`max_tabs` is the setting that turns the gateway from a queue into a pool: with
`max_tabs = 2` an account can answer two requests (or two variants of one `n = 2`
request) at the same time, in two tabs of the same browser.
`capture.markdown` decides where the answer text comes from:
| Value | Behaviour |
|---|---|
| `auto` | the provider's copy button (Markdown through the clipboard) when it has one, the DOM conversion otherwise |
| `clipboard` | always the copy button; falls back to the DOM when the clipboard cannot be read |
| `dom` | always convert the answer HTML to Markdown |
| `text` | the visible text only, formatting flattened (the historical behaviour) |
A provider may override it in its own `[input]` section. Streamed deltas always
stay plain text: only the settled answer is re-read with full fidelity.
Provider files (`~/.llm-gateway/providers/*.toml`) hold the URLs, the CSS
selectors, the timeouts and the capabilities of each web UI. They are watched:
editing one reloads it within a second, without restarting the server or the
browser. See `providers/selectors.md`.
They are **copies** created on the first run, so upgrading the binary does not
change them. Refresh them after an upgrade with
`llm-gateway config init --force` (the previous file is kept next to it as
`<provider>.toml.bak`).
`llm-gateway --version` prints the compilation date
(`0.1.0 (built 2026-09-18T17:26:11Z)`): if it does not match your sources, run
`cargo build --release` again.
## Error mapping
Errors use the OpenAI error shape and stable codes:
| Situation | HTTP | code |
|---|---|---|
| Unknown model | 404 | `model_not_found` |
| Unknown account in the model string | 404 | `account_not_found` (the message lists the known ones) |
| Bad request, ignored-but-unsupported parameter | 400 | `invalid_request`, `unsupported_parameter` |
| Images sent to a provider without upload support | 400 | `unsupported_content_type` |
| Missing or wrong API key | 401 | `invalid_api_key` |
| Signed out | 401 | `requires_login` |
| Captcha challenge | 403 | `captcha_required` |
| Browser profile locked by another instance | 409 | `browser_profile_locked` |
| Tab busy (30 s wait) or upstream throttling | 429 | `provider_busy`, `upstream_rate_limit` |
| Selector gone, upstream error | 500 | `selector_missing`, `upstream_error` |
| Browser missing, provider misconfigured | 503 | `browser_unavailable`, `provider_misconfigured` |
| No answer in time | 504 | `upstream_timeout` |
On a failure the server saves a screenshot and the full DOM in
`~/.llm-gateway/debug/`, plus one line per failure in `debug/index.jsonl`
(`--debug` does it for every request).
## Dashboard
`http://127.0.0.1:8080/dashboard` shows the providers, their live state, the
recent logs (SSE) and lets you send a prompt by hand. If an API key is
configured, paste it in the header field.
## Development
```bash
cargo fmt --all --check
cargo clippy --all-targets -- -D warnings
cargo test # no browser, no network
./scripts/e2e.ps1 # real Chrome, local fixture page
./scripts/ci.ps1 # all of the above, in one go
./scripts/ci.ps1 -E2e # ... plus the browser suite
```
`.github/workflows/ci.yml` runs the same gates on Linux and Windows for every
push and pull request, and builds the release binary. The browser suite needs a
real Chromium, so it is a separate job that runs on demand (or nightly) instead
of blocking a machine that has no browser.
- `tests/api_integration.rs` drives the whole HTTP surface through a mock backend,
including multi-account routing and `n > 1`.
- `tests/e2e_browser.rs` drives a real Chrome against
`tests/fixtures/chatgpt_mock.html`, a static fake of the ChatGPT DOM with
login, captcha, rate-limit, error, timeout, rewrite and attachment modes. That
fixture is what makes the capture loop regression-testable without a network
or an account.
- `demo.ps1` / `demo.sh` run a full cycle; add `-Mock` (or `--mock`) to run
without a browser or a login.
## Limits and honest caveats
- **Streamed deltas are best effort.** The final answer written into a
non-streaming response is always the exact captured text. If the web UI
rewrites text it has already shown (React re-render, code block highlighting),
a streaming client may have received text that the final answer no longer
contains. A warning is logged when that happens.
- **Markdown fidelity depends on the provider.** The clipboard path is exact,
because the provider copies what it rendered; the DOM conversion is a faithful
approximation that covers the constructs a chat UI actually produces. A page
that refuses the clipboard read (or a provider with no copy button) silently
falls back to it. Only the settled answer is re-read: streamed deltas stay
plain text.
- **One turn at a time per provider account** by default: a second request waits
up to `browser.busy_wait_s` (30 s) and then gets a 429. Raise `max_tabs` to
let an account answer several requests at once, in several tabs.
- **Only one tab per browser is rendered** by Chrome. On the other tabs the
prompt is submitted with in-page events rather than a synthetic mouse click,
which every chat UI tested here accepts, and the clipboard read still works
because the permission is granted over CDP.
- **Selectors are the fragile part.** The fixture page covers regressions in our
capture loop; it cannot predict a change on the real site. Run
`llm-gateway selftest` after a UI update. The Claude and DeepSeek files are
starting points collected from public sources, not validated against a
signed-in session.
- **No tool calling, no model switching.**
- Latency is browser latency: a fresh conversation takes a few seconds to start
generating, plus the page load.
## Security and terms of use
- The API binds to `127.0.0.1` by default. If you expose it on a network,
configure `server.api_key` (or `--api-key`) and put it behind TLS.
- Browser profiles contain live session cookies: they live in
`~/.llm-gateway/profiles/<provider>` and are never encrypted by this tool.
Use disk encryption and `llm-gateway logout --all` when you are done.
- The tool only uses **your own** signed-in subscriptions. It does not bypass
paywalls, quotas or captchas: when a challenge appears it stops and tells you.
- You remain responsible for complying with each provider's terms of service.
Automating a web UI may be restricted by them; check before you rely on it.
## Layout of the state directory
```
~/.llm-gateway/
config.toml global configuration
providers/ one TOML per provider (selectors, URLs, timeouts)
profiles/<provider>/user-data-dir/ persistent Chrome profile
profiles/<provider>/accounts/<account>/user-data-dir/ one per declared account
conversations.json client conversation -> web conversation mapping
debug/ screenshots, DOM dumps, index.jsonl
logs/llm-gateway.log JSON logs
```
## Providers and accounts
| Provider | File | Models | Notes |
|---|---|---|---|
| OpenAI ChatGPT | `providers/chatgpt.toml` | `gpt-4o`, `gpt-4o-mini`, `gpt-4.1` | `#prompt-textarea`, per-answer copy button |
| Anthropic Claude | `providers/claude.toml` | `claude-sonnet-4`, `claude-opus-4`, `claude-haiku-4` | ProseMirror editor; selectors from public sources |
| DeepSeek | `providers/deepseek.toml` | `deepseek-chat`, `deepseek-reasoner` | `textarea#chat-input`, `.ds-markdown` answers |
An account is a browser profile: declaring two ChatGPT accounts gives two
independent sessions, each with its own tab pool, driven side by side.
```toml
[[accounts]]
id = "perso"
email = "[email protected]"
default = true
[[accounts]]
id = "pro"
email = "[email protected]"
```
```bash
llm-gateway login chatgpt --account perso
llm-gateway login chatgpt@pro
llm-gateway list # shows every account and its profile
llm-gateway logout chatgpt@pro # forget one account
```
The model string then carries the account:
`{"model": "chatgpt@pro"}`, or `{"model": "[email protected]@gmail.com"}`,
or `{"model": "chatgpt@pro/gpt-4o"}`. Each account keeps its own thread store
entries, so the same prompt sent to two accounts never shares one web
conversation. Adding accounts to a provider that was already signed in changes
its profile layout (`profiles/<provider>/accounts/<id>/…`), so sign that
provider in again once.
build.rs
+44
View File
@@ -0,0 +1,44 @@
//! Build stamp: lets `llm-gateway --version` tell you when the binary was built.
//!
//! Without it, a stale release binary looks exactly like a fresh one, which has
//! already caused a confusing "unknown argument" report.
use std::time::{SystemTime, UNIX_EPOCH};
fn main() {
let seconds = SystemTime::now()
.duration_since(UNIX_EPOCH)
.map(|elapsed| elapsed.as_secs())
.unwrap_or(0);
println!(
"cargo:rustc-env=LLM_GATEWAY_BUILD_TIME={}",
format_utc(seconds)
);
}
/// `YYYY-MM-DDTHH:MM:SSZ`, without pulling a date crate into the build script.
fn format_utc(seconds: u64) -> String {
let days = (seconds / 86_400) as i64;
let rem = seconds % 86_400;
// Howard Hinnant's civil-from-days algorithm.
let z = days + 719_468;
let era = if z >= 0 { z } else { z - 146_096 } / 146_097;
let doe = z - era * 146_097;
let yoe = (doe - doe / 1460 + doe / 36_524 - doe / 146_096) / 365;
let mut year = yoe + era * 400;
let doy = doe - (365 * yoe + yoe / 4 - yoe / 100);
let mp = (5 * doy + 2) / 153;
let day = doy - (153 * mp + 2) / 5 + 1;
let month = if mp < 10 { mp + 3 } else { mp - 9 };
if month <= 2 {
year += 1;
}
format!(
"{year:04}-{month:02}-{day:02}T{:02}:{:02}:{:02}Z",
rem / 3600,
(rem % 3600) / 60,
rem % 60
)
}
config.toml.example
+109
View File
@@ -0,0 +1,109 @@
# llm-gateway global configuration.
# Copied to ~/.llm-gateway/config.toml on first run; edits there win.
# Precedence: CLI flags > LLM_GATEWAY_* environment variables > this file > defaults.
[server]
# Bind address. Keep it on the loopback interface unless you set an api_key.
host = "127.0.0.1"
port = 8080
# Optional bearer token required from clients. Empty disables authentication.
api_key = ""
# Adds an "x_gateway_warnings" array to responses describing ignored parameters.
include_warnings = true
# Maximum accepted request body, in bytes.
request_body_limit = 33554432
[browser]
# Path to the Chrome/Chromium/Edge/Brave executable. Empty = auto-detect
# (honours the CHROME environment variable, then well-known install paths).
executable = ""
# Headless hides the window; keep it false so you can log in and solve captchas.
headless = false
# Root of the persistent browser profiles. Empty = <state dir>/profiles.
profile_dir = ""
# Concurrent turns allowed per provider account, i.e. how many tabs stay open.
# 1 serialises the turns of an account; a larger value lets several requests
# (or the variants of one "n": 4 request) run in parallel.
max_tabs = 1
# How long a request waits for a free tab before it is refused with 429.
busy_wait_s = 30
launch_timeout_s = 60
request_timeout_s = 60
# Extra Chromium command line switches.
extra_args = [
"--disable-blink-features=AutomationControlled",
"--no-first-run",
"--no-default-browser-check",
]
# How much the browser hides that it is automated:
# off leave the page alone (default). The switch above already hides
# navigator.webdriver and the user agent stays genuine, which is
# what sign-in pages expect.
# minimal additionally patch navigator.webdriver from an injected script.
# aggressive full stealth, including a spoofed user agent. Warning: it
# advertises an outdated Chrome version, which some sign-in
# providers reject as an insecure browser.
stealth = "off"
# Attach to a browser you started instead of launching one. Use it when a
# sign-in provider refuses a browser the gateway launched: start Chrome
# yourself with a debugging port (llm-gateway login <provider> --manual prints
# the exact command), keep it open, and set attach = true.
attach = false
debug_port = 9222
[conversation]
# Thread store file. Empty = <state dir>/conversations.json.
store = ""
# auto = reuse the web conversation when possible, otherwise replay the history
# reuse = always reuse (fails if the web conversation is gone)
# replay = always send the full history into a new conversation
strategy = "auto"
# fingerprint = derive an id from the opening messages (works with header-less clients)
# header = only trust the X-Conversation-Id header (else a fresh uuid per request)
# uuid = never reuse a conversation unless the header is supplied
id_source = "fingerprint"
[capture]
# DOM polling interval while waiting for the answer.
poll_interval_ms = 400
# The answer is considered finished after this much silence and no streaming indicator.
quiet_ms = 1500
# Hard budget for one generated answer.
response_timeout_s = 180
# auto | insert_text | exec_command | type_str
inject_method = "auto"
# Fidelity of the captured answer:
# auto the provider's copy button (Markdown through the clipboard) when
# it has one, the DOM conversion otherwise. Default.
# clipboard always the copy button; falls back to the DOM when the clipboard
# cannot be read.
# dom always convert the answer HTML to Markdown.
# text the visible text only, formatting flattened (historical behaviour).
markdown = "auto"
# How long the clipboard read is given before the DOM conversion wins.
clipboard_timeout_ms = 1500
# Upper bound accepted for the OpenAI "n" parameter. Values above it are clamped
# and reported in x_gateway_warnings.
max_variants = 4
[tokens]
# tiktoken encoding used for the estimated usage block.
encoding = "o200k_base"
# Fall back to a characters/4 heuristic when the encoding cannot be loaded.
fallback_heuristic = true
[debug]
# never | on_error | always
screenshots = "on_error"
# Empty = <state dir>/debug.
dir = ""
[logging]
# trace | debug | info | warn | error
level = "info"
# pretty | json
format = "pretty"
# Additional JSON log file. Empty = <state dir>/logs/llm-gateway.log
file = ""
demo.ps1
+140
View File
@@ -0,0 +1,140 @@
<#
.SYNOPSIS
End to end demonstration of llm-gateway.
.DESCRIPTION
Starts the API server on a spare port, runs a full cycle against it
(health, models, non-streaming completion, streaming completion, provider
status, validation, configuration reload) and stops the server.
Without -Mock the server talks to a real browser: log in first with
"llm-gateway login chatgpt" and make sure the provider is configured.
With -Mock everything runs in memory, no browser and no login needed.
.EXAMPLE
./demo.ps1 -Mock
./demo.ps1 -Port 8099
#>
param(
[switch]$Mock,
[int]$Port = 8099,
[string]$Provider = 'chatgpt'
)
$ErrorActionPreference = 'Stop'
$root = Split-Path -Parent $MyInvocation.MyCommand.Path
Push-Location $root
$stateDir = Join-Path $root '.demo-state'
New-Item -ItemType Directory -Force -Path $stateDir | Out-Null
$env:LLM_GATEWAY_HOME = $stateDir
$backend = if ($Mock) { 'mock' } else { 'browser' }
$base = "http://127.0.0.1:$Port"
Write-Host "== building ==" -ForegroundColor Cyan
cargo build --quiet
if ($LASTEXITCODE -ne 0) { throw 'cargo build failed' }
$exe = Join-Path $root 'target/debug/llm-gateway.exe'
Write-Host "== starting llm-gateway (backend: $backend, port: $Port) ==" -ForegroundColor Cyan
$server = Start-Process -FilePath $exe -ArgumentList @('serve', '--port', "$Port", '--backend', $backend) -PassThru -NoNewWindow
try {
$health = $null
foreach ($attempt in 1..60) {
try {
$health = Invoke-RestMethod -Uri "$base/health" -TimeoutSec 2
break
} catch {
Start-Sleep -Milliseconds 500
}
}
if (-not $health) { throw "the server did not become healthy on $base" }
Write-Host "
== /health ==" -ForegroundColor Green
$health | ConvertTo-Json -Depth 6
Write-Host "
== /v1/models ==" -ForegroundColor Green
(Invoke-RestMethod -Uri "$base/v1/models").data | ForEach-Object { " " + $_.id }
$payload = @{
model = $Provider
messages = @(@{ role = 'user'; content = 'Say ok and nothing else.' })
} | ConvertTo-Json -Depth 5
Write-Host "
== POST /v1/chat/completions (non-streaming) ==" -ForegroundColor Green
$answer = Invoke-RestMethod -Uri "$base/v1/chat/completions" -Method Post -Body $payload -ContentType 'application/json'
" id : " + $answer.id
" model : " + $answer.model
" answer : " + $answer.choices[0].message.content
" usage : prompt " + $answer.usage.prompt_tokens + ", completion " + $answer.usage.completion_tokens
Write-Host "
== POST /v1/chat/completions (two completions) ==" -ForegroundColor Green
$variantsPayload = @{
model = $Provider
n = 2
messages = @(@{ role = 'user'; content = 'Answer with one word.' })
} | ConvertTo-Json -Depth 5
$variants = Invoke-RestMethod -Uri "$base/v1/chat/completions" -Method Post -Body $variantsPayload -ContentType 'application/json'
" choices : " + $variants.choices.Count
$variants.choices | ForEach-Object { " [" + $_.index + "] " + $_.message.content }
if ($variants.x_gateway_warnings) { " warning : " + $variants.x_gateway_warnings[0] }
$accounts = ($health.providers | Where-Object { $_.name -eq $Provider }).accounts
if ($accounts) {
Write-Host "
== accounts of $Provider ==" -ForegroundColor Green
$accounts | ForEach-Object {
" " + $_.id + " " + $_.email + $(if ($_.default) { " (default)" } else { "" })
}
$account = $accounts[0].id
$accountPayload = @{
model = "$Provider@$account"
messages = @(@{ role = 'user'; content = 'Say ok and nothing else.' })
} | ConvertTo-Json -Depth 5
$accounted = Invoke-RestMethod -Uri "$base/v1/chat/completions" -Method Post -Body $accountPayload -ContentType 'application/json'
" model $Provider@$account answered: " + $accounted.choices[0].message.content
}
Write-Host "
== POST /v1/chat/completions (streaming) ==" -ForegroundColor Green
$streamPayload = @{
model = $Provider
stream = $true
messages = @(@{ role = 'user'; content = 'Count from one to five.' })
} | ConvertTo-Json -Depth 5
$streamed = Invoke-WebRequest -Uri "$base/v1/chat/completions" -Method Post -Body $streamPayload -ContentType 'application/json'
$streamed.Content -split "`n" | Select-Object -First 12 | ForEach-Object { " " + $_ }
" ... ($((($streamed.Content -split "`n") | Measure-Object).Count) lines of SSE)"
Write-Host "
== GET /v1/providers/$Provider/status ==" -ForegroundColor Green
Invoke-RestMethod -Uri "$base/v1/providers/$Provider/status" | ConvertTo-Json -Depth 5
Write-Host "
== POST /v1/providers/$Provider/validate ==" -ForegroundColor Green
Invoke-RestMethod -Uri "$base/v1/providers/$Provider/validate" -Method Post | ConvertTo-Json -Depth 5
Write-Host "
== POST /v1/admin/reload ==" -ForegroundColor Green
Invoke-RestMethod -Uri "$base/v1/admin/reload" -Method Post | ConvertTo-Json -Depth 5
Write-Host "
== dashboard ==" -ForegroundColor Green
" $base/dashboard"
" state directory: $stateDir"
}
finally {
Write-Host "
== stopping the server ==" -ForegroundColor Cyan
if ($server -and -not $server.HasExited) {
Stop-Process -Id $server.Id -Force
$server.WaitForExit(5000) | Out-Null
}
Pop-Location
}
demo.sh
+91
View File
@@ -0,0 +1,91 @@
#!/usr/bin/env bash
# End to end demonstration of llm-gateway (bash / curl equivalent of demo.ps1).
#
# ./demo.sh --mock
# ./demo.sh --port 8099 --provider chatgpt
set -euo pipefail
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
cd "$ROOT"
PORT=8099
PROVIDER=chatgpt
MOCK=0
while [[ $# -gt 0 ]]; do
case "$1" in
--mock) MOCK=1; shift ;;
--port) PORT="$2"; shift 2 ;;
--provider) PROVIDER="$2"; shift 2 ;;
*) echo "unknown option: $1" >&2; exit 1 ;;
esac
done
BACKEND="browser"
[[ "$MOCK" == "1" ]] && BACKEND="mock"
BASE="http://127.0.0.1:$PORT"
STATE="$ROOT/.demo-state"
mkdir -p "$STATE"
# shellcheck disable=SC2030
export LLM_GATEWAY_HOME="$STATE"
echo "== building =="
cargo build --quiet
EXE="$ROOT/target/debug/llm-gateway"
echo "== starting llm-gateway (backend: $BACKEND, port: $PORT) =="
"$EXE" serve --port "$PORT" --backend "$BACKEND" &
SERVER_PID=$!
trap 'echo; echo "== stopping the server =="; kill "$SERVER_PID" 2>/dev/null || true' EXIT
for _ in $(seq 1 60); do
if curl -sf "$BASE/health" >/dev/null 2>&1; then break; fi
sleep 0.5
done
echo
echo "== /health =="
curl -s "$BASE/health" | python3 -m json.tool 2>/dev/null || curl -s "$BASE/health"
echo
echo "== /v1/models =="
curl -s "$BASE/v1/models"
echo
echo "== POST /v1/chat/completions (non-streaming) =="
curl -s "$BASE/v1/chat/completions" \
-H 'content-type: application/json' \
-d "{\"model\":\"$PROVIDER\",\"messages\":[{\"role\":\"user\",\"content\":\"Say ok and nothing else.\"}]}"
echo
echo "== POST /v1/chat/completions (two completions) =="
curl -s "$BASE/v1/chat/completions" \
-H 'content-type: application/json' \
-d "{\"model\":\"$PROVIDER\",\"n\":2,\"messages\":[{\"role\":\"user\",\"content\":\"Answer with one word.\"}]}"
echo
echo "== accounts of $PROVIDER (when the provider file declares some) =="
curl -s "$BASE/health" | grep -o '"accounts":\[[^]]*\]' || echo " none declared"
echo
echo "== POST /v1/chat/completions (streaming, first lines) =="
curl -sN "$BASE/v1/chat/completions" \
-H 'content-type: application/json' \
-d "{\"model\":\"$PROVIDER\",\"stream\":true,\"messages\":[{\"role\":\"user\",\"content\":\"Count from one to five.\"}]}" | head -n 12 || true
echo
echo "== GET /v1/providers/$PROVIDER/status =="
curl -s "$BASE/v1/providers/$PROVIDER/status"
echo
echo "== POST /v1/providers/$PROVIDER/validate =="
curl -s -X POST "$BASE/v1/providers/$PROVIDER/validate"
echo
echo "== POST /v1/admin/reload =="
curl -s -X POST "$BASE/v1/admin/reload"
echo
echo "== dashboard =="
echo " $BASE/dashboard"
echo " state directory: $STATE"
providers/chatgpt.toml
+86
View File
@@ -0,0 +1,86 @@
# ChatGPT provider configuration, driven entirely by this file.
# Selectors change when the web UI is updated: edit them here (reloaded live),
# never in the Rust source. Validate with: llm-gateway selftest chatgpt
[provider]
name = "chatgpt"
display_name = "OpenAI ChatGPT"
web_url = "https://chatgpt.com/"
new_conversation_url = "https://chatgpt.com/"
# Captures the web conversation id from the current URL (one capture group).
conversation_url_pattern = "^https://chatgpt\\.com/c/([0-9a-zA-Z-]{8,})"
default_models = ["gpt-4o", "gpt-4o-mini", "gpt-4.1"]
enabled = true
[selectors]
input_field = "div#prompt-textarea, div.ProseMirror[contenteditable='true']"
# The id and the data-testid are language independent; the aria-label fallbacks
# only match an English UI (see providers/selectors.md, "Localised user
# interfaces").
send_button = "button[data-testid='send-button'], button#composer-submit-button, button[aria-label='Send prompt'], button[aria-label='Envoyer la requête']"
response_container = "div[data-message-author-role='assistant']"
streaming_indicator = "button[data-testid='stop-button']"
file_input = "input[type='file']"
new_chat_button = "a[data-testid='create-new-chat-button']"
# Clicked once the answer has settled, to read the provider's own Markdown
# through the clipboard. Looked up inside the answer node, so it must be
# relative to it. Leave empty to always convert the DOM instead.
copy_button = "button[data-testid='copy-turn-action-button'], button[aria-label='Copy'], button[aria-label='Copier']"
login_page_indicator = "button[data-testid='login'], form[action*='login'], a[href*='auth/login'], a[href*='auth/0']"
captcha_indicator = "iframe[src*='captcha'], iframe[src*='challenges.cloudflare.com'], div#challenge-form"
error_banner = "[data-testid='error-message'], div[role='alert']"
[login]
url_patterns = ["/auth/login", "/login", "accounts.google.com", "/auth/0"]
# The page a signed-out visitor lands on (title "ChatGPT: Chat, Work, Create &
# Code with AI") has no prompt field and shows these call to action buttons.
# The text patterns are only used when that field is missing, so a signed-in
# conversation is never mistaken for a sign-in screen.
text_patterns = ["log in", "sign up", "create an account"]
[rate_limit]
text_patterns = [
"too many requests",
"rate limit",
"trop de requêtes",
"réessayez plus tard",
"you've reached",
]
[timeouts]
page_load = 30
response_generation = 120
response_timeout = 180
[input]
poll_interval_ms = 400
quiet_ms = 1500
inject_method = "auto"
[options]
supports_images = true
supports_streaming = true
supports_new_chat = true
# Multi-account. A ChatGPT account is a browser profile, so declaring two
# accounts gives two independent sessions, each with its own tab pool, driven
# side by side. Address them with the model string:
#
# {"model": "chatgpt"} -> the default account
# {"model": "chatgpt@perso"} -> by account id
# {"model": "[email protected]@gmail.com"} -> by address
# {"model": "chatgpt@perso/gpt-4o"} -> account and model
#
# Sign each one in once: llm-gateway login chatgpt --account perso
# Their profiles live in profiles/chatgpt/accounts/<account>/user-data-dir.
#
# [[accounts]]
# id = "perso"
# email = "[email protected]"
# label = "Personal"
# default = true
#
# [[accounts]]
# id = "pro"
# email = "[email protected]"
# label = "Work"
providers/claude.toml
+79
View File
@@ -0,0 +1,79 @@
# Claude (claude.ai) provider configuration, driven entirely by this file.
# Selectors change when the web UI is updated: edit them here (reloaded live),
# never in the Rust source. Validate with: llm-gateway selftest claude
#
# The values below come from the public claude.ai DOM observed in 2026 and from
# published selector lists (May 2026). They are a starting point, not a
# guarantee: run 'llm-gateway selftest claude' before relying on them.
[provider]
name = "claude"
display_name = "Anthropic Claude"
web_url = "https://claude.ai/new"
new_conversation_url = "https://claude.ai/new"
# Captures the web conversation id from the current URL (one capture group).
conversation_url_pattern = '^https://claude\.ai/chat/([0-9a-zA-Z-]{8,})'
default_models = ["claude-sonnet-4", "claude-opus-4", "claude-haiku-4"]
enabled = true
[selectors]
# Claude uses ProseMirror, so the editor is a contenteditable div rather than a
# textarea. The fallbacks cover the composer variants seen behind A/B tests.
input_field = "div.ProseMirror[contenteditable='true'], div[contenteditable='true'][data-placeholder], [data-testid='composer-input']"
send_button = "button[aria-label='Send Message'], button[aria-label='Send message'], button[data-testid='send-button'], button[type='submit']"
# One node per assistant answer, in document order.
response_container = "div.font-claude-response, div[data-is-streaming], div.font-claude-message"
streaming_indicator = "button[aria-label='Stop response'], button[aria-label='Stop Response']"
file_input = "input[type='file']"
new_chat_button = "a[href='/new'], button[aria-label='New chat']"
login_page_indicator = "a[href*='/login'], button[data-testid='login-button'], form[action*='login']"
captcha_indicator = "iframe[src*='challenges.cloudflare.com'], iframe[title*='challenge'], div#challenge-form"
error_banner = "[role='alert'], div[data-testid='error']"
# Relative to the answer node: the hover toolbar button that copies the answer
# as Markdown.
copy_button = "button[aria-label='Copy'], button[data-testid='copy-button']"
[login]
url_patterns = ["/login", "/magic-link", "accounts.google.com"]
# Only consulted when the prompt field is missing, so a chat page that happens
# to mention "log in" is never misclassified.
text_patterns = ["sign in", "log in", "continue with google"]
[rate_limit]
text_patterns = [
"message limit",
"rate limit",
"too many requests",
"you've reached",
"limite de messages",
"trop de requêtes",
"réessayez plus tard",
]
[timeouts]
page_load = 30
response_generation = 120
response_timeout = 180
[input]
poll_interval_ms = 400
quiet_ms = 1500
inject_method = "auto"
[options]
supports_images = true
supports_streaming = true
supports_new_chat = true
# Multi-account: each account is a separate browser profile, so several
# subscriptions can be driven side by side. Declare them like this and address
# them with model = "claude@work".
#
# [[accounts]]
# id = "perso"
# email = "[email protected]"
# default = true
#
# [[accounts]]
# id = "work"
# email = "[email protected]"
providers/deepseek.toml
+73
View File
@@ -0,0 +1,73 @@
# DeepSeek (chat.deepseek.com) provider configuration, driven entirely by this
# file. Selectors change when the web UI is updated: edit them here (reloaded
# live), never in the Rust source. Validate with: llm-gateway selftest deepseek
#
# The values below combine the public chat.deepseek.com DOM with published
# selector lists (May 2026). They are a starting point, not a guarantee: run
# 'llm-gateway selftest deepseek' before relying on them.
[provider]
name = "deepseek"
display_name = "DeepSeek"
web_url = "https://chat.deepseek.com/"
new_conversation_url = "https://chat.deepseek.com/"
# Captures the web conversation id from the current URL (one capture group).
conversation_url_pattern = '^https://chat\.deepseek\.com/a/chat/s/([0-9a-zA-Z-]{8,})'
default_models = ["deepseek-chat", "deepseek-reasoner"]
enabled = true
[selectors]
# The composer is a plain <textarea>. DeepSeek localises its placeholder and the
# id seen in older builds is gone, so the wording observed on the live page comes
# first and the older guesses are kept as fallbacks.
input_field = "textarea#chat-input, textarea[placeholder*='Message DeepSeek'], textarea[placeholder*='Envoyer'], textarea[placeholder*='Send a message'], textarea[placeholder*='message']"
send_button = "div[role='button'][aria-label='Send'], button[aria-label='Send message'], button[data-testid='send-button']"
# The answer body is rendered by the "ds-markdown" component; the attribute
# first variant mirrors the ChatGPT markup.
response_container = "div[data-message-author-role='assistant'], div.ds-markdown, div[class*='markdown-body']"
streaming_indicator = "div[role='button'][aria-label*='Stop'], button[aria-label*='Stop'], div[class*='_stop']"
file_input = "input[type='file']"
new_chat_button = "a[href='/'], div[class*='new-chat'], button[aria-label='New chat']"
login_page_indicator = "div[class*='ds-modal'], a[href*='sign_in'], button[data-testid='login-button']"
captcha_indicator = "iframe[src*='captcha'], div[class*='captcha'], div#challenge-form"
error_banner = "[role='alert'], div[class*='ds-toast'], div[class*='error']"
# Relative to the answer node: the toolbar button that copies the answer.
copy_button = "[aria-label='Copy'], [data-testid='copy-button'], div[class*='ds-icon-button'][role='button']"
[login]
url_patterns = ["/sign_in", "/login", "account.deepseek.com"]
text_patterns = ["log in", "sign in", "log in with google", "se connecter"]
[rate_limit]
text_patterns = [
"too many requests",
"rate limit",
"please try again later",
"请求过于频繁",
"服务器繁忙",
"trop de requêtes",
]
[timeouts]
page_load = 30
response_generation = 120
# DeepSeek R1 reasons for a long time before answering: keep a generous budget.
response_timeout = 300
[input]
poll_interval_ms = 400
quiet_ms = 1500
inject_method = "auto"
[options]
supports_images = true
supports_streaming = true
supports_new_chat = true
# Multi-account: each account is a separate browser profile. Address them with
# model = "deepseek@perso".
#
# [[accounts]]
# id = "perso"
# email = "[email protected]"
# default = true
providers/selectors.md
+169
View File
@@ -0,0 +1,169 @@
# Provider selectors
Every selector lives in a provider configuration file
(`~/.llm-gateway/providers/<provider>.toml`), never in the Rust source. When a
web UI changes, edit the TOML: the server picks the change up within a second
(no restart, no browser relaunch). `llm-gateway selftest <provider>` tells you
whether the current selectors still work.
## Why this file exists
Web UIs change without notice. The three failure modes we care about are:
| Symptom | Reported as | What to do |
|---|---|---|
| The prompt field disappeared | HTTP 500, code `selector_missing` | Update `selectors.input_field` |
| The send button disappeared | HTTP 500, code `selector_missing` | Update `selectors.send_button` |
| The answer container moved | empty answer or HTTP 500 | Update `selectors.response_container` |
| The streaming indicator is wrong | answers truncated or 504 | Update `selectors.streaming_indicator` |
| The session expired | HTTP 401, code `requires_login` | Run `llm-gateway login <provider>` |
A failing capture always leaves a screenshot and a DOM dump in
`~/.llm-gateway/debug/`, plus one line per failure in `debug/index.jsonl`.
## How to update a selector
1. Open the provider page in a normal browser, signed in.
2. Open the developer tools, inspect the element you need.
3. Prefer stable hooks in this order: `data-testid` > `id` > semantic attributes
(`aria-label`, `role`, `type`) > a CSS class. Avoid generated class names such
as `.css-1x2y3z`: they change on every deploy.
4. Put a comma separated list in the TOML when a UI has several variants
(`div#prompt-textarea, div.ProseMirror[contenteditable='true']`): the first
match wins and the fallbacks keep you running through a partial rollout.
5. Run `llm-gateway selftest <provider>` and keep the TOML only when it reports
`ok`.
## Required and optional selectors
| Key | Required | Used for |
|---|---|---|
| `input_field` | yes | focusing the editor, verifying the injected text, detecting an empty editor after submit |
| `send_button` | yes | submitting the prompt (Enter is the fallback) |
| `response_container` | yes | one node per assistant answer, in document order; the driver reads the n-th node. It must select the **answer text**, not the whole exchange: a container that also holds the action toolbar would capture the word "Copy" as part of the answer |
| `copy_button` | no | the per-answer "copy" control, which gives the answer as Markdown through the clipboard. It may be inside the answer node or a sibling of it: the driver looks for it there and never elsewhere on the page, so the copy button of a previous answer is never clicked |
| `streaming_indicator` | recommended | element that only exists while the answer is generated (a stop button) |
| `file_input` | for images | hidden input fed through `DOM.setFileInputFiles` |
| `attachment_indicator` | optional | chip shown once an upload has been ingested |
| `new_chat_button` | optional | starting a brand new conversation without reloading |
| `login_page_indicator` | recommended | detecting a sign-in page even when the URL does not say so. On chatgpt.com the signed-out landing page exposes `button[data-testid='login']` |
| `[login] text_patterns` | recommended | lowercase substrings of the visible text that reveal a signed-out page; only consulted when the prompt field is missing, so a chat page that mentions "log in" is never misclassified |
| `captcha_indicator` | recommended | detecting a bot challenge |
| `error_banner` | recommended | surfacing the upstream error message to the API client |
| `model_selector` | reserved | not driven yet: the gateway never changes the model in the UI |
## Answer fidelity (Markdown)
A web UI renders its answer as rich HTML and `innerText` flattens it: a table
collapses into loose lines and a code block loses its fences. Once the answer has
settled the driver therefore reads it a second time, in this order:
1. **the clipboard** — it clicks the provider's own copy button and reads the
clipboard over CDP (`Browser.setPermission` then
`navigator.clipboard.readText()`). The provider copies exactly the Markdown it
rendered, so nothing is guessed. Requires `selectors.copy_button`.
2. **the DOM** — it converts the answer node's `innerHTML` to Markdown
(headings, lists, quotes, links, images, fenced code, GitHub tables).
3. **the visible text** — the historical behaviour, used when both fail.
`[capture] markdown` chooses between them and a provider may override it in its
own `[input]` section. Streamed deltas always come from the DOM, so they stay
plain text; the final answer — the one a non-streaming client receives — is the
converted one, and the source used is logged as `fidelity`.
## Current status
| Provider | File | Last validated | Status |
|---|---|---|---|
| OpenAI ChatGPT | `providers/chatgpt.toml` | 2026-09-18, signed-in session | `selftest` ok; `fidelity = "clipboard"` |
| Anthropic Claude | `providers/claude.toml` | 2026-09-18, signed-in session | `selftest` ok; `fidelity = "clipboard"` |
| DeepSeek | `providers/deepseek.toml` | 2026-09-18, signed-in session | `selftest` ok; **`fidelity = "dom"`**: the copy button has not been observed yet |
| Local fixture (tests only) | generated by `tests/e2e_browser.rs` | 2026-09-18 | validated: full pipeline, streaming, login, captcha, rate limit, error banner, timeout, missing selector, rewrite, attachment, Markdown fidelity |
The Claude and DeepSeek selectors were collected from public sources rather than
observed on a signed-in session. Treat them as a starting point: the first run of
`llm-gateway selftest <provider>` tells you whether they still match, and the
failure modes above tell you which key to fix.
The ChatGPT selectors shipped in this repository are the ones observed on the
public web UI in September 2026. They are a starting point, not a guarantee:
run the selftest before relying on them.
## Upgrading the installed files
The provider files under `~/.llm-gateway/providers/` are **copies** made on the
first run: editing the ones in this repository does not change them. After
upgrading the binary, refresh them with:
```bash
llm-gateway config init --force # keeps the previous file as <provider>.toml.bak
```
## Localised user interfaces
A provider translates its own UI, and the browser locale of the machine does not
change that: the same ChatGPT account serves a French page under `lang="fr-CA"`
and an English one under `lang="en-US"`. Every `aria-label` fallback therefore
only matches one language, which is why the stable hooks come first:
| Observed (French UI, September 2026) | Selector to prefer |
|---|---|
| `<button type="submit" id="composer-submit-button" aria-label="Envoyer la requête" data-testid="send-button">` | `button[data-testid='send-button']`, `button#composer-submit-button` |
| the answer toolbar copy control | `button[data-testid='copy-turn-action-button']` |
| `<textarea placeholder="Message DeepSeek" name="search" rows="2">` | `textarea[placeholder*='Message DeepSeek']` - the English placeholder and the `#chat-input` id of older builds are gone |
The `[login] text_patterns` follow the same rule: they are lowercase substrings of
the page text, so a French sign-in page needs French patterns
(`"se connecter"`, `"s'inscrire"`) next to the English ones. A page that is not
recognised as a sign-in page is reported as a missing prompt field instead, which
is a confusing way to learn that the session expired.
## Several accounts of one provider
A signed-in account is a browser profile, so driving two ChatGPT accounts means
two profiles, two sessions and two tab pools. Declare them in the provider file:
```toml
[[accounts]]
id = "perso" # the short name used in the model string
email = "[email protected]" # also accepted as the account key
label = "Personal" # free-form, shown by 'llm-gateway list'
default = true # used by {"model": "chatgpt"}
[[accounts]]
id = "pro"
email = "[email protected]"
```
Then sign each one in once and address it in the model string:
```bash
llm-gateway login chatgpt --account perso
llm-gateway login chatgpt@pro # the same thing
```
```jsonc
{"model": "chatgpt"} // the default account
{"model": "chatgpt@pro"} // by id
{"model": "[email protected]@gmail.com"} // by address
{"model": "chatgpt@pro/gpt-4o"} // account and model
```
Their profiles live in `profiles/<provider>/accounts/<account>/user-data-dir`.
A provider that declares no account keeps the historical
`profiles/<provider>/user-data-dir`, so adding accounts later means signing in
again for the first one. Each account has its own tab pool, so `max_tabs`
applies per account.
## Adding a new provider
1. Copy `providers/chatgpt.toml` to `~/.llm-gateway/providers/<name>.toml`.
2. Set `provider.name`, `provider.web_url`, `provider.new_conversation_url` and
`provider.conversation_url_pattern` (one capture group, the web conversation
id extracted from the page URL).
3. Fill in the selectors, then `llm-gateway selftest <name>`.
4. `llm-gateway login <name>` once, and the provider is usable as a model:
`{"model": "<name>"}` or `{"model": "<name>/<model>"}`.
Nothing else is provider specific: the capture loop, the error mapping, the
conversation store and the API are shared.
scripts/ci.ps1
+76
View File
@@ -0,0 +1,76 @@
<#
.SYNOPSIS
Run the same gates as the CI workflow, on this machine.
.DESCRIPTION
Mirrors .github/workflows/ci.yml so a change can be checked before pushing:
formatting, clippy with warnings as errors, the unit and integration tests
(no browser, no network), the doc tests and a release build.
The browser end-to-end suite needs a real Chromium and is opt-in: pass -E2e
to add it after the other gates.
.EXAMPLE
./scripts/ci.ps1
./scripts/ci.ps1 -E2e
./scripts/ci.ps1 -SkipRelease
#>
param(
[switch]$E2e,
[switch]$SkipRelease
)
$ErrorActionPreference = 'Stop'
$root = Split-Path -Parent (Split-Path -Parent $MyInvocation.MyCommand.Path)
Push-Location $root
$script:failed = @()
function Invoke-Gate {
param([string]$Name, [string[]]$Arguments)
Write-Host "== $Name" -ForegroundColor Cyan
Write-Host " cargo $($Arguments -join ' ')" -ForegroundColor DarkGray
& cargo @Arguments
if ($LASTEXITCODE -ne 0) {
Write-Host " FAILED ($LASTEXITCODE)" -ForegroundColor Red
$script:failed += $Name
} else {
Write-Host " ok" -ForegroundColor Green
}
}
Invoke-Gate 'formatting' @('fmt', '--all', '--check')
Invoke-Gate 'clippy' @('clippy', '--all-targets', '--locked', '--', '-D', 'warnings')
Invoke-Gate 'tests' @('test', '--locked', '--all-targets')
Invoke-Gate 'doc tests' @('test', '--locked', '--doc')
if (-not $SkipRelease) {
Invoke-Gate 'release build' @('build', '--locked', '--release')
}
if ($E2e) {
# The browser suite is #[ignore]d by default and needs a real browser.
$chrome = @(
"$env:ProgramFiles\Google\Chrome\Application\chrome.exe",
"$env:LOCALAPPDATA\Google\Chrome\Application\chrome.exe",
"$env:ProgramFiles\Microsoft\Edge\Application\msedge.exe",
"$env:LOCALAPPDATA\BraveSoftware\Brave-Browser\Application\brave.exe"
) | Where-Object { Test-Path $_ } | Select-Object -First 1
if (-not $chrome) {
Write-Warning 'no Chromium-based browser found: skipping the browser suite'
} else {
$env:CHROME = $chrome
Invoke-Gate 'browser e2e' @('test', '--test', 'e2e_browser', '--', '--ignored', '--test-threads=1')
}
}
Pop-Location
if ($script:failed.Count -gt 0) {
Write-Host ''
Write-Host "failed gates: $($script:failed -join ', ')" -ForegroundColor Red
exit 1
}
Write-Host ''
Write-Host 'all gates passed' -ForegroundColor Green
exit 0
scripts/e2e.ps1
+45
View File
@@ -0,0 +1,45 @@
<#
.SYNOPSIS
Run the browser end-to-end suite against the local fixture page.
.DESCRIPTION
The suite needs a real Chromium-based browser; it is ignored by default so
that "cargo test" stays usable without a browser. It drives a local static
fake of the ChatGPT page (tests/fixtures/chatgpt_mock.html): full pipeline,
streaming, conversation reuse, login, captcha, rate limit, error banner,
first-token timeout, missing selector, mid-stream rewrite and attachments.
.EXAMPLE
./scripts/e2e.ps1
./scripts/e2e.ps1 -Test fixture_full_pipeline
#>
param(
[string]$Test = ''
)
$ErrorActionPreference = 'Stop'
$root = Split-Path -Parent (Split-Path -Parent $MyInvocation.MyCommand.Path)
Push-Location $root
$chrome = @(
"$env:ProgramFiles\Google\Chrome\Application\chrome.exe",
"$env:LOCALAPPDATA\Google\Chrome\Application\chrome.exe",
"$env:ProgramFiles\Microsoft\Edge\Application\msedge.exe",
"$env:LOCALAPPDATA\BraveSoftware\Brave-Browser\Application\brave.exe"
) | Where-Object { Test-Path $_ } | Select-Object -First 1
if (-not $chrome) {
Write-Warning 'no Chromium-based browser found: the browser suite cannot run'
Pop-Location
exit 2
}
Write-Host "using $chrome" -ForegroundColor Cyan
$arguments = @('test', '--test', 'e2e_browser')
if ($Test) { $arguments += $Test }
$arguments += @('--', '--ignored', '--test-threads=1', '--nocapture')
& cargo @arguments
$code = $LASTEXITCODE
Pop-Location
exit $code
src/browser/cdp.rs
+748
View File
@@ -0,0 +1,748 @@
//! Chromium implementation of the browser engine, driven over CDP.
use std::path::{Path, PathBuf};
use std::time::{Duration, Instant};
use async_trait::async_trait;
use chromiumoxide::browser::{Browser, BrowserConfig};
use chromiumoxide::cdp::browser_protocol::browser::{
PermissionDescriptor, PermissionSetting, SetPermissionParams,
};
use chromiumoxide::error::CdpError;
use chromiumoxide::{Element, Page};
use futures::StreamExt;
use tokio::sync::Mutex;
use tokio::task::JoinHandle;
use crate::config::StealthMode;
use crate::error::{GatewayError, Result};
use super::engine::{BrowserEngine, BrowserSettings, TabStats};
use super::pool::{TabLease, TabPool};
/// Script installed in every new document when stealth is set to "minimal".
///
/// It only hides the automation marker; the user agent is left untouched so the
/// browser still looks like the real Chrome it is. Spoofing an outdated user
/// agent (see StealthMode::Aggressive) is what makes OAuth providers such as
/// Google answer "this browser or app may not be secure".
const MINIMAL_STEALTH_SCRIPT: &str = r#"
Object.defineProperty(Object.getPrototypeOf(navigator), 'webdriver', { get: () => false });
if (!window.chrome) { window.chrome = { runtime: {} }; }
"#;
/// Permissions granted before reading the clipboard over CDP.
const CLIPBOARD_PERMISSIONS: [&str; 2] = ["clipboard-read", "clipboard-write"];
/// How long a real mouse click is given before the scripted fallback runs.
const CLICK_TIMEOUT: Duration = Duration::from_secs(2);
/// Click and focus from inside the page.
///
/// Used when the mouse click cannot complete: it also moves the focus, which a
/// scripted `click()` alone does not do, so the editor accepts the text.
const SCRIPTED_CLICK: &str =
"function() { if (this.focus) { this.focus(); } this.click(); return true; }";
struct Running {
browser: Browser,
handler: JoinHandle<()>,
/// One entry per tab slot of the pool; `None` until that slot is used.
pages: Vec<Option<Page>>,
/// False when the browser belongs to the user (attach mode): it must not be
/// closed when the gateway shuts down.
owned: bool,
}
/// Launches and owns one browser process, or attaches to a running one.
pub struct CdpEngine {
settings: BrowserSettings,
state: Mutex<Option<Running>>,
pool: TabPool,
/// Serialises mouse, key and text input across the tabs of this browser.
input_lock: Mutex<()>,
}
impl CdpEngine {
pub fn new(settings: BrowserSettings) -> Self {
let owner = match &settings.account {
Some(account) => format!("{}@{}", settings.provider, account),
None => settings.provider.clone(),
};
let pool = TabPool::new(owner, settings.max_tabs, settings.busy_wait);
Self {
settings,
state: Mutex::new(None),
pool,
input_lock: Mutex::new(()),
}
}
/// Endpoint of the browser we attach to.
pub fn attach_endpoint(&self) -> String {
format!("http://127.0.0.1:{}", self.settings.debug_port)
}
/// Page of one tab slot, launching the browser and opening the tab when
/// that slot is used for the first time.
async fn ensure_page(&self, slot: usize) -> Result<Page> {
let mut guard = self.state.lock().await;
let stale = guard
.as_ref()
.is_some_and(|running| running.handler.is_finished());
if stale {
if let Some(mut dead) = guard.take() {
let _ = dead.browser.close().await;
dead.handler.abort();
}
tracing::warn!(
provider = %self.settings.provider,
"the browser went away, starting a new one"
);
}
if guard.is_none() {
let running = if self.settings.attach {
self.attach().await?
} else {
self.launch().await?
};
*guard = Some(running);
}
let running = guard.as_mut().expect("the browser was just started");
if let Some(page) = running.pages.get(slot).and_then(|page| page.clone()) {
return Ok(page);
}
let page = running
.browser
.new_page("about:blank")
.await
.map_err(|err| self.classify(err))?;
self.apply_stealth(&page).await;
if running.pages.len() <= slot {
running.pages.resize(slot + 1, None);
}
running.pages[slot] = Some(page.clone());
tracing::debug!(
provider = %self.settings.provider,
slot,
"opened a new tab"
);
Ok(page)
}
/// Start our own browser process.
async fn launch(&self) -> Result<Running> {
std::fs::create_dir_all(&self.settings.profile_dir).map_err(|e| {
GatewayError::BrowserUnavailable(format!(
"cannot create the profile directory {}: {e}",
self.settings.profile_dir.display()
))
})?;
let mut builder = BrowserConfig::builder().user_data_dir(self.settings.profile_dir.clone());
builder = if self.settings.headless {
builder.new_headless_mode()
} else {
builder.with_head()
};
if let Some(executable) = &self.settings.executable {
builder = builder.chrome_executable(executable.clone());
}
for arg in &self.settings.extra_args {
builder = builder.arg(arg.as_str());
}
let config = builder
.launch_timeout(self.settings.launch_timeout)
.request_timeout(self.settings.request_timeout)
.build()
.map_err(|e| GatewayError::BrowserUnavailable(e.to_string()))?;
tracing::debug!(
provider = %self.settings.provider,
account = self.settings.account.as_deref().unwrap_or("-"),
profile = %self.settings.profile_dir.display(),
headless = self.settings.headless,
stealth = ?self.settings.stealth,
launch_timeout_s = self.settings.launch_timeout.as_secs(),
"launching the browser"
);
let (browser, mut handler) = match Browser::launch(config).await {
Ok(pair) => pair,
Err(err) => return Err(self.classify(err)),
};
// The handler drives the websocket: without polling it the connection
// stalls after a few messages.
let handle = tokio::spawn(async move {
while let Some(event) = handler.next().await {
if event.is_err() {
break;
}
}
});
Ok(Running {
browser,
handler: handle,
pages: Vec::new(),
owned: true,
})
}
/// Attach to a browser the user started, and reuse one of its tabs.
async fn attach(&self) -> Result<Running> {
let endpoint = self.attach_endpoint();
tracing::info!(endpoint = %endpoint, "attaching to the running browser");
let (mut browser, mut handler) = match Browser::connect(endpoint.clone()).await {
Ok(pair) => pair,
Err(err) => {
return Err(GatewayError::BrowserUnavailable(format!(
"cannot attach to the browser on {endpoint}: {err}\n\
Start one first, for example:\n {}",
external_launch(
self.settings.executable.as_ref(),
&self.settings.profile_dir,
"about:blank",
Some(self.settings.debug_port),
)
.command_line()
)))
}
};
let handle = tokio::spawn(async move {
while let Some(event) = handler.next().await {
if event.is_err() {
break;
}
}
});
let page = self.pick_page(&mut browser).await?;
self.apply_stealth(&page).await;
Ok(Running {
browser,
handler: handle,
// The tab the user already had open takes the first slot.
pages: vec![Some(page)],
owned: false,
})
}
/// Reuse an already open tab so the user keeps their window and session.
///
/// The tabs of a browser we did not launch are only visible through the
/// target list, not through the page registry of the handler.
async fn pick_page(&self, browser: &mut Browser) -> Result<Page> {
// The handler learns about the tabs of a browser it did not launch
// through Target.targetCreated events, which arrive just after the
// connection: retry briefly instead of giving up on the first miss.
let deadline = Instant::now() + Duration::from_secs(5);
let mut last_error = None;
loop {
match browser.fetch_targets().await {
Ok(targets) => {
let reusable: Vec<_> = targets
.into_iter()
.filter(|target| target.r#type.as_str() == "page")
.filter(|target| {
!target.url.is_empty()
&& target.url != "about:blank"
&& !target.url.starts_with("devtools://")
})
.collect();
for target in reusable {
match browser.get_page(target.target_id.clone()).await {
Ok(page) => {
// A browser that is still starting up hands out
// sessions for tabs it then replaces: the handle
// exists but every command fails with "session
// with given id not found". Prove the tab answers
// before adopting it.
if self.page_answers(&page).await {
tracing::info!(
url = %target.url,
"reusing the tab already open in the browser"
);
return Ok(page);
}
tracing::debug!(
url = %target.url,
"the tab is not answering yet, looking for another one"
);
}
Err(err) => last_error = Some(err),
}
}
}
Err(err) => last_error = Some(err),
}
if Instant::now() >= deadline {
break;
}
tokio::time::sleep(Duration::from_millis(150)).await;
}
if let Some(err) = last_error {
tracing::debug!(error = %err, "no reusable tab found in the running browser");
}
tracing::info!("no tab of yours could be reused, opening one in your browser instead");
browser
.new_page("about:blank")
.await
.map_err(|err| self.classify(err))
}
/// True when Chrome is rendering that tab.
///
/// A hidden tab never delivers the IntersectionObserver callback that
/// chromiumoxide's mouse click waits for, so the click would hang forever.
async fn page_is_visible(&self, page: &Page) -> bool {
match page
.evaluate_expression("document.visibilityState === 'visible'")
.await
{
Ok(result) => result.into_value::<bool>().unwrap_or(false),
Err(_) => false,
}
}
/// True when the tab answers a trivial CDP command.
///
/// Attaching to a browser the user just started races with Chrome creating
/// its first tabs: the target list already advertises them while their
/// session is still being replaced.
async fn page_answers(&self, page: &Page) -> bool {
matches!(
tokio::time::timeout(Duration::from_secs(3), page.evaluate_expression("1")).await,
Ok(Ok(_))
)
}
async fn apply_stealth(&self, page: &Page) {
match self.settings.stealth {
StealthMode::Off => {}
StealthMode::Minimal => {
if let Err(err) = page.evaluate_on_new_document(MINIMAL_STEALTH_SCRIPT).await {
tracing::debug!(error = %err, "cannot install the stealth script");
}
}
StealthMode::Aggressive => {
tracing::warn!(
"aggressive stealth advertises an outdated user agent, which some sign-in pages reject as insecure"
);
if let Err(err) = page.enable_stealth_mode().await {
tracing::debug!(error = %err, "stealth mode unavailable on this browser build");
}
}
}
}
fn classify(&self, err: CdpError) -> GatewayError {
let text = err.to_string();
let lower = text.to_lowercase();
if lower.contains("already in use")
|| lower.contains("singletonlock")
|| lower.contains("processsingleton")
|| lower.contains("cannot create a file when that file already exists")
{
return GatewayError::BrowserProfileLocked {
path: self.settings.profile_dir.clone(),
};
}
if lower.contains("no such file")
|| lower.contains("cannot find")
|| lower.contains("failed to spawn")
|| lower.contains("os error 2")
{
return GatewayError::BrowserUnavailable(format!(
"cannot start the browser executable: {text}"
));
}
GatewayError::BrowserUnavailable(text)
}
}
#[async_trait]
impl BrowserEngine for CdpEngine {
fn kind(&self) -> &'static str {
"chromium-cdp"
}
fn settings(&self) -> &BrowserSettings {
&self.settings
}
async fn start(&self) -> Result<()> {
self.ensure_page(0).await.map(|_| ())
}
async fn acquire_page(&self, wait: Duration) -> Result<TabLease> {
self.acquire_within(wait).await
}
async fn is_running(&self) -> bool {
let guard = self.state.lock().await;
guard
.as_ref()
.is_some_and(|running| !running.handler.is_finished())
}
fn stats(&self) -> TabStats {
TabStats {
max_tabs: self.pool.max_tabs(),
busy: self.pool.busy(),
}
}
async fn grant_permission(&self, name: &str) -> Result<()> {
let guard = self.state.lock().await;
let Some(running) = guard.as_ref() else {
return Ok(());
};
let params =
SetPermissionParams::new(PermissionDescriptor::new(name), PermissionSetting::Granted);
running
.browser
.execute(params)
.await
.map(|_| ())
.map_err(|err| GatewayError::BrowserUnavailable(err.to_string()))
}
async fn click(&self, page: &Page, element: &Element) -> Result<()> {
if self.page_is_visible(page).await {
let _guard = self.input_lock.lock().await;
match tokio::time::timeout(CLICK_TIMEOUT, element.click()).await {
Ok(Ok(_)) => return Ok(()),
Ok(Err(err)) => return Err(dom_error(err)),
Err(_) => tracing::debug!(
"the mouse click did not complete, clicking from the page instead"
),
}
} else {
tracing::debug!("the tab is not rendered, clicking from the page");
}
element
.call_js_fn(SCRIPTED_CLICK, false)
.await
.map(|_| ())
.map_err(dom_error)
}
async fn press_key(&self, element: &Element, key: &str) -> Result<()> {
let _guard = self.input_lock.lock().await;
element.press_key(key).await.map(|_| ()).map_err(dom_error)
}
async fn type_text(&self, element: &Element, text: &str) -> Result<()> {
let _guard = self.input_lock.lock().await;
element.type_str(text).await.map(|_| ()).map_err(dom_error)
}
async fn shutdown(&self) -> Result<()> {
let running = self.state.lock().await.take();
if let Some(mut running) = running {
if running.owned {
if let Err(err) = running.browser.close().await {
tracing::debug!(error = %err, "closing the browser reported an error");
}
} else {
tracing::info!("detaching: the browser you started keeps running");
}
running.handler.abort();
}
Ok(())
}
}
impl CdpEngine {
/// Lease a tab, waiting at most `wait` for one to be free.
async fn acquire_within(&self, wait: Duration) -> Result<TabLease> {
let reservation = self.pool.reserve_within(wait).await?;
match self.ensure_page(reservation.slot()).await {
Ok(page) => Ok(reservation.into_lease(page)),
Err(err) => {
// The reservation gives its slot back on drop.
drop(reservation);
Err(err)
}
}
}
/// Grant every permission the clipboard read needs. Best effort.
pub async fn grant_clipboard_permissions(&self) -> Result<()> {
for name in CLIPBOARD_PERMISSIONS {
if let Err(err) = self.grant_permission(name).await {
tracing::debug!(permission = name, error = %err, "cannot grant a clipboard permission");
}
}
Ok(())
}
}
/// A browser the user starts themselves, described as a command line.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ExternalLaunch {
pub program: String,
pub args: Vec<String>,
}
impl ExternalLaunch {
/// Command line to paste in a shell.
pub fn command_line(&self) -> String {
let mut out = quote(&self.program);
for arg in &self.args {
out.push(' ');
out.push_str(&quote_argument(arg));
}
out
}
/// Start it, detached from the gateway.
pub fn spawn(&self) -> std::io::Result<std::process::Child> {
std::process::Command::new(&self.program)
.args(&self.args)
.spawn()
}
}
/// Command line that opens a browser the user owns, on the gateway profile.
///
/// With a debug port the gateway can attach to that window; without one the
/// sign-in happens in a completely normal browser, which is what OAuth
/// providers that block automation require.
pub fn external_launch(
executable: Option<&PathBuf>,
profile_dir: &Path,
url: &str,
debug_port: Option<u16>,
) -> ExternalLaunch {
let program = executable
.map(|path| path.to_string_lossy().to_string())
.unwrap_or_else(|| "chrome".to_string());
let mut args = vec![
format!("--user-data-dir={}", profile_dir.display()),
"--no-first-run".to_string(),
"--no-default-browser-check".to_string(),
];
if let Some(port) = debug_port {
args.push(format!("--remote-debugging-port={port}"));
}
args.push(url.to_string());
ExternalLaunch { program, args }
}
/// Quote only the value of a --key=value switch, which reads better.
fn quote_argument(arg: &str) -> String {
match arg.split_once('=') {
Some((key, value)) if key.starts_with("--") => format!("{key}={}", quote(value)),
_ => quote(arg),
}
}
fn quote(value: &str) -> String {
if value.contains(' ') || value.contains('\t') {
format!("\"{value}\"")
} else {
value.to_string()
}
}
/// Resolve the browser executable: explicit path, environment, then well-known
/// installation directories.
pub fn resolve_executable(explicit: Option<&PathBuf>, env_var: Option<String>) -> Option<PathBuf> {
if let Some(path) = explicit {
if path.is_file() {
return Some(path.clone());
}
tracing::warn!(path = %path.display(), "configured browser executable does not exist");
}
if let Some(candidate) = env_var {
let path = PathBuf::from(candidate);
if path.is_file() {
return Some(path);
}
}
candidate_executables()
.into_iter()
.find(|path| path.is_file())
}
/// Well-known installation paths of Chromium-based browsers.
pub fn candidate_executables() -> Vec<PathBuf> {
let mut candidates: Vec<PathBuf> = Vec::new();
#[cfg(windows)]
{
for base in ["ProgramFiles", "ProgramFiles(x86)", "LOCALAPPDATA"] {
if let Some(root) = std::env::var_os(base) {
let root = PathBuf::from(root);
candidates.push(
root.join("Google")
.join("Chrome")
.join("Application")
.join("chrome.exe"),
);
candidates.push(root.join("Chromium").join("Application").join("chrome.exe"));
candidates.push(
root.join("Microsoft")
.join("Edge")
.join("Application")
.join("msedge.exe"),
);
candidates.push(
root.join("BraveSoftware")
.join("Brave-Browser")
.join("Application")
.join("brave.exe"),
);
candidates.push(
root.join("BraveSoftware")
.join("Brave-Browser-Beta")
.join("Application")
.join("brave.exe"),
);
}
}
}
#[cfg(not(windows))]
{
for path in [
"/usr/bin/google-chrome",
"/usr/bin/google-chrome-stable",
"/usr/bin/chromium",
"/usr/bin/chromium-browser",
"/opt/google/chrome/chrome",
"/snap/bin/chromium",
"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome",
] {
candidates.push(PathBuf::from(path));
}
}
candidates
}
/// Map the browser family flag onto a concrete executable.
pub fn executable_hint(name: &str) -> Result<Option<PathBuf>> {
let needle = match name.trim().to_lowercase().as_str() {
"" | "auto" | "chrome" | "chromium" => return Ok(None),
"edge" | "msedge" => "msedge",
"brave" => "brave",
other => return Err(GatewayError::UnsupportedBrowser(other.to_string())),
};
Ok(candidate_executables().into_iter().find(|path| {
path.is_file()
&& path
.file_name()
.is_some_and(|file| file.to_string_lossy().to_lowercase().contains(needle))
}))
}
/// Map a CDP failure onto a gateway error, as `browser::dom` does.
fn dom_error(err: CdpError) -> GatewayError {
crate::browser::dom::cdp_error(err)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn unsupported_browsers_are_rejected_explicitly() {
let err = executable_hint("firefox").unwrap_err();
assert_eq!(err.code(), "unsupported_browser");
assert!(executable_hint("chrome").unwrap().is_none());
assert!(executable_hint("auto").unwrap().is_none());
assert!(executable_hint("").unwrap().is_none());
}
#[test]
fn browser_families_match_their_executable_name() {
if let Some(path) = executable_hint("edge").unwrap() {
assert!(path.to_string_lossy().to_lowercase().contains("msedge"));
}
if let Some(path) = executable_hint("brave").unwrap() {
assert!(path.to_string_lossy().to_lowercase().contains("brave"));
}
}
#[test]
fn the_external_launch_command_has_no_automation_flags() {
let profile = PathBuf::from("/home/me/.llm-gateway/profiles/chatgpt/user-data-dir");
let launch = external_launch(
Some(&PathBuf::from("/usr/bin/google-chrome")),
&profile,
"https://chatgpt.com/",
None,
);
let line = launch.command_line();
assert!(line.starts_with("/usr/bin/google-chrome"));
assert!(
line.contains("--user-data-dir=/home/me/.llm-gateway/profiles/chatgpt/user-data-dir")
);
assert!(line.ends_with("https://chatgpt.com/"));
assert!(!line.contains("enable-automation"));
assert!(!line.contains("remote-debugging-port"));
let attach = external_launch(None, &profile, "https://chatgpt.com/", Some(9222));
assert!(attach
.command_line()
.contains("--remote-debugging-port=9222"));
assert_eq!(attach.command_line().split(' ').next(), Some("chrome"));
}
#[test]
fn paths_with_spaces_are_quoted() {
let profile = PathBuf::from("C:/Program Files/profile");
let launch = external_launch(
Some(&PathBuf::from("C:/Program Files/Google/Chrome/chrome.exe")),
&profile,
"https://chatgpt.com/",
Some(9222),
);
assert!(launch
.command_line()
.starts_with("\"C:/Program Files/Google/Chrome/chrome.exe\""));
assert!(launch
.command_line()
.contains("--user-data-dir=\"C:/Program Files/profile\""));
}
#[test]
fn the_minimal_stealth_script_does_not_touch_the_user_agent() {
assert!(MINIMAL_STEALTH_SCRIPT.contains("webdriver"));
assert!(!MINIMAL_STEALTH_SCRIPT.contains("userAgent"));
assert!(!MINIMAL_STEALTH_SCRIPT.contains("Chrome/1"));
}
#[test]
fn an_engine_names_its_owner_and_its_pool() {
let settings = BrowserSettings {
provider: "chatgpt".to_string(),
account: Some("perso".to_string()),
max_tabs: 3,
..BrowserSettings::default()
};
let engine = CdpEngine::new(settings);
assert_eq!(engine.kind(), "chromium-cdp");
assert_eq!(engine.settings().max_tabs, 3);
assert_eq!(engine.stats().max_tabs, 3);
assert_eq!(engine.stats().busy, 0);
assert!(!engine.stats().is_saturated());
}
#[test]
fn the_clipboard_permissions_are_the_ones_the_browser_expects() {
assert_eq!(CLIPBOARD_PERMISSIONS, ["clipboard-read", "clipboard-write"]);
}
}
src/browser/debug.rs
+140
View File
@@ -0,0 +1,140 @@
//! Debug artifacts written when a capture fails.
use std::path::{Path, PathBuf};
use chromiumoxide::Page;
use crate::config::ScreenshotMode;
use crate::util;
use super::dom;
/// Screenshot and DOM dump of a failed capture.
#[derive(Debug, Clone, Default)]
pub struct CapturedArtifacts {
pub screenshot: Option<PathBuf>,
pub html: Option<PathBuf>,
}
impl CapturedArtifacts {
pub fn is_empty(&self) -> bool {
self.screenshot.is_none() && self.html.is_none()
}
}
/// Writes screenshots and DOM dumps next to the state directory.
#[derive(Debug, Clone)]
pub struct DebugRecorder {
dir: PathBuf,
mode: ScreenshotMode,
}
impl DebugRecorder {
pub fn new(dir: PathBuf, mode: ScreenshotMode) -> Self {
Self { dir, mode }
}
pub fn dir(&self) -> &Path {
&self.dir
}
pub fn mode(&self) -> ScreenshotMode {
self.mode
}
/// Whether a capture should happen for this outcome.
pub fn should_capture(&self, on_error: bool) -> bool {
match self.mode {
ScreenshotMode::Never => false,
ScreenshotMode::OnError => on_error,
ScreenshotMode::Always => true,
}
}
/// Save a screenshot and the current DOM. Failures are logged, never returned:
/// debugging must not mask the original error.
pub async fn capture(
&self,
provider: &str,
trace_id: &str,
page: &Page,
reason: &str,
) -> CapturedArtifacts {
let mut artifacts = CapturedArtifacts::default();
if let Err(err) = std::fs::create_dir_all(&self.dir) {
tracing::warn!(error = %err, dir = %self.dir.display(), "cannot create the debug directory");
return artifacts;
}
let stamp = util::format_compact_utc(util::now_secs());
let safe_trace: String = trace_id
.chars()
.filter(|c| c.is_ascii_alphanumeric())
.collect();
let base = format!("{provider}_{safe_trace}_{stamp}");
match dom::screenshot_png(page).await {
Ok(bytes) => {
let path = self.dir.join(format!("{base}.png"));
match std::fs::write(&path, bytes) {
Ok(()) => artifacts.screenshot = Some(path),
Err(err) => tracing::warn!(error = %err, "cannot write the screenshot"),
}
}
Err(err) => tracing::warn!(error = %err, "cannot take a screenshot"),
}
match dom::page_html(page).await {
Ok(html) => {
let path = self.dir.join(format!("{base}.html"));
match std::fs::write(&path, html) {
Ok(()) => artifacts.html = Some(path),
Err(err) => tracing::warn!(error = %err, "cannot write the DOM dump"),
}
}
Err(err) => tracing::warn!(error = %err, "cannot read the page DOM"),
}
let index_line = format!(
"{{\"ts\":{},\"provider\":\"{provider}\",\"trace_id\":\"{trace_id}\",\"reason\":{},\"screenshot\":{},\"html\":{}}}\n",
util::now_secs(),
serde_json::to_string(reason).unwrap_or_else(|_| "\"\"".into()),
serde_json::to_string(&artifacts.screenshot.as_ref().map(|p| p.display().to_string()))
.unwrap_or_else(|_| "null".into()),
serde_json::to_string(&artifacts.html.as_ref().map(|p| p.display().to_string()))
.unwrap_or_else(|_| "null".into()),
);
let index = self.dir.join("index.jsonl");
if let Err(err) = std::fs::OpenOptions::new()
.create(true)
.append(true)
.open(&index)
.and_then(|mut file| {
use std::io::Write;
file.write_all(index_line.as_bytes())
})
{
tracing::warn!(error = %err, "cannot append to the debug index");
}
artifacts
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn capture_policy_follows_the_mode() {
let recorder = DebugRecorder::new(PathBuf::from("/tmp"), ScreenshotMode::OnError);
assert!(!recorder.should_capture(false));
assert!(recorder.should_capture(true));
let always = DebugRecorder::new(PathBuf::from("/tmp"), ScreenshotMode::Always);
assert!(always.should_capture(false));
let never = DebugRecorder::new(PathBuf::from("/tmp"), ScreenshotMode::Never);
assert!(!never.should_capture(true));
}
}
src/browser/dom.rs
+193
View File
@@ -0,0 +1,193 @@
//! Thin, error-mapped helpers around a Chromium page handle.
use std::path::PathBuf;
use std::time::{Duration, Instant};
use chromiumoxide::cdp::browser_protocol::dom::SetFileInputFilesParams;
use chromiumoxide::error::CdpError;
use chromiumoxide::page::ScreenshotParams;
use chromiumoxide::{Element, Page};
use serde::de::DeserializeOwned;
use crate::error::{GatewayError, Result};
/// Map a CDP failure onto a gateway error.
pub fn cdp_error(err: CdpError) -> GatewayError {
let text = err.to_string();
let lower = text.to_lowercase();
if lower.contains("already in use")
|| lower.contains("singletonlock")
|| lower.contains("processsingleton")
{
return GatewayError::BrowserProfileLocked {
path: PathBuf::new(),
};
}
GatewayError::BrowserUnavailable(text)
}
/// Evaluate a JavaScript expression and decode the returned JSON value.
pub async fn evaluate_json<T: DeserializeOwned>(page: &Page, script: &str) -> Result<T> {
let result = page.evaluate_expression(script).await.map_err(cdp_error)?;
result
.into_value::<T>()
.map_err(|e| GatewayError::Internal(format!("decoding the page result: {e}")))
}
/// Evaluate JavaScript, ignoring a decoding failure.
pub async fn evaluate_ok(page: &Page, script: &str) -> Result<()> {
page.evaluate_expression(script).await.map_err(cdp_error)?;
Ok(())
}
/// Navigate to a URL with a hard timeout.
///
/// Navigating to a URL that only differs by its fragment is a same-document
/// navigation: Chrome never fires the load event the CDP helper waits for, so
/// the fragment is set directly instead of hanging.
pub async fn goto(page: &Page, url: &str, timeout: Duration) -> Result<()> {
let current = page.url().await.ok().flatten().unwrap_or_default();
if !current.is_empty() && strip_fragment(&current) == strip_fragment(url) {
let current_fragment = fragment(&current);
let wanted = fragment(url);
if current_fragment != wanted {
let value = if wanted.is_empty() {
String::new()
} else {
format!("#{wanted}")
};
let script = format!("location.hash = {}", crate::browser::js::js_string(&value));
evaluate_ok(page, &script).await?;
}
tracing::debug!(url, "already on the requested page, navigation skipped");
return Ok(());
}
match tokio::time::timeout(timeout, page.goto(url)).await {
Ok(Ok(_)) => Ok(()),
Ok(Err(err)) => Err(cdp_error(err)),
Err(_) => Err(GatewayError::CaptureTimeout {
stage: format!("loading {url}"),
timeout_s: timeout.as_secs(),
}),
}
}
/// URL without its fragment.
fn strip_fragment(url: &str) -> &str {
url.split('#').next().unwrap_or(url)
}
/// Fragment of a URL, without the leading hash.
fn fragment(url: &str) -> &str {
match url.split_once('#') {
Some((_, fragment)) => fragment,
None => "",
}
}
/// Poll for a selector until it appears.
pub async fn wait_for_selector(
page: &Page,
selector: &str,
step: &str,
timeout: Duration,
) -> Result<Element> {
if selector.trim().is_empty() {
return Err(GatewayError::SelectorMissing {
selector: selector.to_string(),
step: format!("{step}: no selector configured"),
});
}
let started = Instant::now();
loop {
match page.find_element(selector).await {
Ok(element) => return Ok(element),
Err(_) => {
if started.elapsed() >= timeout {
break;
}
tokio::time::sleep(Duration::from_millis(250)).await;
}
}
}
tracing::debug!(
selector,
timeout_s = timeout.as_secs(),
"selector never appeared"
);
Err(GatewayError::SelectorMissing {
selector: selector.to_string(),
step: step.to_string(),
})
}
/// True when the selector currently matches at least one node.
pub async fn selector_exists(page: &Page, selector: &str) -> bool {
if selector.trim().is_empty() {
return false;
}
page.find_element(selector).await.is_ok()
}
/// Attach files to a hidden file input (images, PDFs).
pub async fn set_file_input_files(page: &Page, selector: &str, files: &[PathBuf]) -> Result<()> {
if files.is_empty() {
return Ok(());
}
let element = wait_for_selector(
page,
selector,
"uploading an attachment",
Duration::from_secs(15),
)
.await?;
let params = SetFileInputFilesParams {
files: files
.iter()
.map(|path| path.to_string_lossy().to_string())
.collect(),
node_id: Some(element.node_id),
backend_node_id: None,
object_id: None,
};
page.execute(params).await.map_err(cdp_error)?;
Ok(())
}
/// Current URL of the page.
pub async fn page_url(page: &Page) -> Result<String> {
Ok(page.url().await.map_err(cdp_error)?.unwrap_or_default())
}
/// Full HTML of the page.
pub async fn page_html(page: &Page) -> Result<String> {
page.content().await.map_err(cdp_error)
}
/// PNG screenshot of the current viewport.
pub async fn screenshot_png(page: &Page) -> Result<Vec<u8>> {
page.screenshot(ScreenshotParams::default())
.await
.map_err(cdp_error)
}
/// Bring the page to the front, best effort.
pub async fn focus_page(page: &Page) {
if let Err(err) = page.bring_to_front().await {
tracing::trace!(error = %err, "cannot bring the page to the front");
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn fragments_are_split_off_the_url() {
assert_eq!(strip_fragment("https://x/c/1#c=abc"), "https://x/c/1");
assert_eq!(fragment("https://x/c/1#c=abc"), "c=abc");
assert_eq!(fragment("https://x/c/1"), "");
assert_eq!(strip_fragment("https://x/c/1"), "https://x/c/1");
}
}
src/browser/engine.rs
+126
View File
@@ -0,0 +1,126 @@
//! Browser engine abstraction.
//!
//! The trait isolates process lifecycle (launch, reuse, shutdown) and tab
//! admission so the business logic never talks to Chrome directly. DOM
//! interaction goes through the helpers of `browser::dom`, which take a page
//! handle; a future WebDriver backend would add a page-ops trait there without
//! touching the providers.
use std::path::PathBuf;
use std::time::Duration;
use async_trait::async_trait;
use chromiumoxide::{Element, Page};
use serde::Serialize;
use crate::config::StealthMode;
use crate::error::Result;
use super::pool::TabLease;
/// Everything needed to spawn one browser process.
///
/// One instance describes one provider *account*: the account owns its browser
/// profile, its tab pool and therefore its signed-in session.
#[derive(Debug, Clone)]
pub struct BrowserSettings {
/// Provider this browser belongs to, for logs and errors.
pub provider: String,
/// Account id, when the provider declares several.
pub account: Option<String>,
pub executable: Option<PathBuf>,
pub headless: bool,
pub profile_dir: PathBuf,
/// Concurrent turns allowed on this account, i.e. tabs kept open.
pub max_tabs: usize,
/// How long a request waits for a free tab before it gets a 429.
pub busy_wait: Duration,
pub launch_timeout: Duration,
pub request_timeout: Duration,
pub extra_args: Vec<String>,
pub stealth: StealthMode,
/// Attach to a browser the user started instead of launching our own.
pub attach: bool,
pub debug_port: u16,
}
impl Default for BrowserSettings {
fn default() -> Self {
Self {
provider: "provider".to_string(),
account: None,
executable: None,
headless: false,
profile_dir: PathBuf::new(),
max_tabs: 1,
busy_wait: Duration::from_secs(30),
launch_timeout: Duration::from_secs(60),
request_timeout: Duration::from_secs(60),
extra_args: Vec::new(),
stealth: StealthMode::Off,
attach: false,
debug_port: 9222,
}
}
}
/// Occupancy of a provider account's tab pool.
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Serialize)]
pub struct TabStats {
pub max_tabs: usize,
pub busy: usize,
}
impl TabStats {
pub fn is_saturated(&self) -> bool {
self.busy >= self.max_tabs
}
}
/// Lifecycle of the browser backing one provider account.
#[async_trait]
pub trait BrowserEngine: Send + Sync {
/// Backend name, for logs and status reports.
fn kind(&self) -> &'static str;
fn settings(&self) -> &BrowserSettings;
/// Launch the browser if needed. Idempotent.
async fn start(&self) -> Result<()>;
/// Lease one tab for a turn.
///
/// Returns `provider_busy` when every tab stays busy for longer than
/// `wait`; the caller chooses the budget so that a status probe does not
/// wait as long as a real turn.
async fn acquire_page(&self, wait: Duration) -> Result<TabLease>;
async fn is_running(&self) -> bool;
/// Current tab occupancy.
fn stats(&self) -> TabStats;
/// Grant a CDP permission to every origin, best effort.
///
/// Used for the clipboard: reading it is what gives the gateway the
/// provider's own Markdown instead of the flattened visible text.
async fn grant_permission(&self, name: &str) -> Result<()>;
/// Click an element.
///
/// A real mouse event is preferred, because web UIs trust it. It is only
/// usable on a tab Chrome actually renders, so a hidden tab (the second tab
/// of a `max_tabs > 1` account) is clicked from the page instead: waiting
/// for a mouse event there hangs forever. Real mouse events are serialised
/// browser wide, which costs milliseconds while a generation costs seconds.
async fn click(&self, page: &Page, element: &Element) -> Result<()>;
/// Type a key into an element, with real key events.
async fn press_key(&self, element: &Element, key: &str) -> Result<()>;
/// Type text into an element, character by character.
async fn type_text(&self, element: &Element, text: &str) -> Result<()>;
/// Close the browser process started by this engine.
async fn shutdown(&self) -> Result<()>;
}
src/browser/js.rs
+406
View File
@@ -0,0 +1,406 @@
//! JavaScript snippets injected into the provider page.
//!
//! Every selector comes from the provider TOML configuration: this module only
//! builds strings, it never hardcodes a site specific selector.
use serde::{Deserialize, Serialize};
use crate::providers::config::Selectors;
/// Helpers shared by every snippet. All selector lookups are guarded so an
/// invalid or stale selector degrades into a missing element instead of an
/// exception.
const HELPERS: &str = r#"
const q = (s) => { if (!s) return null; try { return document.querySelector(s); } catch (e) { return null; } };
const qa = (s) => { if (!s) return []; try { return Array.prototype.slice.call(document.querySelectorAll(s)); } catch (e) { return []; } };
const txt = (el) => { if (!el) return ''; const t = el.innerText; if (typeof t === 'string' && t.length) return t; const c = el.textContent; return typeof c === 'string' ? c : ''; };
const val = (s) => { const el = q(s); return el ? txt(el).trim().slice(0, 500) : null; };
"#;
/// Encode a Rust string as a JavaScript string literal.
pub fn js_string(raw: &str) -> String {
serde_json::to_string(raw).unwrap_or_else(|_| "\"\"".to_string())
}
/// State of the page before a prompt is sent.
#[derive(Debug, Clone, Default, Deserialize, Serialize)]
#[serde(default)]
pub struct ProbeResult {
pub url: String,
pub title: String,
pub ready_state: String,
pub has_input: bool,
pub input_len: usize,
pub has_send: bool,
pub send_disabled: bool,
pub has_login: bool,
pub has_captcha: bool,
pub response_count: usize,
pub streaming: bool,
pub error_text: Option<String>,
pub body_text: String,
}
/// State of the answer container while polling.
#[derive(Debug, Clone, Default, Deserialize, Serialize)]
#[serde(default)]
pub struct ReadResponseResult {
pub count: usize,
pub text: String,
pub streaming: bool,
pub error_text: Option<String>,
pub has_login: bool,
pub has_captcha: bool,
}
/// Outcome of a text injection attempt.
#[derive(Debug, Clone, Default, Deserialize, Serialize)]
#[serde(default)]
pub struct InjectionResult {
pub ok: bool,
pub len: usize,
pub reason: Option<String>,
}
/// Rich content of a settled answer node.
#[derive(Debug, Clone, Default, Deserialize, Serialize)]
#[serde(default)]
pub struct AnswerHtml {
/// `innerHTML` of the answer node: the input of the Markdown conversion.
pub html: String,
/// `innerText` of the same node, the historical fallback.
pub text: String,
/// True when the node itself was found.
pub found: bool,
}
/// Outcome of clicking the provider copy button and reading the clipboard.
#[derive(Debug, Clone, Default, Deserialize, Serialize)]
#[serde(default)]
pub struct ClipboardResult {
/// True when a copy button was found and clicked.
pub clicked: bool,
/// Markdown the provider itself wrote to the clipboard.
pub clipboard: Option<String>,
/// Why the clipboard could not be read, when it could not.
pub error: Option<String>,
}
/// Probe the page: presence of the input, the send button, the answer container
/// and any login/captcha/error banner.
pub fn probe_script(sel: &Selectors) -> String {
let mut s = String::with_capacity(2048);
s.push_str("(() => {\n");
s.push_str(HELPERS);
s.push_str(&format!(
" const input = q({});\n",
js_string(&sel.input_field)
));
s.push_str(&format!(
" const send = q({});\n",
js_string(&sel.send_button)
));
s.push_str(&format!(
" const login = q({});\n",
js_string(&sel.login_page_indicator)
));
s.push_str(&format!(
" const captcha = q({});\n",
js_string(&sel.captcha_indicator)
));
s.push_str(&format!(
" const banner = q({});\n",
js_string(&sel.error_banner)
));
s.push_str(&format!(
" const stop = q({});\n",
js_string(&sel.streaming_indicator)
));
s.push_str(" return {\n");
s.push_str(" url: String(location.href),\n");
s.push_str(" title: String(document.title || ''),\n");
s.push_str(" ready_state: String(document.readyState || ''),\n");
s.push_str(" has_input: !!input,\n");
s.push_str(" input_len: txt(input).trim().length,\n");
s.push_str(" has_send: !!send,\n");
s.push_str(" send_disabled: send ? (send.disabled === true || send.getAttribute('aria-disabled') === 'true') : false,\n");
s.push_str(" has_login: !!login,\n");
s.push_str(" has_captcha: !!captcha,\n");
s.push_str(&format!(
" response_count: qa({}).length,\n",
js_string(&sel.response_container)
));
s.push_str(" streaming: !!stop,\n");
s.push_str(&format!(
" error_text: val({}),\n",
js_string(&sel.error_banner)
));
s.push_str(&format!(
" body_text: (document.body ? txt(document.body) : '').slice(0, {}).toLowerCase()\n",
BODY_SAMPLE_CHARS
));
s.push_str(" };\n})()");
s
}
/// Read the n-th answer container of the page.
pub fn read_response_script(sel: &Selectors, index: usize) -> String {
let mut s = String::with_capacity(1024);
s.push_str("(() => {\n");
s.push_str(HELPERS);
s.push_str(&format!(
" const nodes = qa({});\n",
js_string(&sel.response_container)
));
s.push_str(" const node = (nodes.length > ");
s.push_str(&index.to_string());
s.push_str(") ? nodes[");
s.push_str(&index.to_string());
s.push_str("] : null;\n");
s.push_str(" return {\n");
s.push_str(" count: nodes.length,\n");
s.push_str(" text: txt(node),\n");
s.push_str(&format!(
" streaming: !!(q({})),\n",
js_string(&sel.streaming_indicator)
));
s.push_str(&format!(
" error_text: val({}),\n",
js_string(&sel.error_banner)
));
s.push_str(&format!(
" has_login: !!(q({})),\n",
js_string(&sel.login_page_indicator)
));
s.push_str(&format!(
" has_captcha: !!(q({}))\n",
js_string(&sel.captcha_indicator)
));
s.push_str(" };\n})()");
s
}
/// Read the HTML and the visible text of the n-th answer node.
///
/// Used once, after the answer has settled: the HTML is what makes tables and
/// code fences survive, and it is too big to fetch on every poll.
pub fn read_answer_html_script(sel: &Selectors, index: usize) -> String {
let mut s = String::with_capacity(768);
s.push_str("(() => {\n");
s.push_str(HELPERS);
s.push_str(&format!(
" const nodes = qa({});\n",
js_string(&sel.response_container)
));
s.push_str(" const node = (nodes.length > ");
s.push_str(&index.to_string());
s.push_str(") ? nodes[");
s.push_str(&index.to_string());
s.push_str("] : null;\n");
s.push_str(" if (!node) return { html: '', text: '', found: false };\n");
s.push_str(" return { html: String(node.innerHTML || ''), text: txt(node), found: true };\n");
s.push_str("})()");
s
}
/// Click the provider copy button of the n-th answer node and read the
/// clipboard.
///
/// This is the highest fidelity source available: the web UI copies exactly the
/// Markdown it rendered, so tables, code fences and lists arrive intact. The
/// expression is asynchronous, and Chromium is asked to await its promise.
pub fn copy_answer_script(sel: &Selectors, index: usize, timeout_ms: u64) -> String {
let mut s = String::with_capacity(1280);
s.push_str("(async () => {\n");
s.push_str(HELPERS);
s.push_str(&format!(
" const nodes = qa({});\n",
js_string(&sel.response_container)
));
s.push_str(" const node = (nodes.length > ");
s.push_str(&index.to_string());
s.push_str(") ? nodes[");
s.push_str(&index.to_string());
s.push_str("] : null;\n");
s.push_str(" if (!node) return { clicked: false, clipboard: null, error: 'the answer node is gone' };\n");
// Toolbars of several web UIs only exist once the message has been hovered.
s.push_str(" try {\n");
s.push_str(" node.dispatchEvent(new MouseEvent('mouseover', { bubbles: true }));\n");
s.push_str(" node.dispatchEvent(new MouseEvent('mouseenter', { bubbles: true }));\n");
s.push_str(" } catch (e) {}\n");
s.push_str(" await new Promise((resolve) => setTimeout(resolve, 60));\n");
// The toolbar of a real web UI sits next to the answer text rather than
// inside it, so the button is searched from the answer node outwards: a
// descendant first, then the nearest button that follows it in document
// order within a few levels. Searching the whole page would click the copy
// button of a previous answer.
s.push_str(&format!(
" const candidates = qa({});\n",
js_string(&sel.copy_button)
));
s.push_str(" let button = null;\n");
s.push_str(" for (const candidate of candidates) {\n");
s.push_str(" if (node.contains(candidate)) { button = candidate; break; }\n");
s.push_str(" if (!(node.compareDocumentPosition(candidate) & 4)) continue;\n");
s.push_str(" let scope = candidate.parentElement;\n");
s.push_str(" let depth = 1;\n");
s.push_str(" while (scope && !scope.contains(node) && depth < 5) { scope = scope.parentElement; depth += 1; }\n");
s.push_str(" if (scope && depth <= 4) { button = candidate; break; }\n");
s.push_str(" }\n");
s.push_str(
" if (!button) return { clicked: false, clipboard: null, error: 'no copy button' };\n",
);
s.push_str(" try { button.click(); } catch (e) { return { clicked: false, clipboard: null, error: String(e) }; }\n");
s.push_str(" await new Promise((resolve) => setTimeout(resolve, 80));\n");
s.push_str(" if (!navigator.clipboard || !navigator.clipboard.readText) {\n");
s.push_str(" return { clicked: true, clipboard: null, error: 'navigator.clipboard is unavailable on this page' };\n");
s.push_str(" }\n");
s.push_str(" try {\n");
s.push_str(" const text = await Promise.race([\n");
s.push_str(" navigator.clipboard.readText(),\n");
s.push_str(&format!(
" new Promise((_, reject) => setTimeout(() => reject(new Error('the clipboard read timed out')), {timeout_ms}))\n"
));
s.push_str(" ]);\n");
s.push_str(" return { clicked: true, clipboard: typeof text === 'string' ? text : null, error: null };\n");
s.push_str(" } catch (e) {\n");
s.push_str(" return { clicked: true, clipboard: null, error: String(e && e.message ? e.message : e) };\n");
s.push_str(" }\n");
s.push_str("})()");
s
}
/// Insert text into the focused element using the DOM editing command.
pub fn exec_command_insert_script(text: &str) -> String {
let mut s = String::with_capacity(512);
s.push_str("(() => {\n");
s.push_str(HELPERS);
s.push_str(&format!(" const value = {};\n", js_string(text)));
s.push_str(" const el = document.activeElement;\n");
s.push_str(" if (!el) return { ok: false, len: 0, reason: 'no active element' };\n");
s.push_str(" try { document.execCommand('insertText', false, value); } catch (e) { return { ok: false, len: 0, reason: String(e) }; }\n");
s.push_str(" return { ok: true, len: txt(el).trim().length, reason: null };\n");
s.push_str("})()");
s
}
/// Check that the editor really contains the text we tried to inject.
pub fn verify_input_script(sel: &Selectors, expected_len: usize) -> String {
let mut s = String::with_capacity(512);
s.push_str("(() => {\n");
s.push_str(HELPERS);
s.push_str(&format!(
" const el = q({});\n",
js_string(&sel.input_field)
));
s.push_str(" if (!el) return { ok: false, len: 0, reason: 'input field missing' };\n");
s.push_str(
" const text = (typeof el.value === 'string' && el.value.length) ? el.value : txt(el);\n",
);
s.push_str(" const len = text.trim().length;\n");
s.push_str(&format!(
" return {{ ok: len >= {}, len: len, reason: len === 0 ? 'editor is empty' : null }};\n",
expected_len.min(8)
));
s.push_str("})()");
s
}
/// Remove whatever is already in the editor before injecting the prompt.
pub fn clear_input_script(sel: &Selectors) -> String {
let mut s = String::with_capacity(512);
s.push_str("(() => {\n");
s.push_str(HELPERS);
s.push_str(&format!(
" const el = q({});\n",
js_string(&sel.input_field)
));
s.push_str(" if (!el) return { ok: false, len: 0, reason: 'input field missing' };\n");
s.push_str(" el.focus();\n");
s.push_str(" try { document.execCommand('selectAll', false, null); document.execCommand('delete', false, null); } catch (e) { return { ok: false, len: 0, reason: String(e) }; }\n");
s.push_str(" const len = (typeof el.value === 'string' && el.value.length) ? el.value.length : txt(el).trim().length;\n");
s.push_str(" return { ok: true, len: len, reason: null };\n");
s.push_str("})()");
s
}
/// Number of body characters sampled for rate-limit detection.
pub const BODY_SAMPLE_CHARS: usize = 40_000;
#[cfg(test)]
mod tests {
use super::*;
fn selectors() -> Selectors {
Selectors {
input_field: "div#prompt-textarea".into(),
send_button: "button[data-testid='send-button']".into(),
response_container: "div[data-message-author-role='assistant']".into(),
streaming_indicator: "button[data-testid='stop-button']".into(),
file_input: "input[type='file']".into(),
new_chat_button: String::new(),
login_page_indicator: "form[action*='login']".into(),
captcha_indicator: "iframe[src*='captcha']".into(),
error_banner: "[role='alert']".into(),
model_selector: String::new(),
attachment_indicator: String::new(),
copy_button: String::new(),
}
}
#[test]
fn scripts_embed_selectors_and_are_syntactically_closed() {
let sel = selectors();
let probe = probe_script(&sel);
assert!(probe.contains("div#prompt-textarea"));
assert!(probe.contains("button[data-testid='send-button']"));
assert!(probe.starts_with("(() => {"));
assert!(probe.trim_end().ends_with("})()"));
assert!(!probe.contains("{input}"));
let read = read_response_script(&sel, 2);
assert!(read.contains("nodes[2]"));
assert!(read.trim_end().ends_with("})()"));
}
#[test]
fn text_is_escaped_not_interpolated() {
let script = exec_command_insert_script("a \" b\nline 'quote' </script>");
assert!(script.contains(r#""a \" b\nline 'quote' </script>""#));
assert!(script.trim_end().ends_with("})()"));
}
#[test]
fn the_copy_script_awaits_the_clipboard_and_has_a_deadline() {
let mut sel = selectors();
sel.copy_button = "button[aria-label='Copy']".into();
let script = copy_answer_script(&sel, 1, 2000);
assert!(script.starts_with("(async () => {"));
assert!(script.trim_end().ends_with("})()"));
assert!(script.contains("nodes[1]"));
assert!(script.contains("navigator.clipboard.readText()"));
assert!(script.contains("setTimeout"));
assert!(script.contains("2000"));
assert!(script.contains("button[aria-label='Copy']"));
// The toolbar of several web UIs only exists after a hover.
assert!(script.contains("mouseover"));
// The button is searched from the answer node outwards, never from the
// whole page, so a previous answer's button is never clicked.
assert!(script.contains("node.compareDocumentPosition"));
assert!(script.contains("node.contains(candidate)"));
}
#[test]
fn the_html_script_returns_the_node_markup() {
let script = read_answer_html_script(&selectors(), 0);
assert!(script.contains("innerHTML"));
assert!(script.contains("nodes[0]"));
assert!(script.trim_end().ends_with("})()"));
}
#[test]
fn verification_uses_a_small_threshold() {
let script = verify_input_script(&selectors(), 400);
assert!(script.contains("len >= 8"));
}
}
src/browser/mod.rs
+15
View File
@@ -0,0 +1,15 @@
//! Browser automation: engine lifecycle, DOM helpers and session state.
pub mod cdp;
pub mod debug;
pub mod dom;
pub mod engine;
pub mod js;
pub mod pool;
pub mod session;
pub use cdp::{executable_hint, external_launch, resolve_executable, CdpEngine, ExternalLaunch};
pub use debug::{CapturedArtifacts, DebugRecorder};
pub use engine::{BrowserEngine, BrowserSettings, TabStats};
pub use pool::{TabLease, TabPool};
pub use session::{Session, SessionState, SessionStatus};
src/browser/pool.rs
+237
View File
@@ -0,0 +1,237 @@
//! Tab admission control.
//!
//! One provider account can run up to `max_tabs` turns at the same time. A turn
//! leases a tab for its whole duration: the lease is returned to the pool when
//! it is dropped, which is also what happens when a turn fails or its client
//! disappears mid-generation. A request that cannot get a tab within the
//! configured budget is rejected with 429 instead of queueing forever.
use std::sync::{Arc, Mutex};
use std::time::{Duration, Instant};
use chromiumoxide::Page;
use tokio::sync::{OwnedSemaphorePermit, Semaphore};
use crate::error::{GatewayError, Result};
/// Slots of one pool, shared with the leases handed out from it.
#[derive(Debug)]
struct Slots {
idle: Mutex<Vec<usize>>,
}
/// Bounded pool of browser tabs belonging to one provider account.
#[derive(Debug, Clone)]
pub struct TabPool {
owner: Arc<str>,
max_tabs: usize,
wait: Duration,
permits: Arc<Semaphore>,
slots: Arc<Slots>,
}
impl TabPool {
pub fn new(owner: impl Into<String>, max_tabs: usize, wait: Duration) -> Self {
let max_tabs = max_tabs.max(1);
Self {
owner: Arc::from(owner.into().as_str()),
max_tabs,
wait,
permits: Arc::new(Semaphore::new(max_tabs)),
slots: Arc::new(Slots {
idle: Mutex::new((0..max_tabs).collect()),
}),
}
}
pub fn max_tabs(&self) -> usize {
self.max_tabs
}
/// Number of tabs currently leased.
pub fn busy(&self) -> usize {
self.max_tabs - self.permits.available_permits()
}
/// Reserve a slot, waiting at most the configured budget.
pub async fn reserve(&self) -> Result<Reservation> {
self.reserve_within(self.wait).await
}
/// Reserve a slot, waiting at most `wait`.
///
/// A status probe uses a much shorter budget than a turn: it must answer
/// quickly rather than queue behind a generation in progress.
pub async fn reserve_within(&self, wait: Duration) -> Result<Reservation> {
let started = Instant::now();
let permit = match tokio::time::timeout(wait, self.permits.clone().acquire_owned()).await {
Ok(Ok(permit)) => permit,
Ok(Err(_)) => return Err(GatewayError::internal("the tab pool is closed")),
Err(_) => {
return Err(GatewayError::ProviderBusy {
provider: self.owner.to_string(),
waited_ms: started.elapsed().as_millis() as u64,
})
}
};
// Holding a permit guarantees that a slot is free: a slot is out of the
// idle list exactly while its permit is held.
let slot = self
.slots
.idle
.lock()
.map_err(|_| GatewayError::internal("the tab pool is poisoned"))?
.pop()
.ok_or_else(|| GatewayError::internal("the tab pool lost a slot"))?;
Ok(Reservation {
slot,
pool: self.clone(),
permit: Some(permit),
})
}
fn return_slot(&self, slot: usize) {
match self.slots.idle.lock() {
Ok(mut idle) => idle.push(slot),
Err(err) => tracing::warn!(error = %err, "cannot return a tab slot to the pool"),
}
}
}
/// A reserved slot, not bound to a page yet.
///
/// Creating the page can fail, so an unconsumed reservation gives its slot back
/// on drop rather than leaking it for the life of the process.
#[derive(Debug)]
pub struct Reservation {
slot: usize,
pool: TabPool,
permit: Option<OwnedSemaphorePermit>,
}
impl Reservation {
pub fn slot(&self) -> usize {
self.slot
}
/// Bind the reservation to the page created for it.
pub fn into_lease(mut self, page: Page) -> TabLease {
let permit = self
.permit
.take()
.expect("a reservation is consumed at most once");
TabLease {
page,
slot: self.slot,
pool: self.pool.clone(),
_permit: permit,
}
}
}
impl Drop for Reservation {
fn drop(&mut self) {
// Only when the reservation was never turned into a lease: the permit
// is released after the slot is back in the list.
if self.permit.is_some() {
self.pool.return_slot(self.slot);
}
}
}
/// One browser tab, leased for the duration of a turn.
pub struct TabLease {
page: Page,
slot: usize,
pool: TabPool,
_permit: OwnedSemaphorePermit,
}
impl TabLease {
pub fn page(&self) -> &Page {
&self.page
}
pub fn slot(&self) -> usize {
self.slot
}
}
impl std::fmt::Debug for TabLease {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("TabLease")
.field("slot", &self.slot)
.field("max_tabs", &self.pool.max_tabs())
.finish()
}
}
impl Drop for TabLease {
fn drop(&mut self) {
// The slot goes back before the permit is released, so a waiter that
// grabs the permit always finds a slot in the idle list.
self.pool.return_slot(self.slot);
}
}
#[cfg(test)]
mod tests {
use super::*;
#[tokio::test]
async fn a_second_request_waits_then_times_out() {
let pool = TabPool::new("chatgpt", 1, Duration::from_millis(120));
let held = pool.reserve().await.unwrap();
assert_eq!(pool.busy(), 1);
let err = pool.reserve().await.unwrap_err();
assert_eq!(err.http_status(), 429);
assert_eq!(err.code(), "provider_busy");
drop(held);
assert_eq!(pool.busy(), 0);
assert!(pool.reserve().await.is_ok());
}
#[tokio::test]
async fn parallel_tabs_are_allowed_up_to_the_limit() {
let pool = TabPool::new("chatgpt", 3, Duration::from_millis(50));
let a = pool.reserve().await.unwrap();
let b = pool.reserve().await.unwrap();
let c = pool.reserve().await.unwrap();
assert_eq!(pool.busy(), 3);
assert!(pool.reserve().await.is_err());
// Every reservation got a distinct slot.
let mut slots = vec![a.slot(), b.slot(), c.slot()];
slots.sort_unstable();
assert_eq!(slots, vec![0, 1, 2]);
drop(b);
assert_eq!(pool.busy(), 2);
let d = pool.reserve().await.unwrap();
assert_eq!(d.slot(), 1);
}
#[tokio::test]
async fn a_dropped_reservation_returns_its_slot() {
let pool = TabPool::new("chatgpt", 2, Duration::from_millis(10));
{
let _reserved = pool.reserve().await.unwrap();
assert_eq!(pool.busy(), 1);
}
// Creating the page failed: the slot must not be lost forever.
assert_eq!(pool.busy(), 0);
let a = pool.reserve().await.unwrap();
let b = pool.reserve().await.unwrap();
assert_ne!(a.slot(), b.slot());
}
#[test]
fn a_zero_tab_pool_still_serves_one_request() {
let pool = TabPool::new("chatgpt", 0, Duration::from_millis(10));
assert_eq!(pool.max_tabs(), 1);
}
}
src/browser/session.rs
+453
View File
@@ -0,0 +1,453 @@
//! A provider browser session: the reusable tab plus its observed state.
use std::sync::Arc;
use std::time::{Duration, Instant};
use chromiumoxide::Page;
use serde::{Deserialize, Serialize};
use tokio::sync::RwLock;
use crate::error::{GatewayError, Result};
use crate::providers::config::ProviderConfig;
use crate::util;
use super::dom;
use super::engine::BrowserEngine;
use super::js::{self, ProbeResult};
use super::pool::TabLease;
/// Coarse state of a provider session.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum SessionState {
Available,
RequiresLogin,
Captcha,
Busy,
Misconfigured,
Error,
}
impl SessionState {
pub fn as_str(self) -> &'static str {
match self {
Self::Available => "available",
Self::RequiresLogin => "requires_login",
Self::Captcha => "captcha",
Self::Busy => "busy",
Self::Misconfigured => "misconfigured",
Self::Error => "error",
}
}
}
/// Detailed status returned by the status and validate endpoints.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct SessionStatus {
pub provider: String,
pub state: SessionState,
#[serde(skip_serializing_if = "Option::is_none")]
pub detail: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub url: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub browser: Option<String>,
pub checked_at: u64,
}
impl SessionStatus {
fn build(
provider: &str,
state: SessionState,
detail: Option<String>,
url: Option<String>,
) -> Self {
Self {
provider: provider.to_string(),
state,
detail,
url,
browser: None,
checked_at: util::now_secs(),
}
}
pub fn available(provider: &str, url: Option<String>) -> Self {
Self::build(provider, SessionState::Available, None, url)
}
pub fn requires_login(provider: &str, detail: impl Into<String>, url: Option<String>) -> Self {
Self::build(
provider,
SessionState::RequiresLogin,
Some(detail.into()),
url,
)
}
pub fn captcha(provider: &str, url: Option<String>) -> Self {
Self::build(
provider,
SessionState::Captcha,
Some("captcha challenge displayed".into()),
url,
)
}
pub fn error(provider: &str, detail: impl Into<String>) -> Self {
Self::build(provider, SessionState::Error, Some(detail.into()), None)
}
pub fn misconfigured(provider: &str, detail: impl Into<String>) -> Self {
Self::build(
provider,
SessionState::Misconfigured,
Some(detail.into()),
None,
)
}
pub fn busy(provider: &str) -> Self {
Self::build(provider, SessionState::Busy, None, None)
}
pub fn with_browser(mut self, browser: &str) -> Self {
self.browser = Some(browser.to_string());
self
}
pub fn is_available(&self) -> bool {
self.state == SessionState::Available
}
/// Translate the status into the error a client should receive.
pub fn as_error(&self) -> GatewayError {
match self.state {
SessionState::Available => GatewayError::internal("session is available"),
SessionState::RequiresLogin => GatewayError::RequiresLogin {
provider: self.provider.clone(),
},
SessionState::Captcha => GatewayError::Captcha {
provider: self.provider.clone(),
},
SessionState::Busy => GatewayError::ProviderBusy {
provider: self.provider.clone(),
waited_ms: 0,
},
SessionState::Misconfigured => GatewayError::ProviderMisconfigured {
provider: self.provider.clone(),
detail: self.detail.clone().unwrap_or_default(),
},
SessionState::Error => GatewayError::UpstreamError {
provider: self.provider.clone(),
message: self
.detail
.clone()
.unwrap_or_else(|| "unknown error".into()),
},
}
}
}
/// A live page bound to one provider account. Cheap to clone: every field is
/// shared, and the tab is released only when the last clone goes away.
#[derive(Clone)]
pub struct Session {
provider: Arc<str>,
account: Option<Arc<str>>,
browser: Arc<str>,
engine: Arc<dyn BrowserEngine>,
lease: Arc<TabLease>,
web_url: Arc<RwLock<Option<String>>>,
}
impl std::fmt::Debug for Session {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("Session")
.field("provider", &self.provider)
.field("account", &self.account)
.field("browser", &self.browser)
.field("tab_slot", &self.lease.slot())
.finish()
}
}
impl Session {
/// Bind a leased tab to its provider account.
pub fn new(
provider: impl Into<String>,
browser: impl Into<String>,
engine: Arc<dyn BrowserEngine>,
lease: TabLease,
) -> Self {
let account = engine
.settings()
.account
.as_deref()
.filter(|value| !value.is_empty())
.map(Arc::<str>::from);
Self {
provider: Arc::from(provider.into().as_str()),
account,
browser: Arc::from(browser.into().as_str()),
engine,
lease: Arc::new(lease),
web_url: Arc::new(RwLock::new(None)),
}
}
/// Shared page handle, for spawning a turn into its own task.
pub fn page_handle(&self) -> Arc<Page> {
Arc::new(self.lease.page().clone())
}
pub fn provider(&self) -> &str {
&self.provider
}
/// Account of this session, when the provider declares several.
pub fn account(&self) -> Option<&str> {
self.account.as_deref()
}
/// Provider and account, as used in logs and error messages.
pub fn owner(&self) -> String {
match self.account() {
Some(account) => format!("{}@{account}", self.provider),
None => self.provider.to_string(),
}
}
pub fn browser(&self) -> &str {
&self.browser
}
pub fn page(&self) -> &Page {
self.lease.page()
}
/// Slot of the tab pool this session holds, for diagnostics.
pub fn tab_slot(&self) -> usize {
self.lease.slot()
}
/// Engine that owns the browser of this session.
pub fn engine(&self) -> &Arc<dyn BrowserEngine> {
&self.engine
}
/// Allow the page to read and write the clipboard.
///
/// The gateway reads the provider's own Markdown through the clipboard, and
/// Chrome only allows that when the permission was granted over CDP (the
/// page cannot show a permission prompt during automation).
pub async fn grant_clipboard_access(&self) -> Result<()> {
for name in ["clipboard-read", "clipboard-write"] {
if let Err(err) = self.engine.grant_permission(name).await {
tracing::debug!(permission = name, error = %err, "cannot grant a clipboard permission");
}
}
Ok(())
}
/// URL of the web conversation currently bound to this session.
pub async fn web_url(&self) -> Option<String> {
self.web_url.read().await.clone()
}
pub async fn set_web_url(&self, url: Option<String>) {
let mut guard = self.web_url.write().await;
*guard = url;
}
/// Read the page state with a bounded timeout.
pub async fn probe(&self, cfg: &ProviderConfig, timeout: Duration) -> Result<ProbeResult> {
let script = js::probe_script(cfg.selectors());
match tokio::time::timeout(
timeout,
dom::evaluate_json::<ProbeResult>(self.page(), &script),
)
.await
{
Ok(result) => result,
Err(_) => Err(GatewayError::CaptureTimeout {
stage: "probing the provider page".into(),
timeout_s: timeout.as_secs(),
}),
}
}
/// Probe the page until it shows something meaningful, or the budget runs out.
///
/// A page that is still hydrating looks empty, and classifying it too early
/// reports a missing selector on a perfectly healthy provider.
pub async fn settled_probe(
&self,
cfg: &ProviderConfig,
budget: Duration,
) -> Result<ProbeResult> {
let deadline = Instant::now() + budget;
let mut probe = self.probe(cfg, budget).await?;
while Instant::now() < deadline {
if page_is_meaningful(cfg, &probe) {
break;
}
tokio::time::sleep(Duration::from_millis(250)).await;
probe = self.probe(cfg, budget).await?;
}
Ok(probe)
}
/// Classify the current page into a status.
pub async fn status(&self, cfg: &ProviderConfig, timeout: Duration) -> SessionStatus {
if !cfg.is_usable() {
return SessionStatus::misconfigured(cfg.name(), cfg.problems.join("; "))
.with_browser(&self.browser);
}
match self.settled_probe(cfg, timeout).await {
Ok(probe) => classify(cfg, &probe, &self.browser),
Err(err) => {
SessionStatus::error(cfg.name(), err.to_string()).with_browser(&self.browser)
}
}
}
}
/// True when the probe says something usable about the page: the chat editor,
/// a challenge, or a sign-in screen. An empty shell means "still loading".
pub fn page_is_meaningful(cfg: &ProviderConfig, probe: &ProbeResult) -> bool {
probe.has_input
|| probe.has_captcha
|| cfg.is_signed_out_page(
&probe.url,
&probe.body_text,
probe.has_login,
probe.has_input,
)
}
/// Turn a page probe into a status.
pub fn classify(cfg: &ProviderConfig, probe: &ProbeResult, browser: &str) -> SessionStatus {
let provider = cfg.name();
let url = if probe.url.is_empty() {
None
} else {
Some(probe.url.clone())
};
let status = if !cfg.is_usable() {
SessionStatus::misconfigured(provider, cfg.problems.join("; "))
} else if probe.has_captcha {
SessionStatus::captcha(provider, url.clone())
} else if cfg.is_signed_out_page(
&probe.url,
&probe.body_text,
probe.has_login,
probe.has_input,
) {
SessionStatus::requires_login(
provider,
format!(
"not signed in: run 'llm-gateway login {provider} --manual', sign in inside the \
window it opens, then close that window"
),
url.clone(),
)
} else if !probe.has_input {
SessionStatus::error(
provider,
format!(
"prompt field not found on {} (selectors.input_field = {})",
if probe.url.is_empty() {
"an unknown page"
} else {
probe.url.as_str()
},
cfg.selectors().input_field
),
)
} else {
SessionStatus::available(provider, url.clone())
};
status.with_browser(browser)
}
#[cfg(test)]
mod tests {
use super::*;
use crate::providers::config::ProviderConfig;
const CHATGPT: &str = include_str!("../../providers/chatgpt.toml");
fn cfg() -> ProviderConfig {
ProviderConfig::parse(CHATGPT, "chatgpt.toml").unwrap()
}
fn probe(url: &str) -> ProbeResult {
ProbeResult {
url: url.to_string(),
has_input: true,
has_send: true,
..Default::default()
}
}
#[test]
fn a_ready_page_is_available() {
let status = classify(&cfg(), &probe("https://chatgpt.com/"), "chromium-cdp");
assert_eq!(status.state, SessionState::Available);
assert!(status.is_available());
assert_eq!(status.browser.as_deref(), Some("chromium-cdp"));
}
#[test]
fn login_pages_are_detected_by_url_or_marker() {
let by_url = classify(&cfg(), &probe("https://chatgpt.com/auth/login"), "cdp");
assert_eq!(by_url.state, SessionState::RequiresLogin);
assert_eq!(by_url.as_error().code(), "requires_login");
let mut markup = probe("https://chatgpt.com/");
markup.has_login = true;
assert_eq!(
classify(&cfg(), &markup, "cdp").state,
SessionState::RequiresLogin
);
}
#[test]
fn captcha_wins_over_everything() {
let mut markup = probe("https://chatgpt.com/");
markup.has_captcha = true;
markup.has_login = true;
let status = classify(&cfg(), &markup, "cdp");
assert_eq!(status.state, SessionState::Captcha);
assert_eq!(status.as_error().http_status(), 403);
}
#[test]
fn the_real_signed_out_landing_page_is_reported_as_a_login_page() {
// Markup captured from chatgpt.com while signed out: no prompt field,
// "Log in" / "Sign up" buttons, no login URL.
let mut landing = probe("https://chatgpt.com/");
landing.has_input = false;
landing.body_text = "chatgpt: chat, work, create & code with ai log in sign up".to_string();
let status = classify(&cfg(), &landing, "cdp");
assert_eq!(status.state, SessionState::RequiresLogin);
assert!(status.detail.as_deref().unwrap().contains("--manual"));
assert!(page_is_meaningful(&cfg(), &landing));
}
#[test]
fn a_missing_input_field_is_reported_with_the_selector() {
let mut markup = probe("https://chatgpt.com/");
markup.has_input = false;
let status = classify(&cfg(), &markup, "cdp");
assert_eq!(status.state, SessionState::Error);
assert!(status
.detail
.as_deref()
.unwrap()
.contains("prompt-textarea"));
}
}
src/cli.rs
+380
View File
@@ -0,0 +1,380 @@
//! Command line interface definition.
use std::path::PathBuf;
use clap::{Args, Parser, Subcommand};
/// OpenAI-compatible gateway driving a real browser session on LLM web UIs.
#[derive(Debug, Parser)]
#[command(
name = "llm-gateway",
version = crate::VERSION,
about = "Expose an OpenAI-compatible API backed by an authenticated browser session",
long_about = None,
propagate_version = true
)]
pub struct Cli {
#[command(flatten)]
pub global: GlobalArgs,
#[command(subcommand)]
pub command: Option<Command>,
}
/// Flags accepted by every subcommand.
#[derive(Debug, Clone, Args, Default)]
pub struct GlobalArgs {
/// Address to bind (default 127.0.0.1).
#[arg(long, global = true, env = "LLM_GATEWAY_HOST", value_name = "HOST")]
pub host: Option<String>,
/// Port of the API server (default 8080).
#[arg(
long,
short,
global = true,
env = "LLM_GATEWAY_PORT",
value_name = "PORT"
)]
pub port: Option<u16>,
/// Browser family: chrome, chromium, edge, brave.
#[arg(long, global = true, value_name = "NAME")]
pub browser: Option<String>,
/// Explicit path to the browser executable.
#[arg(long, global = true, value_name = "PATH")]
pub browser_path: Option<PathBuf>,
/// Root directory of the persistent browser profiles.
#[arg(long, global = true, value_name = "PATH")]
pub profile_dir: Option<PathBuf>,
/// Restrict the server to a single provider.
#[arg(long, global = true, value_name = "NAME")]
pub provider: Option<String>,
/// Run the browser without a window (no manual login possible).
#[arg(long, global = true)]
pub headless: bool,
/// Attach to a browser you started instead of launching one.
#[arg(long, global = true)]
pub attach: bool,
/// Debug port of the browser to attach to.
#[arg(long, global = true, value_name = "PORT")]
pub debug_port: Option<u16>,
/// Automation hiding: off (default), minimal or aggressive.
#[arg(long, global = true, value_name = "MODE")]
pub stealth: Option<String>,
/// Log level: trace, debug, info, warn, error.
#[arg(long, global = true, env = "LLM_GATEWAY_LOG", value_name = "LEVEL")]
pub log_level: Option<String>,
/// Log format: pretty or json.
#[arg(long, global = true, value_name = "FORMAT")]
pub log_format: Option<String>,
/// Configuration file (default <state dir>/config.toml).
#[arg(long, global = true, env = "LLM_GATEWAY_CONFIG", value_name = "PATH")]
pub config: Option<PathBuf>,
/// Save screenshots and DOM dumps for every request.
#[arg(long, global = true)]
pub debug: bool,
/// Maximum number of tabs per provider.
#[arg(long, global = true, value_name = "N")]
pub max_tabs: Option<usize>,
/// Require this bearer token from API clients.
#[arg(long, global = true, value_name = "TOKEN")]
pub api_key: Option<String>,
/// Fidelity of the captured answer: auto, clipboard, dom or text.
#[arg(long, global = true, value_name = "MODE")]
pub markdown: Option<String>,
/// Skip the confirmation prompts.
#[arg(long, short = 'y', global = true)]
pub yes: bool,
}
#[derive(Debug, Subcommand)]
pub enum Command {
/// Start the OpenAI-compatible API server.
Serve(ServeArgs),
/// Send a prompt to a provider and print the captured answer.
Test(TestArgs),
/// Check that the configured selectors and the session still work.
Selftest(SelftestArgs),
/// List the configured providers.
List(ListArgs),
/// Open a browser window to sign in to a provider.
Login(LoginArgs),
/// Delete the persistent browser profile of a provider.
Logout(LogoutArgs),
/// Inspect or clear the persisted conversations.
Conversations(ConversationsArgs),
/// Ask a running server to reload its configuration.
Reload,
/// Configuration helpers.
Config(ConfigArgs),
}
#[derive(Debug, Args)]
pub struct ServeArgs {
/// Provider backend: browser (default) or mock (no browser at all).
#[arg(long, value_name = "KIND", default_value = "browser")]
pub backend: String,
}
#[derive(Debug, Args)]
pub struct TestArgs {
/// Provider to test, optionally one of its accounts: `chatgpt@perso`.
pub provider: Option<String>,
/// Test every configured provider.
#[arg(long)]
pub all: bool,
/// Prompt to send.
#[arg(long, value_name = "TEXT")]
pub prompt: Option<String>,
/// Print the result as JSON.
#[arg(long)]
pub json: bool,
}
#[derive(Debug, Args)]
pub struct SelftestArgs {
/// Provider to check, optionally one of its accounts.
pub provider: Option<String>,
/// Check every configured provider.
#[arg(long)]
pub all: bool,
/// Print the report as JSON.
#[arg(long)]
pub json: bool,
}
#[derive(Debug, Args)]
pub struct ListArgs {
/// Print the provider list as JSON.
#[arg(long)]
pub json: bool,
}
#[derive(Debug, Args)]
pub struct LoginArgs {
/// Provider name, optionally followed by the account: `chatgpt@perso`.
pub provider: String,
/// Account to sign in, when the provider declares several.
#[arg(long, value_name = "ID")]
pub account: Option<String>,
/// Print how to sign in from a browser you start yourself.
///
/// This is the configuration OAuth providers such as Google accept, since
/// no automation flag and no debugging port is involved.
#[arg(long)]
pub manual: bool,
/// Start the printed command for you.
#[arg(long, requires = "manual")]
pub launch: bool,
}
#[derive(Debug, Args)]
pub struct LogoutArgs {
/// Provider to forget, optionally followed by the account.
pub provider: Option<String>,
/// Account to forget, when the provider declares several.
#[arg(long, value_name = "ID")]
pub account: Option<String>,
/// Forget every provider, or every account of the named one.
#[arg(long)]
pub all: bool,
}
#[derive(Debug, Args)]
pub struct ConversationsArgs {
#[command(subcommand)]
pub command: ConversationsCommand,
}
#[derive(Debug, Subcommand)]
pub enum ConversationsCommand {
/// List the stored conversations.
List {
/// Provider to filter on, optionally with its account: chatgpt@perso.
#[arg(long, value_name = "NAME")]
provider: Option<String>,
/// Account to filter on; "default" selects the threads of a provider
/// that declares no account.
#[arg(long, value_name = "ID")]
account: Option<String>,
#[arg(long)]
json: bool,
},
/// Remove stored conversations.
Clear {
/// Provider to filter on, optionally with its account: chatgpt@perso.
#[arg(long, value_name = "NAME")]
provider: Option<String>,
/// Account to filter on; "default" selects the threads of a provider
/// that declares no account.
#[arg(long, value_name = "ID")]
account: Option<String>,
},
}
#[derive(Debug, Args)]
pub struct ConfigArgs {
#[command(subcommand)]
pub command: ConfigCommand,
}
#[derive(Debug, Subcommand)]
pub enum ConfigCommand {
/// Print the configuration file path.
Path,
/// Create the state directory and the default files.
Init {
/// Overwrite the installed provider files with the shipped defaults.
///
/// The previous file is kept as <provider>.toml.bak. Use it after an
/// upgrade to pick up refreshed selectors.
#[arg(long)]
force: bool,
},
/// Print the effective configuration.
Show,
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn global_flags_are_accepted_before_and_after_the_subcommand() {
let cli = Cli::try_parse_from(["llm-gateway", "--port", "9000", "list"]).unwrap();
assert_eq!(cli.global.port, Some(9000));
assert!(matches!(cli.command, Some(Command::List(_))));
let cli = Cli::try_parse_from(["llm-gateway", "list", "--port", "9001"]).unwrap();
assert_eq!(cli.global.port, Some(9001));
}
#[test]
fn serve_defaults_to_the_browser_backend() {
let cli = Cli::try_parse_from(["llm-gateway", "serve"]).unwrap();
match cli.command {
Some(Command::Serve(args)) => assert_eq!(args.backend, "browser"),
other => panic!("unexpected command: {other:?}"),
}
}
#[test]
fn login_accepts_the_manual_flags() {
let cli = Cli::try_parse_from(["llm-gateway", "login", "chatgpt", "--manual", "--launch"])
.unwrap();
match cli.command {
Some(Command::Login(args)) => {
assert_eq!(args.provider, "chatgpt");
assert!(args.manual);
assert!(args.launch);
}
other => panic!("unexpected: {other:?}"),
}
assert!(Cli::try_parse_from(["llm-gateway", "login", "chatgpt", "--launch"]).is_err());
}
#[test]
fn accounts_can_be_named_on_login_and_logout() {
let cli =
Cli::try_parse_from(["llm-gateway", "login", "chatgpt", "--account", "perso"]).unwrap();
match cli.command {
Some(Command::Login(args)) => assert_eq!(args.account.as_deref(), Some("perso")),
other => panic!("unexpected: {other:?}"),
}
// The account may also ride along with the provider name.
let cli = Cli::try_parse_from(["llm-gateway", "logout", "chatgpt@perso"]).unwrap();
match cli.command {
Some(Command::Logout(args)) => {
assert_eq!(args.provider.as_deref(), Some("chatgpt@perso"));
assert_eq!(args.account, None);
}
other => panic!("unexpected: {other:?}"),
}
}
#[test]
fn the_markdown_mode_is_a_global_flag() {
let cli = Cli::try_parse_from(["llm-gateway", "serve", "--markdown", "clipboard"]).unwrap();
assert_eq!(cli.global.markdown.as_deref(), Some("clipboard"));
}
#[test]
fn attach_flags_are_global() {
let cli = Cli::try_parse_from(["llm-gateway", "serve", "--attach", "--debug-port", "9333"])
.unwrap();
assert!(cli.global.attach);
assert_eq!(cli.global.debug_port, Some(9333));
}
#[test]
fn conversations_has_nested_subcommands() {
let cli = Cli::try_parse_from(["llm-gateway", "conversations", "clear", "--provider", "x"])
.unwrap();
match cli.command {
Some(Command::Conversations(args)) => match args.command {
ConversationsCommand::Clear { provider, account } => {
assert_eq!(provider.as_deref(), Some("x"));
assert_eq!(account, None);
}
other => panic!("unexpected: {other:?}"),
},
other => panic!("unexpected: {other:?}"),
}
}
#[test]
fn conversations_can_be_filtered_by_account() {
let cli = Cli::try_parse_from([
"llm-gateway",
"conversations",
"list",
"--provider",
"chatgpt",
"--account",
"perso",
"--json",
])
.unwrap();
match cli.command {
Some(Command::Conversations(args)) => match args.command {
ConversationsCommand::List {
provider,
account,
json,
} => {
assert_eq!(provider.as_deref(), Some("chatgpt"));
assert_eq!(account.as_deref(), Some("perso"));
assert!(json);
}
other => panic!("unexpected: {other:?}"),
},
other => panic!("unexpected: {other:?}"),
}
}
#[test]
fn no_subcommand_means_serve() {
let cli = Cli::try_parse_from(["llm-gateway"]).unwrap();
assert!(cli.command.is_none());
}
}
src/config.rs
+562
View File
@@ -0,0 +1,562 @@
//! Global configuration: TOML file, defaults and CLI overrides.
use std::path::{Path, PathBuf};
use serde::{Deserialize, Serialize};
use crate::error::{GatewayError, Result};
use crate::paths::{self, Layout};
/// When to save screenshots and DOM dumps for debugging.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize, Serialize, Default)]
#[serde(rename_all = "snake_case")]
pub enum ScreenshotMode {
Never,
#[default]
OnError,
Always,
}
/// How a thread is continued on the provider web UI.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize, Serialize, Default)]
#[serde(rename_all = "snake_case")]
pub enum ConversationStrategy {
#[default]
Auto,
Reuse,
Replay,
}
/// Where the client conversation id comes from.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize, Serialize, Default)]
#[serde(rename_all = "snake_case")]
pub enum ConversationIdSource {
#[default]
Fingerprint,
Header,
Uuid,
}
/// How much the browser hides the fact that it is automated.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize, Serialize, Default)]
#[serde(rename_all = "snake_case")]
pub enum StealthMode {
/// Leave the page alone. Chromium is launched with a switch that already
/// hides navigator.webdriver and the user agent stays genuine, which is
/// what sign-in providers such as Google expect.
#[default]
Off,
/// Also patch navigator.webdriver from an injected script.
Minimal,
/// Full chromiumoxide stealth, including a spoofed user agent.
///
/// Warning: it advertises an outdated Chrome version, which is itself a
/// bot signal and makes OAuth providers refuse the sign-in.
Aggressive,
}
impl StealthMode {
/// Parse the --stealth flag.
pub fn parse(raw: &str) -> Result<Self> {
match raw.trim().to_lowercase().as_str() {
"off" | "none" => Ok(Self::Off),
"minimal" | "light" => Ok(Self::Minimal),
"aggressive" | "full" => Ok(Self::Aggressive),
other => Err(GatewayError::Config(format!(
"unknown stealth mode '{other}', expected off, minimal or aggressive"
))),
}
}
}
/// How the prompt text is pushed into the web editor.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize, Serialize, Default)]
#[serde(rename_all = "snake_case")]
pub enum InjectMethod {
#[default]
Auto,
InsertText,
ExecCommand,
TypeStr,
}
/// Where the final answer text comes from.
///
/// A web UI renders its answer as rich HTML, and `innerText` flattens it: code
/// fences, tables and lists all collapse into plain text. The fidelity pass
/// runs once, when the answer has settled, and this setting decides how.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize, Serialize, Default)]
#[serde(rename_all = "snake_case")]
pub enum MarkdownMode {
/// Clipboard when the provider has a copy button, otherwise the DOM.
#[default]
Auto,
/// Always click the provider copy button and read the clipboard over CDP.
/// Falls back to the DOM when the clipboard cannot be read.
Clipboard,
/// Always convert the answer node's HTML to Markdown.
Dom,
/// Keep the historical behaviour: the visible text, formatting flattened.
Text,
}
impl MarkdownMode {
/// Parse the --markdown flag.
pub fn parse(raw: &str) -> Result<Self> {
match raw.trim().to_lowercase().as_str() {
"auto" => Ok(Self::Auto),
"clipboard" | "copy" => Ok(Self::Clipboard),
"dom" | "html" => Ok(Self::Dom),
"text" | "plain" | "off" => Ok(Self::Text),
other => Err(GatewayError::Config(format!(
"unknown markdown mode '{other}', expected auto, clipboard, dom or text"
))),
}
}
}
#[derive(Debug, Clone, Deserialize, Serialize)]
#[serde(default)]
pub struct ServerSection {
pub host: String,
pub port: u16,
pub api_key: String,
pub include_warnings: bool,
pub request_body_limit: usize,
}
impl Default for ServerSection {
fn default() -> Self {
Self {
host: "127.0.0.1".to_string(),
port: 8080,
api_key: String::new(),
include_warnings: true,
request_body_limit: 32 * 1024 * 1024,
}
}
}
impl ServerSection {
/// True when the server is reachable from another host.
pub fn is_public_bind(&self) -> bool {
let host = self.host.trim();
!(host == "127.0.0.1" || host == "localhost" || host == "::1")
}
}
#[derive(Debug, Clone, Deserialize, Serialize)]
#[serde(default)]
pub struct BrowserSection {
pub executable: String,
pub headless: bool,
pub profile_dir: String,
/// Concurrent turns allowed per provider account, i.e. tabs kept open.
pub max_tabs: usize,
/// How long a request waits for a free tab before it is refused with 429.
pub busy_wait_s: u64,
pub launch_timeout_s: u64,
pub request_timeout_s: u64,
pub extra_args: Vec<String>,
pub stealth: StealthMode,
/// Attach to a browser the user started instead of launching one.
pub attach: bool,
/// Debug port of that browser.
pub debug_port: u16,
}
impl Default for BrowserSection {
fn default() -> Self {
Self {
executable: String::new(),
headless: false,
profile_dir: String::new(),
max_tabs: 1,
busy_wait_s: 30,
launch_timeout_s: 60,
request_timeout_s: 60,
extra_args: vec![
"--disable-blink-features=AutomationControlled".to_string(),
"--no-first-run".to_string(),
"--no-default-browser-check".to_string(),
],
stealth: StealthMode::Off,
attach: false,
debug_port: 9222,
}
}
}
#[derive(Debug, Clone, Deserialize, Serialize)]
#[serde(default)]
pub struct ConversationSection {
pub store: String,
pub strategy: ConversationStrategy,
pub id_source: ConversationIdSource,
}
impl Default for ConversationSection {
fn default() -> Self {
Self {
store: String::new(),
strategy: ConversationStrategy::Auto,
id_source: ConversationIdSource::Fingerprint,
}
}
}
#[derive(Debug, Clone, Deserialize, Serialize)]
#[serde(default)]
pub struct CaptureSection {
pub poll_interval_ms: u64,
pub quiet_ms: u64,
pub response_timeout_s: u64,
pub inject_method: InjectMethod,
/// Fidelity of the captured answer (see `MarkdownMode`).
pub markdown: MarkdownMode,
/// How long the clipboard read is given before the DOM conversion wins.
pub clipboard_timeout_ms: u64,
/// Upper bound accepted for the OpenAI `n` parameter.
pub max_variants: u32,
}
impl Default for CaptureSection {
fn default() -> Self {
Self {
poll_interval_ms: 400,
quiet_ms: 1500,
response_timeout_s: 180,
inject_method: InjectMethod::Auto,
markdown: MarkdownMode::Auto,
clipboard_timeout_ms: 1500,
max_variants: 4,
}
}
}
#[derive(Debug, Clone, Deserialize, Serialize)]
#[serde(default)]
pub struct TokensSection {
pub encoding: String,
pub fallback_heuristic: bool,
}
impl Default for TokensSection {
fn default() -> Self {
Self {
encoding: "o200k_base".to_string(),
fallback_heuristic: true,
}
}
}
#[derive(Debug, Clone, Deserialize, Serialize)]
#[serde(default)]
pub struct DebugSection {
pub screenshots: ScreenshotMode,
pub dir: String,
}
impl Default for DebugSection {
fn default() -> Self {
Self {
screenshots: ScreenshotMode::OnError,
dir: String::new(),
}
}
}
#[derive(Debug, Clone, Deserialize, Serialize)]
#[serde(default)]
pub struct LoggingSection {
pub level: String,
pub format: String,
pub file: String,
}
impl Default for LoggingSection {
fn default() -> Self {
Self {
level: "info".to_string(),
format: "pretty".to_string(),
file: String::new(),
}
}
}
/// Root configuration object.
#[derive(Debug, Clone, Default, Deserialize, Serialize)]
#[serde(default)]
pub struct GlobalConfig {
pub server: ServerSection,
pub browser: BrowserSection,
pub conversation: ConversationSection,
pub capture: CaptureSection,
pub tokens: TokensSection,
pub debug: DebugSection,
pub logging: LoggingSection,
}
impl GlobalConfig {
/// Read the configuration file, returning defaults when it does not exist.
pub fn load(path: &Path) -> Result<Self> {
if !path.exists() {
return Ok(Self::default());
}
let raw = std::fs::read_to_string(path)
.map_err(|e| GatewayError::Config(format!("reading {}: {e}", path.display())))?;
Self::from_toml(&raw)
}
pub fn from_toml(raw: &str) -> Result<Self> {
toml::from_str::<Self>(raw).map_err(|e| GatewayError::Config(e.to_string()))
}
pub fn validate(&self) -> Result<()> {
if self.server.port == 0 {
return Err(GatewayError::Config("server.port must not be 0".into()));
}
if self.browser.max_tabs == 0 {
return Err(GatewayError::Config(
"browser.max_tabs must be at least 1".into(),
));
}
if self.capture.poll_interval_ms < 50 {
return Err(GatewayError::Config(
"capture.poll_interval_ms must be at least 50".into(),
));
}
if self.capture.response_timeout_s == 0 {
return Err(GatewayError::Config(
"capture.response_timeout_s must not be 0".into(),
));
}
if self.capture.max_variants == 0 {
return Err(GatewayError::Config(
"capture.max_variants must be at least 1".into(),
));
}
if self.browser.attach && self.browser.debug_port == 0 {
return Err(GatewayError::Config(
"browser.debug_port must be set when browser.attach is true".into(),
));
}
if self.browser.headless && self.server.api_key.is_empty() {
// Not fatal: headless mode simply cannot ask the user to log in.
tracing::debug!("headless mode cannot prompt for a manual login");
}
Ok(())
}
pub fn resolved_profile_root(&self, layout: &Layout) -> PathBuf {
non_empty_path(&self.browser.profile_dir)
.map(|raw| paths::resolve_against(layout.root(), &raw))
.unwrap_or_else(|| layout.profiles_dir())
}
pub fn profile_dir_for(&self, layout: &Layout, provider: &str) -> PathBuf {
self.resolved_profile_root(layout)
.join(provider)
.join("user-data-dir")
}
pub fn resolved_debug_dir(&self, layout: &Layout) -> PathBuf {
non_empty_path(&self.debug.dir)
.map(|raw| paths::resolve_against(layout.root(), &raw))
.unwrap_or_else(|| layout.debug_dir())
}
pub fn resolved_store_path(&self, layout: &Layout) -> PathBuf {
non_empty_path(&self.conversation.store)
.map(|raw| paths::resolve_against(layout.root(), &raw))
.unwrap_or_else(|| layout.conversations_file())
}
pub fn resolved_log_file(&self, layout: &Layout) -> PathBuf {
non_empty_path(&self.logging.file)
.map(|raw| paths::resolve_against(layout.root(), &raw))
.unwrap_or_else(|| layout.logs_dir().join("llm-gateway.log"))
}
}
fn non_empty_path(raw: &str) -> Option<String> {
let trimmed = raw.trim();
if trimmed.is_empty() {
None
} else {
Some(trimmed.to_string())
}
}
/// Values coming from the command line that override the configuration file.
#[derive(Debug, Clone, Default)]
pub struct CliOverrides {
pub host: Option<String>,
pub port: Option<u16>,
pub browser_executable: Option<String>,
pub headless: Option<bool>,
pub profile_dir: Option<String>,
pub max_tabs: Option<usize>,
pub debug_screenshots: Option<bool>,
pub log_level: Option<String>,
pub log_format: Option<String>,
pub api_key: Option<String>,
pub attach: Option<bool>,
pub debug_port: Option<u16>,
pub stealth: Option<StealthMode>,
pub markdown: Option<MarkdownMode>,
}
impl GlobalConfig {
/// Apply CLI flags on top of the file configuration.
pub fn apply(&mut self, overrides: &CliOverrides) {
if let Some(host) = &overrides.host {
self.server.host = host.clone();
}
if let Some(port) = overrides.port {
self.server.port = port;
}
if let Some(key) = &overrides.api_key {
self.server.api_key = key.clone();
}
if let Some(exe) = &overrides.browser_executable {
self.browser.executable = exe.clone();
}
if let Some(headless) = overrides.headless {
self.browser.headless = headless;
}
if let Some(dir) = &overrides.profile_dir {
self.browser.profile_dir = dir.clone();
}
if let Some(max_tabs) = overrides.max_tabs {
self.browser.max_tabs = max_tabs;
}
if let Some(debug) = overrides.debug_screenshots {
self.debug.screenshots = if debug {
ScreenshotMode::Always
} else {
ScreenshotMode::OnError
};
}
if let Some(level) = &overrides.log_level {
self.logging.level = level.clone();
}
if let Some(format) = &overrides.log_format {
self.logging.format = format.clone();
}
if let Some(attach) = overrides.attach {
self.browser.attach = attach;
}
if let Some(port) = overrides.debug_port {
self.browser.debug_port = port;
}
if let Some(stealth) = overrides.stealth {
self.browser.stealth = stealth;
}
if let Some(markdown) = overrides.markdown {
self.capture.markdown = markdown;
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn defaults_are_loopback_and_serialised() {
let cfg = GlobalConfig::default();
assert_eq!(cfg.server.host, "127.0.0.1");
assert_eq!(cfg.server.port, 8080);
assert_eq!(cfg.browser.max_tabs, 1);
assert_eq!(cfg.conversation.strategy, ConversationStrategy::Auto);
assert_eq!(cfg.capture.poll_interval_ms, 400);
cfg.validate().unwrap();
assert!(!cfg.server.is_public_bind());
}
#[test]
fn toml_round_trip_and_partial_files() {
let full = toml::to_string(&GlobalConfig::default()).unwrap();
let parsed = GlobalConfig::from_toml(&full).unwrap();
assert_eq!(parsed.server.port, 8080);
let partial = "[server]\nport = 9999\n";
let parsed = GlobalConfig::from_toml(partial).unwrap();
assert_eq!(parsed.server.port, 9999);
assert_eq!(parsed.server.host, "127.0.0.1");
}
#[test]
fn invalid_toml_is_a_config_error() {
let err = GlobalConfig::from_toml("port = ").unwrap_err();
assert_eq!(err.code(), "invalid_configuration");
}
#[test]
fn attach_mode_requires_a_port() {
let mut cfg = GlobalConfig::default();
assert_eq!(cfg.browser.stealth, StealthMode::Off);
assert_eq!(cfg.browser.debug_port, 9222);
cfg.browser.attach = true;
cfg.validate().unwrap();
cfg.browser.debug_port = 0;
assert!(cfg.validate().is_err());
}
#[test]
fn stealth_mode_is_parsed_from_the_file() {
let cfg = GlobalConfig::from_toml("[browser]\nstealth = \"aggressive\"\n").unwrap();
assert_eq!(cfg.browser.stealth, StealthMode::Aggressive);
let cfg = GlobalConfig::from_toml("[browser]\nstealth = \"minimal\"\n").unwrap();
assert_eq!(cfg.browser.stealth, StealthMode::Minimal);
}
#[test]
fn markdown_modes_are_parsed_and_default_to_auto() {
assert_eq!(
MarkdownMode::parse("clipboard").unwrap(),
MarkdownMode::Clipboard
);
assert_eq!(MarkdownMode::parse("DOM").unwrap(), MarkdownMode::Dom);
assert_eq!(MarkdownMode::parse(" text ").unwrap(), MarkdownMode::Text);
assert!(MarkdownMode::parse("rich").is_err());
assert_eq!(GlobalConfig::default().capture.markdown, MarkdownMode::Auto);
assert_eq!(GlobalConfig::default().capture.max_variants, 4);
let cfg = GlobalConfig::from_toml("[capture]\nmarkdown = \"text\"\n").unwrap();
assert_eq!(cfg.capture.markdown, MarkdownMode::Text);
}
#[test]
fn cli_overrides_win() {
let mut cfg = GlobalConfig::default();
cfg.apply(&CliOverrides {
port: Some(1234),
headless: Some(true),
debug_screenshots: Some(true),
..Default::default()
});
assert_eq!(cfg.server.port, 1234);
assert!(cfg.browser.headless);
assert_eq!(cfg.debug.screenshots, ScreenshotMode::Always);
}
#[test]
fn paths_fall_back_to_the_layout() {
let layout = Layout::rooted_at(PathBuf::from("/state"));
let cfg = GlobalConfig::default();
assert_eq!(
cfg.resolved_profile_root(&layout),
PathBuf::from("/state/profiles")
);
assert_eq!(
cfg.profile_dir_for(&layout, "chatgpt"),
PathBuf::from("/state/profiles/chatgpt/user-data-dir")
);
assert_eq!(
cfg.resolved_store_path(&layout),
PathBuf::from("/state/conversations.json")
);
}
}
src/conversation/fingerprint.rs
+99
View File
@@ -0,0 +1,99 @@
//! Stable client-side conversation identifiers.
use crate::config::ConversationIdSource;
use sha2::{Digest, Sha256};
/// Separator that cannot appear in user supplied text.
const SEP: char = '\u{1f}';
/// Derive a deterministic id from the opening messages of a thread.
///
/// OpenAI clients resend the whole history on every call and do not send a
/// conversation header, so deriving the id from the opening prompt is what
/// gives them continuity across calls and across server restarts.
pub fn fingerprint(owner: &str, system: &str, first_user: &str) -> String {
let mut hasher = Sha256::new();
hasher.update(owner.as_bytes());
hasher.update(SEP.to_string().as_bytes());
hasher.update(system.as_bytes());
hasher.update(SEP.to_string().as_bytes());
hasher.update(first_user.as_bytes());
let digest = hasher.finalize();
let hex = digest
.iter()
.map(|b| format!("{b:02x}"))
.collect::<String>();
format!("fp-{}", &hex[..16])
}
/// Decide which conversation id a request belongs to.
///
/// `owner` identifies the provider *and* the account (`chatgpt@perso`): the same
/// prompt sent to two accounts must map to two different web conversations.
pub fn resolve_conversation_id(
source: ConversationIdSource,
header: Option<&str>,
owner: &str,
system: &str,
first_user: &str,
) -> String {
if let Some(id) = header.map(str::trim).filter(|id| !id.is_empty()) {
return id.to_string();
}
match source {
ConversationIdSource::Fingerprint => fingerprint(owner, system, first_user),
ConversationIdSource::Header | ConversationIdSource::Uuid => {
format!("sess-{}", uuid::Uuid::new_v4())
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn fingerprints_are_stable_and_distinct() {
let a = fingerprint("chatgpt", "sys", "hello");
let b = fingerprint("chatgpt", "sys", "hello");
let c = fingerprint("chatgpt", "sys", "hello there");
let d = fingerprint("claude", "sys", "hello");
assert_eq!(a, b);
assert_ne!(a, c);
assert_ne!(a, d);
// Two accounts of the same provider never share a thread.
assert_ne!(
fingerprint("chatgpt", "s", "m"),
fingerprint("chatgpt@perso", "s", "m")
);
assert!(a.starts_with("fp-"));
assert_eq!(a.len(), 19);
}
#[test]
fn separator_prevents_ambiguous_joins() {
let a = fingerprint("p", "ab", "c");
let b = fingerprint("p", "a", "bc");
assert_ne!(a, b);
}
#[test]
fn header_wins_over_the_configured_source() {
let id = resolve_conversation_id(
ConversationIdSource::Fingerprint,
Some(" client-thread-1 "),
"chatgpt",
"",
"hi",
);
assert_eq!(id, "client-thread-1");
let id = resolve_conversation_id(ConversationIdSource::Header, None, "chatgpt", "", "hi");
assert!(id.starts_with("sess-"));
let id =
resolve_conversation_id(ConversationIdSource::Fingerprint, None, "chatgpt", "", "hi");
assert_eq!(id, fingerprint("chatgpt", "", "hi"));
}
}
src/conversation/mod.rs
+7
View File
@@ -0,0 +1,7 @@
//! Multi-turn conversation continuity between an OpenAI client and a web thread.
pub mod fingerprint;
pub mod store;
pub use fingerprint::{fingerprint, resolve_conversation_id};
pub use store::{StoreFileSnapshot, ThreadRecord, ThreadState, ThreadStore};
src/conversation/store.rs
+386
View File
@@ -0,0 +1,386 @@
//! JSON-backed thread store mapping client conversations to web conversations.
use std::collections::BTreeMap;
use std::io::Write;
use std::path::{Path, PathBuf};
use std::time::{SystemTime, UNIX_EPOCH};
use serde::{Deserialize, Serialize};
use tokio::sync::Mutex;
use crate::error::{GatewayError, Result};
use crate::providers::config::IMPLICIT_ACCOUNT;
/// Whether the web thread can be reused safely.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum ThreadState {
/// The last turn finished cleanly: the thread can be continued.
#[default]
Ready,
/// The last turn was interrupted: replay the history instead of reusing.
Incomplete,
}
/// One client conversation and the web conversation backing it.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ThreadRecord {
pub provider: String,
/// Account of the provider, when it declares several.
#[serde(default)]
pub account: Option<String>,
/// Full URL of the web conversation, when the provider exposes one.
#[serde(default)]
pub web_url: Option<String>,
/// Provider-side conversation identifier extracted from the URL.
#[serde(default)]
pub web_id: Option<String>,
pub created_at: u64,
pub last_used_at: u64,
#[serde(default)]
pub turns: u64,
#[serde(default)]
pub messages_sent: u64,
#[serde(default)]
pub state: ThreadState,
#[serde(default)]
pub note: Option<String>,
}
impl ThreadRecord {
pub fn new(provider: &str) -> Self {
let now = now_secs();
Self {
provider: provider.to_string(),
account: None,
web_url: None,
web_id: None,
created_at: now,
last_used_at: now,
turns: 0,
messages_sent: 0,
state: ThreadState::Ready,
note: None,
}
}
/// True when the stored web conversation can be continued.
pub fn is_reusable(&self) -> bool {
self.state == ThreadState::Ready && self.web_url.is_some()
}
}
/// On-disk representation, versioned so it can evolve.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct StoreFileSnapshot {
pub version: u32,
#[serde(default)]
pub threads: BTreeMap<String, ThreadRecord>,
}
impl Default for StoreFileSnapshot {
fn default() -> Self {
Self {
version: 1,
threads: BTreeMap::new(),
}
}
}
/// Thread storage, readable and editable by hand.
#[derive(Debug)]
pub struct ThreadStore {
path: Option<PathBuf>,
inner: Mutex<StoreFileSnapshot>,
}
impl ThreadStore {
/// Open the store, tolerating a missing file.
pub async fn open(path: &Path) -> Result<Self> {
let snapshot = if path.exists() {
let raw = std::fs::read_to_string(path)?;
if raw.trim().is_empty() {
StoreFileSnapshot::default()
} else {
serde_json::from_str::<StoreFileSnapshot>(&raw)
.map_err(|e| GatewayError::Config(format!("parsing {}: {e}", path.display())))?
}
} else {
StoreFileSnapshot::default()
};
Ok(Self {
path: Some(path.to_path_buf()),
inner: Mutex::new(snapshot),
})
}
/// In-memory store, used by tests and by the mock backend.
pub fn in_memory() -> Self {
Self {
path: None,
inner: Mutex::new(StoreFileSnapshot::default()),
}
}
pub fn path(&self) -> Option<&Path> {
self.path.as_deref()
}
pub async fn get(&self, id: &str) -> Option<ThreadRecord> {
self.inner.lock().await.threads.get(id).cloned()
}
pub async fn upsert(&self, id: &str, record: ThreadRecord) -> Result<()> {
{
let mut guard = self.inner.lock().await;
guard.threads.insert(id.to_string(), record);
}
self.flush().await
}
/// Record a completed turn.
pub async fn record_turn(
&self,
id: &str,
provider: &str,
account: Option<&str>,
web_url: Option<String>,
web_id: Option<String>,
messages_sent: u64,
) -> Result<()> {
{
let mut guard = self.inner.lock().await;
let entry = guard
.threads
.entry(id.to_string())
.or_insert_with(|| ThreadRecord::new(provider));
entry.provider = provider.to_string();
if account.is_some() {
entry.account = account.map(str::to_string);
}
if web_url.is_some() {
entry.web_url = web_url;
}
if web_id.is_some() {
entry.web_id = web_id;
}
entry.last_used_at = now_secs();
entry.turns += 1;
entry.messages_sent += messages_sent;
entry.state = ThreadState::Ready;
entry.note = None;
}
self.flush().await
}
/// Mark a thread as interrupted so the next turn replays instead of reusing.
pub async fn mark_incomplete(&self, id: &str, note: &str) -> Result<()> {
{
let mut guard = self.inner.lock().await;
if let Some(entry) = guard.threads.get_mut(id) {
entry.state = ThreadState::Incomplete;
entry.note = Some(note.to_string());
entry.last_used_at = now_secs();
}
}
self.flush().await
}
/// Stored threads, optionally restricted to one provider and one account.
pub async fn list(
&self,
provider: Option<&str>,
account: Option<&str>,
) -> Vec<(String, ThreadRecord)> {
self.inner
.lock()
.await
.threads
.iter()
.filter(|(_, record)| {
provider.is_none_or(|p| record.provider == p) && matches_account(record, account)
})
.map(|(id, record)| (id.clone(), record.clone()))
.collect()
}
/// Remove threads, optionally restricted to one provider and one account.
/// Returns the count.
pub async fn clear(&self, provider: Option<&str>, account: Option<&str>) -> Result<usize> {
let removed = {
let mut guard = self.inner.lock().await;
let before = guard.threads.len();
if provider.is_none() && account.is_none() {
guard.threads.clear();
} else {
guard.threads.retain(|_, record| {
!(provider.is_none_or(|p| record.provider == p)
&& matches_account(record, account))
});
}
before - guard.threads.len()
};
self.flush().await?;
Ok(removed)
}
/// Persist the current snapshot atomically.
pub async fn flush(&self) -> Result<()> {
let Some(path) = self.path.clone() else {
return Ok(());
};
let snapshot = self.inner.lock().await.clone();
write_atomic(&path, &snapshot)
}
}
/// Does a stored thread belong to the account a command asked for?
///
/// A thread recorded without an account was served by the single implicit
/// account of its provider, which the CLI names `default` everywhere else.
fn matches_account(record: &ThreadRecord, account: Option<&str>) -> bool {
let Some(wanted) = account.map(str::trim).filter(|value| !value.is_empty()) else {
return true;
};
match record.account.as_deref() {
Some(account) => account.eq_ignore_ascii_case(wanted),
None => wanted.eq_ignore_ascii_case(IMPLICIT_ACCOUNT),
}
}
fn write_atomic(path: &Path, snapshot: &StoreFileSnapshot) -> Result<()> {
let parent = path.parent().unwrap_or_else(|| Path::new("."));
std::fs::create_dir_all(parent)?;
let body = serde_json::to_vec_pretty(snapshot)
.map_err(|e| GatewayError::Internal(format!("serialising the thread store: {e}")))?;
let mut tmp = tempfile::NamedTempFile::new_in(parent)?;
tmp.write_all(&body)?;
tmp.flush()?;
tmp.persist(path).map_err(|e| GatewayError::Io(e.error))?;
Ok(())
}
pub fn now_secs() -> u64 {
SystemTime::now()
.duration_since(UNIX_EPOCH)
.map(|d| d.as_secs())
.unwrap_or(0)
}
#[cfg(test)]
mod tests {
use super::*;
#[tokio::test]
async fn records_turns_and_reloads_from_disk() {
let tmp = tempfile::tempdir().unwrap();
let path = tmp.path().join("conversations.json");
let store = ThreadStore::open(&path).await.unwrap();
store
.record_turn(
"fp-1",
"chatgpt",
None,
Some("https://chatgpt.com/c/abc".into()),
Some("abc".into()),
3,
)
.await
.unwrap();
let reloaded = ThreadStore::open(&path).await.unwrap();
let record = reloaded.get("fp-1").await.unwrap();
assert_eq!(record.turns, 1);
assert_eq!(record.messages_sent, 3);
assert_eq!(record.web_id.as_deref(), Some("abc"));
assert!(record.is_reusable());
}
#[tokio::test]
async fn incomplete_threads_are_not_reusable() {
let store = ThreadStore::in_memory();
store
.record_turn(
"t",
"chatgpt",
None,
Some("https://x/c/1".into()),
Some("1".into()),
1,
)
.await
.unwrap();
store
.mark_incomplete("t", "client disconnected")
.await
.unwrap();
let record = store.get("t").await.unwrap();
assert!(!record.is_reusable());
assert_eq!(record.note.as_deref(), Some("client disconnected"));
}
#[tokio::test]
async fn clear_filters_by_provider() {
let store = ThreadStore::in_memory();
store
.record_turn("a", "chatgpt", Some("perso"), None, None, 1)
.await
.unwrap();
store
.record_turn("b", "claude", None, None, None, 1)
.await
.unwrap();
assert_eq!(
store.get("a").await.unwrap().account.as_deref(),
Some("perso")
);
assert_eq!(store.clear(Some("chatgpt"), None).await.unwrap(), 1);
assert_eq!(store.list(None, None).await.len(), 1);
assert_eq!(store.clear(None, None).await.unwrap(), 1);
assert!(store.list(None, None).await.is_empty());
}
#[tokio::test]
async fn listing_and_clearing_filter_by_account() {
let store = ThreadStore::in_memory();
store
.record_turn("perso-1", "chatgpt", Some("perso"), None, None, 1)
.await
.unwrap();
store
.record_turn("pro-1", "chatgpt", Some("pro"), None, None, 1)
.await
.unwrap();
// A provider that declares no account stores no account at all.
store
.record_turn("solo-1", "claude", None, None, None, 1)
.await
.unwrap();
assert_eq!(store.list(None, Some("perso")).await.len(), 1);
assert_eq!(store.list(Some("chatgpt"), Some("pro")).await.len(), 1);
assert!(store.list(Some("claude"), Some("perso")).await.is_empty());
// "default" is the name of the implicit account in the rest of the CLI.
let implicit = store.list(None, Some("default")).await;
assert_eq!(implicit.len(), 1);
assert_eq!(implicit[0].0, "solo-1");
assert_eq!(
store.clear(Some("chatgpt"), Some("perso")).await.unwrap(),
1
);
assert_eq!(store.list(Some("chatgpt"), None).await.len(), 1);
}
#[tokio::test]
async fn corrupt_store_reports_a_config_error() {
let tmp = tempfile::tempdir().unwrap();
let path = tmp.path().join("conversations.json");
std::fs::write(&path, "{not json").unwrap();
let err = ThreadStore::open(&path).await.unwrap_err();
assert_eq!(err.code(), "invalid_configuration");
}
}
src/dashboard/index.html
+194
View File
@@ -0,0 +1,194 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8" />
<title>llm-gateway dashboard</title>
<style>
:root { color-scheme: dark; }
body { margin: 0; font: 14px/1.5 ui-monospace, SFMono-Regular, Menlo, Consolas, monospace;
background: #14161a; color: #d8dee9; }
header { padding: 12px 18px; background: #1c1f26; border-bottom: 1px solid #2b303b;
display: flex; gap: 16px; align-items: center; flex-wrap: wrap; }
h1 { font-size: 15px; margin: 0; font-weight: 600; }
main { display: grid; grid-template-columns: minmax(320px, 1fr) minmax(360px, 1.2fr); gap: 18px; padding: 18px; }
section { background: #1c1f26; border: 1px solid #2b303b; border-radius: 8px; padding: 14px; }
h2 { font-size: 13px; text-transform: uppercase; letter-spacing: .08em; color: #8f9bab; margin: 0 0 10px; }
table { width: 100%; border-collapse: collapse; }
td, th { text-align: left; padding: 5px 6px; border-bottom: 1px solid #262b34; font-size: 13px; }
.dot { display: inline-block; width: 9px; height: 9px; border-radius: 50%; margin-right: 7px; background: #6b7280; }
.available { background: #22c55e; } .requires_login { background: #f59e0b; }
.captcha, .error, .misconfigured { background: #ef4444; } .busy { background: #3b82f6; }
#logs { height: 340px; overflow: auto; white-space: pre-wrap; word-break: break-word;
background: #101216; border: 1px solid #262b34; border-radius: 6px; padding: 8px; font-size: 12px; }
input, textarea, select, button { font: inherit; background: #101216; color: #d8dee9;
border: 1px solid #2b303b; border-radius: 5px; padding: 6px 8px; }
button { cursor: pointer; background: #23303f; }
button:hover { background: #2c3d51; }
.row { display: flex; gap: 8px; align-items: center; margin-bottom: 8px; flex-wrap: wrap; }
textarea { width: 100%; min-height: 64px; resize: vertical; }
pre { background: #101216; border: 1px solid #262b34; border-radius: 6px; padding: 8px;
max-height: 240px; overflow: auto; font-size: 12px; }
a { color: #7aa2f7; }
</style>
</head>
<body>
<header>
<h1>llm-gateway</h1>
<span id="health">loading…</span>
<label>API key <input id="apiKey" placeholder="only if configured" size="18" /></label>
<button id="reload">Reload configuration</button>
<a href="/v1/models" target="_blank">/v1/models</a>
<a href="/health" target="_blank">/health</a>
</header>
<main>
<section>
<h2>Providers</h2>
<table><thead><tr><th>provider</th><th>state</th><th>url</th><th></th></tr></thead>
<tbody id="providers"></tbody></table>
<div class="row" style="margin-top:12px">
<button data-action="status">Refresh status</button>
<button data-action="validate">Validate all</button>
</div>
<pre id="report"></pre>
</section>
<section>
<h2>Recent logs</h2>
<div id="logs"></div>
</section>
<section style="grid-column: 1 / -1">
<h2>Manual test</h2>
<div class="row">
<select id="model"></select>
<label><input type="checkbox" id="stream" checked /> stream</label>
<button id="send">Send</button>
</div>
<textarea id="prompt">Say 'ok'</textarea>
<pre id="answer"></pre>
</section>
</main>
<script>
const key = () => document.getElementById('apiKey').value.trim();
const headers = () => key() ? { 'authorization': 'Bearer ' + key() } : {};
const apiKeyInput = document.getElementById('apiKey');
apiKeyInput.value = localStorage.getItem('llm-gateway-key') || '';
apiKeyInput.addEventListener('change', () => localStorage.setItem('llm-gateway-key', apiKeyInput.value));
async function getJson(path, init) {
const response = await fetch(path, { ...init, headers: { ...headers(), ...(init?.headers || {}) } });
const text = await response.text();
try { return { ok: response.ok, status: response.status, body: JSON.parse(text) }; }
catch { return { ok: response.ok, status: response.status, body: text }; }
}
let models = [];
async function refreshHealth() {
const { body } = await getJson('/health');
document.getElementById('health').textContent =
body && body.status ? body.status + ' · up ' + body.uptime_s + 's · ' + (body.providers || []).length + ' provider(s)' : 'unreachable';
models = ((body && body.providers) || []).flatMap(p => [
p.name,
...(p.accounts || []).map(a => p.name + '@' + a.id),
...(p.models || []).map(m => p.name + '/' + m),
]);
const select = document.getElementById('model');
select.innerHTML = models.map(m => '<option>' + m + '</option>').join('');
}
async function refreshStatus() {
const { body } = await getJson('/health');
const rows = [];
for (const provider of (body && body.providers) || []) {
// One row per account: each has its own browser, session and tab pool, so
// each is probed on its own - a signed-out second account shows up here
// instead of hiding behind the state of the default one.
const accounts = provider.accounts && provider.accounts.length
? provider.accounts
: [{ id: null, default: true }];
for (const account of accounts) {
const label = account.id ? provider.name + '@' + account.id : provider.name;
const status = await getJson('/v1/providers/' + label + '/status');
const state = (status.body && status.body.state) || 'unknown';
const identity = account.email || account.label || '';
rows.push('<tr><td>' + label + (account.default ? ' *' : '') +
(identity ? '<br><small>' + identity + '</small>' : '') + '</td>' +
'<td><span class="dot ' + state + '"></span>' + state + '</td>' +
'<td>' + (status.body && status.body.url || provider.web_url) + '</td>' +
'<td>' + (status.body && status.body.detail ? '<small>' + status.body.detail + '</small>' : '') + '</td></tr>');
}
}
document.getElementById('providers').innerHTML = rows.join('') || '<tr><td colspan="4">no provider configured</td></tr>';
}
async function validateAll() {
const { body } = await getJson('/health');
const reports = [];
for (const provider of (body && body.providers) || []) {
const owners = provider.accounts && provider.accounts.length
? provider.accounts.map(a => provider.name + '@' + a.id)
: [provider.name];
for (const owner of owners) {
const result = await getJson('/v1/providers/' + owner + '/validate', { method: 'POST' });
reports.push(result.body);
}
}
document.getElementById('report').textContent = JSON.stringify(reports, null, 2);
}
async function send() {
const model = document.getElementById('model').value;
const prompt = document.getElementById('prompt').value;
const stream = document.getElementById('stream').checked;
const out = document.getElementById('answer');
out.textContent = '';
const response = await fetch('/v1/chat/completions', {
method: 'POST',
headers: { ...headers(), 'content-type': 'application/json' },
body: JSON.stringify({ model, messages: [{ role: 'user', content: prompt }], stream }),
});
if (!stream) { out.textContent = JSON.stringify(await response.json(), null, 2); return; }
const reader = response.body.getReader();
const decoder = new TextDecoder();
let buffer = '';
for (;;) {
const { value, done } = await reader.read();
if (done) break;
buffer += decoder.decode(value, { stream: true });
const parts = buffer.split('\n\n');
buffer = parts.pop();
for (const part of parts) {
const line = part.split('\n').find(l => l.startsWith('data: '));
if (!line) continue;
const payload = line.slice(6);
if (payload === '[DONE]') continue;
try {
const chunk = JSON.parse(payload);
const delta = chunk.choices && chunk.choices[0] && chunk.choices[0].delta && chunk.choices[0].delta.content;
if (delta) out.textContent += delta;
else if (chunk.error) out.textContent += '\n[error] ' + chunk.error.message;
} catch (e) { /* keep the raw stream on screen */ }
}
}
}
document.getElementById('reload').addEventListener('click', async () => {
const result = await getJson('/v1/admin/reload', { method: 'POST' });
document.getElementById('report').textContent = JSON.stringify(result.body, null, 2);
await refreshHealth();
});
document.querySelector('[data-action="status"]').addEventListener('click', refreshStatus);
document.querySelector('[data-action="validate"]').addEventListener('click', validateAll);
document.getElementById('send').addEventListener('click', send);
const logs = document.getElementById('logs');
const events = new EventSource('/dashboard/events');
events.onmessage = (event) => {
logs.textContent += event.data + '\n';
logs.scrollTop = logs.scrollHeight;
};
refreshHealth();
refreshStatus();
setInterval(refreshHealth, 15000);
</script>
</body>
</html>
src/dashboard/mod.rs
+4
View File
@@ -0,0 +1,4 @@
//! The static dashboard page embedded in the binary.
/// Raw HTML of the dashboard.
pub const INDEX_HTML: &str = include_str!("index.html");
src/error.rs
+210
View File
@@ -0,0 +1,210 @@
//! Typed errors mapped onto the OpenAI error payload.
use std::path::PathBuf;
/// Convenience alias used across the crate.
pub type Result<T, E = GatewayError> = std::result::Result<T, E>;
/// Every failure mode the gateway can surface to a client.
#[derive(Debug, thiserror::Error)]
pub enum GatewayError {
#[error("model '{0}' does not match any configured provider")]
UnknownProvider(String),
/// The model string named a provider, but not one of its accounts.
#[error("{0}")]
UnknownAccount(String),
#[error("parameter '{param}' {detail}")]
UnsupportedParameter { param: String, detail: String },
#[error("{0}")]
UnsupportedContentType(String),
#[error("{0}")]
InvalidRequest(String),
#[error("invalid API key")]
InvalidApiKey,
#[error(
"provider '{provider}' is not signed in: run 'llm-gateway login {provider} --manual', sign in inside the window it opens, then close it (Google refuses browsers it sees as automated, so this path avoids that)"
)]
RequiresLogin { provider: String },
#[error(
"provider '{provider}' is showing a captcha challenge; solve it in the browser window"
)]
Captcha { provider: String },
#[error("provider '{provider}' is busy, retry later")]
ProviderBusy { provider: String, waited_ms: u64 },
#[error("provider '{provider}' is rate limiting: {detail}")]
UpstreamRateLimited { provider: String, detail: String },
#[error("browser profile {path} is locked by another browser instance; close it and retry")]
BrowserProfileLocked { path: PathBuf },
#[error("selector '{selector}' not found ({step})")]
SelectorMissing { selector: String, step: String },
#[error("provider '{provider}' reported an error: {message}")]
UpstreamError { provider: String, message: String },
#[error("browser engine unavailable: {0}")]
BrowserUnavailable(String),
#[error("timed out while {stage} after {timeout_s}s")]
CaptureTimeout { stage: String, timeout_s: u64 },
#[error("provider '{provider}' is misconfigured: {detail}")]
ProviderMisconfigured { provider: String, detail: String },
#[error("unsupported browser '{0}': only Chromium-based browsers speak CDP")]
UnsupportedBrowser(String),
#[error("provider '{0}' is not configured")]
ProviderNotConfigured(String),
#[error("HTTP request to {url} failed: {source}")]
Http {
url: String,
#[source]
source: reqwest::Error,
},
#[error("I/O error: {0}")]
Io(#[from] std::io::Error),
#[error("invalid configuration: {0}")]
Config(String),
#[error("internal error: {0}")]
Internal(String),
/// The caller went away before the answer was captured. Never serialised:
/// there is no client left to read it.
#[error("client disconnected before the answer was captured")]
Cancelled,
}
impl GatewayError {
pub fn invalid_request(msg: impl Into<String>) -> Self {
Self::InvalidRequest(msg.into())
}
pub fn unsupported_param(param: impl Into<String>, detail: impl Into<String>) -> Self {
Self::UnsupportedParameter {
param: param.into(),
detail: detail.into(),
}
}
pub fn internal(msg: impl Into<String>) -> Self {
Self::Internal(msg.into())
}
/// HTTP status returned to the OpenAI-compatible client.
pub fn http_status(&self) -> u16 {
match self {
Self::UnknownProvider(_) | Self::UnknownAccount(_) => 404,
Self::UnsupportedParameter { .. }
| Self::UnsupportedContentType(_)
| Self::InvalidRequest(_) => 400,
Self::InvalidApiKey | Self::RequiresLogin { .. } => 401,
Self::Captcha { .. } => 403,
Self::ProviderBusy { .. } | Self::UpstreamRateLimited { .. } => 429,
Self::BrowserProfileLocked { .. } => 409,
Self::BrowserUnavailable(_) | Self::ProviderMisconfigured { .. } => 503,
Self::CaptureTimeout { .. } => 504,
Self::Cancelled => 499,
_ => 500,
}
}
/// OpenAI-compatible error type field.
pub fn error_type(&self) -> &'static str {
match self.http_status() {
400 | 401 | 403 | 404 => "invalid_request_error",
429 => "rate_limit_error",
_ => "api_error",
}
}
/// Stable machine-readable code, also used as a log field.
pub fn code(&self) -> &'static str {
match self {
Self::UnknownProvider(_) => "model_not_found",
Self::UnknownAccount(_) => "account_not_found",
Self::UnsupportedParameter { .. } => "unsupported_parameter",
Self::UnsupportedContentType(_) => "unsupported_content_type",
Self::InvalidRequest(_) => "invalid_request",
Self::InvalidApiKey => "invalid_api_key",
Self::RequiresLogin { .. } => "requires_login",
Self::Captcha { .. } => "captcha_required",
Self::ProviderBusy { .. } => "provider_busy",
Self::UpstreamRateLimited { .. } => "upstream_rate_limit",
Self::BrowserProfileLocked { .. } => "browser_profile_locked",
Self::SelectorMissing { .. } => "selector_missing",
Self::UpstreamError { .. } => "upstream_error",
Self::BrowserUnavailable(_) => "browser_unavailable",
Self::CaptureTimeout { .. } => "upstream_timeout",
Self::ProviderMisconfigured { .. } => "provider_misconfigured",
Self::UnsupportedBrowser(_) => "unsupported_browser",
Self::ProviderNotConfigured(_) => "provider_not_configured",
Self::Http { .. } => "upstream_http_error",
Self::Io(_) => "io_error",
Self::Config(_) => "invalid_configuration",
Self::Internal(_) => "internal_error",
Self::Cancelled => "client_disconnected",
}
}
/// Name of the offending parameter, when the error is parameter-specific.
pub fn param(&self) -> Option<&str> {
match self {
Self::UnsupportedParameter { param, .. } => Some(param.as_str()),
_ => None,
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::path::PathBuf;
#[test]
fn codes_and_statuses_are_stable() {
let e = GatewayError::RequiresLogin {
provider: "chatgpt".into(),
};
assert_eq!(e.http_status(), 401);
assert_eq!(e.code(), "requires_login");
assert_eq!(e.error_type(), "invalid_request_error");
assert!(e.to_string().contains("llm-gateway login chatgpt"));
let e = GatewayError::CaptureTimeout {
stage: "waiting for the answer".into(),
timeout_s: 180,
};
assert_eq!(e.http_status(), 504);
assert_eq!(e.code(), "upstream_timeout");
let e = GatewayError::unsupported_param("tools", "is not supported by the browser backend");
assert_eq!(e.param(), Some("tools"));
assert_eq!(e.http_status(), 400);
let e = GatewayError::BrowserProfileLocked {
path: PathBuf::from("/p"),
};
assert_eq!(e.http_status(), 409);
let e = GatewayError::UnknownAccount("provider 'chatgpt' has no account 'x'".into());
assert_eq!(e.http_status(), 404);
assert_eq!(e.code(), "account_not_found");
assert_eq!(e.error_type(), "invalid_request_error");
assert!(e.to_string().contains("no account 'x'"));
}
}
src/lib.rs
+29
View File
@@ -0,0 +1,29 @@
//! llm-bridge: an OpenAI-compatible gateway driven by a real browser session.
pub mod browser;
pub mod cli;
pub mod config;
pub mod conversation;
pub mod dashboard;
pub mod error;
pub mod markdown;
pub mod paths;
pub mod providers;
pub mod selftest;
pub mod server;
pub mod tokens;
pub mod util;
/// Version reported by the API.
pub const API_VERSION: &str = env!("CARGO_PKG_VERSION");
/// Version plus the moment this binary was compiled.
///
/// Printed by `llm-gateway --version` and in the start-up banner, so it is
/// obvious whether an installed binary matches the sources it came from.
pub const VERSION: &str = concat!(
env!("CARGO_PKG_VERSION"),
" (built ",
env!("LLM_GATEWAY_BUILD_TIME"),
")"
);
src/main.rs
+1245
View File
File diff suppressed because it is too large Load Diff
src/markdown.rs
+1380
View File
File diff suppressed because it is too large Load Diff
src/paths.rs
+251
View File
@@ -0,0 +1,251 @@
//! Filesystem layout of the gateway state directory (default: `~/.llm-gateway`).
use std::path::{Path, PathBuf};
use anyhow::{Context, Result};
/// Environment variable overriding the state directory.
pub const HOME_ENV: &str = "LLM_GATEWAY_HOME";
/// Default provider configurations, materialised on first run.
const DEFAULT_PROVIDER_CONFIGS: &[(&str, &str)] = &[
("chatgpt", include_str!("../providers/chatgpt.toml")),
("claude", include_str!("../providers/claude.toml")),
("deepseek", include_str!("../providers/deepseek.toml")),
];
/// Names of the provider files shipped with this build.
pub fn default_provider_names() -> Vec<&'static str> {
DEFAULT_PROVIDER_CONFIGS
.iter()
.map(|(name, _)| *name)
.collect()
}
/// Default global configuration, materialised on first run.
const DEFAULT_CONFIG: &str = include_str!("../config.toml.example");
/// Resolved locations of every file the gateway reads or writes.
#[derive(Debug, Clone)]
pub struct Layout {
root: PathBuf,
}
impl Layout {
/// Discover the layout, honouring the environment override then `~/.llm-gateway`.
pub fn discover() -> Result<Self> {
if let Some(raw) = std::env::var_os(HOME_ENV) {
let raw = raw.to_string_lossy().to_string();
if !raw.trim().is_empty() {
return Ok(Self {
root: expand_tilde(&raw),
});
}
}
let home = dirs::home_dir().context("cannot determine the home directory")?;
Ok(Self {
root: home.join(".llm-gateway"),
})
}
/// Build a layout rooted at an explicit directory (used by tests).
pub fn rooted_at(root: impl Into<PathBuf>) -> Self {
Self { root: root.into() }
}
pub fn root(&self) -> &Path {
&self.root
}
/// Root of all browser profiles.
pub fn profiles_dir(&self) -> PathBuf {
self.root.join("profiles")
}
/// Chrome user-data-dir of a provider. A dedicated directory is used so the
/// personal browser profile of the user is never touched.
///
/// This is the layout of a provider without declared accounts; a provider
/// that declares accounts keeps one directory per account under
/// `profiles/<provider>/accounts/` (see `account_profile_dir`).
pub fn profile_dir(&self, provider: &str) -> PathBuf {
self.profiles_dir().join(provider).join("user-data-dir")
}
/// Chrome user-data-dir of one account of a provider.
pub fn account_profile_dir(&self, provider: &str, account: &str) -> PathBuf {
self.profiles_dir()
.join(provider)
.join("accounts")
.join(account)
.join("user-data-dir")
}
/// Directory holding the accounts of a provider.
pub fn accounts_dir(&self, provider: &str) -> PathBuf {
self.profiles_dir().join(provider).join("accounts")
}
pub fn providers_dir(&self) -> PathBuf {
self.root.join("providers")
}
pub fn provider_config(&self, provider: &str) -> PathBuf {
self.providers_dir().join(format!("{provider}.toml"))
}
pub fn conversations_file(&self) -> PathBuf {
self.root.join("conversations.json")
}
pub fn debug_dir(&self) -> PathBuf {
self.root.join("debug")
}
pub fn logs_dir(&self) -> PathBuf {
self.root.join("logs")
}
pub fn config_file(&self) -> PathBuf {
self.root.join("config.toml")
}
/// Create the directory tree and materialise the embedded defaults.
///
/// Existing files are never overwritten: user edits always win.
pub fn ensure(&self) -> Result<()> {
for dir in [
self.root.clone(),
self.profiles_dir(),
self.providers_dir(),
self.debug_dir(),
self.logs_dir(),
] {
std::fs::create_dir_all(&dir).with_context(|| format!("creating {}", dir.display()))?;
}
let config = self.config_file();
if !config.exists() {
std::fs::write(&config, DEFAULT_CONFIG)
.with_context(|| format!("writing {}", config.display()))?;
}
self.install_default_providers(false)?;
Ok(())
}
/// Write the embedded provider defaults.
///
/// Without `force`, an already installed file is left alone: your edits win.
/// With `force`, the previous file is kept as `<name>.toml.bak`, which is
/// how an upgraded build ships refreshed selectors.
pub fn install_default_providers(&self, force: bool) -> Result<Vec<PathBuf>> {
let mut written = Vec::new();
for (name, body) in DEFAULT_PROVIDER_CONFIGS {
let path = self.provider_config(name);
if path.exists() {
if !force {
continue;
}
let backup = path.with_extension("toml.bak");
std::fs::copy(&path, &backup)
.with_context(|| format!("backing up {}", path.display()))?;
}
std::fs::write(&path, body).with_context(|| format!("writing {}", path.display()))?;
written.push(path);
}
Ok(written)
}
}
/// Expand a leading tilde into the user home directory.
pub fn expand_tilde(raw: &str) -> PathBuf {
let trimmed = raw.trim();
if trimmed == "~" {
return dirs::home_dir().unwrap_or_else(|| PathBuf::from(trimmed));
}
let rest = trimmed
.strip_prefix("~/")
.or_else(|| trimmed.strip_prefix("~\\"));
if let Some(rest) = rest {
if let Some(home) = dirs::home_dir() {
return home.join(rest);
}
}
PathBuf::from(trimmed)
}
/// Resolve a configuration path against the layout root when it is relative.
pub fn resolve_against(root: &Path, raw: &str) -> PathBuf {
let expanded = expand_tilde(raw);
if expanded.is_absolute() {
expanded
} else {
root.join(expanded)
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn tilde_is_expanded() {
let home = dirs::home_dir().unwrap();
assert_eq!(expand_tilde("~"), home);
assert_eq!(expand_tilde("~/.llm-gateway"), home.join(".llm-gateway"));
assert_eq!(expand_tilde("/tmp/x"), PathBuf::from("/tmp/x"));
}
#[test]
fn ensure_creates_layout_without_overwriting() {
let tmp = tempfile::tempdir().unwrap();
let layout = Layout::rooted_at(tmp.path().join("state"));
layout.ensure().unwrap();
assert!(layout.providers_dir().join("chatgpt.toml").is_file());
for name in default_provider_names() {
assert!(
layout
.providers_dir()
.join(format!("{name}.toml"))
.is_file(),
"{name}.toml was not installed"
);
}
assert!(layout.profiles_dir().is_dir());
assert!(layout.debug_dir().is_dir());
assert!(layout.logs_dir().is_dir());
let custom = layout.provider_config("chatgpt");
std::fs::write(&custom, "# edited by the user\n").unwrap();
layout.ensure().unwrap();
assert_eq!(
std::fs::read_to_string(&custom).unwrap(),
"# edited by the user\n"
);
}
#[test]
fn accounts_get_their_own_profile_directory() {
let layout = Layout::rooted_at(PathBuf::from("/state"));
assert_eq!(
layout.profile_dir("chatgpt"),
PathBuf::from("/state/profiles/chatgpt/user-data-dir")
);
assert_eq!(
layout.account_profile_dir("chatgpt", "perso"),
PathBuf::from("/state/profiles/chatgpt/accounts/perso/user-data-dir")
);
}
#[test]
fn relative_paths_resolve_against_root() {
let root = PathBuf::from("/state");
assert_eq!(
resolve_against(&root, "debug"),
PathBuf::from("/state/debug")
);
assert_eq!(resolve_against(&root, "/abs"), PathBuf::from("/abs"));
}
}
src/providers/backend.rs
+127
View File
@@ -0,0 +1,127 @@
//! The seam the HTTP layer talks to.
//!
//! `RegistryBackend` drives real browsers; `MockBackend` answers from memory so
//! the API can be exercised without Chrome, network or a login.
use async_trait::async_trait;
use serde::{Deserialize, Serialize};
use crate::browser::{SessionState, SessionStatus};
use crate::error::Result;
use crate::providers::config::Capabilities;
use crate::providers::{AccountInfo, TurnRequest, TurnStream};
/// Timing of one validation step.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct StepTiming {
pub step: String,
pub ms: u64,
pub ok: bool,
#[serde(skip_serializing_if = "Option::is_none")]
pub detail: Option<String>,
}
/// Result of a provider validation run.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ValidationReport {
pub provider: String,
pub ok: bool,
/// ok | login_required | captcha | selector_missing | rate_limited | timeout | error | misconfigured
pub status: String,
#[serde(skip_serializing_if = "Option::is_none")]
pub message: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub response_excerpt: Option<String>,
pub duration_ms: u64,
pub checked_at: u64,
pub steps: Vec<StepTiming>,
}
/// Public description of a configured provider.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ProviderDescriptor {
pub name: String,
pub display_name: String,
pub web_url: String,
pub models: Vec<String>,
pub enabled: bool,
pub usable: bool,
#[serde(skip_serializing_if = "Vec::is_empty", default)]
pub problems: Vec<String>,
pub capabilities: Capabilities,
#[serde(skip_serializing_if = "Option::is_none")]
pub browser: Option<String>,
/// Declared accounts; empty when the provider has a single implicit one.
#[serde(skip_serializing_if = "Vec::is_empty", default)]
pub accounts: Vec<AccountInfo>,
}
/// A `model` string resolved to one provider and one of its accounts.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ResolvedModel {
pub provider: ProviderDescriptor,
/// Account id, when the provider declares accounts.
#[serde(skip_serializing_if = "Option::is_none")]
pub account: Option<String>,
/// Model name asked for after the `/`, echoed back to the client.
#[serde(skip_serializing_if = "Option::is_none")]
pub model: Option<String>,
}
impl ResolvedModel {
/// Provider and account, as used in logs and errors.
pub fn owner(&self) -> String {
match &self.account {
Some(account) => format!("{}@{account}", self.provider.name),
None => self.provider.name.clone(),
}
}
}
/// Outcome of a configuration reload.
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
pub struct ReloadReport {
pub added: Vec<String>,
pub updated: Vec<String>,
pub removed: Vec<String>,
pub problems: Vec<String>,
}
/// Everything the HTTP layer needs from the provider layer.
#[async_trait]
pub trait ChatBackend: Send + Sync {
/// Run one turn on `provider`, optionally on one of its accounts.
async fn complete(
&self,
provider: &str,
account: Option<&str>,
request: TurnRequest,
) -> Result<TurnStream>;
/// Session state of one account, or of the default one when `account` is
/// None. An unknown account is reported as a misconfigured status rather
/// than an error, so a monitor still gets a body to look at.
async fn status(&self, provider: &str, account: Option<&str>) -> SessionStatus;
/// Validate one account, or the default one when `account` is None.
async fn validate(&self, provider: &str, account: Option<&str>) -> Result<ValidationReport>;
async fn reload(&self) -> Result<ReloadReport>;
fn providers(&self) -> Vec<ProviderDescriptor>;
/// Resolve a model string to a provider, an account and a model name.
fn resolve(&self, model: &str) -> Result<ResolvedModel>;
async fn shutdown(&self);
}
/// `provider` or `provider@account`, as printed in logs and reports.
pub fn owner_label(provider: &str, account: Option<&str>) -> String {
match account.map(str::trim).filter(|value| !value.is_empty()) {
Some(account) => format!("{provider}@{account}"),
None => provider.to_string(),
}
}
/// Build the status of a provider that is not configured at all.
pub fn unknown_provider_status(provider: &str) -> SessionStatus {
let mut status =
SessionStatus::error(provider, format!("provider '{provider}' is not configured"));
status.state = SessionState::Misconfigured;
status
}
src/providers/config.rs
+882
View File
@@ -0,0 +1,882 @@
//! Provider configuration loaded from `~/.llm-gateway/providers/*.toml`.
//!
//! Selectors live here and nowhere else: a provider UI change must never
//! require a code change.
use std::path::{Path, PathBuf};
use serde::{Deserialize, Serialize};
use crate::error::{GatewayError, Result};
/// CSS selectors used to drive the provider page.
#[derive(Debug, Clone, Default, Deserialize, Serialize)]
#[serde(default)]
pub struct Selectors {
/// Editable prompt area.
pub input_field: String,
/// Submit button.
pub send_button: String,
/// One node per assistant answer, in document order.
pub response_container: String,
/// Element that only exists while the answer is being generated.
pub streaming_indicator: String,
/// Hidden `input[type=file]` used for attachments.
pub file_input: String,
/// Control that starts a brand new conversation.
pub new_chat_button: String,
/// Element that only exists on the sign-in page.
pub login_page_indicator: String,
/// Bot challenge frame.
pub captcha_indicator: String,
/// Visible upstream error message.
pub error_banner: String,
/// Model picker (reserved: not driven yet).
pub model_selector: String,
/// Attachment chip shown after an upload completed.
pub attachment_indicator: String,
/// Copy button inside one answer node, looked up relative to that node.
///
/// Clicking it is what gives the gateway the provider's own Markdown
/// rendering instead of the flattened visible text.
pub copy_button: String,
}
impl Selectors {
/// Selectors without which no turn can be executed.
pub fn missing_required(&self) -> Vec<&'static str> {
let mut missing = Vec::new();
if self.input_field.trim().is_empty() {
missing.push("selectors.input_field");
}
if self.send_button.trim().is_empty() {
missing.push("selectors.send_button");
}
if self.response_container.trim().is_empty() {
missing.push("selectors.response_container");
}
missing
}
}
/// Identity and entry points of the web UI.
#[derive(Debug, Clone, Deserialize, Serialize)]
pub struct ProviderMeta {
pub name: String,
#[serde(default)]
pub display_name: Option<String>,
pub web_url: String,
#[serde(default)]
pub new_conversation_url: Option<String>,
/// Regex with one capture group extracting the web conversation id.
#[serde(default)]
pub conversation_url_pattern: Option<String>,
#[serde(default)]
pub default_models: Vec<String>,
#[serde(default = "default_true")]
pub enabled: bool,
}
fn default_true() -> bool {
true
}
#[derive(Debug, Clone, Default, Deserialize, Serialize)]
#[serde(default)]
pub struct LoginRules {
/// Substrings that identify a sign-in URL.
pub url_patterns: Vec<String>,
/// Lowercase substrings of the visible page text that reveal a signed-out
/// page. They are only consulted when no prompt field is present, so a chat
/// page that happens to mention "log in" somewhere is never misclassified.
pub text_patterns: Vec<String>,
}
#[derive(Debug, Clone, Default, Deserialize, Serialize)]
#[serde(default)]
pub struct RateLimitRules {
/// Lowercase substrings that reveal upstream throttling.
pub text_patterns: Vec<String>,
}
#[derive(Debug, Clone, Deserialize, Serialize)]
#[serde(default)]
pub struct Timeouts {
pub page_load: u64,
pub response_generation: u64,
pub response_timeout: u64,
}
impl Default for Timeouts {
fn default() -> Self {
Self {
page_load: 30,
response_generation: 120,
response_timeout: 180,
}
}
}
#[derive(Debug, Clone, Deserialize, Serialize)]
#[serde(default)]
pub struct InputOptions {
pub poll_interval_ms: u64,
pub quiet_ms: u64,
pub inject_method: crate::config::InjectMethod,
/// Overrides the global `[capture] markdown` for this provider.
pub markdown: crate::config::MarkdownMode,
}
impl Default for InputOptions {
fn default() -> Self {
Self {
poll_interval_ms: 400,
quiet_ms: 1500,
inject_method: crate::config::InjectMethod::Auto,
markdown: crate::config::MarkdownMode::Auto,
}
}
}
/// One signed-in account of a provider.
///
/// A web UI account is a browser profile: two ChatGPT accounts are two
/// profiles, two sessions and two tabs. Declaring them here is what lets a
/// single provider file serve `model = "chatgpt"` and
/// `model = "[email protected]@gmail.com"` at the same time.
#[derive(Debug, Clone, Default, Deserialize, Serialize)]
#[serde(default)]
pub struct AccountConfig {
/// Short name used in the model string, e.g. "perso" in `chatgpt@perso`.
pub id: String,
/// Address of the account, also accepted in the model string.
pub email: Option<String>,
/// Free-form human label, shown by `llm-gateway list`.
pub label: Option<String>,
/// Directory name of the browser profile. Defaults to the account id.
pub profile: Option<String>,
/// The account used when a request names the provider alone.
#[serde(default)]
pub default: bool,
#[serde(default = "default_true")]
pub enabled: bool,
}
impl AccountConfig {
/// Directory name of this account's browser profile.
pub fn profile_name(&self) -> String {
let raw = self
.profile
.as_deref()
.map(str::trim)
.filter(|value| !value.is_empty())
.unwrap_or(&self.id);
sanitise_profile_name(raw)
}
/// Every string that may be used to address this account.
pub fn aliases(&self) -> Vec<String> {
let mut out = vec![self.id.clone()];
for extra in [self.email.as_deref(), self.label.as_deref()] {
if let Some(value) = extra.map(str::trim).filter(|value| !value.is_empty()) {
out.push(value.to_string());
}
}
out
}
/// True when `key` designates this account (case-insensitive).
pub fn matches(&self, key: &str) -> bool {
let key = key.trim();
!key.is_empty()
&& self
.aliases()
.iter()
.any(|alias| alias.eq_ignore_ascii_case(key))
}
}
/// Name of the single implicit account of a provider that declares none.
pub const IMPLICIT_ACCOUNT: &str = "default";
/// Keep a profile directory name usable on every platform.
///
/// A profile name becomes one path segment under `profiles/<provider>/accounts`,
/// so separators and reserved characters are folded into underscores rather
/// than escaping the directory.
fn sanitise_profile_name(raw: &str) -> String {
raw.chars()
.map(|c| match c {
'/' | '\\' | ':' | '*' | '?' | '"' | '<' | '>' | '|' => '_',
other => other,
})
.collect()
}
/// What the web UI of this provider can do.
#[derive(Debug, Clone, Deserialize, Serialize)]
#[serde(default)]
pub struct Capabilities {
pub supports_images: bool,
pub supports_streaming: bool,
pub supports_new_chat: bool,
}
impl Default for Capabilities {
fn default() -> Self {
Self {
supports_images: false,
supports_streaming: true,
supports_new_chat: true,
}
}
}
/// Raw `providers/<name>.toml` document.
#[derive(Debug, Clone, Deserialize, Serialize)]
pub struct ProviderFile {
pub provider: ProviderMeta,
/// Signed-in accounts. Empty means a single implicit one.
#[serde(default)]
pub accounts: Vec<AccountConfig>,
#[serde(default)]
pub selectors: Selectors,
#[serde(default)]
pub login: LoginRules,
#[serde(default)]
pub rate_limit: RateLimitRules,
#[serde(default)]
pub timeouts: Timeouts,
#[serde(default)]
pub input: InputOptions,
#[serde(default)]
pub options: Capabilities,
}
impl ProviderFile {
/// Canonical TOML of this document, used to detect real changes on reload.
pub fn serialized(&self) -> String {
toml::to_string(self).unwrap_or_default()
}
}
/// Validated provider configuration, ready to be handed to a provider.
#[derive(Debug, Clone)]
pub struct ProviderConfig {
pub file: ProviderFile,
pub source: PathBuf,
pattern: Option<regex::Regex>,
pub problems: Vec<String>,
}
impl ProviderConfig {
pub fn name(&self) -> &str {
&self.file.provider.name
}
pub fn display_name(&self) -> &str {
self.file
.provider
.display_name
.as_deref()
.unwrap_or(&self.file.provider.name)
}
pub fn web_url(&self) -> &str {
&self.file.provider.web_url
}
pub fn new_conversation_url(&self) -> &str {
self.file
.provider
.new_conversation_url
.as_deref()
.unwrap_or(&self.file.provider.web_url)
}
pub fn enabled(&self) -> bool {
self.file.provider.enabled
}
pub fn default_models(&self) -> &[String] {
&self.file.provider.default_models
}
pub fn default_model(&self) -> String {
self.file
.provider
.default_models
.first()
.cloned()
.unwrap_or_else(|| self.name().to_string())
}
pub fn selectors(&self) -> &Selectors {
&self.file.selectors
}
pub fn capabilities(&self) -> &Capabilities {
&self.file.options
}
/// Declared accounts, enabled ones only, in configuration order.
pub fn accounts(&self) -> impl Iterator<Item = &AccountConfig> {
self.file.accounts.iter().filter(|account| account.enabled)
}
/// True when the provider file declares its own accounts.
pub fn has_explicit_accounts(&self) -> bool {
!self.file.accounts.is_empty()
}
/// Account id used by a request that names the provider alone.
pub fn default_account_id(&self) -> Option<&str> {
if !self.has_explicit_accounts() {
return None;
}
self.accounts()
.find(|account| account.default)
.or_else(|| self.accounts().next())
.map(|account| account.id.as_str())
}
/// Look up an account by id, e-mail address or label.
pub fn find_account(&self, key: &str) -> Option<&AccountConfig> {
self.accounts().find(|account| account.matches(key))
}
/// Resolve the account of a request into an account id.
///
/// `None` (or an empty key) means "the default account"; it is accepted
/// only when the provider actually has one, so that a typo in a model name
/// is never silently routed to the wrong profile.
pub fn resolve_account_id(
&self,
key: Option<&str>,
) -> std::result::Result<Option<String>, String> {
let explicit = key.map(str::trim).filter(|value| !value.is_empty());
if !self.has_explicit_accounts() {
return match explicit {
None => Ok(None),
Some(name) if name.eq_ignore_ascii_case(IMPLICIT_ACCOUNT) => Ok(None),
Some(name) => Err(format!(
"provider '{}' has a single account; '{name}' does not select one. \
Declare it in an [[accounts]] table of {} to give it its own profile",
self.name(),
self.source.display()
)),
};
}
match explicit {
None => Ok(self.default_account_id().map(str::to_string)),
Some(name) => self
.find_account(name)
.map(|account| Some(account.id.clone()))
.ok_or_else(|| {
format!(
"provider '{}' has no account '{name}' (known: {})",
self.name(),
self.accounts()
.map(|account| match &account.email {
Some(email) => format!("{} <{email}>", account.id),
None => account.id.clone(),
})
.collect::<Vec<_>>()
.join(", ")
)
}),
}
}
pub fn is_usable(&self) -> bool {
self.problems.is_empty()
}
/// True when the page is a sign-in screen rather than a chat UI.
///
/// The URL and the markup marker are trusted at any time; the text patterns
/// are a fallback for the marketing page a signed-out visitor lands on, and
/// are only used when the prompt field is missing.
pub fn is_signed_out_page(
&self,
url: &str,
body_text_lowercase: &str,
login_marker: bool,
has_input: bool,
) -> bool {
if login_marker || self.is_login_url(url) {
return true;
}
if has_input {
return false;
}
self.file.login.text_patterns.iter().any(|pattern| {
let pattern = pattern.trim().to_lowercase();
!pattern.is_empty() && body_text_lowercase.contains(&pattern)
})
}
/// True when the page URL identifies a sign-in screen.
pub fn is_login_url(&self, url: &str) -> bool {
let lower = url.to_lowercase();
self.file
.login
.url_patterns
.iter()
.any(|p| !p.is_empty() && lower.contains(&p.to_lowercase()))
}
/// Extract the provider-side conversation id from a page URL.
pub fn conversation_id_from_url(&self, url: &str) -> Option<String> {
let pattern = self.pattern.as_ref()?;
pattern
.captures(url)?
.get(1)
.map(|m| m.as_str().to_string())
}
/// Effective answer fidelity: the provider file wins when it sets one.
pub fn markdown_mode(
&self,
global: crate::config::MarkdownMode,
) -> crate::config::MarkdownMode {
match self.file.input.markdown {
crate::config::MarkdownMode::Auto => global,
explicit => explicit,
}
}
/// True when the provider exposes a per-answer copy button.
pub fn has_copy_button(&self) -> bool {
!self.file.selectors.copy_button.trim().is_empty()
}
/// Detect an upstream throttling notice in the visible page text.
pub fn rate_limit_hit(&self, body_text_lowercase: &str) -> Option<String> {
self.file
.rate_limit
.text_patterns
.iter()
.find(|p| !p.is_empty() && body_text_lowercase.contains(&p.to_lowercase()))
.cloned()
}
/// Parse a TOML document and validate it.
pub fn parse(raw: &str, source: impl Into<PathBuf>) -> Result<Self> {
let file: ProviderFile =
toml::from_str(raw).map_err(|e| GatewayError::Config(e.to_string()))?;
Ok(Self::from_file(file, source.into()))
}
pub fn from_file(file: ProviderFile, source: PathBuf) -> Self {
let mut problems = Vec::new();
if file.provider.name.trim().is_empty() {
problems.push("provider.name must not be empty".to_string());
}
let url = file.provider.web_url.as_str();
if !(url.starts_with("http://")
|| url.starts_with("https://")
|| url.starts_with("file://"))
{
problems.push("provider.web_url must be an absolute http(s) or file URL".to_string());
}
for missing in file.selectors.missing_required() {
problems.push(format!("{missing} is required"));
}
if file.timeouts.response_timeout == 0 {
problems.push("timeouts.response_timeout must not be 0".to_string());
}
if file.input.poll_interval_ms < 50 {
problems.push("input.poll_interval_ms must be at least 50".to_string());
}
// Accounts: unique ids, and at most one default so that a request which
// names the provider alone is never ambiguous.
let mut seen_ids: Vec<String> = Vec::new();
let mut defaults = 0usize;
let mut profile_names: Vec<String> = Vec::new();
for (index, account) in file.accounts.iter().enumerate() {
let id = account.id.trim();
if id.is_empty() {
problems.push(format!("accounts[{index}].id must not be empty"));
continue;
}
let lower = id.to_lowercase();
if seen_ids.contains(&lower) {
problems.push(format!("duplicate account id '{id}'"));
}
seen_ids.push(lower);
if account.default {
defaults += 1;
}
// Two accounts sharing a profile would share one signed-in session.
let profile = account.profile_name();
if profile_names.contains(&profile) {
problems.push(format!(
"accounts[{index}] reuses the profile directory '{profile}'"
));
}
profile_names.push(profile);
}
if defaults > 1 {
problems.push("at most one account may set default = true".to_string());
}
let pattern = match file.provider.conversation_url_pattern.as_deref() {
Some(raw) if !raw.trim().is_empty() => match regex::Regex::new(raw) {
Ok(re) => {
if re.captures_len() < 2 {
problems.push(
"provider.conversation_url_pattern needs one capture group".to_string(),
);
None
} else {
Some(re)
}
}
Err(e) => {
problems.push(format!("provider.conversation_url_pattern is invalid: {e}"));
None
}
},
_ => None,
};
Self {
file,
source,
pattern,
problems,
}
}
/// Load one provider file.
pub fn load(path: &Path) -> Result<Self> {
let raw = std::fs::read_to_string(path)
.map_err(|e| GatewayError::Config(format!("reading {}: {e}", path.display())))?;
Self::parse(&raw, path.to_path_buf())
}
/// Load every `*.toml` of a directory, sorted by file name for stable output.
pub fn load_dir(dir: &Path) -> Vec<(PathBuf, Result<Self>)> {
let mut out = Vec::new();
let Ok(entries) = std::fs::read_dir(dir) else {
return out;
};
let mut paths: Vec<PathBuf> = entries
.flatten()
.map(|entry| entry.path())
.filter(|path| path.extension().is_some_and(|ext| ext == "toml"))
.collect();
paths.sort();
for path in paths {
let result = Self::load(&path);
out.push((path, result));
}
out
}
}
#[cfg(test)]
mod tests {
use super::*;
const CHATGPT: &str = include_str!("../../providers/chatgpt.toml");
#[test]
fn embedded_chatgpt_config_is_valid() {
let cfg = ProviderConfig::parse(CHATGPT, "chatgpt.toml").unwrap();
assert_eq!(cfg.name(), "chatgpt");
assert!(cfg.is_usable(), "problems: {:?}", cfg.problems);
assert!(cfg.enabled());
assert_eq!(cfg.default_model(), "gpt-4o");
assert!(cfg.capabilities().supports_images);
assert_eq!(
cfg.selectors().input_field,
"div#prompt-textarea, div.ProseMirror[contenteditable='true']"
);
assert_eq!(cfg.new_conversation_url(), "https://chatgpt.com/");
}
#[test]
fn conversation_ids_are_extracted_from_the_url() {
let cfg = ProviderConfig::parse(CHATGPT, "chatgpt.toml").unwrap();
let id = cfg
.conversation_id_from_url("https://chatgpt.com/c/68f0a1b2-1111-2222-3333-abcdefabcdef");
assert_eq!(id.as_deref(), Some("68f0a1b2-1111-2222-3333-abcdefabcdef"));
assert!(cfg
.conversation_id_from_url("https://chatgpt.com/")
.is_none());
}
#[test]
fn a_signed_out_landing_page_is_recognised_without_a_marker() {
let raw = r#"
[provider]
name = "x"
web_url = "https://example.com"
[selectors]
input_field = "div#prompt"
send_button = "button#send"
response_container = "div.answer"
[login]
url_patterns = ["/auth/login"]
text_patterns = ["log in", "sign up"]
"#;
let cfg = ProviderConfig::parse(raw, "x.toml").unwrap();
// The real chatgpt.com landing page: no prompt field, "Log in" buttons.
let body = "chatgpt: chat, work, create & code with ai log in sign up";
assert!(cfg.is_signed_out_page("https://example.com/", body, false, false));
// A chat page that merely mentions "log in" somewhere stays available.
assert!(!cfg.is_signed_out_page("https://example.com/", body, false, true));
// A neutral page with the editor present is obviously fine.
assert!(!cfg.is_signed_out_page("https://example.com/", "hello", false, true));
// Nothing matches: not a login page.
assert!(!cfg.is_signed_out_page("https://example.com/", "hello", false, false));
// Markers and URLs are always trusted.
assert!(cfg.is_signed_out_page("https://example.com/", "", true, true));
assert!(cfg.is_signed_out_page("https://example.com/auth/login", "", false, true));
}
#[test]
fn login_urls_and_rate_limits_are_detected() {
let cfg = ProviderConfig::parse(CHATGPT, "chatgpt.toml").unwrap();
assert!(cfg.is_login_url("https://chatgpt.com/auth/login"));
assert!(cfg.is_login_url("https://accounts.google.com/o/oauth2"));
assert!(!cfg.is_login_url("https://chatgpt.com/c/abc"));
assert!(cfg
.rate_limit_hit("you are sending too many requests")
.is_some());
assert!(cfg.rate_limit_hit("hello there").is_none());
}
#[test]
fn only_absolute_urls_are_accepted() {
let raw = r#"
[provider]
name = "x"
web_url = "chatgpt.com"
[selectors]
input_field = "a"
send_button = "b"
response_container = "c"
"#;
let cfg = ProviderConfig::parse(raw, "x.toml").unwrap();
assert!(cfg.problems.iter().any(|p| p.contains("web_url")));
let local = ProviderConfig::parse(
&raw.replace("chatgpt.com", "file:///tmp/fixture.html"),
"x.toml",
)
.unwrap();
assert!(local.is_usable(), "{:?}", local.problems);
}
#[test]
fn missing_selectors_are_reported_not_fatal() {
let raw = r#"
[provider]
name = "broken"
web_url = "https://example.com"
[selectors]
send_button = "button#send"
"#;
let cfg = ProviderConfig::parse(raw, "broken.toml").unwrap();
assert!(!cfg.is_usable());
assert!(cfg.problems.iter().any(|p| p.contains("input_field")));
assert!(cfg
.problems
.iter()
.any(|p| p.contains("response_container")));
}
const WITH_ACCOUNTS: &str = r#"
[provider]
name = "chatgpt"
web_url = "https://chatgpt.com/"
[selectors]
input_field = "div#prompt"
send_button = "button#send"
response_container = "div.answer"
[[accounts]]
id = "perso"
email = "[email protected]"
default = true
[[accounts]]
id = "pro"
email = "[email protected]"
label = "Work"
"#;
#[test]
fn a_provider_without_accounts_has_a_single_implicit_one() {
let cfg = ProviderConfig::parse(CHATGPT, "chatgpt.toml").unwrap();
assert!(!cfg.has_explicit_accounts());
assert_eq!(cfg.accounts().count(), 0);
assert_eq!(cfg.default_account_id(), None);
assert_eq!(cfg.resolve_account_id(None).unwrap(), None);
assert_eq!(cfg.resolve_account_id(Some("default")).unwrap(), None);
assert_eq!(cfg.resolve_account_id(Some(" ")).unwrap(), None);
let err = cfg.resolve_account_id(Some("perso")).unwrap_err();
assert!(err.contains("single account"), "{err}");
}
#[test]
fn declared_accounts_are_addressed_by_id_email_or_label() {
let cfg = ProviderConfig::parse(WITH_ACCOUNTS, "chatgpt.toml").unwrap();
assert!(cfg.is_usable(), "problems: {:?}", cfg.problems);
assert_eq!(cfg.accounts().count(), 2);
assert_eq!(cfg.default_account_id(), Some("perso"));
assert_eq!(
cfg.resolve_account_id(None).unwrap().as_deref(),
Some("perso")
);
assert_eq!(
cfg.resolve_account_id(Some("PRO")).unwrap().as_deref(),
Some("pro")
);
assert_eq!(
cfg.resolve_account_id(Some("[email protected]"))
.unwrap()
.as_deref(),
Some("pro")
);
assert_eq!(
cfg.resolve_account_id(Some("Work")).unwrap().as_deref(),
Some("pro")
);
let err = cfg.resolve_account_id(Some("nobody")).unwrap_err();
assert!(err.contains("no account 'nobody'"), "{err}");
assert!(err.contains("perso"), "{err}");
}
#[test]
fn the_first_enabled_account_is_the_default_when_none_is_marked() {
let raw = WITH_ACCOUNTS.replace("default = true", "enabled = false");
let cfg = ProviderConfig::parse(&raw, "chatgpt.toml").unwrap();
// "perso" is disabled, so the default is the first enabled account.
assert_eq!(cfg.default_account_id(), Some("pro"));
assert_eq!(cfg.accounts().count(), 1);
}
#[test]
fn accounts_reuse_is_reported() {
let raw = WITH_ACCOUNTS.replace("id = \"pro\"", "id = \"perso\"");
let cfg = ProviderConfig::parse(&raw, "chatgpt.toml").unwrap();
assert!(!cfg.is_usable());
assert!(cfg
.problems
.iter()
.any(|p| p.contains("duplicate account id")));
assert!(cfg.problems.iter().any(|p| p.contains("profile directory")));
let two_defaults = WITH_ACCOUNTS.replace("label = \"Work\"", "default = true");
let cfg = ProviderConfig::parse(&two_defaults, "chatgpt.toml").unwrap();
assert!(cfg
.problems
.iter()
.any(|p| p.contains("at most one account")));
let nameless = WITH_ACCOUNTS.replace("id = \"pro\"", "id = \"\"");
let cfg = ProviderConfig::parse(&nameless, "chatgpt.toml").unwrap();
assert!(cfg.problems.iter().any(|p| p.contains("must not be empty")));
}
#[test]
fn profile_directory_names_are_safe_path_segments() {
let account = AccountConfig {
id: "a/b:c".to_string(),
..Default::default()
};
assert_eq!(account.profile_name(), "a_b_c");
let explicit = AccountConfig {
id: "perso".to_string(),
profile: Some(" my profile ".to_string()),
..Default::default()
};
assert_eq!(explicit.profile_name(), "my profile");
}
#[test]
fn the_markdown_mode_falls_back_to_the_global_one() {
use crate::config::MarkdownMode;
let cfg = ProviderConfig::parse(CHATGPT, "chatgpt.toml").unwrap();
assert_eq!(cfg.markdown_mode(MarkdownMode::Dom), MarkdownMode::Dom);
let raw = r#"
[provider]
name = "x"
web_url = "https://example.com"
[selectors]
input_field = "a"
send_button = "b"
response_container = "c"
[input]
markdown = "text"
"#;
let cfg = ProviderConfig::parse(raw, "x.toml").unwrap();
assert_eq!(
cfg.markdown_mode(MarkdownMode::Clipboard),
MarkdownMode::Text
);
}
#[test]
fn the_copy_button_is_optional() {
let cfg = ProviderConfig::parse(CHATGPT, "chatgpt.toml").unwrap();
assert!(cfg.has_copy_button());
// Blank the key by name rather than by matching the whole line: the
// shipped selector list is edited as the provider UIs change, and this
// test is about the key being optional, not about its current value.
let raw = CHATGPT
.lines()
.map(|line| {
if line.trim_start().starts_with("copy_button") {
"copy_button = \"\""
} else {
line
}
})
.collect::<Vec<_>>()
.join("\n");
let cfg = ProviderConfig::parse(&raw, "chatgpt.toml").unwrap();
assert!(!cfg.has_copy_button());
assert!(cfg.is_usable(), "the copy button is never required");
}
#[test]
fn invalid_regex_is_reported() {
let raw = r#"
[provider]
name = "x"
web_url = "https://example.com"
conversation_url_pattern = "("
[selectors]
input_field = "a"
send_button = "b"
response_container = "c"
"#;
let cfg = ProviderConfig::parse(raw, "x.toml").unwrap();
assert!(cfg
.problems
.iter()
.any(|p| p.contains("conversation_url_pattern")));
}
}
src/providers/mock.rs
+290
View File
@@ -0,0 +1,290 @@
//! An in-memory backend: no browser, no network, no login.
//!
//! It serves the CLI demo and the HTTP integration tests, and keeps the API
//! exercisable on a machine without Chrome.
use std::time::Duration;
use async_trait::async_trait;
use tokio::sync::mpsc;
use crate::browser::{SessionState, SessionStatus};
use crate::error::{GatewayError, Result};
use crate::providers::backend::{
owner_label, ChatBackend, ProviderDescriptor, ReloadReport, ResolvedModel, StepTiming,
ValidationReport,
};
use crate::providers::config::{Capabilities, IMPLICIT_ACCOUNT};
use crate::providers::model::parse_model;
use crate::providers::{AccountInfo, TurnEvent, TurnOutcome, TurnRequest, TurnStream};
use crate::util;
/// Canned backend used by tests and by `serve --backend mock`.
pub struct MockBackend {
providers: Vec<ProviderDescriptor>,
per_chunk: Duration,
answers: Vec<String>,
}
impl Default for MockBackend {
fn default() -> Self {
Self::new(
vec![("chatgpt", "gpt-4o"), ("claude", "claude-sonnet-4")],
Duration::from_millis(20),
)
}
}
impl MockBackend {
/// Mirror the providers that are actually configured.
pub fn from_descriptors(mut descriptors: Vec<ProviderDescriptor>, per_chunk: Duration) -> Self {
for descriptor in &mut descriptors {
descriptor.browser = Some("mock".to_string());
descriptor.enabled = true;
descriptor.usable = true;
}
Self {
providers: descriptors,
per_chunk,
answers: MockBackend::default().answers,
}
}
pub fn new(providers: Vec<(&str, &str)>, per_chunk: Duration) -> Self {
let providers = providers
.into_iter()
.map(|(name, model)| ProviderDescriptor {
name: name.to_string(),
display_name: name.to_string(),
web_url: format!("https://{name}.example/"),
models: vec![model.to_string()],
enabled: true,
usable: true,
problems: Vec::new(),
capabilities: Capabilities {
supports_images: true,
supports_streaming: true,
supports_new_chat: true,
},
browser: Some("mock".to_string()),
accounts: Vec::new(),
})
.collect();
Self {
providers,
per_chunk,
answers: vec![
"ok".to_string(),
"This answer comes from the mock backend: no browser was launched.".to_string(),
],
}
}
/// Declare accounts on one provider, to exercise multi-account routing.
pub fn with_accounts(mut self, provider: &str, accounts: Vec<AccountInfo>) -> Self {
if let Some(descriptor) = self.providers.iter_mut().find(|p| p.name == provider) {
descriptor.accounts = accounts;
}
self
}
fn find(&self, provider: &str) -> Result<&ProviderDescriptor> {
self.providers
.iter()
.find(|p| p.name == provider)
.ok_or_else(|| GatewayError::UnknownProvider(provider.to_string()))
}
/// Resolve an account key against a descriptor, tolerating an id, an
/// e-mail address or a label.
fn account_of(descriptor: &ProviderDescriptor, key: Option<&str>) -> Result<Option<String>> {
if descriptor.accounts.is_empty() {
return match key.map(str::trim).filter(|value| !value.is_empty()) {
None => Ok(None),
Some(name) if name.eq_ignore_ascii_case(IMPLICIT_ACCOUNT) => Ok(None),
Some(name) => Err(GatewayError::UnknownAccount(format!(
"provider '{}' has a single account; '{name}' does not select one",
descriptor.name
))),
};
}
let default = descriptor
.accounts
.iter()
.find(|account| account.default)
.or_else(|| descriptor.accounts.first())
.map(|account| account.id.clone());
match key.map(str::trim).filter(|value| !value.is_empty()) {
None => Ok(default),
Some(name) => descriptor
.accounts
.iter()
.find(|account| {
account.id.eq_ignore_ascii_case(name)
|| account
.email
.as_deref()
.is_some_and(|email| email.eq_ignore_ascii_case(name))
|| account
.label
.as_deref()
.is_some_and(|label| label.eq_ignore_ascii_case(name))
})
.map(|account| Some(account.id.clone()))
.ok_or_else(|| {
GatewayError::UnknownAccount(format!(
"provider '{}' has no account '{name}' (known: {})",
descriptor.name,
descriptor
.accounts
.iter()
.map(|account| account.id.clone())
.collect::<Vec<_>>()
.join(", ")
))
}),
}
}
}
#[async_trait]
impl ChatBackend for MockBackend {
async fn complete(
&self,
provider: &str,
account: Option<&str>,
request: TurnRequest,
) -> Result<TurnStream> {
let descriptor = self.find(provider)?;
MockBackend::account_of(descriptor, account)?;
let answer = if request.probe {
self.answers[0].clone()
} else {
self.answers.last().cloned().unwrap_or_default()
};
let per_chunk = self.per_chunk;
let reused = request.reuse_url.is_some();
let web_id = request
.reuse_url
.as_deref()
.and_then(|url| url.rsplit('/').next().map(str::to_string))
.unwrap_or_else(|| format!("mock-{}", uuid::Uuid::new_v4().simple()));
let web_url = format!("{}c/{}", descriptor.web_url, web_id);
let (tx, rx) = mpsc::channel(16);
tokio::spawn(async move {
let started = std::time::Instant::now();
let mut deltas = 0u32;
for chunk in answer.as_bytes().chunks(8) {
let piece = String::from_utf8_lossy(chunk).to_string();
if tx.send(TurnEvent::Delta(piece)).await.is_err() {
return;
}
deltas += 1;
tokio::time::sleep(per_chunk).await;
}
let _ = tx
.send(TurnEvent::Completed(TurnOutcome {
text: answer,
web_url: Some(web_url),
web_id: Some(web_id),
elapsed_ms: started.elapsed().as_millis() as u64,
deltas,
fidelity: "text".to_string(),
}))
.await;
});
let _ = reused;
Ok(TurnStream::new(rx))
}
async fn status(&self, provider: &str, account: Option<&str>) -> SessionStatus {
let descriptor = match self.find(provider) {
Ok(descriptor) => descriptor,
Err(_) => {
let mut status = SessionStatus::error(provider, "provider is not configured");
status.state = SessionState::Misconfigured;
return status;
}
};
let account = match MockBackend::account_of(descriptor, account) {
Ok(account) => account,
Err(err) => {
let label = owner_label(provider, account);
let mut status = SessionStatus::error(&label, err.to_string());
status.state = SessionState::Misconfigured;
return status;
}
};
SessionStatus::available(
&owner_label(provider, account.as_deref()),
Some(descriptor.web_url.clone()),
)
.with_browser("mock")
}
async fn validate(&self, provider: &str, account: Option<&str>) -> Result<ValidationReport> {
let descriptor = self.find(provider)?;
let account = MockBackend::account_of(descriptor, account)?;
let started = std::time::Instant::now();
Ok(ValidationReport {
provider: owner_label(provider, account.as_deref()),
ok: true,
status: "ok".to_string(),
message: None,
response_excerpt: Some("ok".to_string()),
duration_ms: started.elapsed().as_millis() as u64,
checked_at: util::now_secs(),
steps: vec![
StepTiming {
step: "session".into(),
ms: 0,
ok: true,
detail: None,
},
StepTiming {
step: "probe_prompt".into(),
ms: 1,
ok: true,
detail: None,
},
],
})
}
async fn reload(&self) -> Result<ReloadReport> {
Ok(ReloadReport::default())
}
fn providers(&self) -> Vec<ProviderDescriptor> {
self.providers.clone()
}
fn resolve(&self, model: &str) -> Result<ResolvedModel> {
let reference = parse_model(model);
let descriptor = if let Ok(descriptor) = self.find(&reference.provider) {
descriptor.clone()
} else if self.providers.len() == 1 {
self.providers[0].clone()
} else {
return Err(GatewayError::UnknownProvider(format!(
"{} (mock providers: {})",
reference.original(),
self.providers
.iter()
.map(|p| p.name.clone())
.collect::<Vec<_>>()
.join(", ")
)));
};
let account = MockBackend::account_of(&descriptor, reference.account.as_deref())?;
Ok(ResolvedModel {
provider: descriptor,
account,
model: reference.model,
})
}
async fn shutdown(&self) {}
}
src/providers/mod.rs
+23
View File
@@ -0,0 +1,23 @@
//! Providers: configuration, the browser-driven implementation and the registry.
pub mod backend;
pub mod config;
pub mod mock;
pub mod model;
pub mod registry;
pub mod r#trait;
pub mod webui;
pub use backend::{
owner_label, ChatBackend, ProviderDescriptor, ReloadReport, ResolvedModel, StepTiming,
ValidationReport,
};
pub use config::{AccountConfig, Capabilities, ProviderConfig, Selectors, IMPLICIT_ACCOUNT};
pub use mock::MockBackend;
pub use model::{parse_model, ModelRef};
pub use r#trait::{
AccountInfo, Attachment, AttachmentKind, LlmProvider, TurnEvent, TurnOutcome, TurnRequest,
TurnStream,
};
pub use registry::ProviderRegistry;
pub use webui::{CaptureDefaults, WebUiProvider};
src/providers/model.rs
+132
View File
@@ -0,0 +1,132 @@
//! Parsing of the `model` string a client sends.
//!
//! The OpenAI API has a single field to select a model, and this gateway uses it
//! for two things: the provider and, when the provider declares several, the
//! account to answer from.
//!
//! ```text
//! chatgpt provider, default account
//! chatgpt@perso provider, account "perso"
//! [email protected]@gmail.com provider, account by e-mail
//! chatgpt/gpt-4o provider, model name
//! chatgpt@perso/gpt-4o all three
//! ```
//!
//! The account is parsed before the model name, so an address that itself
//! contains an `@` still works.
/// A `model` string broken into its parts.
#[derive(Debug, Clone, PartialEq, Eq, Default)]
pub struct ModelRef {
/// Everything before the account marker: the provider name or a model name.
pub provider: String,
/// Account requested with `@`, when one is.
pub account: Option<String>,
/// Model name after the first `/`, when one is.
pub model: Option<String>,
}
impl ModelRef {
/// Rebuild the string, for error messages.
pub fn original(&self) -> String {
let mut out = self.provider.clone();
if let Some(account) = &self.account {
out.push('@');
out.push_str(account);
}
if let Some(model) = &self.model {
out.push('/');
out.push_str(model);
}
out
}
}
/// Split a model string into provider, account and model name.
pub fn parse_model(raw: &str) -> ModelRef {
let trimmed = raw.trim();
let (head, model) = match trimmed.split_once('/') {
Some((head, model)) => (head, Some(model.trim()).filter(|value| !value.is_empty())),
None => (trimmed, None),
};
let (provider, account) = match head.split_once('@') {
Some((provider, account)) => (
provider.trim(),
Some(account.trim()).filter(|value| !value.is_empty()),
),
None => (head.trim(), None),
};
ModelRef {
provider: provider.to_string(),
account: account.map(str::to_string),
model: model.map(str::to_string),
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn a_bare_provider_has_no_account_and_no_model() {
let parsed = parse_model("chatgpt");
assert_eq!(parsed.provider, "chatgpt");
assert_eq!(parsed.account, None);
assert_eq!(parsed.model, None);
assert_eq!(parsed.original(), "chatgpt");
}
#[test]
fn the_model_name_follows_a_slash() {
let parsed = parse_model("chatgpt/gpt-4o");
assert_eq!(parsed.provider, "chatgpt");
assert_eq!(parsed.account, None);
assert_eq!(parsed.model.as_deref(), Some("gpt-4o"));
}
#[test]
fn the_account_follows_an_at_sign() {
let parsed = parse_model("chatgpt@perso");
assert_eq!(parsed.provider, "chatgpt");
assert_eq!(parsed.account.as_deref(), Some("perso"));
assert_eq!(parsed.model, None);
let both = parse_model("chatgpt@perso/gpt-4o");
assert_eq!(both.provider, "chatgpt");
assert_eq!(both.account.as_deref(), Some("perso"));
assert_eq!(both.model.as_deref(), Some("gpt-4o"));
assert_eq!(both.original(), "chatgpt@perso/gpt-4o");
}
#[test]
fn an_e_mail_address_survives_parsing() {
let parsed = parse_model("[email protected]@gmail.com");
assert_eq!(parsed.provider, "chatgpt");
assert_eq!(parsed.account.as_deref(), Some("[email protected]"));
assert_eq!(parsed.model, None);
let with_model = parse_model("[email protected]@gmail.com/gpt-4o");
assert_eq!(with_model.provider, "chatgpt");
assert_eq!(
with_model.account.as_deref(),
Some("[email protected]")
);
assert_eq!(with_model.model.as_deref(), Some("gpt-4o"));
}
#[test]
fn blanks_and_trailing_slashes_are_tolerated() {
let parsed = parse_model(" chatgpt ");
assert_eq!(parsed.provider, "chatgpt");
let parsed = parse_model("chatgpt/");
assert_eq!(parsed.provider, "chatgpt");
assert_eq!(parsed.model, None);
let parsed = parse_model("chatgpt@/gpt-4o");
assert_eq!(parsed.provider, "chatgpt");
assert_eq!(parsed.account, None);
assert_eq!(parsed.model.as_deref(), Some("gpt-4o"));
}
}
src/providers/registry.rs
+591
View File
@@ -0,0 +1,591 @@
//! Provider registry: loads provider files and builds browser-driven providers.
use std::collections::{BTreeMap, BTreeSet};
use std::sync::{Arc, RwLock};
use std::time::Duration;
use arc_swap::ArcSwap;
use async_trait::async_trait;
use crate::browser::{resolve_executable, DebugRecorder, SessionStatus};
use crate::config::GlobalConfig;
use crate::error::{GatewayError, Result};
use crate::paths::{expand_tilde, Layout};
use crate::providers::backend::{
owner_label, ChatBackend, ProviderDescriptor, ReloadReport, ResolvedModel, ValidationReport,
};
use crate::providers::config::ProviderConfig;
use crate::providers::model::parse_model;
use crate::providers::webui::{CaptureDefaults, WebUiProvider};
use crate::providers::{LlmProvider, TurnRequest, TurnStream};
use crate::selftest::runner::validate_provider;
/// Public description of one configured provider, with its accounts.
pub fn descriptor_of(provider: &WebUiProvider) -> ProviderDescriptor {
let cfg = provider.cfg();
ProviderDescriptor {
name: cfg.name().to_string(),
display_name: cfg.display_name().to_string(),
web_url: cfg.web_url().to_string(),
models: cfg.default_models().to_vec(),
enabled: cfg.enabled(),
usable: cfg.is_usable(),
problems: cfg.problems.clone(),
capabilities: cfg.capabilities().clone(),
browser: Some(provider.browser().to_string()),
accounts: provider.accounts(),
}
}
/// Owns every configured provider and their browser processes.
pub struct ProviderRegistry {
layout: Layout,
config: Arc<ArcSwap<GlobalConfig>>,
providers: RwLock<BTreeMap<String, Arc<WebUiProvider>>>,
}
impl ProviderRegistry {
pub fn new(layout: Layout, config: GlobalConfig) -> Self {
Self {
layout,
config: Arc::new(ArcSwap::from_pointee(config)),
providers: RwLock::new(BTreeMap::new()),
}
}
/// Shared configuration handle: the HTTP layer reads the same snapshot.
pub fn config_handle(&self) -> Arc<ArcSwap<GlobalConfig>> {
Arc::clone(&self.config)
}
pub fn layout(&self) -> &Layout {
&self.layout
}
pub fn global(&self) -> Arc<GlobalConfig> {
self.config.load_full()
}
/// Swap in a reloaded global configuration (browser settings need a restart
/// to take effect; capture settings apply immediately).
pub fn replace_global(&self, config: GlobalConfig) {
self.config.store(Arc::new(config));
}
pub fn get(&self, name: &str) -> Option<Arc<WebUiProvider>> {
self.providers
.read()
.ok()
.and_then(|map| map.get(name).cloned())
}
pub fn all(&self) -> Vec<Arc<WebUiProvider>> {
self.providers
.read()
.map(|map| map.values().cloned().collect())
.unwrap_or_default()
}
fn browser_settings(&self, provider: &str) -> crate::browser::BrowserSettings {
let global = self.config.load();
let configured = global.browser.executable.trim();
let executable = if configured.is_empty() {
resolve_executable(None, std::env::var("CHROME").ok())
} else {
let path = expand_tilde(configured);
Some(path)
};
crate::browser::BrowserSettings {
provider: provider.to_string(),
account: None,
executable,
headless: global.browser.headless,
// The historical, single-account profile directory. A provider that
// declares accounts replaces it with a per-account one.
profile_dir: global.profile_dir_for(&self.layout, provider),
max_tabs: global.browser.max_tabs.max(1),
busy_wait: Duration::from_secs(global.browser.busy_wait_s.max(1)),
launch_timeout: Duration::from_secs(global.browser.launch_timeout_s.max(5)),
request_timeout: Duration::from_secs(global.browser.request_timeout_s.max(5)),
extra_args: global.browser.extra_args.clone(),
stealth: global.browser.stealth,
attach: global.browser.attach,
debug_port: global.browser.debug_port,
}
}
fn build_provider(&self, cfg: ProviderConfig) -> Arc<WebUiProvider> {
let name = cfg.name().to_string();
let global = self.config.load_full();
let settings = self.browser_settings(&name);
let profiles_root = settings
.profile_dir
.parent()
.map(|path| path.to_path_buf())
.unwrap_or_else(|| global.resolved_profile_root(&self.layout).join(&name));
let recorder = DebugRecorder::new(
global.resolved_debug_dir(&self.layout),
global.debug.screenshots,
);
let defaults = CaptureDefaults::from_config(&global);
if global.browser.max_tabs > 1 {
tracing::info!(
provider = %name,
max_tabs = global.browser.max_tabs,
"concurrent turns per account"
);
}
Arc::new(WebUiProvider::new(
cfg,
settings,
profiles_root,
recorder,
defaults,
))
}
/// Load (or reload) every provider file. Returns what changed.
///
/// The provider map is never held across an await: reloading an account can
/// stop a browser, which is asynchronous work.
pub async fn reload(&self) -> Result<ReloadReport> {
let mut report = ReloadReport::default();
let mut seen: BTreeSet<String> = BTreeSet::new();
let mut to_update: Vec<(Arc<WebUiProvider>, ProviderConfig)> = Vec::new();
let mut to_add: Vec<ProviderConfig> = Vec::new();
for (path, result) in ProviderConfig::load_dir(&self.layout.providers_dir()) {
let cfg = match result {
Ok(cfg) => cfg,
Err(err) => {
report.problems.push(format!("{}: {err}", path.display()));
tracing::error!(path = %path.display(), error = %err, "cannot load the provider configuration");
continue;
}
};
if !cfg.enabled() {
tracing::debug!(provider = %cfg.name(), "provider disabled in its configuration file");
continue;
}
if !cfg.is_usable() {
for problem in &cfg.problems {
report.problems.push(format!("{}: {problem}", cfg.name()));
}
}
let name = cfg.name().to_string();
if !seen.insert(name.clone()) {
report
.problems
.push(format!("duplicate provider '{name}' in {}", path.display()));
continue;
}
match self.get(&name) {
Some(existing) => {
if existing.cfg().file.serialized() != cfg.file.serialized() {
report.updated.push(name);
to_update.push((existing, cfg));
}
}
None => {
report.added.push(name);
to_add.push(cfg);
}
}
}
let mut removed: Vec<Arc<WebUiProvider>> = Vec::new();
{
let mut map = self.providers.write().expect("provider map poisoned");
for cfg in to_add {
let provider = self.build_provider(cfg);
map.insert(provider.name().to_string(), provider);
}
let stale: Vec<String> = map
.keys()
.filter(|key| !seen.contains(*key))
.cloned()
.collect();
for name in stale {
if let Some(provider) = map.remove(&name) {
removed.push(provider);
report.removed.push(name);
}
}
}
for (provider, cfg) in to_update {
provider.replace_config(cfg).await;
}
// Shut the removed providers down outside the lock: it awaits.
for provider in removed {
if let Err(err) = provider.shutdown().await {
tracing::warn!(provider = %provider.name(), error = %err, "cannot stop the removed provider");
}
}
for name in &report.added {
tracing::info!(provider = %name, "provider registered");
}
if !report.problems.is_empty() {
tracing::warn!(problems = ?report.problems, "provider configuration problems");
}
Ok(report)
}
/// Drop every provider except one. Returns false when the name is unknown.
pub async fn retain_only(&self, keep: &str) -> bool {
let mut removed = Vec::new();
{
let mut map = self.providers.write().expect("provider map poisoned");
let names: Vec<String> = map
.keys()
.filter(|name| name.as_str() != keep)
.cloned()
.collect();
for name in names {
if let Some(provider) = map.remove(&name) {
removed.push(provider);
}
}
}
let found = self.get(keep).is_some();
for provider in removed {
let _ = provider.shutdown().await;
}
found
}
/// Stop every browser process owned by the registry.
pub async fn shutdown_all(&self) {
for provider in self.all() {
if let Err(err) = provider.shutdown().await {
tracing::debug!(provider = %provider.name(), error = %err, "shutdown failed");
}
}
}
/// Human readable status of one provider account, for the CLI and the API.
pub async fn status_of(&self, provider: &str, account: Option<&str>) -> SessionStatus {
let Some(handle) = self.get(provider) else {
let mut status =
SessionStatus::error(provider, format!("provider '{provider}' is not configured"));
status.state = crate::browser::SessionState::Misconfigured;
return status;
};
// A bare provider name selects its default account; the report says
// which account was actually probed, exactly like a validation report.
let cfg = handle.cfg();
let account = match cfg.resolve_account_id(account) {
Ok(account) => account,
Err(detail) => return SessionStatus::error(&owner_label(provider, account), detail),
};
let label = owner_label(provider, account.as_deref());
let mut status = match handle.probe_session(account.as_deref()).await {
Ok(session) => handle
.validate_session(account.as_deref(), &session)
.await
.unwrap_or_else(|err| SessionStatus::error(&label, err.to_string())),
Err(err) => SessionStatus::error(&label, err.to_string()),
};
// The page probe only knows the provider name; the label is what tells
// two accounts of the same provider apart.
status.provider = label;
status
}
/// Occupancy of every tab pool, for the status endpoints.
pub fn tab_stats(&self) -> Vec<(String, crate::browser::TabStats)> {
self.all()
.iter()
.flat_map(|provider| {
let name = provider.name().to_string();
provider
.tab_stats()
.into_iter()
.map(move |(account, stats)| (format!("{name}@{account}"), stats))
.collect::<Vec<_>>()
})
.collect()
}
}
#[async_trait]
impl ChatBackend for ProviderRegistry {
async fn complete(
&self,
provider: &str,
account: Option<&str>,
request: TurnRequest,
) -> Result<TurnStream> {
let handle = self
.get(provider)
.ok_or_else(|| GatewayError::UnknownProvider(provider.to_string()))?;
let cfg = handle.cfg();
if !cfg.is_usable() {
return Err(GatewayError::ProviderMisconfigured {
provider: provider.to_string(),
detail: cfg.problems.join("; "),
});
}
// Admission control lives in the provider account's tab pool: the lease
// is held for the whole capture and released when it ends.
let wait = request
.wait_for_tab
.unwrap_or_else(|| handle.busy_wait(account));
let session = handle.session_with_wait(account, wait).await?;
handle.run_turn(session, request).await
}
async fn status(&self, provider: &str, account: Option<&str>) -> SessionStatus {
self.status_of(provider, account).await
}
async fn validate(&self, provider: &str, account: Option<&str>) -> Result<ValidationReport> {
let handle = self
.get(provider)
.ok_or_else(|| GatewayError::UnknownProvider(provider.to_string()))?;
let cfg = handle.cfg();
// An unknown account is a 404, not a report: the caller named an
// account that does not exist rather than one that fails to answer.
let account = cfg
.resolve_account_id(account)
.map_err(GatewayError::UnknownAccount)?;
validate_provider(handle.as_ref(), &cfg, account.as_deref()).await
}
async fn reload(&self) -> Result<ReloadReport> {
ProviderRegistry::reload(self).await
}
fn providers(&self) -> Vec<ProviderDescriptor> {
self.all()
.into_iter()
.map(|provider| descriptor_of(&provider))
.collect()
}
fn resolve(&self, model: &str) -> Result<ResolvedModel> {
let reference = parse_model(model);
let map = self.providers.read().expect("provider map poisoned");
let provider = match map.get(&reference.provider) {
Some(provider) => provider,
None if map.len() == 1 => {
let only = map.values().next().expect("one provider");
tracing::debug!(
model = %reference.original(),
provider = %only.name(),
"routing to the only configured provider"
);
only
}
None if map.is_empty() => {
return Err(GatewayError::UnknownProvider(format!(
"{} (no provider is configured: check {})",
reference.original(),
self.layout.providers_dir().display()
)))
}
None => {
return Err(GatewayError::UnknownProvider(format!(
"{} (configured providers: {})",
reference.original(),
map.keys().cloned().collect::<Vec<_>>().join(", ")
)))
}
};
let cfg = provider.cfg();
let account = cfg
.resolve_account_id(reference.account.as_deref())
.map_err(|detail| {
GatewayError::UnknownAccount(format!("{detail} (model '{}')", reference.original()))
})?;
Ok(ResolvedModel {
provider: descriptor_of(provider),
account,
model: reference.model,
})
}
async fn shutdown(&self) {
self.shutdown_all().await;
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::providers::config::IMPLICIT_ACCOUNT;
/// A registry with two providers, one of them multi-account.
fn registry() -> (ProviderRegistry, tempfile::TempDir) {
let tmp = tempfile::tempdir().unwrap();
let layout = Layout::rooted_at(tmp.path().join("state"));
layout.ensure().unwrap();
std::fs::remove_file(layout.provider_config("chatgpt")).ok();
std::fs::remove_file(layout.provider_config("claude")).ok();
std::fs::remove_file(layout.provider_config("deepseek")).ok();
let accounts = r#"
[[accounts]]
id = "perso"
email = "[email protected]"
default = true
[[accounts]]
id = "pro"
email = "[email protected]"
"#;
std::fs::write(
layout.provider_config("chatgpt"),
format!("{}{accounts}", include_str!("../../providers/chatgpt.toml")),
)
.unwrap();
std::fs::write(
layout.provider_config("claude"),
include_str!("../../providers/claude.toml"),
)
.unwrap();
(ProviderRegistry::new(layout, GlobalConfig::default()), tmp)
}
#[tokio::test]
async fn the_shipped_providers_are_registered() {
let (registry, _tmp) = registry();
let report = registry.reload().await.unwrap();
assert!(report.problems.is_empty(), "{:?}", report.problems);
let mut names: Vec<String> = report.added;
names.sort();
assert_eq!(names, vec!["chatgpt".to_string(), "claude".to_string()]);
}
#[tokio::test]
async fn a_model_string_selects_provider_account_and_model() {
let (registry, _tmp) = registry();
registry.reload().await.unwrap();
let plain = registry.resolve("claude").unwrap();
assert_eq!(plain.provider.name, "claude");
assert_eq!(plain.account, None);
assert_eq!(plain.owner(), "claude");
let named = registry.resolve("chatgpt@pro/claude-sonnet-4").unwrap();
assert_eq!(named.provider.name, "chatgpt");
assert_eq!(named.account.as_deref(), Some("pro"));
assert_eq!(named.model.as_deref(), Some("claude-sonnet-4"));
assert_eq!(named.owner(), "chatgpt@pro");
// The default account is used when none is named.
let default = registry.resolve("chatgpt").unwrap();
assert_eq!(default.account.as_deref(), Some("perso"));
// An address works as well as an id.
let by_email = registry
.resolve("[email protected]@gmail.com")
.unwrap();
assert_eq!(by_email.account.as_deref(), Some("pro"));
}
#[tokio::test]
async fn unknown_providers_and_accounts_are_404s() {
let (registry, _tmp) = registry();
registry.reload().await.unwrap();
let err = registry.resolve("gemini").unwrap_err();
assert_eq!(err.code(), "model_not_found");
assert_eq!(err.http_status(), 404);
assert!(err.to_string().contains("chatgpt"), "{err}");
let err = registry.resolve("chatgpt@nobody").unwrap_err();
assert_eq!(err.code(), "account_not_found");
assert_eq!(err.http_status(), 404);
assert!(err.to_string().contains("no account 'nobody'"), "{err}");
assert!(err.to_string().contains("chatgpt@nobody"), "{err}");
}
#[tokio::test]
async fn every_account_is_described_with_its_own_profile() {
let (registry, _tmp) = registry();
registry.reload().await.unwrap();
let descriptor = registry.resolve("chatgpt").unwrap().provider;
assert_eq!(descriptor.accounts.len(), 2);
assert_eq!(
descriptor.accounts[0].email.as_deref(),
Some("[email protected]")
);
assert!(descriptor.accounts[0].default);
assert!(!descriptor.accounts[1].default);
assert_ne!(
descriptor.accounts[0].profile,
descriptor.accounts[1].profile
);
// A single-account provider reports no account list at all.
let claude = registry.resolve("claude").unwrap().provider;
assert!(claude.accounts.is_empty());
}
#[tokio::test]
async fn a_single_provider_still_answers_to_a_bare_model_name() {
let tmp = tempfile::tempdir().unwrap();
let layout = Layout::rooted_at(tmp.path().join("state"));
layout.ensure().unwrap();
std::fs::remove_file(layout.provider_config("chatgpt")).ok();
std::fs::remove_file(layout.provider_config("claude")).ok();
std::fs::remove_file(layout.provider_config("deepseek")).ok();
std::fs::write(
layout.provider_config("chatgpt"),
include_str!("../../providers/chatgpt.toml"),
)
.unwrap();
let registry = ProviderRegistry::new(layout, GlobalConfig::default());
registry.reload().await.unwrap();
let resolved = registry.resolve("gpt-4o").unwrap();
assert_eq!(resolved.provider.name, "chatgpt");
assert_eq!(resolved.account, None);
// A provider that declares no account refuses a named one.
let err = registry.resolve("chatgpt@perso").unwrap_err();
assert!(err.to_string().contains("single account"), "{err}");
}
#[tokio::test]
async fn the_implicit_account_keeps_the_legacy_profile_directory() {
let tmp = tempfile::tempdir().unwrap();
let layout = Layout::rooted_at(tmp.path().join("state"));
layout.ensure().unwrap();
std::fs::remove_file(layout.provider_config("chatgpt")).ok();
std::fs::remove_file(layout.provider_config("claude")).ok();
std::fs::remove_file(layout.provider_config("deepseek")).ok();
std::fs::write(
layout.provider_config("chatgpt"),
include_str!("../../providers/chatgpt.toml"),
)
.unwrap();
let registry = ProviderRegistry::new(layout.clone(), GlobalConfig::default());
registry.reload().await.unwrap();
let provider = registry.get("chatgpt").unwrap();
assert_eq!(
provider.profile_dir(None).unwrap(),
layout.profile_dir("chatgpt")
);
assert_eq!(
provider.profile_dir(None).unwrap(),
layout.profiles_dir().join("chatgpt").join("user-data-dir")
);
}
#[test]
fn the_implicit_account_name_is_stable() {
assert_eq!(IMPLICIT_ACCOUNT, "default");
}
}
src/providers/trait.rs
+199
View File
@@ -0,0 +1,199 @@
//! The provider contract: one browser-driven chat turn.
use std::path::PathBuf;
use std::sync::Arc;
use std::time::Duration;
use async_trait::async_trait;
use serde::{Deserialize, Serialize};
use tokio::sync::mpsc;
use crate::browser::{Session, SessionStatus};
use crate::error::Result;
use crate::providers::config::{Capabilities, ProviderConfig};
/// A file materialised on disk, ready to be attached in the web UI.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Attachment {
pub path: PathBuf,
pub mime: String,
pub kind: AttachmentKind,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum AttachmentKind {
Image,
Document,
}
/// Everything needed to run one turn.
#[derive(Debug, Clone)]
pub struct TurnRequest {
/// Text actually pushed into the editor (history replayed when needed).
pub prompt: String,
pub attachments: Vec<Attachment>,
/// Web conversation to reopen, when the thread is being continued.
pub reuse_url: Option<String>,
/// Force a fresh web conversation and inject the whole history.
pub replay_history: bool,
/// Probe turns (selftest/validate) skip the thread bookkeeping.
pub probe: bool,
pub trace_id: String,
/// How long this turn may wait for a free tab.
///
/// `None` uses the configured `browser.busy_wait_s`: a client request that
/// arrives while the account is busy should get a 429 rather than queue for
/// minutes. The extra variants of one request are our own concurrency, so
/// they wait for their turn instead of failing.
pub wait_for_tab: Option<Duration>,
}
impl TurnRequest {
pub fn probe(trace_id: impl Into<String>) -> Self {
Self {
prompt: "Answer with the single word: ok".to_string(),
attachments: Vec::new(),
reuse_url: None,
replay_history: true,
probe: true,
trace_id: trace_id.into(),
wait_for_tab: None,
}
}
}
/// Incremental result of a turn.
#[derive(Debug)]
pub enum TurnEvent {
/// Newly generated text, appended to the previous deltas.
Delta(String),
/// Terminal event carrying the full answer.
Completed(TurnOutcome),
/// Terminal error: the capture failed after the stream started.
Failed(crate::error::GatewayError),
}
/// Final result of a turn.
#[derive(Debug, Clone, Default)]
pub struct TurnOutcome {
pub text: String,
pub web_url: Option<String>,
pub web_id: Option<String>,
pub elapsed_ms: u64,
pub deltas: u32,
/// Where the final text comes from: `clipboard`, `dom` or `text`.
pub fidelity: String,
}
/// One declared account of a provider, as reported by the API.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct AccountInfo {
pub id: String,
#[serde(skip_serializing_if = "Option::is_none")]
pub email: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub label: Option<String>,
/// True for the account a request naming the provider alone reaches.
pub default: bool,
/// Directory of the browser profile holding this account's session.
pub profile: String,
}
/// Stream of turn events.
pub struct TurnStream {
rx: mpsc::Receiver<TurnEvent>,
}
impl Unpin for TurnStream {}
impl futures::Stream for TurnStream {
type Item = TurnEvent;
fn poll_next(
self: std::pin::Pin<&mut Self>,
cx: &mut std::task::Context<'_>,
) -> std::task::Poll<Option<Self::Item>> {
self.get_mut().rx.poll_recv(cx)
}
}
impl TurnStream {
pub fn new(rx: mpsc::Receiver<TurnEvent>) -> Self {
Self { rx }
}
pub async fn recv(&mut self) -> Option<TurnEvent> {
self.rx.recv().await
}
pub fn into_receiver(self) -> mpsc::Receiver<TurnEvent> {
self.rx
}
/// Drain the stream, returning the final outcome.
pub async fn collect(mut self) -> Result<TurnOutcome> {
while let Some(event) = self.rx.recv().await {
match event {
TurnEvent::Completed(outcome) => return Ok(outcome),
TurnEvent::Failed(err) => return Err(err),
TurnEvent::Delta(_) => {}
}
}
Err(crate::error::GatewayError::internal(
"the provider task ended without producing an answer",
))
}
}
/// A chat provider backed by a web UI.
#[async_trait]
pub trait LlmProvider: Send + Sync {
/// Configured name, e.g. "chatgpt".
fn name(&self) -> &str;
/// Current configuration snapshot (hot reloaded).
fn cfg(&self) -> Arc<ProviderConfig>;
/// What the web UI supports.
fn capabilities(&self) -> Capabilities {
self.cfg().capabilities().clone()
}
/// True when the configuration has no problems.
fn is_configured(&self) -> bool {
self.cfg().is_usable()
}
/// Accounts declared by the configuration. Empty when the provider has a
/// single implicit one.
fn accounts(&self) -> Vec<AccountInfo> {
Vec::new()
}
/// Browser session of one account, starting the browser when needed.
///
/// `account` is an account id; `None` selects the default account.
async fn session(&self, account: Option<&str>) -> Result<Session>;
/// Read the current page status.
async fn validate_session(
&self,
account: Option<&str>,
session: &Session,
) -> Result<SessionStatus>;
/// Execute one turn. Returns immediately with a stream of events.
async fn run_turn(&self, session: Session, request: TurnRequest) -> Result<TurnStream>;
/// Start a brand new web conversation, returning its URL when known.
async fn new_conversation(&self, session: &Session) -> Result<Option<String>>;
/// Attach files in the web UI.
async fn upload_files(&self, session: &Session, files: &[Attachment]) -> Result<()>;
/// Close the browser process owned by this provider, every account.
async fn shutdown(&self) -> Result<()>;
/// Close the browser of one account only.
async fn shutdown_account(&self, account: &str) -> Result<()>;
}
src/providers/webui.rs
+1247
View File
File diff suppressed because it is too large Load Diff
src/selftest/mod.rs
+79
View File
@@ -0,0 +1,79 @@
//! Selector and session validation, exposed by the CLI and the HTTP API.
pub mod runner;
pub use runner::{validate_provider, PROBE_PROMPT};
use crate::error::{GatewayError, Result};
use crate::providers::{parse_model, LlmProvider, ProviderRegistry, ValidationReport};
/// Validate one provider, or every provider when `only` is None.
///
/// `only` accepts the same `provider[@account]` spelling as the command line,
/// and every report names the account it probed.
pub async fn run(registry: &ProviderRegistry, only: Option<&str>) -> Result<Vec<ValidationReport>> {
let wanted = only.map(parse_model);
let mut reports = Vec::new();
for provider in registry.all() {
let account = match &wanted {
Some(reference) => {
if provider.name() != reference.provider {
continue;
}
reference.account.clone()
}
None => None,
};
let cfg = provider.cfg();
let account = cfg
.resolve_account_id(account.as_deref())
.map_err(GatewayError::Config)?;
reports.push(validate_provider(provider.as_ref(), &cfg, account.as_deref()).await?);
}
Ok(reports)
}
/// Human readable report, one line per provider plus the step timings.
pub fn render_text(reports: &[ValidationReport]) -> String {
let mut out = String::new();
if reports.is_empty() {
out.push_str("no provider configured\n");
return out;
}
for report in reports {
let marker = if report.ok { "OK " } else { "FAIL" };
out.push_str(&format!(
"{marker} {:<12} {:<18} {} ms\n",
report.provider, report.status, report.duration_ms
));
for step in &report.steps {
out.push_str(&format!(
" - {:<14} {:>7} ms {}{}\n",
step.step,
step.ms,
if step.ok { "ok" } else { "failed" },
step.detail
.as_ref()
.map(|d| format!(" ({d})"))
.unwrap_or_default()
));
}
if let Some(message) = &report.message {
out.push_str(&format!(" ! {message}\n"));
}
if let Some(excerpt) = &report.response_excerpt {
out.push_str(&format!(" = {excerpt}\n"));
}
}
out
}
/// JSON report, for scripts.
pub fn render_json(reports: &[ValidationReport]) -> String {
serde_json::to_string_pretty(reports).unwrap_or_else(|_| "[]".to_string())
}
/// True when every report succeeded.
pub fn all_ok(reports: &[ValidationReport]) -> bool {
reports.iter().all(|report| report.ok)
}
src/selftest/runner.rs
+271
View File
@@ -0,0 +1,271 @@
//! Provider validation: proves that a configured web UI can still be driven.
//!
//! This is the selector non-regression test: it opens the provider page, checks
//! the login state and the required selectors, then sends a probe prompt and
//! verifies that a non-empty answer comes back.
use std::time::Instant;
use crate::browser::SessionState;
use crate::error::{GatewayError, Result};
use crate::providers::backend::{StepTiming, ValidationReport};
use crate::providers::config::ProviderConfig;
use crate::providers::{LlmProvider, TurnRequest};
use crate::util;
/// Prompt used to check that generation still works end to end.
pub const PROBE_PROMPT: &str = "Answer with the single word: ok";
/// Validate one provider account without any HTTP involvement.
pub async fn validate_provider(
provider: &dyn LlmProvider,
cfg: &ProviderConfig,
account: Option<&str>,
) -> Result<ValidationReport> {
let started = Instant::now();
let name = match account {
Some(account) => format!("{}@{account}", cfg.name()),
None => cfg.name().to_string(),
};
let mut steps: Vec<StepTiming> = Vec::new();
if !cfg.is_usable() {
return Ok(finish(
name,
false,
"misconfigured",
Some(cfg.problems.join("; ")),
None,
steps,
started,
));
}
// 1. Browser session.
let step_started = Instant::now();
let session = match provider.session(account).await {
Ok(session) => {
steps.push(StepTiming {
step: "session".to_string(),
ms: step_started.elapsed().as_millis() as u64,
ok: true,
detail: Some(format!("{} browser ready", session.browser())),
});
session
}
Err(err) => {
steps.push(failed_step("session", step_started, &err));
return Ok(failure_from_error(name, err, steps, started));
}
};
// 2. Page state: login, captcha, prompt field.
let step_started = Instant::now();
match provider.validate_session(account, &session).await {
Ok(status) if status.is_available() => steps.push(StepTiming {
step: "page_probe".to_string(),
ms: step_started.elapsed().as_millis() as u64,
ok: true,
detail: status.url.clone(),
}),
Ok(status) => {
steps.push(StepTiming {
step: "page_probe".to_string(),
ms: step_started.elapsed().as_millis() as u64,
ok: false,
detail: status.detail.clone(),
});
return Ok(finish(
name,
false,
state_to_status(status.state),
status.detail,
None,
steps,
started,
));
}
Err(err) => {
steps.push(failed_step("page_probe", step_started, &err));
return Ok(failure_from_error(name, err, steps, started));
}
}
// 3. Real turn with a probe prompt.
let step_started = Instant::now();
let request = TurnRequest {
prompt: PROBE_PROMPT.to_string(),
attachments: Vec::new(),
reuse_url: None,
replay_history: true,
probe: true,
trace_id: format!("selftest-{}", uuid::Uuid::new_v4().simple()),
wait_for_tab: None,
};
let stream = match provider.run_turn(session, request).await {
Ok(stream) => stream,
Err(err) => {
steps.push(failed_step("probe_prompt", step_started, &err));
return Ok(failure_from_error(name, err, steps, started));
}
};
match stream.collect().await {
Ok(outcome) => {
let text = outcome.text.trim().to_string();
let answered = !text.is_empty();
let matched = text.to_lowercase().contains("ok");
steps.push(StepTiming {
step: "probe_prompt".to_string(),
ms: step_started.elapsed().as_millis() as u64,
ok: answered,
detail: Some(format!("{} characters captured", text.chars().count())),
});
let (ok, status, message) = if !answered {
(
false,
"empty_answer",
Some("the provider returned an empty answer".to_string()),
)
} else if !matched {
(
true,
"ok",
Some(
"the answer did not contain the probe word but generation works"
.to_string(),
),
)
} else {
(true, "ok", None)
};
Ok(finish(
name,
ok,
status,
message,
Some(util::truncate(&text, 200)),
steps,
started,
))
}
Err(err) => {
steps.push(failed_step("probe_prompt", step_started, &err));
Ok(failure_from_error(name, err, steps, started))
}
}
}
fn failed_step(step: &str, started: Instant, err: &GatewayError) -> StepTiming {
StepTiming {
step: step.to_string(),
ms: started.elapsed().as_millis() as u64,
ok: false,
detail: Some(err.to_string()),
}
}
fn failure_from_error(
provider: String,
err: GatewayError,
steps: Vec<StepTiming>,
started: Instant,
) -> ValidationReport {
let status = error_to_status(&err);
finish(
provider,
false,
status,
Some(err.to_string()),
None,
steps,
started,
)
}
fn finish(
provider: String,
ok: bool,
status: &str,
message: Option<String>,
response_excerpt: Option<String>,
steps: Vec<StepTiming>,
started: Instant,
) -> ValidationReport {
ValidationReport {
provider,
ok,
status: status.to_string(),
message,
response_excerpt,
duration_ms: started.elapsed().as_millis() as u64,
checked_at: util::now_secs(),
steps,
}
}
/// Map a gateway error onto a selftest status.
pub fn error_to_status(err: &GatewayError) -> &'static str {
match err.code() {
"requires_login" => "login_required",
"captcha_required" => "captcha",
"upstream_rate_limit" => "rate_limited",
"upstream_timeout" => "timeout",
"selector_missing" => "selector_missing",
"provider_misconfigured" => "misconfigured",
"browser_profile_locked" | "browser_unavailable" => "browser_unavailable",
_ => "error",
}
}
fn state_to_status(state: SessionState) -> &'static str {
match state {
SessionState::Available => "ok",
SessionState::RequiresLogin => "login_required",
SessionState::Captcha => "captcha",
SessionState::Busy => "busy",
SessionState::Misconfigured => "misconfigured",
SessionState::Error => "error",
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn errors_map_to_selftest_statuses() {
let err = GatewayError::RequiresLogin {
provider: "chatgpt".into(),
};
assert_eq!(error_to_status(&err), "login_required");
let err = GatewayError::SelectorMissing {
selector: "a".into(),
step: "b".into(),
};
assert_eq!(error_to_status(&err), "selector_missing");
let err = GatewayError::CaptureTimeout {
stage: "x".into(),
timeout_s: 1,
};
assert_eq!(error_to_status(&err), "timeout");
let err = GatewayError::Captcha {
provider: "p".into(),
};
assert_eq!(error_to_status(&err), "captcha");
}
#[test]
fn states_map_to_selftest_statuses() {
assert_eq!(
state_to_status(SessionState::RequiresLogin),
"login_required"
);
assert_eq!(state_to_status(SessionState::Available), "ok");
assert_eq!(
state_to_status(SessionState::Misconfigured),
"misconfigured"
);
}
}
src/server/error.rs
+88
View File
@@ -0,0 +1,88 @@
//! Mapping gateway errors onto OpenAI error responses.
use axum::http::StatusCode;
use axum::response::{IntoResponse, Response};
use axum::Json;
use serde::Serialize;
use crate::error::GatewayError;
#[derive(Debug, Clone, Serialize)]
pub struct OpenAiError {
pub message: String,
#[serde(rename = "type")]
pub kind: &'static str,
#[serde(skip_serializing_if = "Option::is_none")]
pub param: Option<String>,
pub code: &'static str,
}
#[derive(Debug, Clone, Serialize)]
pub struct OpenAiErrorBody {
pub error: OpenAiError,
}
impl OpenAiErrorBody {
pub fn from_error(err: &GatewayError) -> Self {
Self {
error: OpenAiError {
message: err.to_string(),
kind: err.error_type(),
param: err.param().map(str::to_string),
code: err.code(),
},
}
}
pub fn new(message: impl Into<String>, kind: &'static str, code: &'static str) -> Self {
Self {
error: OpenAiError {
message: message.into(),
kind,
param: None,
code,
},
}
}
pub fn to_json(&self) -> String {
serde_json::to_string(self).unwrap_or_else(|_| {
r#"{"error":{"message":"internal error","type":"api_error","code":"internal_error"}}"#
.to_string()
})
}
}
impl IntoResponse for GatewayError {
fn into_response(self) -> Response {
let status =
StatusCode::from_u16(self.http_status()).unwrap_or(StatusCode::INTERNAL_SERVER_ERROR);
tracing::warn!(code = self.code(), status = self.http_status(), error = %self, "request failed");
(status, Json(OpenAiErrorBody::from_error(&self))).into_response()
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn errors_serialise_into_the_openai_shape() {
let err = GatewayError::RequiresLogin {
provider: "chatgpt".into(),
};
let body = OpenAiErrorBody::from_error(&err);
let value = serde_json::to_value(&body).unwrap();
assert_eq!(value["error"]["type"], "invalid_request_error");
assert_eq!(value["error"]["code"], "requires_login");
assert!(value["error"]["message"]
.as_str()
.unwrap()
.contains("llm-gateway login"));
let err = GatewayError::unsupported_param("tools", "is not supported");
let body = OpenAiErrorBody::from_error(&err);
assert_eq!(body.error.param.as_deref(), Some("tools"));
assert_eq!(body.error.code, "unsupported_parameter");
}
}
src/server/logbuf.rs
+141
View File
@@ -0,0 +1,141 @@
//! Recent log lines, replayed on the dashboard.
use std::collections::VecDeque;
use std::sync::{Arc, Mutex};
use tokio::sync::broadcast;
use tracing::field::{Field, Visit};
use tracing::{Event, Subscriber};
use tracing_subscriber::layer::{Context, Layer};
/// Ring buffer plus live fan-out of the formatted log lines.
#[derive(Clone)]
pub struct LogBuffer {
capacity: usize,
recent: Arc<Mutex<VecDeque<String>>>,
sender: broadcast::Sender<String>,
}
impl LogBuffer {
pub fn new(capacity: usize) -> Self {
let (sender, _) = broadcast::channel(256);
Self {
capacity,
recent: Arc::new(Mutex::new(VecDeque::with_capacity(capacity))),
sender,
}
}
pub fn push(&self, line: String) {
if let Ok(mut recent) = self.recent.lock() {
if recent.len() == self.capacity {
recent.pop_front();
}
recent.push_back(line.clone());
}
let _ = self.sender.send(line);
}
pub fn subscribe(&self) -> broadcast::Receiver<String> {
self.sender.subscribe()
}
pub fn recent(&self) -> Vec<String> {
self.recent
.lock()
.map(|recent| recent.iter().cloned().collect())
.unwrap_or_default()
}
}
/// Tracing layer feeding a LogBuffer.
pub struct BufferLayer {
buffer: LogBuffer,
}
impl BufferLayer {
pub fn new(buffer: LogBuffer) -> Self {
Self { buffer }
}
}
impl<S: Subscriber> Layer<S> for BufferLayer {
fn on_event(&self, event: &Event<'_>, _context: Context<'_, S>) {
let metadata = event.metadata();
let mut visitor = LineVisitor::default();
event.record(&mut visitor);
let mut line = format!("{} {}", metadata.level(), metadata.target());
if let Some(message) = visitor.message {
line.push_str(": ");
line.push_str(&message);
}
for (key, value) in visitor.fields {
line.push(' ');
line.push_str(&key);
line.push('=');
line.push_str(&value);
}
self.buffer.push(line);
}
}
#[derive(Default)]
struct LineVisitor {
message: Option<String>,
fields: Vec<(String, String)>,
}
impl Visit for LineVisitor {
fn record_str(&mut self, field: &Field, value: &str) {
self.record(field, value.to_string());
}
fn record_debug(&mut self, field: &Field, value: &dyn std::fmt::Debug) {
self.record(field, format!("{value:?}"));
}
fn record_i64(&mut self, field: &Field, value: i64) {
self.record(field, value.to_string());
}
fn record_u64(&mut self, field: &Field, value: u64) {
self.record(field, value.to_string());
}
fn record_bool(&mut self, field: &Field, value: bool) {
self.record(field, value.to_string());
}
}
impl LineVisitor {
fn record(&mut self, field: &Field, value: String) {
if field.name() == "message" {
self.message = Some(value);
} else {
self.fields.push((field.name().to_string(), value));
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn ring_buffer_keeps_the_most_recent_lines() {
let buffer = LogBuffer::new(3);
for index in 0..5 {
buffer.push(format!("line {index}"));
}
assert_eq!(buffer.recent(), vec!["line 2", "line 3", "line 4"]);
}
#[test]
fn subscribers_receive_new_lines() {
let buffer = LogBuffer::new(4);
let mut receiver = buffer.subscribe();
buffer.push("hello".to_string());
assert_eq!(receiver.try_recv().unwrap(), "hello");
}
}
src/server/middleware.rs
+69
View File
@@ -0,0 +1,69 @@
//! Optional bearer-token authentication.
use axum::extract::{Request, State};
use axum::http::header::AUTHORIZATION;
use axum::middleware::Next;
use axum::response::{IntoResponse, Response};
use crate::error::GatewayError;
use crate::server::state::AppState;
/// Reject requests that do not carry the configured API key.
pub async fn require_api_key(
State(state): State<AppState>,
request: Request,
next: Next,
) -> Response {
let expected = state.config().server.api_key.clone();
if expected.is_empty() || is_open_path(request.uri().path()) {
return next.run(request).await;
}
let provided = request
.headers()
.get(AUTHORIZATION)
.and_then(|value| value.to_str().ok())
.and_then(|value| value.strip_prefix("Bearer "))
.map(str::trim);
match provided {
Some(token) if constant_time_eq(token, &expected) => next.run(request).await,
_ => GatewayError::InvalidApiKey.into_response(),
}
}
/// Paths that stay reachable without a key: health and the dashboard.
fn is_open_path(path: &str) -> bool {
matches!(path, "/health" | "/dashboard" | "/dashboard/events")
}
/// Compare two secrets without leaking their length through timing.
fn constant_time_eq(a: &str, b: &str) -> bool {
let (a, b) = (a.as_bytes(), b.as_bytes());
if a.len() != b.len() {
return false;
}
let mut diff = 0u8;
for (x, y) in a.iter().zip(b.iter()) {
diff |= x ^ y;
}
diff == 0
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn secrets_are_compared_exactly() {
assert!(constant_time_eq("secret", "secret"));
assert!(!constant_time_eq("secret", "secrez"));
assert!(!constant_time_eq("secret", "secret-longer"));
}
#[test]
fn open_paths_are_limited() {
assert!(is_open_path("/health"));
assert!(!is_open_path("/v1/models"));
}
}
src/server/mod.rs
+53
View File
@@ -0,0 +1,53 @@
//! HTTP layer: OpenAI-compatible routes, SSE and the local dashboard.
pub mod error;
pub mod logbuf;
pub mod middleware;
pub mod models;
pub mod routes;
pub mod sse;
pub mod state;
pub use logbuf::{BufferLayer, LogBuffer};
pub use state::{AppState, Metrics};
use axum::extract::DefaultBodyLimit;
use axum::routing::{get, post};
use axum::Router;
use tower_http::cors::CorsLayer;
use tower_http::trace::TraceLayer;
/// Build the application router with every route wired up.
pub fn router(state: AppState) -> Router {
let body_limit = state.config().server.request_body_limit.max(1024);
let open = Router::new()
.route("/health", get(routes::health))
.route("/metrics", get(routes::metrics))
.route("/dashboard", get(routes::dashboard))
.route("/dashboard/events", get(routes::dashboard_events));
let api = Router::new()
.route("/v1/chat/completions", post(routes::chat_completions))
.route("/v1/completions", post(routes::completions))
.route("/v1/models", get(routes::models))
.route(
"/v1/providers/{provider}/status",
get(routes::provider_status),
)
.route(
"/v1/providers/{provider}/validate",
post(routes::provider_validate),
)
.route("/v1/admin/reload", post(routes::admin_reload))
.route_layer(axum::middleware::from_fn_with_state(
state.clone(),
middleware::require_api_key,
));
open.merge(api)
.layer(DefaultBodyLimit::max(body_limit))
.layer(CorsLayer::permissive())
.layer(TraceLayer::new_for_http())
.with_state(state)
}
src/server/models.rs
+449
View File
@@ -0,0 +1,449 @@
//! OpenAI-compatible request and response types.
use serde::{Deserialize, Serialize};
/// Content of a message: either a plain string or an array of typed parts.
#[derive(Debug, Clone, Deserialize, Serialize)]
#[serde(untagged)]
pub enum MessageContent {
Text(String),
Parts(Vec<ContentPart>),
}
impl Default for MessageContent {
fn default() -> Self {
Self::Text(String::new())
}
}
impl MessageContent {
/// Concatenated text of every text part.
pub fn text(&self) -> String {
match self {
Self::Text(text) => text.clone(),
Self::Parts(parts) => parts
.iter()
.filter(|part| part.kind == "text" || part.kind == "input_text")
.filter_map(|part| part.text.clone())
.collect::<Vec<_>>()
.join("\n"),
}
}
/// Image parts, in order.
pub fn images(&self) -> Vec<&ImageUrl> {
match self {
Self::Text(_) => Vec::new(),
Self::Parts(parts) => parts
.iter()
.filter(|part| part.kind == "image_url" || part.kind == "input_image")
.filter_map(|part| part.image_url.as_ref())
.collect(),
}
}
pub fn has_images(&self) -> bool {
!self.images().is_empty()
}
}
/// One element of a multimodal content array.
#[derive(Debug, Clone, Deserialize, Serialize)]
pub struct ContentPart {
#[serde(rename = "type")]
pub kind: String,
#[serde(default)]
pub text: Option<String>,
#[serde(default)]
pub image_url: Option<ImageUrl>,
}
#[derive(Debug, Clone, Deserialize, Serialize)]
pub struct ImageUrl {
pub url: String,
#[serde(default)]
pub detail: Option<String>,
}
#[derive(Debug, Clone, Deserialize, Serialize)]
pub struct ChatMessage {
pub role: String,
#[serde(default)]
pub content: MessageContent,
#[serde(default)]
pub name: Option<String>,
#[serde(default)]
pub tool_call_id: Option<String>,
}
/// `POST /v1/chat/completions` body.
#[derive(Debug, Clone, Deserialize)]
pub struct ChatCompletionRequest {
pub model: String,
#[serde(default)]
pub messages: Vec<ChatMessage>,
#[serde(default)]
pub stream: bool,
#[serde(default)]
pub n: Option<u32>,
#[serde(default)]
pub tools: Option<serde_json::Value>,
#[serde(default)]
pub functions: Option<serde_json::Value>,
#[serde(default)]
pub tool_choice: Option<serde_json::Value>,
#[serde(default)]
pub temperature: Option<f64>,
#[serde(default)]
pub top_p: Option<f64>,
#[serde(default)]
pub max_tokens: Option<u32>,
#[serde(default)]
pub max_completion_tokens: Option<u32>,
#[serde(default)]
pub stop: Option<serde_json::Value>,
#[serde(default)]
pub response_format: Option<serde_json::Value>,
#[serde(default)]
pub seed: Option<i64>,
#[serde(default)]
pub presence_penalty: Option<f64>,
#[serde(default)]
pub frequency_penalty: Option<f64>,
#[serde(default)]
pub logit_bias: Option<serde_json::Value>,
#[serde(default)]
pub logprobs: Option<bool>,
#[serde(default)]
pub user: Option<String>,
#[serde(default)]
pub stream_options: Option<serde_json::Value>,
}
impl ChatCompletionRequest {
/// Parameters the browser backend accepts but cannot honour.
pub fn ignored_parameters(&self) -> Vec<String> {
let mut out = Vec::new();
let mut note = |present: bool, name: &str| {
if present {
out.push(format!("{name} is ignored by the browser backend"));
}
};
note(self.temperature.is_some(), "temperature");
note(self.top_p.is_some(), "top_p");
note(self.max_tokens.is_some(), "max_tokens");
note(
self.max_completion_tokens.is_some(),
"max_completion_tokens",
);
note(self.stop.is_some(), "stop");
note(self.response_format.is_some(), "response_format");
note(self.seed.is_some(), "seed");
note(self.presence_penalty.is_some(), "presence_penalty");
note(self.frequency_penalty.is_some(), "frequency_penalty");
note(self.logit_bias.is_some(), "logit_bias");
note(self.logprobs.is_some(), "logprobs");
note(self.user.is_some(), "user");
note(self.stream_options.is_some(), "stream_options");
out
}
/// Parameters that make the request impossible to serve.
pub fn unsupported_parameter(&self) -> Option<(&'static str, &'static str)> {
if self.tools.as_ref().is_some_and(|value| !value.is_null()) {
return Some((
"tools",
"tool calling is not available through the browser backend",
));
}
if self
.functions
.as_ref()
.is_some_and(|value| !value.is_null())
{
return Some((
"functions",
"function calling is not available through the browser backend",
));
}
if self
.tool_choice
.as_ref()
.is_some_and(|value| !value.is_null())
{
return Some((
"tool_choice",
"tool calling is not available through the browser backend",
));
}
if self.n == Some(0) {
return Some(("n", "must be at least 1"));
}
None
}
/// Number of completions to produce, clamped to `max_variants`.
///
/// Returns the count and, when the request asked for more than the gateway
/// accepts, the warning explaining the clamp.
pub fn variants(&self, max_variants: u32) -> (u32, Option<String>) {
let requested = self.n.unwrap_or(1).max(1);
let limit = max_variants.max(1);
if requested > limit {
return (
limit,
Some(format!(
"n = {requested} was clamped to {limit} (capture.max_variants)"
)),
);
}
(requested, None)
}
}
/// Legacy `POST /v1/completions` body.
#[derive(Debug, Clone, Deserialize)]
pub struct CompletionRequest {
pub model: String,
#[serde(default)]
pub prompt: PromptInput,
#[serde(default)]
pub stream: bool,
#[serde(default)]
pub max_tokens: Option<u32>,
#[serde(default)]
pub temperature: Option<f64>,
#[serde(default)]
pub stop: Option<serde_json::Value>,
}
#[derive(Debug, Clone, Deserialize)]
#[serde(untagged)]
pub enum PromptInput {
Text(String),
Parts(Vec<String>),
}
impl Default for PromptInput {
fn default() -> Self {
Self::Text(String::new())
}
}
impl PromptInput {
pub fn joined(&self) -> String {
match self {
Self::Text(text) => text.clone(),
Self::Parts(parts) => parts.join("\n"),
}
}
}
#[derive(Debug, Clone, Serialize)]
pub struct Usage {
pub prompt_tokens: usize,
pub completion_tokens: usize,
pub total_tokens: usize,
}
#[derive(Debug, Clone, Serialize)]
pub struct ResponseMessage {
pub role: &'static str,
pub content: String,
}
#[derive(Debug, Clone, Serialize)]
pub struct ChatChoice {
pub index: u32,
pub message: ResponseMessage,
pub finish_reason: &'static str,
#[serde(skip_serializing_if = "Option::is_none")]
pub logprobs: Option<serde_json::Value>,
}
/// Non-streaming chat completion payload.
#[derive(Debug, Clone, Serialize)]
pub struct ChatCompletion {
pub id: String,
pub object: &'static str,
pub created: u64,
pub model: String,
pub choices: Vec<ChatChoice>,
pub usage: Usage,
#[serde(skip_serializing_if = "Option::is_none")]
pub system_fingerprint: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub x_gateway_warnings: Option<Vec<String>>,
}
#[derive(Debug, Clone, Serialize)]
pub struct ChunkDelta {
#[serde(skip_serializing_if = "Option::is_none")]
pub role: Option<&'static str>,
#[serde(skip_serializing_if = "Option::is_none")]
pub content: Option<String>,
}
#[derive(Debug, Clone, Serialize)]
pub struct ChunkChoice {
pub index: u32,
pub delta: ChunkDelta,
pub finish_reason: Option<&'static str>,
}
/// One SSE chunk of a streamed chat completion.
#[derive(Debug, Clone, Serialize)]
pub struct ChatCompletionChunk {
pub id: String,
pub object: &'static str,
pub created: u64,
pub model: String,
pub choices: Vec<ChunkChoice>,
#[serde(skip_serializing_if = "Option::is_none")]
pub usage: Option<Usage>,
}
#[derive(Debug, Clone, Serialize)]
pub struct TextChoice {
pub text: String,
pub index: u32,
pub finish_reason: &'static str,
pub logprobs: Option<serde_json::Value>,
}
/// Non-streaming legacy completion payload.
#[derive(Debug, Clone, Serialize)]
pub struct TextCompletion {
pub id: String,
pub object: &'static str,
pub created: u64,
pub model: String,
pub choices: Vec<TextChoice>,
pub usage: Usage,
}
#[derive(Debug, Clone, Serialize)]
pub struct ModelEntry {
pub id: String,
pub object: &'static str,
pub created: u64,
pub owned_by: String,
#[serde(skip_serializing_if = "Option::is_none")]
pub x_status: Option<String>,
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn multimodal_content_is_flattened() {
let raw = r#"[
{"type": "text", "text": "describe this"},
{"type": "image_url", "image_url": {"url": "data:image/png;base64,AAA"}}
]"#;
let content: MessageContent = serde_json::from_str(raw).unwrap();
assert_eq!(content.text(), "describe this");
assert!(content.has_images());
assert_eq!(content.images()[0].url, "data:image/png;base64,AAA");
}
#[test]
fn plain_string_content_still_works() {
let content: MessageContent = serde_json::from_str("\"hello\"").unwrap();
assert_eq!(content.text(), "hello");
assert!(!content.has_images());
}
#[test]
fn request_parses_a_realistic_payload() {
let raw = r#"{
"model": "chatgpt",
"messages": [
{"role": "system", "content": "be brief"},
{"role": "user", "content": "hi"}
],
"stream": true,
"temperature": 0.2,
"max_tokens": 100,
"user": "abc"
}"#;
let request: ChatCompletionRequest = serde_json::from_str(raw).unwrap();
assert_eq!(request.model, "chatgpt");
assert!(request.stream);
assert_eq!(request.messages.len(), 2);
assert!(request.unsupported_parameter().is_none());
let ignored = request.ignored_parameters();
assert!(ignored.iter().any(|w| w.contains("temperature")));
assert!(ignored.iter().any(|w| w.contains("max_tokens")));
}
#[test]
fn tool_calling_is_rejected() {
let raw = r#"{"model": "chatgpt", "messages": [], "tools": [{"type": "function"}]}"#;
let request: ChatCompletionRequest = serde_json::from_str(raw).unwrap();
assert_eq!(request.unsupported_parameter().unwrap().0, "tools");
}
#[test]
fn n_is_accepted_clamped_and_never_zero() {
let request = |n: u32| {
serde_json::from_str::<ChatCompletionRequest>(&format!(
r#"{{"model": "chatgpt", "messages": [], "n": {n}}}"#
))
.unwrap()
};
assert!(request(1).unsupported_parameter().is_none());
assert_eq!(request(1).variants(4), (1, None));
assert_eq!(request(3).variants(4), (3, None));
let (count, warning) = request(9).variants(4);
assert_eq!(count, 4);
assert!(warning.unwrap().contains("clamped"));
// n = 0 is a client mistake, not a clamp.
assert_eq!(request(0).unsupported_parameter().unwrap().0, "n");
// No n at all means one completion.
let absent: ChatCompletionRequest =
serde_json::from_str(r#"{"model": "chatgpt", "messages": []}"#).unwrap();
assert_eq!(absent.variants(4), (1, None));
}
#[test]
fn responses_serialise_with_the_openai_shape() {
let body = ChatCompletion {
id: "chatcmpl-1".into(),
object: "chat.completion",
created: 1,
model: "chatgpt".into(),
choices: vec![ChatChoice {
index: 0,
message: ResponseMessage {
role: "assistant",
content: "hi".to_string(),
},
finish_reason: "stop",
logprobs: None,
}],
usage: Usage {
prompt_tokens: 1,
completion_tokens: 1,
total_tokens: 2,
},
system_fingerprint: None,
x_gateway_warnings: Some(vec!["temperature is ignored by the browser backend".into()]),
};
let value = serde_json::to_value(&body).unwrap();
assert_eq!(value["object"], "chat.completion");
assert_eq!(value["choices"][0]["message"]["role"], "assistant");
assert_eq!(value["choices"][0]["finish_reason"], "stop");
assert_eq!(value["usage"]["total_tokens"], 2);
assert!(value["choices"][0].get("logprobs").is_none());
assert_eq!(
value["x_gateway_warnings"][0],
"temperature is ignored by the browser backend"
);
}
}
src/server/routes.rs
+1137
View File
File diff suppressed because it is too large Load Diff
src/server/sse.rs
+404
View File
@@ -0,0 +1,404 @@
//! Server-sent events for streamed completions.
use std::convert::Infallible;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::Arc;
use axum::response::sse::Event;
use futures::{Stream, StreamExt};
use serde_json::json;
use crate::config::TokensSection;
use crate::conversation::ThreadStore;
use crate::providers::TurnEvent;
use crate::server::models::Usage;
use crate::tokens;
use crate::util;
/// Identity shared by every chunk of one completion.
#[derive(Debug, Clone)]
pub struct StreamMeta {
pub id: String,
pub model: String,
pub created: u64,
/// How many completions the stream carries (`n`, at least 1).
pub choices: u32,
}
/// Token accounting for the final usage block.
#[derive(Debug, Clone)]
pub struct TokenCounts {
pub prompt_tokens: usize,
pub encoding: TokensSection,
pub include_usage: bool,
}
/// Shared flag telling the cancellation guard that the turn finished.
#[derive(Clone, Default)]
pub struct CancelFlag(Arc<AtomicBool>);
impl CancelFlag {
pub fn new() -> Self {
Self::default()
}
pub fn mark_done(&self) {
self.0.store(true, Ordering::SeqCst);
}
pub fn is_done(&self) -> bool {
self.0.load(Ordering::SeqCst)
}
}
/// Marks a thread as interrupted when the client vanishes mid-generation.
pub struct CancelGuard {
store: Arc<ThreadStore>,
conversation_id: String,
flag: CancelFlag,
}
impl CancelGuard {
pub fn new(store: Arc<ThreadStore>, conversation_id: String, flag: CancelFlag) -> Self {
Self {
store,
conversation_id,
flag,
}
}
}
impl Drop for CancelGuard {
fn drop(&mut self) {
if self.flag.is_done() {
return;
}
let store = Arc::clone(&self.store);
let conversation_id = self.conversation_id.clone();
if let Ok(handle) = tokio::runtime::Handle::try_current() {
handle.spawn(async move {
let _ = store
.mark_incomplete(&conversation_id, "client disconnected mid-generation")
.await;
});
}
}
}
fn chunk_event(value: &serde_json::Value) -> Result<Event, Infallible> {
Ok(Event::default().data(value.to_string()))
}
/// One event tagged with the completion (`n`) it belongs to.
pub type IndexedEvent = (u32, TurnEvent);
/// Build the SSE stream of a chat completion.
///
/// With `n > 1` several completions are produced at once: every chunk carries
/// the index of the choice it belongs to, exactly like the OpenAI API, and the
/// stream ends once the last one has finished.
pub fn chat_event_stream<S>(
mut stream: S,
meta: StreamMeta,
counts: TokenCounts,
flag: CancelFlag,
) -> impl Stream<Item = Result<Event, Infallible>>
where
S: Stream<Item = IndexedEvent> + Unpin,
{
let choices = meta.choices.max(1);
async_stream::stream! {
// One opening chunk per choice, each announcing its assistant role.
for index in 0..choices {
yield chunk_event(&json!({
"id": meta.id,
"object": "chat.completion.chunk",
"created": meta.created,
"model": meta.model,
"choices": [{ "index": index, "delta": { "role": "assistant" }, "finish_reason": null }],
}));
}
let mut texts: Vec<String> = vec![String::new(); choices as usize];
let mut finished = 0u32;
while let Some((index, event)) = stream.next().await {
let slot = texts.get_mut(index as usize);
match event {
TurnEvent::Delta(delta) => {
if let Some(text) = slot {
text.push_str(&delta);
}
yield chunk_event(&json!({
"id": meta.id,
"object": "chat.completion.chunk",
"created": meta.created,
"model": meta.model,
"choices": [{ "index": index, "delta": { "content": delta }, "finish_reason": null }],
}));
}
TurnEvent::Completed(outcome) => {
if let Some(text) = slot {
if !outcome.text.is_empty() {
*text = outcome.text;
}
}
finished += 1;
yield chunk_event(&json!({
"id": meta.id,
"object": "chat.completion.chunk",
"created": meta.created,
"model": meta.model,
"choices": [{ "index": index, "delta": {}, "finish_reason": "stop" }],
}));
if finished >= choices {
flag.mark_done();
if counts.include_usage {
let completion_tokens: usize = texts
.iter()
.map(|text| tokens::count_tokens(text, &counts.encoding))
.sum();
let usage = Usage {
prompt_tokens: counts.prompt_tokens,
completion_tokens,
total_tokens: counts.prompt_tokens + completion_tokens,
};
yield chunk_event(&json!({
"id": meta.id,
"object": "chat.completion.chunk",
"created": meta.created,
"model": meta.model,
"choices": [],
"usage": usage,
}));
}
break;
}
}
TurnEvent::Failed(err) => {
tracing::warn!(
index,
code = err.code(),
error = %err,
"stream failed after it started"
);
yield chunk_event(&json!({
"error": {
"message": util::truncate(&err.to_string(), 500),
"type": err.error_type(),
"code": err.code(),
}
}));
break;
}
}
}
yield Ok(Event::default().data("[DONE]"));
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::providers::TurnOutcome;
use axum::response::sse::Sse;
use axum::response::IntoResponse;
use tokio::sync::mpsc;
use tokio_stream::wrappers::ReceiverStream;
/// A stream of events tagged with the choice they belong to.
fn indexed(rx: mpsc::Receiver<IndexedEvent>) -> ReceiverStream<IndexedEvent> {
ReceiverStream::new(rx)
}
fn counts() -> TokenCounts {
TokenCounts {
prompt_tokens: 3,
encoding: TokensSection {
encoding: "heuristic".into(),
fallback_heuristic: true,
},
include_usage: false,
}
}
/// Render every SSE event of a stream into the raw wire payload.
async fn collect_sse<S>(stream: S) -> String
where
S: Stream<Item = Result<Event, Infallible>> + Send + 'static,
{
let response = Sse::new(stream).into_response();
let bytes = axum::body::to_bytes(response.into_body(), usize::MAX)
.await
.unwrap();
String::from_utf8_lossy(&bytes).to_string()
}
#[tokio::test]
async fn deltas_become_chunks_and_the_stream_ends_with_done() {
let (tx, rx) = mpsc::channel(8);
tokio::spawn(async move {
tx.send((0, TurnEvent::Delta("Hel".into()))).await.unwrap();
tx.send((0, TurnEvent::Delta("lo".into()))).await.unwrap();
tx.send((
0,
TurnEvent::Completed(TurnOutcome {
text: "Hello".into(),
..Default::default()
}),
))
.await
.unwrap();
});
let flag = CancelFlag::new();
let stream = chat_event_stream(
indexed(rx),
StreamMeta {
id: "chatcmpl-1".into(),
model: "chatgpt".into(),
created: 1,
choices: 1,
},
counts(),
flag.clone(),
);
let joined = collect_sse(stream).await;
assert!(joined.contains(r#""content":"Hel""#), "{joined}");
assert!(joined.contains(r#""content":"lo""#), "{joined}");
assert!(joined.contains(r#""role":"assistant""#), "{joined}");
assert!(joined.contains(r#""finish_reason":"stop""#), "{joined}");
assert!(joined.contains("data: [DONE]"), "{joined}");
assert!(flag.is_done());
}
#[tokio::test]
async fn an_error_after_the_stream_started_is_reported_in_band() {
let (tx, rx) = mpsc::channel(8);
tokio::spawn(async move {
tx.send((0, TurnEvent::Delta("partial".into())))
.await
.unwrap();
tx.send((
0,
TurnEvent::Failed(crate::error::GatewayError::Captcha {
provider: "chatgpt".into(),
}),
))
.await
.unwrap();
});
let flag = CancelFlag::new();
let stream = chat_event_stream(
indexed(rx),
StreamMeta {
id: "chatcmpl-2".into(),
model: "chatgpt".into(),
created: 1,
choices: 1,
},
counts(),
flag,
);
let joined = collect_sse(stream).await;
assert!(joined.contains(r#""content":"partial""#), "{joined}");
assert!(joined.contains("captcha_required"), "{joined}");
// The stream is still terminated properly so that clients do not hang.
assert!(joined.contains("data: [DONE]"), "{joined}");
}
#[tokio::test]
async fn two_variants_stream_under_their_own_index_and_end_when_both_do() {
let (tx, rx) = mpsc::channel(8);
tokio::spawn(async move {
tx.send((0, TurnEvent::Delta("first".into())))
.await
.unwrap();
tx.send((1, TurnEvent::Delta("second".into())))
.await
.unwrap();
tx.send((
0,
TurnEvent::Completed(TurnOutcome {
text: "first".into(),
..Default::default()
}),
))
.await
.unwrap();
tx.send((
1,
TurnEvent::Completed(TurnOutcome {
text: "second".into(),
..Default::default()
}),
))
.await
.unwrap();
});
let flag = CancelFlag::new();
let stream = chat_event_stream(
indexed(rx),
StreamMeta {
id: "chatcmpl-3".into(),
model: "chatgpt".into(),
created: 1,
choices: 2,
},
counts(),
flag.clone(),
);
let joined = collect_sse(stream).await;
// Both choices announce their role, and each delta keeps its index.
assert!(joined.contains(r#""index":0"#), "{joined}");
assert!(joined.contains(r#""index":1"#), "{joined}");
assert!(joined.contains(r#""content":"first""#), "{joined}");
assert!(joined.contains(r#""content":"second""#), "{joined}");
assert_eq!(joined.matches(r#""finish_reason":"stop""#).count(), 2);
// The flag is only set once nothing more can arrive.
assert!(flag.is_done());
assert!(joined.contains("data: [DONE]"), "{joined}");
}
#[tokio::test]
async fn usage_counts_every_variant() {
let (tx, rx) = mpsc::channel(8);
tokio::spawn(async move {
for index in 0..2u32 {
tx.send((
index,
TurnEvent::Completed(TurnOutcome {
text: "hello world hello world".into(),
..Default::default()
}),
))
.await
.unwrap();
}
});
let mut counts = counts();
counts.include_usage = true;
let stream = chat_event_stream(
indexed(rx),
StreamMeta {
id: "chatcmpl-4".into(),
model: "chatgpt".into(),
created: 1,
choices: 2,
},
counts,
CancelFlag::new(),
);
let joined = collect_sse(stream).await;
let usage: serde_json::Value = joined
.lines()
.filter_map(|line| line.strip_prefix("data: "))
.filter(|payload| *payload != "[DONE]")
.filter_map(|payload| serde_json::from_str::<serde_json::Value>(payload).ok())
.find(|chunk| chunk.get("usage").is_some())
.expect("a usage chunk");
let completion_tokens = usage["usage"]["completion_tokens"].as_u64().unwrap();
assert!(completion_tokens >= 2, "{usage}");
}
}
src/server/state.rs
+107
View File
@@ -0,0 +1,107 @@
//! Shared server state.
use std::collections::BTreeMap;
use std::sync::atomic::{AtomicU64, Ordering};
use std::sync::{Arc, Mutex};
use std::time::Instant;
use arc_swap::ArcSwap;
use crate::config::{CliOverrides, GlobalConfig};
use crate::conversation::ThreadStore;
use crate::paths::Layout;
use crate::providers::ChatBackend;
use crate::server::logbuf::LogBuffer;
/// Cheap counters exposed by GET /metrics.
#[derive(Debug, Default)]
pub struct Metrics {
pub requests: AtomicU64,
pub errors: AtomicU64,
pub completions: AtomicU64,
pub streamed_chunks: AtomicU64,
pub captured_chars: AtomicU64,
error_codes: Mutex<BTreeMap<String, u64>>,
}
impl Metrics {
pub fn record_request(&self) {
self.requests.fetch_add(1, Ordering::Relaxed);
}
pub fn record_error(&self, code: &str) {
self.errors.fetch_add(1, Ordering::Relaxed);
if let Ok(mut map) = self.error_codes.lock() {
*map.entry(code.to_string()).or_insert(0) += 1;
}
}
pub fn snapshot(&self) -> serde_json::Value {
let codes = self
.error_codes
.lock()
.map(|map| map.clone())
.unwrap_or_default();
serde_json::json!({
"requests_total": self.requests.load(Ordering::Relaxed),
"errors_total": self.errors.load(Ordering::Relaxed),
"completions_total": self.completions.load(Ordering::Relaxed),
"streamed_chunks_total": self.streamed_chunks.load(Ordering::Relaxed),
"captured_characters_total": self.captured_chars.load(Ordering::Relaxed),
"errors_by_code": codes,
})
}
}
/// Everything the HTTP handlers need.
#[derive(Clone)]
pub struct AppState {
pub backend: Arc<dyn ChatBackend>,
pub store: Arc<ThreadStore>,
pub layout: Arc<Layout>,
pub config: Arc<ArcSwap<GlobalConfig>>,
/// CLI flags are re-applied after every configuration reload.
pub cli_overrides: Arc<CliOverrides>,
pub logs: LogBuffer,
pub metrics: Arc<Metrics>,
pub started_at: Instant,
}
impl AppState {
/// Assemble the state from its parts (used by the CLI and by tests).
#[allow(clippy::too_many_arguments)]
pub fn new(
backend: Arc<dyn ChatBackend>,
store: Arc<ThreadStore>,
layout: Layout,
config: Arc<ArcSwap<GlobalConfig>>,
cli_overrides: CliOverrides,
logs: LogBuffer,
) -> Self {
Self {
backend,
store,
layout: Arc::new(layout),
config,
cli_overrides: Arc::new(cli_overrides),
logs,
metrics: Arc::new(Metrics::default()),
started_at: Instant::now(),
}
}
pub fn config(&self) -> Arc<GlobalConfig> {
self.config.load_full()
}
pub fn uptime_secs(&self) -> u64 {
self.started_at.elapsed().as_secs()
}
pub fn provider_exists(&self, name: &str) -> bool {
self.backend
.providers()
.iter()
.any(|provider| provider.name == name)
}
}
src/tokens.rs
+101
View File
@@ -0,0 +1,101 @@
//! Token estimation for the OpenAI-compatible usage block.
use std::sync::{Mutex, OnceLock};
use crate::config::TokensSection;
/// Encoding name and its lazily loaded BPE table.
type EncoderCache = Mutex<Option<(String, Option<tiktoken_rs::CoreBPE>)>>;
static CACHE: OnceLock<EncoderCache> = OnceLock::new();
static WARNED: OnceLock<()> = OnceLock::new();
/// Estimate the number of tokens in a piece of text.
///
/// Uses the real BPE encoding when it can be loaded and falls back to a
/// characters/4 heuristic otherwise: tiktoken fetches its vocabulary files on
/// first use, which fails on an offline machine.
pub fn count_tokens(text: &str, cfg: &TokensSection) -> usize {
if text.is_empty() {
return 0;
}
if cfg.encoding != "heuristic" {
if let Some(count) = bpe_count(text, &cfg.encoding) {
return count;
}
}
heuristic(text)
}
/// Characters/4 approximation, never below 1 for non-empty text.
pub fn heuristic(text: &str) -> usize {
let chars = text.chars().count();
if chars == 0 {
0
} else {
chars.div_ceil(4).max(1)
}
}
fn bpe_count(text: &str, encoding: &str) -> Option<usize> {
let cache = CACHE.get_or_init(|| Mutex::new(None));
let mut guard = cache.lock().ok()?;
let stale = guard
.as_ref()
.map(|(name, _)| name != encoding)
.unwrap_or(true);
if stale {
let loaded = load_bpe(encoding);
if loaded.is_none() {
WARNED.get_or_init(|| {
tracing::warn!(
encoding,
"token encoding unavailable (offline?); falling back to a characters/4 estimate"
);
});
}
*guard = Some((encoding.to_string(), loaded));
}
guard
.as_ref()
.and_then(|(_, bpe)| bpe.as_ref())
.map(|bpe| bpe.encode_with_special_tokens(text).len())
}
fn load_bpe(encoding: &str) -> Option<tiktoken_rs::CoreBPE> {
let result = match encoding {
"cl100k_base" => tiktoken_rs::cl100k_base(),
"p50k_base" => tiktoken_rs::p50k_base(),
"r50k_base" => tiktoken_rs::r50k_base(),
_ => tiktoken_rs::o200k_base(),
};
match result {
Ok(bpe) => Some(bpe),
Err(err) => {
tracing::debug!(error = %err, "could not load the BPE encoding");
None
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn heuristic_is_monotonic() {
assert_eq!(heuristic(""), 0);
assert_eq!(heuristic("a"), 1);
assert!(heuristic("a longer sentence with words") >= 6);
}
#[test]
fn counting_never_returns_zero_for_text() {
let cfg = TokensSection {
encoding: "heuristic".into(),
fallback_heuristic: true,
};
assert_eq!(count_tokens("", &cfg), 0);
assert!(count_tokens("bonjour le monde", &cfg) >= 1);
}
}
src/util.rs
+100
View File
@@ -0,0 +1,100 @@
//! Small shared helpers (time formatting, text diffing).
use std::time::{SystemTime, UNIX_EPOCH};
/// Seconds since the Unix epoch.
pub fn now_secs() -> u64 {
SystemTime::now()
.duration_since(UNIX_EPOCH)
.map(|d| d.as_secs())
.unwrap_or(0)
}
/// Convert days since 1970-01-01 into a civil date (Howard Hinnant's algorithm).
fn civil_from_days(days: i64) -> (i64, u32, u32) {
let z = days + 719_468;
let era = if z >= 0 { z } else { z - 146_096 } / 146_097;
let doe = z - era * 146_097; // [0, 146096]
let yoe = (doe - doe / 1460 + doe / 36_524 - doe / 146_096) / 365;
let y = yoe + era * 400;
let doy = doe - (365 * yoe + yoe / 4 - yoe / 100);
let mp = (5 * doy + 2) / 153;
let d = (doy - (153 * mp + 2) / 5 + 1) as u32;
let m = if mp < 10 { mp + 3 } else { mp - 9 } as u32;
(if m <= 2 { y + 1 } else { y }, m, d)
}
/// `YYYYmmdd_HHMMSS` in UTC, used in debug file names.
pub fn format_compact_utc(secs: u64) -> String {
let days = (secs / 86_400) as i64;
let rem = secs % 86_400;
let (year, month, day) = civil_from_days(days);
format!(
"{year:04}{month:02}{day:02}_{:02}{:02}{:02}",
rem / 3600,
(rem % 3600) / 60,
rem % 60
)
}
/// `YYYY-MM-DDTHH:MM:SSZ`, used in reports and logs.
pub fn format_rfc3339(secs: u64) -> String {
let days = (secs / 86_400) as i64;
let rem = secs % 86_400;
let (year, month, day) = civil_from_days(days);
format!(
"{year:04}-{month:02}-{day:02}T{:02}:{:02}:{:02}Z",
rem / 3600,
(rem % 3600) / 60,
rem % 60
)
}
/// Truncate on a character boundary, appending an ellipsis when cut.
pub fn truncate(text: &str, max_chars: usize) -> String {
if text.chars().count() <= max_chars {
return text.to_string();
}
let mut out: String = text.chars().take(max_chars).collect();
out.push('…');
out
}
/// Length in bytes of the longest common prefix of two strings.
pub fn longest_common_prefix_len(a: &str, b: &str) -> usize {
let mut len = 0;
for (x, y) in a.chars().zip(b.chars()) {
if x != y {
break;
}
len += x.len_utf8();
}
len
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn timestamps_are_human_readable() {
assert_eq!(format_compact_utc(0), "19700101_000000");
assert_eq!(format_compact_utc(1_735_689_600), "20250101_000000");
assert_eq!(format_rfc3339(1_735_689_600), "2025-01-01T00:00:00Z");
assert_eq!(format_compact_utc(1_735_693_261), "20250101_010101");
}
#[test]
fn lcp_is_on_char_boundaries() {
assert_eq!(longest_common_prefix_len("abc", "abd"), 2);
assert_eq!(longest_common_prefix_len("éé", "éa"), 2);
assert_eq!(longest_common_prefix_len("", "abc"), 0);
assert_eq!(longest_common_prefix_len("abc", "abc"), 3);
}
#[test]
fn truncate_respects_chars() {
assert_eq!(truncate("hello", 10), "hello");
assert_eq!(truncate("hello", 3), "hel…");
}
}
tests/api_integration.rs
+764
View File
@@ -0,0 +1,764 @@
//! HTTP integration tests against the mock backend: no browser, no network.
use std::sync::Arc;
use std::time::Duration;
use arc_swap::ArcSwap;
use axum::body::Body;
use axum::http::{HeaderMap, Request, StatusCode};
use axum::Router;
use http_body_util::BodyExt;
use llm_bridge::config::{CliOverrides, GlobalConfig, ScreenshotMode};
use llm_bridge::conversation::ThreadStore;
use llm_bridge::paths::Layout;
use llm_bridge::providers::{AccountInfo, ChatBackend, MockBackend};
use llm_bridge::server::{router, AppState, LogBuffer};
use serde_json::{json, Value};
use tower::ServiceExt;
struct Harness {
app: Router,
store: Arc<ThreadStore>,
state: AppState,
}
/// The two accounts of the mock ChatGPT provider.
fn mock_accounts() -> Vec<AccountInfo> {
vec![
AccountInfo {
id: "perso".to_string(),
email: Some("[email protected]".to_string()),
label: None,
default: true,
profile: "profiles/chatgpt/accounts/perso/user-data-dir".to_string(),
},
AccountInfo {
id: "pro".to_string(),
email: Some("[email protected]".to_string()),
label: Some("Work".to_string()),
default: false,
profile: "profiles/chatgpt/accounts/pro/user-data-dir".to_string(),
},
]
}
fn mock_backend(accounts: bool) -> MockBackend {
let backend = MockBackend::new(
vec![("chatgpt", "gpt-4o"), ("claude", "claude-sonnet-4")],
Duration::from_millis(1),
);
if accounts {
backend.with_accounts("chatgpt", mock_accounts())
} else {
backend
}
}
fn harness_with(config: GlobalConfig) -> Harness {
harness_with_backend(config, mock_backend(false))
}
fn harness_with_backend(config: GlobalConfig, backend: MockBackend) -> Harness {
let backend: Arc<dyn ChatBackend> = Arc::new(backend);
let store = Arc::new(ThreadStore::in_memory());
let layout = Layout::rooted_at(std::env::temp_dir().join("llm-gateway-tests"));
let state = AppState::new(
backend,
Arc::clone(&store),
layout,
Arc::new(ArcSwap::from_pointee(config)),
CliOverrides::default(),
LogBuffer::new(32),
);
Harness {
app: router(state.clone()),
store,
state,
}
}
fn harness() -> Harness {
harness_with(GlobalConfig::default())
}
/// A harness whose ChatGPT provider declares two accounts.
fn harness_with_accounts(config: GlobalConfig) -> Harness {
harness_with_backend(config, mock_backend(true))
}
async fn send(app: &Router, request: Request<Body>) -> (StatusCode, HeaderMap, String) {
let response = app
.clone()
.oneshot(request)
.await
.expect("router call failed");
let status = response.status();
let headers = response.headers().clone();
let bytes = response
.into_body()
.collect()
.await
.expect("body")
.to_bytes();
(status, headers, String::from_utf8_lossy(&bytes).to_string())
}
async fn get_json(app: &Router, path: &str) -> (StatusCode, Value) {
let request = Request::builder().uri(path).body(Body::empty()).unwrap();
let (status, _, body) = send(app, request).await;
(status, serde_json::from_str(&body).unwrap_or(Value::Null))
}
fn post_chat(body: Value) -> Request<Body> {
Request::builder()
.method("POST")
.uri("/v1/chat/completions")
.header("content-type", "application/json")
.body(Body::from(body.to_string()))
.unwrap()
}
#[tokio::test]
async fn health_and_models_describe_every_provider() {
let harness = harness();
let (status, health) = get_json(&harness.app, "/health").await;
assert_eq!(status, StatusCode::OK);
assert_eq!(health["status"], "ok");
assert_eq!(health["providers"].as_array().unwrap().len(), 2);
let (status, models) = get_json(&harness.app, "/v1/models").await;
assert_eq!(status, StatusCode::OK);
assert_eq!(models["object"], "list");
let ids: Vec<String> = models["data"]
.as_array()
.unwrap()
.iter()
.map(|entry| entry["id"].as_str().unwrap().to_string())
.collect();
assert!(ids.contains(&"chatgpt".to_string()));
assert!(ids.contains(&"chatgpt/gpt-4o".to_string()));
assert!(ids.contains(&"claude/claude-sonnet-4".to_string()));
}
#[tokio::test]
async fn a_chat_completion_returns_the_openai_shape_and_a_conversation_id() {
let harness = harness();
let (status, headers, body) = send(
&harness.app,
post_chat(json!({
"model": "chatgpt",
"messages": [{"role": "user", "content": "hello"}],
})),
)
.await;
assert_eq!(status, StatusCode::OK);
let value: Value = serde_json::from_str(&body).unwrap();
assert_eq!(value["object"], "chat.completion");
assert_eq!(value["model"], "chatgpt");
assert_eq!(value["choices"][0]["message"]["role"], "assistant");
assert_eq!(value["choices"][0]["finish_reason"], "stop");
let content = value["choices"][0]["message"]["content"].as_str().unwrap();
assert!(!content.is_empty());
assert!(value["usage"]["total_tokens"].as_u64().unwrap() >= 1);
assert!(value["id"].as_str().unwrap().starts_with("chatcmpl-"));
let conversation = headers
.get("x-conversation-id")
.unwrap()
.to_str()
.unwrap()
.to_string();
assert!(conversation.starts_with("fp-"), "{conversation}");
assert!(headers.get("x-trace-id").is_some());
// The thread mapping was persisted for the next turn.
let record = harness
.store
.get(&conversation)
.await
.expect("thread recorded");
assert_eq!(record.provider, "chatgpt");
assert!(record.web_url.is_some());
}
#[tokio::test]
async fn the_same_conversation_is_reused_when_the_header_is_repeated() {
let harness = harness();
let request = |text: &str| {
post_chat(json!({
"model": "chatgpt",
"messages": [
{"role": "user", "content": "opening"},
{"role": "assistant", "content": "answer"},
{"role": "user", "content": text},
],
}))
};
let (_, headers, _) = send(&harness.app, request("second")).await;
let conversation = headers
.get("x-conversation-id")
.unwrap()
.to_str()
.unwrap()
.to_string();
let (status, _, _) = send(&harness.app, request("third")).await;
assert_eq!(status, StatusCode::OK);
let record = harness.store.get(&conversation).await.unwrap();
assert_eq!(
record.turns, 2,
"the second call must reuse the stored thread"
);
}
#[tokio::test]
async fn streaming_emits_sse_chunks_and_terminates_with_done() {
let harness = harness();
let (status, headers, body) = send(
&harness.app,
post_chat(json!({
"model": "chatgpt",
"messages": [{"role": "user", "content": "hello"}],
"stream": true,
})),
)
.await;
assert_eq!(status, StatusCode::OK);
let content_type = headers.get("content-type").unwrap().to_str().unwrap();
assert!(
content_type.starts_with("text/event-stream"),
"{content_type}"
);
assert!(body.contains("chat.completion.chunk"), "{body}");
assert!(body.contains(r#""role":"assistant""#), "{body}");
assert!(body.contains("data: [DONE]"), "{body}");
let deltas: String = body
.lines()
.filter_map(|line| line.strip_prefix("data: "))
.filter(|payload| *payload != "[DONE]")
.filter_map(|payload| serde_json::from_str::<Value>(payload).ok())
.filter_map(|chunk| {
chunk["choices"][0]["delta"]["content"]
.as_str()
.map(str::to_string)
})
.collect();
assert!(!deltas.is_empty());
assert!(deltas.contains("mock backend"), "{deltas}");
}
#[tokio::test]
async fn the_conversation_header_wins_over_the_fingerprint() {
let harness = harness();
let request = Request::builder()
.method("POST")
.uri("/v1/chat/completions")
.header("content-type", "application/json")
.header("x-conversation-id", "client-thread-42")
.body(Body::from(
json!({"model": "chatgpt", "messages": [{"role": "user", "content": "hi"}]})
.to_string(),
))
.unwrap();
let (status, headers, _) = send(&harness.app, request).await;
assert_eq!(status, StatusCode::OK);
assert_eq!(
headers.get("x-conversation-id").unwrap(),
"client-thread-42"
);
assert!(harness.store.get("client-thread-42").await.is_some());
}
#[tokio::test]
async fn unsupported_parameters_are_rejected_with_a_clear_error() {
let harness = harness();
let (status, _, body) = send(
&harness.app,
post_chat(json!({
"model": "chatgpt",
"messages": [{"role": "user", "content": "hi"}],
"tools": [{"type": "function", "function": {"name": "f"}}],
})),
)
.await;
assert_eq!(status, StatusCode::BAD_REQUEST);
let value: Value = serde_json::from_str(&body).unwrap();
assert_eq!(value["error"]["code"], "unsupported_parameter");
assert_eq!(value["error"]["param"], "tools");
assert_eq!(value["error"]["type"], "invalid_request_error");
// n = 0 is a client mistake, unlike n > 1 which is now supported.
let (status, _, body) = send(
&harness.app,
post_chat(json!({
"model": "chatgpt",
"messages": [{"role": "user", "content": "hi"}],
"n": 0,
})),
)
.await;
assert_eq!(status, StatusCode::BAD_REQUEST);
let value: Value = serde_json::from_str(&body).unwrap();
assert_eq!(value["error"]["param"], "n");
}
#[tokio::test]
async fn several_completions_come_back_as_several_choices() {
let harness = harness();
let (status, _, body) = send(
&harness.app,
post_chat(json!({
"model": "chatgpt",
"messages": [{"role": "user", "content": "hello"}],
"n": 3,
})),
)
.await;
assert_eq!(status, StatusCode::OK);
let value: Value = serde_json::from_str(&body).unwrap();
let choices = value["choices"].as_array().unwrap();
assert_eq!(choices.len(), 3);
for (index, choice) in choices.iter().enumerate() {
assert_eq!(choice["index"], index as u64);
assert_eq!(choice["message"]["role"], "assistant");
assert_eq!(choice["finish_reason"], "stop");
assert!(!choice["message"]["content"].as_str().unwrap().is_empty());
}
assert!(value["usage"]["total_tokens"].as_u64().unwrap() >= 1);
let warnings = value["x_gateway_warnings"].as_array().unwrap();
assert!(
warnings.iter().any(|w| w
.as_str()
.unwrap()
.contains("independent web conversations")),
"{warnings:?}"
);
}
#[tokio::test]
async fn n_is_clamped_to_the_configured_maximum() {
let mut config = GlobalConfig::default();
config.capture.max_variants = 2;
let harness = harness_with(config);
let (status, _, body) = send(
&harness.app,
post_chat(json!({
"model": "chatgpt",
"messages": [{"role": "user", "content": "hello"}],
"n": 9,
})),
)
.await;
assert_eq!(status, StatusCode::OK);
let value: Value = serde_json::from_str(&body).unwrap();
assert_eq!(value["choices"].as_array().unwrap().len(), 2);
let warnings = value["x_gateway_warnings"].as_array().unwrap();
assert!(
warnings
.iter()
.any(|w| w.as_str().unwrap().contains("clamped to 2")),
"{warnings:?}"
);
}
#[tokio::test]
async fn streamed_choices_are_tagged_with_their_index() {
let harness = harness();
let (status, _, body) = send(
&harness.app,
post_chat(json!({
"model": "chatgpt",
"messages": [{"role": "user", "content": "hello"}],
"stream": true,
"n": 2,
})),
)
.await;
assert_eq!(status, StatusCode::OK);
let chunks: Vec<Value> = body
.lines()
.filter_map(|line| line.strip_prefix("data: "))
.filter(|payload| *payload != "[DONE]")
.filter_map(|payload| serde_json::from_str::<Value>(payload).ok())
.collect();
assert!(body.contains("data: [DONE]"), "{body}");
let indexes: Vec<u64> = chunks
.iter()
.flat_map(|chunk| chunk["choices"].as_array().cloned().unwrap_or_default())
.filter_map(|choice| choice["index"].as_u64())
.collect();
assert!(indexes.contains(&0), "{indexes:?}");
assert!(indexes.contains(&1), "{indexes:?}");
let stops = chunks
.iter()
.flat_map(|chunk| chunk["choices"].as_array().cloned().unwrap_or_default())
.filter(|choice| choice["finish_reason"] == "stop")
.count();
assert_eq!(stops, 2, "each choice must finish: {body}");
}
#[tokio::test]
async fn an_account_can_be_selected_in_the_model_string() {
let harness = harness_with_accounts(GlobalConfig::default());
// The declared accounts show up in /v1/models and /health.
let (_, models) = get_json(&harness.app, "/v1/models").await;
let ids: Vec<String> = models["data"]
.as_array()
.unwrap()
.iter()
.map(|entry| entry["id"].as_str().unwrap().to_string())
.collect();
assert!(ids.contains(&"chatgpt@perso".to_string()), "{ids:?}");
assert!(ids.contains(&"chatgpt@pro".to_string()), "{ids:?}");
let (_, health) = get_json(&harness.app, "/health").await;
let chatgpt = health["providers"]
.as_array()
.unwrap()
.iter()
.find(|provider| provider["name"] == "chatgpt")
.unwrap();
assert_eq!(chatgpt["accounts"].as_array().unwrap().len(), 2);
// By account id, and by e-mail address.
for model in ["chatgpt@pro", "[email protected]@gmail.com"] {
let (status, _, body) = send(
&harness.app,
post_chat(json!({
"model": model,
"messages": [{"role": "user", "content": "hello"}],
})),
)
.await;
assert_eq!(status, StatusCode::OK, "{body}");
let value: Value = serde_json::from_str(&body).unwrap();
assert_eq!(value["model"], model);
}
}
#[tokio::test]
async fn an_unknown_account_is_a_404_listing_the_known_ones() {
let harness = harness_with_accounts(GlobalConfig::default());
let (status, _, body) = send(
&harness.app,
post_chat(json!({
"model": "chatgpt@nobody",
"messages": [{"role": "user", "content": "hello"}],
})),
)
.await;
assert_eq!(status, StatusCode::NOT_FOUND);
let value: Value = serde_json::from_str(&body).unwrap();
assert_eq!(value["error"]["code"], "account_not_found");
let message = value["error"]["message"].as_str().unwrap();
assert!(message.contains("no account 'nobody'"), "{message}");
assert!(message.contains("perso"), "{message}");
assert!(
!message.contains("does not match any configured provider"),
"the message must name the account problem: {message}"
);
}
#[tokio::test]
async fn two_accounts_never_share_a_thread() {
let harness = harness_with_accounts(GlobalConfig::default());
let request = |model: &str| {
post_chat(json!({
"model": model,
"messages": [{"role": "user", "content": "hello"}],
}))
};
let (_, first, _) = send(&harness.app, request("chatgpt@perso")).await;
let (_, second, _) = send(&harness.app, request("chatgpt@pro")).await;
let first = first
.get("x-conversation-id")
.unwrap()
.to_str()
.unwrap()
.to_string();
let second = second
.get("x-conversation-id")
.unwrap()
.to_str()
.unwrap()
.to_string();
assert_ne!(first, second, "the account is part of the thread identity");
}
#[tokio::test]
async fn an_unknown_model_is_a_404() {
let harness = harness();
let (status, _, body) = send(
&harness.app,
post_chat(json!({
"model": "gemini",
"messages": [{"role": "user", "content": "hi"}],
})),
)
.await;
assert_eq!(status, StatusCode::NOT_FOUND);
let value: Value = serde_json::from_str(&body).unwrap();
assert_eq!(value["error"]["code"], "model_not_found");
assert!(value["error"]["message"]
.as_str()
.unwrap()
.contains("chatgpt"));
}
#[tokio::test]
async fn ignored_parameters_are_reported_in_the_response() {
let harness = harness();
let (status, _, body) = send(
&harness.app,
post_chat(json!({
"model": "chatgpt",
"messages": [{"role": "user", "content": "hi"}],
"temperature": 0.1,
"max_tokens": 10,
})),
)
.await;
assert_eq!(status, StatusCode::OK);
let value: Value = serde_json::from_str(&body).unwrap();
let warnings = value["x_gateway_warnings"].as_array().unwrap();
assert!(warnings
.iter()
.any(|w| w.as_str().unwrap().contains("temperature")));
assert!(warnings
.iter()
.any(|w| w.as_str().unwrap().contains("max_tokens")));
}
#[tokio::test]
async fn warnings_can_be_switched_off() {
let mut config = GlobalConfig::default();
config.server.include_warnings = false;
let harness = harness_with(config);
let (status, _, body) = send(
&harness.app,
post_chat(json!({
"model": "chatgpt",
"messages": [{"role": "user", "content": "hi"}],
"top_p": 0.5,
})),
)
.await;
assert_eq!(status, StatusCode::OK);
let value: Value = serde_json::from_str(&body).unwrap();
assert!(value.get("x_gateway_warnings").is_none());
}
#[tokio::test]
async fn the_legacy_completions_endpoint_is_supported() {
let harness = harness();
let request = Request::builder()
.method("POST")
.uri("/v1/completions")
.header("content-type", "application/json")
.body(Body::from(
json!({"model": "chatgpt", "prompt": "say ok"}).to_string(),
))
.unwrap();
let (status, headers, body) = send(&harness.app, request).await;
assert_eq!(status, StatusCode::OK);
let value: Value = serde_json::from_str(&body).unwrap();
assert_eq!(value["object"], "text_completion");
assert!(!value["choices"][0]["text"].as_str().unwrap().is_empty());
assert!(headers.get("x-conversation-id").is_some());
let request = Request::builder()
.method("POST")
.uri("/v1/completions")
.header("content-type", "application/json")
.body(Body::from(
json!({"model": "chatgpt", "prompt": "say ok", "stream": true}).to_string(),
))
.unwrap();
let (status, _, body) = send(&harness.app, request).await;
assert_eq!(status, StatusCode::BAD_REQUEST);
assert!(body.contains("unsupported_parameter"));
}
#[tokio::test]
async fn provider_status_and_validation_are_exposed() {
let harness = harness();
let (status, body) = get_json(&harness.app, "/v1/providers/chatgpt/status").await;
assert_eq!(status, StatusCode::OK);
assert_eq!(body["state"], "available");
assert_eq!(body["provider"], "chatgpt");
let request = Request::builder()
.method("POST")
.uri("/v1/providers/chatgpt/validate")
.body(Body::empty())
.unwrap();
let (status, _, body) = send(&harness.app, request).await;
assert_eq!(status, StatusCode::OK);
let value: Value = serde_json::from_str(&body).unwrap();
assert_eq!(value["ok"], true);
assert_eq!(value["status"], "ok");
assert!(!value["steps"].as_array().unwrap().is_empty());
let (status, body) = get_json(&harness.app, "/v1/providers/nope/status").await;
assert_eq!(status, StatusCode::NOT_FOUND);
assert_eq!(body["error"]["code"], "model_not_found");
}
#[tokio::test]
async fn provider_status_and_validation_can_name_an_account() {
let harness = harness_with_accounts(GlobalConfig::default());
// A named account is probed, and the answer says which one.
let (status, body) = get_json(&harness.app, "/v1/providers/chatgpt@pro/status").await;
assert_eq!(status, StatusCode::OK);
assert_eq!(body["state"], "available");
assert_eq!(body["provider"], "chatgpt@pro");
// The bare provider name means "its default account", and says so rather
// than leaving the caller to guess which profile answered.
let (status, body) = get_json(&harness.app, "/v1/providers/chatgpt/status").await;
assert_eq!(status, StatusCode::OK);
assert_eq!(body["provider"], "chatgpt@perso");
let (status, body) = post_account_validate(&harness, "chatgpt@pro").await;
assert_eq!(status, StatusCode::OK);
assert_eq!(body["ok"], true);
assert_eq!(body["provider"], "chatgpt@pro");
// The status route keeps working for a provider that declares no account.
let (status, body) = get_json(&harness.app, "/v1/providers/claude/status").await;
assert_eq!(status, StatusCode::OK);
assert_eq!(body["provider"], "claude");
}
#[tokio::test]
async fn an_unknown_account_is_a_404_on_the_status_routes() {
let harness = harness_with_accounts(GlobalConfig::default());
let (status, body) = get_json(&harness.app, "/v1/providers/chatgpt@nobody/status").await;
assert_eq!(status, StatusCode::NOT_FOUND);
assert_eq!(body["error"]["code"], "account_not_found");
assert!(body["error"]["message"].as_str().unwrap().contains("perso"));
let (status, body) = post_account_validate(&harness, "chatgpt@nobody").await;
assert_eq!(status, StatusCode::NOT_FOUND);
assert_eq!(body["error"]["code"], "account_not_found");
// A provider that declares no account refuses a named one instead of
// silently probing its only profile.
let (status, body) = get_json(&harness.app, "/v1/providers/claude@perso/status").await;
assert_eq!(status, StatusCode::NOT_FOUND);
assert_eq!(body["error"]["code"], "account_not_found");
}
/// POST /v1/providers/{owner}/validate and parse the JSON body.
async fn post_account_validate(harness: &Harness, owner: &str) -> (StatusCode, Value) {
let request = Request::builder()
.method("POST")
.uri(format!("/v1/providers/{owner}/validate"))
.body(Body::empty())
.unwrap();
let (status, _, body) = send(&harness.app, request).await;
(status, serde_json::from_str(&body).unwrap_or(Value::Null))
}
#[tokio::test]
async fn reload_reports_the_configuration_diff() {
let harness = harness();
let request = Request::builder()
.method("POST")
.uri("/v1/admin/reload")
.body(Body::empty())
.unwrap();
let (status, _, body) = send(&harness.app, request).await;
assert_eq!(status, StatusCode::OK);
let value: Value = serde_json::from_str(&body).unwrap();
assert_eq!(value["config"], "reloaded");
}
#[tokio::test]
async fn an_api_key_protects_the_v1_routes_only() {
let mut config = GlobalConfig::default();
config.server.api_key = "s3cret".to_string();
let harness = harness_with(config);
let (status, _, body) = send(
&harness.app,
post_chat(json!({
"model": "chatgpt",
"messages": [{"role": "user", "content": "hi"}],
})),
)
.await;
assert_eq!(status, StatusCode::UNAUTHORIZED);
let value: Value = serde_json::from_str(&body).unwrap();
assert_eq!(value["error"]["code"], "invalid_api_key");
let request = Request::builder()
.method("POST")
.uri("/v1/chat/completions")
.header("content-type", "application/json")
.header("authorization", "Bearer s3cret")
.body(Body::from(
json!({"model": "chatgpt", "messages": [{"role": "user", "content": "hi"}]})
.to_string(),
))
.unwrap();
let (status, _, _) = send(&harness.app, request).await;
assert_eq!(status, StatusCode::OK);
// /health stays open so that monitors and the dashboard can reach it.
let (status, _) = get_json(&harness.app, "/health").await;
assert_eq!(status, StatusCode::OK);
}
#[tokio::test]
async fn a_malformed_body_is_a_400_with_the_openai_error_shape() {
let harness = harness();
let request = Request::builder()
.method("POST")
.uri("/v1/chat/completions")
.header("content-type", "application/json")
.body(Body::from("{not json"))
.unwrap();
let (status, _, body) = send(&harness.app, request).await;
assert_eq!(status, StatusCode::BAD_REQUEST);
let value: Value = serde_json::from_str(&body).unwrap();
assert_eq!(value["error"]["code"], "invalid_request");
}
#[tokio::test]
async fn the_dashboard_is_served_and_exposes_metrics() {
let harness = harness();
let request = Request::builder()
.uri("/dashboard")
.body(Body::empty())
.unwrap();
let (status, _, body) = send(&harness.app, request).await;
assert_eq!(status, StatusCode::OK);
assert!(body.contains("llm-gateway"));
let (status, metrics) = get_json(&harness.app, "/metrics").await;
assert_eq!(status, StatusCode::OK);
assert!(metrics["requests_total"].is_number());
// The screenshot policy is only reachable through the configuration.
let config = harness.state.config();
assert_eq!(config.debug.screenshots, ScreenshotMode::OnError);
}
tests/e2e_browser.rs
+898
View File
@@ -0,0 +1,898 @@
//! Browser end-to-end tests against the local ChatGPT fixture page.
//!
//! They need a real Chromium-based browser and are therefore ignored by default:
//!
//! cargo test --test e2e_browser -- --ignored --test-threads=1
//!
//! The fixture is a static HTML file that imitates the real page (editor, send
//! button, streaming indicator, login form, captcha, error banner), so the whole
//! capture pipeline can be exercised without a network or a login.
use std::path::PathBuf;
use std::sync::Arc;
use std::time::Duration;
use llm_bridge::browser::{external_launch, resolve_executable, Session};
use llm_bridge::config::{GlobalConfig, ScreenshotMode};
use llm_bridge::error::GatewayError;
use llm_bridge::paths::Layout;
use llm_bridge::providers::config::ProviderConfig;
use llm_bridge::providers::{
Attachment, AttachmentKind, LlmProvider, ProviderRegistry, TurnOutcome, TurnRequest,
WebUiProvider,
};
use llm_bridge::selftest;
use tokio::time::timeout;
/// Install a tracing subscriber once so the tests are diagnosable.
fn init_logs() {
static ONCE: std::sync::Once = std::sync::Once::new();
ONCE.call_once(|| {
let filter = std::env::var("RUST_LOG").unwrap_or_else(|_| "info".to_string());
let _ = tracing_subscriber::fmt()
.with_env_filter(filter)
.with_test_writer()
.try_init();
});
}
/// Absolute file:// URL of the fixture, optionally with a query string.
fn fixture_url(query: &str) -> String {
let path = PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("tests")
.join("fixtures")
.join("chatgpt_mock.html");
let path = path.to_string_lossy().replace('\\', "/");
let url = format!("file:///{path}");
if query.is_empty() {
url
} else {
format!("{url}?{query}")
}
}
/// Provider configuration pointing at the fixture.
fn provider_toml(base: &str) -> String {
format!(
r##"
[provider]
name = "fixture"
display_name = "ChatGPT fixture"
web_url = "{base}"
new_conversation_url = "{base}"
conversation_url_pattern = "#c=([0-9a-zA-Z-]{{8,}})"
default_models = ["fixture-1"]
enabled = true
[selectors]
input_field = "div#prompt-textarea"
send_button = "button[data-testid='send-button']"
response_container = "div[data-message-author-role='assistant']"
streaming_indicator = "button[data-testid='stop-button']"
file_input = "input[type='file']"
new_chat_button = "a[data-testid='create-new-chat-button']"
login_page_indicator = "form[action*='login']"
captcha_indicator = "div#challenge-form"
error_banner = "[data-testid='error-message'], div[role='alert']"
attachment_indicator = "div[data-testid='attachment-chip']"
copy_button = "button[data-testid='copy-button']"
[login]
url_patterns = ["login=1"]
[rate_limit]
text_patterns = ["too many requests"]
[timeouts]
page_load = 20
response_generation = 15
response_timeout = 30
[input]
poll_interval_ms = 100
quiet_ms = 500
[options]
supports_images = true
supports_streaming = true
supports_new_chat = true
"##
)
}
struct Harness {
registry: Arc<ProviderRegistry>,
provider: Arc<WebUiProvider>,
_tmp: tempfile::TempDir,
}
/// Boot a registry with a single fixture provider, or None when no browser exists.
async fn harness(toml: &str) -> Option<Harness> {
harness_with(toml, |_| {}).await
}
/// Same, with a chance to tweak the global configuration.
async fn harness_with(toml: &str, configure: impl FnOnce(&mut GlobalConfig)) -> Option<Harness> {
init_logs();
let chrome = resolve_executable(None, std::env::var("CHROME").ok())?;
let tmp = tempfile::tempdir().expect("temp dir");
let layout = Layout::rooted_at(tmp.path().join("state"));
layout.ensure().expect("layout");
// Only the fixture provider may be registered: the state directory is
// materialised with the providers shipped in the build.
if let Ok(entries) = std::fs::read_dir(layout.providers_dir()) {
for entry in entries.flatten() {
std::fs::remove_file(entry.path()).ok();
}
}
std::fs::write(layout.provider_config("fixture"), toml).expect("provider config");
let mut config = GlobalConfig::default();
config.browser.headless = true;
config.browser.executable = chrome.to_string_lossy().to_string();
config.capture.poll_interval_ms = 100;
config.capture.quiet_ms = 500;
config.capture.response_timeout_s = 30;
config.debug.screenshots = ScreenshotMode::OnError;
config.debug.dir = tmp.path().join("debug").to_string_lossy().to_string();
configure(&mut config);
let registry = Arc::new(ProviderRegistry::new(layout, config));
registry.reload().await.expect("registry reload");
let provider = registry
.get("fixture")
.expect("fixture provider registered");
Some(Harness {
registry,
provider,
_tmp: tmp,
})
}
/// Run one turn and return the deltas plus the final result.
async fn run_turn(
provider: &Arc<WebUiProvider>,
session: &Session,
prompt: &str,
reuse_url: Option<String>,
attachments: Vec<Attachment>,
) -> (Vec<String>, Result<TurnOutcome, GatewayError>) {
let request = TurnRequest {
prompt: prompt.to_string(),
attachments,
replay_history: reuse_url.is_none(),
reuse_url,
probe: false,
trace_id: "e2e".to_string(),
wait_for_tab: None,
};
let mut stream = match provider.run_turn(session.clone(), request).await {
Ok(stream) => stream,
Err(err) => return (Vec::new(), Err(err)),
};
let mut deltas = Vec::new();
loop {
match stream.recv().await {
Some(llm_bridge::providers::TurnEvent::Delta(delta)) => deltas.push(delta),
Some(llm_bridge::providers::TurnEvent::Completed(outcome)) => {
return (deltas, Ok(outcome))
}
Some(llm_bridge::providers::TurnEvent::Failed(err)) => return (deltas, Err(err)),
None => {
return (
deltas,
Err(GatewayError::internal(
"the turn ended without a terminal event",
)),
)
}
}
}
}
/// Assert that the error carries the expected machine readable code.
fn expect_code(result: Result<TurnOutcome, GatewayError>, code: &str) {
match result {
Ok(outcome) => panic!("expected {code}, got an answer: {}", outcome.text),
Err(err) => assert_eq!(err.code(), code, "unexpected error: {err}"),
}
}
#[tokio::test]
#[ignore = "requires a real Chromium-based browser"]
async fn fixture_full_pipeline() {
let Some(harness) = harness(&provider_toml(&fixture_url("conv=1"))).await else {
eprintln!("no browser found, skipping");
return;
};
assert_eq!(
harness.registry.all().len(),
1,
"only the fixture provider is configured"
);
let session = harness.provider.session(None).await.expect("session");
let (deltas, result) = timeout(
Duration::from_secs(90),
run_turn(
&harness.provider,
&session,
"hello fixture",
None,
Vec::new(),
),
)
.await
.expect("the first turn timed out");
let outcome = result.expect("the first turn failed");
assert!(
outcome.text.starts_with("Echo: hello fixture"),
"{}",
outcome.text
);
assert!(outcome.text.contains("fixture page"), "{}", outcome.text);
assert!(
deltas.len() >= 2,
"expected several streamed deltas, got {deltas:?}"
);
assert_eq!(deltas.concat().trim(), outcome.text.trim());
let web_id = outcome
.web_id
.clone()
.expect("the conversation id must be extracted");
assert!(
web_id.len() >= 8,
"suspicious conversation id {web_id:?} from {:?}",
outcome.web_url
);
// Continuing the thread reopens the same web conversation.
let reuse = outcome.web_url.clone().expect("web url");
let (_, second) = timeout(
Duration::from_secs(90),
run_turn(
&harness.provider,
&session,
"second question",
Some(reuse),
Vec::new(),
),
)
.await
.expect("the second turn timed out");
let second = second.expect("the second turn failed");
assert!(
second.text.starts_with("Echo: second question"),
"{}",
second.text
);
assert_eq!(
second.web_id, outcome.web_id,
"the web conversation changed"
);
// The provider passes its own validation. The validation opens its own
// tab, so the one this test holds is released first.
drop(session);
let cfg = harness.provider.cfg();
let report = selftest::validate_provider(harness.provider.as_ref(), &cfg, None)
.await
.expect("validation");
assert!(report.ok, "selftest failed: {report:?}");
harness.provider.shutdown().await.expect("shutdown");
}
#[tokio::test]
#[ignore = "requires a real Chromium-based browser"]
async fn fixture_selftest_works_on_a_brand_new_session() {
let Some(harness) = harness(&provider_toml(&fixture_url(""))).await else {
eprintln!("no browser found, skipping");
return;
};
let session = harness.provider.session(None).await.expect("session");
// This is what the CLI does: a session that never navigated anywhere yet.
let start_page = session
.page()
.url()
.await
.unwrap_or_default()
.unwrap_or_default();
assert!(
start_page.is_empty() || start_page == "about:blank",
"a new session should start on about:blank, got {start_page:?}"
);
// The selftest acquires its own tab: release this one first, otherwise a
// single-tab provider (max_tabs = 1) answers "busy" instead.
drop(session);
let cfg = harness.provider.cfg();
let report = selftest::validate_provider(harness.provider.as_ref(), &cfg, None)
.await
.expect("validation on a fresh session");
assert!(report.ok, "selftest on a fresh session failed: {report:?}");
assert_eq!(report.status, "ok");
// The check must be quick: it opens the page instead of waiting for a
// selector to appear on about:blank.
assert!(
report.duration_ms < 20_000,
"selftest took {} ms",
report.duration_ms
);
let probe_step = report
.steps
.iter()
.find(|step| step.step == "page_probe")
.expect("page_probe step");
assert!(probe_step.ok, "page_probe failed: {probe_step:?}");
harness.provider.shutdown().await.expect("shutdown");
}
#[tokio::test]
#[ignore = "requires a real Chromium-based browser"]
async fn fixture_reports_login_captcha_error_and_rate_limit() {
let Some(harness) = harness(&provider_toml(&fixture_url(""))).await else {
eprintln!("no browser found, skipping");
return;
};
let session = harness.provider.session(None).await.expect("session");
let cases = [
("login=1", "requires_login"),
("captcha=1", "captcha_required"),
("error=1", "upstream_error"),
("ratelimit=1", "upstream_rate_limit"),
];
for (query, code) in cases {
let url = fixture_url(query);
let (_, result) = timeout(
Duration::from_secs(60),
run_turn(&harness.provider, &session, "hello", Some(url), Vec::new()),
)
.await
.unwrap_or_else(|_| panic!("the {query} case timed out"));
expect_code(result, code);
}
harness.provider.shutdown().await.expect("shutdown");
}
#[tokio::test]
#[ignore = "requires a real Chromium-based browser"]
async fn fixture_timeouts_and_missing_selectors() {
// A page that never produces text trips the first-token budget.
let impatient = provider_toml(&fixture_url(""))
.replace("response_generation = 15", "response_generation = 3");
let Some(harness) = harness(&impatient).await else {
eprintln!("no browser found, skipping");
return;
};
let session = harness.provider.session(None).await.expect("session");
let (_, result) = timeout(
Duration::from_secs(60),
run_turn(
&harness.provider,
&session,
"hello",
Some(fixture_url("empty=1")),
Vec::new(),
),
)
.await
.expect("the empty case timed out");
expect_code(result, "upstream_timeout");
// A page whose editor appears late is waited for, not failed.
let (_, result) = timeout(
Duration::from_secs(90),
run_turn(
&harness.provider,
&session,
"late editor",
Some(fixture_url("loading=1")),
Vec::new(),
),
)
.await
.expect("the loading case timed out");
let outcome = result.expect("the late editor should still be driven");
assert!(
outcome.text.starts_with("Echo: late editor"),
"{}",
outcome.text
);
// A stale selector is reported as such.
let broken = provider_toml(&fixture_url(""))
.replace(
"input_field = \"div#prompt-textarea\"",
"input_field = \"div#this-does-not-exist\"",
)
.replace("page_load = 20", "page_load = 4");
harness
.provider
.replace_config(ProviderConfig::parse(&broken, "fixture.toml").expect("broken config"))
.await;
let (_, result) = timeout(
Duration::from_secs(60),
run_turn(&harness.provider, &session, "hello", None, Vec::new()),
)
.await
.expect("the broken selector case timed out");
expect_code(result, "selector_missing");
harness.provider.shutdown().await.expect("shutdown");
}
#[tokio::test]
#[ignore = "requires a real Chromium-based browser"]
async fn two_tabs_of_one_browser_answer_concurrently() {
let toml = provider_toml(&fixture_url("conv=1"));
let Some(harness) = harness_with(&toml, |config| {
config.browser.max_tabs = 2;
})
.await
else {
eprintln!("no browser found, skipping");
return;
};
let first = harness.provider.session(None).await.expect("first tab");
let second = harness.provider.session(None).await.expect("second tab");
let url = fixture_url("conv=1");
let outcome = timeout(Duration::from_secs(30), async {
// Navigate both tabs at once...
let (a, b) = tokio::join!(
llm_bridge::browser::dom::goto(first.page(), &url, Duration::from_secs(10)),
llm_bridge::browser::dom::goto(second.page(), &url, Duration::from_secs(10)),
);
a.expect("the first tab did not navigate");
b.expect("the second tab did not navigate");
// ... then hammer both, the way the capture loop does.
for _ in 0..20 {
let (a, b) = tokio::join!(
first.page().evaluate_expression("document.title"),
second.page().evaluate_expression("document.title"),
);
a.expect("the first tab stopped answering");
b.expect("the second tab stopped answering");
}
})
.await;
assert!(
outcome.is_ok(),
"two tabs of one browser stopped answering: {outcome:?}"
);
harness.provider.shutdown().await.expect("shutdown");
}
#[tokio::test]
#[ignore = "requires a real Chromium-based browser"]
async fn two_tabs_serve_two_turns_in_parallel() {
let toml = provider_toml(&fixture_url("conv=1&delay=5000"));
let Some(harness) = harness_with(&toml, |config| {
config.browser.max_tabs = 2;
})
.await
else {
eprintln!("no browser found, skipping");
return;
};
// Two tabs of the same account can be leased at the same time.
let first = harness.provider.session(None).await.expect("first tab");
let second = harness.provider.session(None).await.expect("second tab");
assert_ne!(first.tab_slot(), second.tab_slot());
let stats = harness.provider.tab_stats();
assert_eq!(stats[0].1.max_tabs, 2);
assert_eq!(stats[0].1.busy, 2);
let started = std::time::Instant::now();
let (a, b) = tokio::join!(
run_turn(&harness.provider, &first, "first", None, Vec::new()),
run_turn(&harness.provider, &second, "second", None, Vec::new()),
);
let elapsed = started.elapsed();
let a = a.1.expect("the first turn failed");
let b = b.1.expect("the second turn failed");
assert!(a.text.contains("first"), "{}", a.text);
assert!(b.text.contains("second"), "{}", b.text);
// Each generation takes five seconds: serving them one after the other
// takes more than ten, two tabs take about six.
assert!(
elapsed < Duration::from_secs(9),
"two tabs did not run in parallel: {elapsed:?}"
);
harness.provider.shutdown().await.expect("shutdown");
}
#[tokio::test]
#[ignore = "requires a real Chromium-based browser"]
async fn a_busy_single_tab_account_refuses_a_second_request() {
let toml = provider_toml(&fixture_url("conv=1"));
let Some(harness) = harness_with(&toml, |config| {
config.browser.max_tabs = 1;
config.browser.busy_wait_s = 1;
})
.await
else {
eprintln!("no browser found, skipping");
return;
};
let held = harness.provider.session(None).await.expect("the only tab");
let started = std::time::Instant::now();
let err = harness
.provider
.session(None)
.await
.expect_err("the second request must not get a tab");
assert_eq!(err.code(), "provider_busy");
assert_eq!(err.http_status(), 429);
assert!(
started.elapsed() >= Duration::from_secs(1),
"the request did not wait for the configured budget"
);
drop(held);
assert!(harness.provider.session(None).await.is_ok());
harness.provider.shutdown().await.expect("shutdown");
}
#[tokio::test]
#[ignore = "requires a real Chromium-based browser"]
async fn each_account_has_its_own_browser_and_tab_pool() {
let mut toml = provider_toml(&fixture_url("conv=1&delay=3000"));
toml.push_str("\n[[accounts]]\nid = \"un\"\ndefault = true\n\n[[accounts]]\nid = \"deux\"\n");
let Some(harness) = harness(&toml).await else {
eprintln!("no browser found, skipping");
return;
};
let accounts = harness.provider.accounts();
assert_eq!(accounts.len(), 2);
assert_ne!(accounts[0].profile, accounts[1].profile);
// max_tabs is 1 for both accounts, yet one tab per account is available at
// the same time: the pools are per account, not per provider.
let one = harness
.provider
.session(Some("un"))
.await
.expect("account un");
let two = harness
.provider
.session(Some("deux"))
.await
.expect("account deux");
assert_eq!(one.account(), Some("un"));
assert_eq!(two.account(), Some("deux"));
let started = std::time::Instant::now();
let (a, b) = tokio::join!(
run_turn(&harness.provider, &one, "un", None, Vec::new()),
run_turn(&harness.provider, &two, "deux", None, Vec::new()),
);
let elapsed = started.elapsed();
assert!(a.1.is_ok(), "account un failed: {:?}", a.1.err());
assert!(b.1.is_ok(), "account deux failed: {:?}", b.1.err());
assert!(
elapsed < Duration::from_secs(6),
"the two accounts did not answer at the same time: {elapsed:?}"
);
harness.provider.shutdown().await.expect("shutdown");
}
#[tokio::test]
#[ignore = "requires a real Chromium-based browser"]
async fn fixture_returns_markdown_and_never_the_toolbar_text() {
let toml = provider_toml(&fixture_url("conv=1&rich=1"));
let Some(harness) = harness(&toml).await else {
eprintln!("no browser found, skipping");
return;
};
let session = harness.provider.session(None).await.expect("session");
// Auto mode: the provider copy button first, the HTML conversion otherwise.
let (_, result) = timeout(
Duration::from_secs(90),
run_turn(&harness.provider, &session, "rich answer", None, Vec::new()),
)
.await
.expect("the rich turn timed out");
let outcome = result.expect("the rich turn failed");
assert!(
outcome.fidelity == "clipboard" || outcome.fidelity == "dom",
"unexpected fidelity {:?}: {}",
outcome.fidelity,
outcome.text
);
eprintln!("rich answer captured through the {} path", outcome.fidelity);
// Structure the plain text capture used to lose.
assert!(outcome.text.contains("## Answer"), "{}", outcome.text);
assert!(outcome.text.contains("- first point"), "{}", outcome.text);
assert!(outcome.text.contains("```rust"), "{}", outcome.text);
assert!(
outcome.text.contains("| Name | Value |"),
"{}",
outcome.text
);
assert!(outcome.text.contains("| alpha | 1 |"), "{}", outcome.text);
// The toolbar sits next to the answer, so its label is not answer text.
assert!(!outcome.text.contains("Copy"), "{}", outcome.text);
// Forcing the DOM conversion must give the same structure.
let forced = provider_toml(&fixture_url("conv=1&rich=1")).replace(
"poll_interval_ms = 100",
"poll_interval_ms = 100\nmarkdown = \"dom\"",
);
harness
.provider
.replace_config(ProviderConfig::parse(&forced, "fixture.toml").expect("forced config"))
.await;
let (_, result) = timeout(
Duration::from_secs(90),
run_turn(&harness.provider, &session, "dom answer", None, Vec::new()),
)
.await
.expect("the forced DOM turn timed out");
let forced_outcome = result.expect("the forced DOM turn failed");
assert_eq!(forced_outcome.fidelity, "dom");
assert!(
forced_outcome.text.contains("## Answer"),
"{}",
forced_outcome.text
);
assert!(
forced_outcome.text.contains("```rust"),
"{}",
forced_outcome.text
);
assert!(
forced_outcome.text.contains("| alpha | 1 |"),
"{}",
forced_outcome.text
);
assert!(
!forced_outcome.text.contains("Copy"),
"{}",
forced_outcome.text
);
harness.provider.shutdown().await.expect("shutdown");
}
#[tokio::test]
#[ignore = "requires a real Chromium-based browser"]
async fn fixture_handles_rewrites_and_attachments() {
let Some(harness) = harness(&provider_toml(&fixture_url(""))).await else {
eprintln!("no browser found, skipping");
return;
};
let session = harness.provider.session(None).await.expect("session");
// A mid-stream rewrite must not corrupt the final answer.
let (_, result) = timeout(
Duration::from_secs(90),
run_turn(
&harness.provider,
&session,
"rewrite please",
Some(fixture_url("rewrite=1")),
Vec::new(),
),
)
.await
.expect("the rewrite case timed out");
let outcome = result.expect("the rewrite case failed");
assert!(
outcome.text.starts_with("Echo: rewrite please"),
"{}",
outcome.text
);
assert!(!outcome.text.contains("Draft answer"), "{}", outcome.text);
// Attachments are pushed into the file input and acknowledged by the page.
let dir = std::env::temp_dir().join("llm-gateway-e2e");
std::fs::create_dir_all(&dir).expect("temp dir");
let image = dir.join("pixel.png");
std::fs::write(&image, b"\x89PNG\r\n\x1a\n").expect("write png");
let (_, result) = timeout(
Duration::from_secs(90),
run_turn(
&harness.provider,
&session,
"with an image",
None,
vec![Attachment {
path: image,
mime: "image/png".to_string(),
kind: AttachmentKind::Image,
}],
),
)
.await
.expect("the attachment case timed out");
let outcome = result.expect("the attachment case failed");
assert!(outcome.text.contains("pixel.png"), "{}", outcome.text);
harness.provider.shutdown().await.expect("shutdown");
}
/// Wait until something accepts TCP connections on a port.
async fn wait_for_port(port: u16, budget: Duration) -> bool {
let deadline = std::time::Instant::now() + budget;
while std::time::Instant::now() < deadline {
if std::net::TcpStream::connect(("127.0.0.1", port)).is_ok() {
return true;
}
tokio::time::sleep(Duration::from_millis(200)).await;
}
false
}
/// Wait until the browser is really up: its debugging endpoint answers and the
/// page we asked it to open is loaded.
async fn wait_for_ready_browser(port: u16, expected_url: &str, budget: Duration) -> bool {
let client = reqwest::Client::new();
let endpoint = format!("http://127.0.0.1:{port}/json/list");
let deadline = std::time::Instant::now() + budget;
while std::time::Instant::now() < deadline {
if let Ok(response) = client.get(&endpoint).send().await {
if let Ok(targets) = response.json::<Vec<serde_json::Value>>().await {
let ready = targets.iter().any(|target| {
target["type"] == "page"
&& target["url"]
.as_str()
.is_some_and(|url| url.contains(expected_url))
});
if ready {
return true;
}
}
}
tokio::time::sleep(Duration::from_millis(250)).await;
}
false
}
/// Kills and reaps a browser process the test started, on every path.
struct BrowserProcess {
child: std::process::Child,
}
impl BrowserProcess {
fn spawn(launch: &llm_bridge::browser::ExternalLaunch) -> Self {
Self {
child: launch.spawn().expect("spawning the browser"),
}
}
fn is_running(&mut self) -> bool {
self.child.try_wait().ok().flatten().is_none()
}
}
impl Drop for BrowserProcess {
fn drop(&mut self) {
let _ = self.child.kill();
let _ = self.child.wait();
}
}
/// Ask the OS for a free port.
fn free_port() -> u16 {
let listener = std::net::TcpListener::bind("127.0.0.1:0").expect("bind an ephemeral port");
listener.local_addr().expect("local address").port()
}
#[tokio::test]
#[ignore = "requires a real Chromium-based browser"]
async fn attach_mode_uses_a_browser_the_user_started() {
init_logs();
let Some(chrome) = resolve_executable(None, std::env::var("CHROME").ok()) else {
eprintln!("no browser found, skipping");
return;
};
let tmp = tempfile::tempdir().expect("temp dir");
let profile = tmp.path().join("attached-profile");
let port = free_port();
// The user starts the browser: no automation switch, no user agent spoofing.
let launch = external_launch(Some(&chrome), &profile, &fixture_url("conv=1"), Some(port));
assert!(!launch.command_line().contains("enable-automation"));
let mut browser = BrowserProcess::spawn(&launch);
if !wait_for_port(port, Duration::from_secs(30)).await {
panic!("the browser never opened its debugging port");
}
// Attaching to a browser that is still starting up races with Chrome
// creating its first tab: wait until the page is really there.
if !wait_for_ready_browser(port, "chatgpt_mock.html", Duration::from_secs(30)).await {
panic!("the browser never finished opening the fixture page");
}
let Some(harness) = harness_with(&provider_toml(&fixture_url("conv=1")), |config| {
config.browser.attach = true;
config.browser.debug_port = port;
})
.await
else {
return;
};
let session = harness
.provider
.session(None)
.await
.expect("attach to the running browser");
// The gateway reuses a tab the browser exposes to us, and otherwise opens
// one of its own in that same browser: either way the session, the cookies
// and the window belong to the user and are never closed by the gateway.
let url = session
.page()
.url()
.await
.unwrap_or_default()
.unwrap_or_default();
assert!(
!url.starts_with("devtools://"),
"unexpected page attached: {url:?}"
);
let (deltas, result) = timeout(
Duration::from_secs(90),
run_turn(
&harness.provider,
&session,
"attached hello",
None,
Vec::new(),
),
)
.await
.expect("the attached turn timed out");
let outcome = result.expect("the attached turn failed");
assert!(
outcome.text.starts_with("Echo: attached hello"),
"{}",
outcome.text
);
assert!(
deltas.len() >= 2,
"expected streamed deltas, got {deltas:?}"
);
// Shutting the gateway down must leave the user's browser alone.
harness.provider.shutdown().await.expect("shutdown");
tokio::time::sleep(Duration::from_millis(500)).await;
assert!(
browser.is_running(),
"attach mode must not close the browser it connected to"
);
}
#[test]
fn the_fixture_configuration_is_valid() {
let cfg = ProviderConfig::parse(&provider_toml(&fixture_url("")), "fixture.toml").unwrap();
assert!(cfg.is_usable(), "{:?}", cfg.problems);
assert!(cfg.capabilities().supports_images);
assert_eq!(cfg.file.timeouts.response_generation, 15);
}
tests/fixtures/chatgpt_mock.html
Vendored
+387
View File
@@ -0,0 +1,387 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8" />
<title>ChatGPT fixture</title>
<!--
Local fake of the ChatGPT web UI used by the browser end-to-end tests.
The Rust driver opens this file with a file:// URL and drives it over CDP:
it inserts text through Input.insertText / execCommand / key events, clicks
button[data-testid=send-button], then polls the DOM until the answer settles.
DOM contract (selectors live in the test provider TOML):
- editor div#prompt-textarea.ProseMirror[contenteditable=true]
- send button button[data-testid=send-button] (disabled while empty)
- answer containers div[data-message-author-role=assistant] (one per answer)
the text lives in div.answer-body inside it, and the
toolbar (with the copy button) is a sibling of the
answer node, exactly like the real page: the capture
must not pick the word "Copy" up as answer text
- answer toolbar div.message-toolbar > button[data-testid=copy-button]
copies the answer as Markdown, like the real UI
- streaming indicator button[data-testid=stop-button] (only while generating)
- file input input[type=file] (hidden)
- attachment chips div[data-testid=attachment-chip]
- new chat a[data-testid=create-new-chat-button]
- login mode form[action="/auth/login"]
- captcha mode div#challenge-form
- error mode div[role=alert]
Query parameters:
conv=1 after the first send, set location.hash to #c=<uuid> and keep it
login=1 render the sign-in form instead of the editor
captcha=1 render a bot challenge
error=1 render an error banner; sending produces no answer
ratelimit=1 the page text contains "too many requests"; sending produces no answer
empty=1 sending shows the stop button but never writes any text
delay=<ms> minimum generation time (default 1200)
loading=1 the editor only appears after 3 seconds
rewrite=1 the answer is rewritten mid-stream before settling
rich=1 the answer is rendered as rich HTML (heading, list, code
block, table) and the copy button copies it as Markdown
-->
<style>
body { font: 15px/1.5 system-ui, sans-serif; margin: 0; background: #212121; color: #ececec; }
header { padding: 10px 16px; border-bottom: 1px solid #3a3a3a; display: flex; gap: 12px; }
#messages { padding: 16px; height: 320px; overflow: auto; }
.msg { max-width: 70%; margin: 8px 0; padding: 8px 12px; border-radius: 10px; white-space: pre-wrap; }
[data-message-author-role="user"] { background: #2f2f2f; margin-left: auto; }
[data-message-author-role="assistant"] { background: #303030; }
footer { padding: 12px 16px; border-top: 1px solid #3a3a3a; }
#prompt-textarea { min-height: 48px; padding: 10px; background: #303030; border-radius: 12px;
border: 1px solid #4a4a4a; outline: none; }
#prompt-textarea:focus { border-color: #7aa2f7; }
button { font: inherit; padding: 6px 12px; border-radius: 8px; border: 1px solid #4a4a4a;
background: #303030; color: #ececec; cursor: pointer; }
button[disabled] { opacity: .45; cursor: default; }
#attachments { display: flex; gap: 6px; flex-wrap: wrap; margin-bottom: 8px; }
[data-testid="attachment-chip"] { background: #3a3a3a; border-radius: 6px; padding: 2px 8px; font-size: 13px; }
.message-toolbar { display: flex; gap: 6px; margin: 2px 0 10px; }
.message-toolbar button { font-size: 12px; padding: 2px 8px; }
.answer-body table { border-collapse: collapse; margin: 6px 0; }
.answer-body th, .answer-body td { border: 1px solid #555; padding: 2px 8px; }
.answer-body pre { background: #1b1b1b; padding: 8px; border-radius: 6px; overflow: auto; }
#toolbar { display: flex; justify-content: space-between; align-items: center; margin-top: 8px; }
.banner { margin: 12px 16px; padding: 10px 12px; border-radius: 8px; background: #4a2020; }
#challenge-form { margin: 24px 16px; padding: 20px; background: #3a3a3a; border-radius: 10px; }
form { margin: 40px auto; width: 320px; padding: 24px; background: #2a2a2a; border-radius: 12px; }
input[type="email"], input[type="password"] { width: 100%; padding: 8px; margin: 4px 0 12px;
background: #303030; border: 1px solid #4a4a4a;
border-radius: 8px; color: inherit; }
</style>
</head>
<body>
<div id="app"></div>
<script>
(function () {
"use strict";
var params = new URLSearchParams(location.search);
var mode = function (name) { return params.get(name) === "1"; };
var state = {
generating: false,
token: 0,
attachments: [],
conversation: null,
answerCount: 0
};
// A reopened conversation keeps its identifier so that the driver can prove
// that it is continuing the same web thread.
var hashMatch = /#c=([0-9a-zA-Z-]+)/.exec(location.hash);
if (hashMatch) { state.conversation = hashMatch[1]; }
function answer(prompt) {
var trimmed = prompt.trim();
var text;
if (trimmed.toLowerCase().indexOf("ok") !== -1) {
text = "ok";
} else {
text = "Echo: " + trimmed + " — this answer was generated by the local ChatGPT fixture page.";
}
if (state.attachments.length) {
text = text + " (attached: " + state.attachments.join(", ") + ")";
}
return text;
}
function escapeHtml(value) {
return String(value)
.replace(/&/g, "&amp;")
.replace(/</g, "&lt;")
.replace(/>/g, "&gt;");
}
// The rich rendering of one answer: a heading, a list, a fenced code block
// and a table, i.e. exactly the constructs a plain innerText capture loses.
function richHtml(prompt) {
return (
"<h2>Answer</h2>" +
"<p>Echo: <strong>" + escapeHtml(prompt.trim()) + "</strong></p>" +
"<ul><li>first point</li><li>second point</li></ul>" +
'<pre><code class="language-rust">fn main() {\n println!("hi");\n}</code></pre>' +
"<table><thead><tr><th>Name</th><th>Value</th></tr></thead>" +
"<tbody><tr><td>alpha</td><td>1</td></tr><tr><td>beta</td><td>2</td></tr></tbody></table>"
);
}
// What the real copy button puts in the clipboard: the Markdown rendering.
function richMarkdown(prompt) {
return (
"## Answer\n\n" +
"Echo: **" + prompt.trim() + "**\n\n" +
"- first point\n" +
"- second point\n\n" +
"```rust\n" +
'fn main() {\n println!("hi");\n}\n' +
"```\n\n" +
"| Name | Value |\n" +
"| --- | --- |\n" +
"| alpha | 1 |\n" +
"| beta | 2 |\n"
);
}
// Give one assistant message the toolbar the real page shows next to it.
// The copy button is a sibling of the answer node, not a child, so a capture
// that reads the whole turn would pick the word "Copy" up as answer text.
function attachToolbar(turn, prompt) {
var toolbar = document.createElement("div");
toolbar.className = "message-toolbar";
var copy = document.createElement("button");
copy.setAttribute("data-testid", "copy-button");
copy.setAttribute("aria-label", "Copy");
copy.textContent = "Copy";
copy.addEventListener("click", function (event) {
event.preventDefault();
var markdown = richMarkdown(prompt);
if (navigator.clipboard && navigator.clipboard.writeText) {
navigator.clipboard.writeText(markdown)["catch"](function () {});
}
});
toolbar.appendChild(copy);
turn.appendChild(toolbar);
}
// Give one assistant message the toolbar the real page shows next to it.
// The copy button is a sibling of the answer node, not a child, so a capture
// that reads the whole turn would pick the word "Copy" up as answer text.
function attachToolbar(turn, prompt) {
var toolbar = document.createElement("div");
toolbar.className = "message-toolbar";
var copy = document.createElement("button");
copy.setAttribute("data-testid", "copy-button");
copy.setAttribute("aria-label", "Copy");
copy.textContent = "Copy";
copy.addEventListener("click", function (event) {
event.preventDefault();
var markdown = richMarkdown(prompt);
if (navigator.clipboard && navigator.clipboard.writeText) {
navigator.clipboard.writeText(markdown)["catch"](function () {});
}
});
toolbar.appendChild(copy);
turn.appendChild(toolbar);
}
function render() {
var app = document.getElementById("app");
if (mode("login")) {
app.innerHTML =
'<form action="/auth/login" method="post">' +
'<h2>Sign in (fixture)</h2>' +
'<label>Email<input type="email" name="email" /></label>' +
'<label>Password<input type="password" name="password" /></label>' +
'<button type="submit">Continue</button>' +
'</form>';
return;
}
var banner = "";
if (mode("error")) {
banner = '<div role="alert">Something went wrong. Please try again.</div>';
}
if (mode("ratelimit")) {
banner += '<div class="banner">Too many requests right now, please try again later.</div>';
}
var challenge = mode("captcha") ? '<div id="challenge-form">Verify you are human</div>' : "";
app.innerHTML =
'<header><a data-testid="create-new-chat-button" href="#">New chat</a><span>ChatGPT fixture</span></header>' +
banner + challenge +
'<div id="messages"></div>' +
'<footer>' +
'<div id="attachments"></div>' +
'<div id="prompt-textarea" class="ProseMirror" contenteditable="true"></div>' +
'<div id="toolbar">' +
'<input type="file" multiple hidden />' +
'<button data-testid="send-button" disabled>Send</button>' +
'</div>' +
'</footer>';
var editor = document.getElementById("prompt-textarea");
var send = document.querySelector("[data-testid='send-button']");
var fileInput = document.querySelector("input[type='file']");
function refresh() { send.disabled = editor.innerText.trim().length === 0 || state.generating; }
editor.addEventListener("input", refresh);
editor.addEventListener("keyup", refresh);
editor.addEventListener("blur", refresh);
window.setInterval(refresh, 100);
editor.addEventListener("keydown", function (event) {
if (event.key === "Enter" && !event.shiftKey) {
event.preventDefault();
submit();
}
});
send.addEventListener("click", function (event) { event.preventDefault(); submit(); });
fileInput.addEventListener("change", function () {
var strip = document.getElementById("attachments");
for (var i = 0; i < fileInput.files.length; i += 1) {
var file = fileInput.files[i];
state.attachments.push(file.name);
var chip = document.createElement("div");
chip.setAttribute("data-testid", "attachment-chip");
chip.textContent = file.name;
strip.appendChild(chip);
}
});
var newChat = document.querySelector("[data-testid='create-new-chat-button']");
newChat.addEventListener("click", function (event) {
event.preventDefault();
state.token += 1;
state.generating = false;
state.answerCount = 0;
state.attachments = [];
document.getElementById("messages").innerHTML = "";
document.getElementById("attachments").innerHTML = "";
if (state.conversation) { history.replaceState(null, "", location.pathname + location.search); }
state.conversation = null;
editor.innerText = "";
refresh();
});
refresh();
}
function stopButton() { return document.querySelector("[data-testid='stop-button']"); }
function setStopButton(visible) {
var existing = stopButton();
if (visible && !existing) {
var button = document.createElement("button");
button.setAttribute("data-testid", "stop-button");
button.textContent = "Stop";
button.addEventListener("click", function () { state.token += 1; state.generating = false; setStopButton(false); });
document.getElementById("toolbar").appendChild(button);
} else if (!visible && existing) {
existing.remove();
}
}
function submit() {
var editor = document.getElementById("prompt-textarea");
var prompt = editor.innerText;
if (!prompt.trim() || state.generating) { return; }
var messages = document.getElementById("messages");
var user = document.createElement("div");
user.setAttribute("data-message-author-role", "user");
user.textContent = prompt;
messages.appendChild(user);
editor.innerText = "";
editor.dispatchEvent(new Event("input", { bubbles: true }));
if (mode("conv") && !state.conversation) {
state.conversation = (crypto.randomUUID ? crypto.randomUUID() : String(Date.now()));
location.hash = "#c=" + state.conversation;
}
if (mode("login") || mode("captcha")) { return; }
if (mode("error") || mode("ratelimit")) { return; }
state.generating = true;
state.token += 1;
var token = state.token;
setStopButton(true);
// The page wraps one exchange in a "turn" element, like the real UI: the
// answer node holds the text only, and the toolbar is its sibling.
var turn = document.createElement("div");
turn.className = "turn";
turn.setAttribute("data-testid", "conversation-turn");
var node = document.createElement("div");
node.setAttribute("data-message-author-role", "assistant");
var body = document.createElement("div");
body.className = "answer-body";
node.appendChild(body);
turn.appendChild(node);
messages.appendChild(turn);
state.answerCount += 1;
if (mode("rich")) {
attachToolbar(turn, prompt);
}
var full = answer(prompt);
var draft = mode("rewrite") ? "Draft answer being rewritten" : null;
var delay = parseInt(params.get("delay") || "1200", 10);
var steps = 22;
var written = "";
if (mode("empty")) {
window.setTimeout(function () {
if (token !== state.token) { return; }
state.generating = false;
setStopButton(false);
}, delay);
return;
}
var index = 0;
var timer = window.setInterval(function () {
if (token !== state.token) { window.clearInterval(timer); return; }
index += 1;
var target = full;
if (draft && index * 2 < steps) { target = draft; }
else if (draft && index === Math.floor(steps / 2) + 1) { written = ""; }
var chars = Math.ceil(target.length * (index / steps));
written = target.slice(0, chars);
body.innerText = written;
if (index >= steps) {
window.clearInterval(timer);
// The settled answer is rendered: the rich modes turn it into HTML,
// which is what a plain text capture cannot represent.
try {
if (mode("rich")) {
body.innerHTML = richHtml(prompt);
} else {
body.innerText = full;
}
} catch (e) {
body.innerText = full;
}
state.generating = false;
setStopButton(false);
var send = document.querySelector("[data-testid='send-button']");
if (send) { send.disabled = document.getElementById("prompt-textarea").innerText.trim().length === 0; }
}
}, Math.max(20, Math.floor(delay / steps)));
}
window.fixtureState = state;
if (mode("loading")) {
window.setTimeout(render, 3000);
} else {
render();
}
})();
</script>
</body>
</html>