Tests and verification

Two kinds of checks exist, and they prove different things.

Automated tests

pnpm test

Runs the server test suite with Vitest against a real PostgreSQL 17 server started for the run (no Docker needed). Each test file gets its own database. What the files cover:

File	Covers
lifecycle.test.ts	A task end to end, the plan gate, invalid and unpermitted tool calls, pause at a safe boundary, cancel (queued, running, in-flight request), limits and resuming past them, provider failures, discarded turns
recovery.test.ts	Six pollers never claiming the same task; crash after a completed step; crash during a read-only, an idempotent and an external operation; review by retry and by skip; fencing of a worker that lost its lease; shutdown hand-back
approvals.test.ts	Approval bound to exact arguments, denial, a changed operation needing a new approval, tampered arguments refused, several approvals in one turn, allowed domains, content that tries to grant itself a tool
memory.test.ts	Persistence across a restart, retrieval order and scope, a preference reaching the model and being recorded, deletion and scrubbing, audit references, proposals
scheduler.test.ts	Cron compilation and time zones, persistence, a due run, two schedulers ticking at once, overlap, missed runs under both policies, pause and edit
tools.test.ts	Workspace path escapes including links, file tools, the CSV parser and statistics, artifact names, address classification, DNS rebinding at connect time, redirects, size and request limits
security.test.ts	Redaction in data, logs, events, transcripts and artifacts; encryption of stored keys; password hashing; operation hashes
providers.test.ts	Both adapters through the official SDKs against local stand-ins: request shapes, replay of saved turns, error mapping, abort
api.test.ts	Setup token, sessions, CSRF and origin checks, throttling, provider connections never showing connected after a failure, tasks and approvals over HTTP, authenticated artifact access, uploads, event stream replay

In these tests the model is a deterministic double: either a scripted list of turns inside the test, or a local HTTP server that imitates a provider's wire format. They prove the runner, the queue, the tools, the permissions and the adapters' handling of requests and responses. They do not prove anything about a real model.

End-to-end run

pnpm e2e:local

Builds the server, then starts PostgreSQL 17, the API process, a worker process and the rule-based model double, and drives them over HTTP like the console does:

• A. File summary: upload a text file, run a task, read its events, download the summary.
• B. CSV report: upload a CSV, check the statistics computed by the tool, check the report quotes them.
• C. Memory reuse: save a preference, run a task, check the entry was supplied, recorded and used; delete it.
• D. Restart recovery: kill the worker process after a completed step, start a new one, check the step was not repeated.
• E. Scheduled run: create a schedule, keep no client connected, check the scheduler starts the task.
• F. Approval gate: check a protected write does not proceed until its exact arguments are approved, and that an approved fetch of a cloud-metadata address is still refused.

It also checks cancellation of an in-flight request, the connection test, stored-key encryption, and that no credential appears in any process log.

Console checks

The console was exercised in headless Chrome against pnpm dev and the model double, with the scripts in scripts/flows/: first-run setup with a wrong and a right token, connecting the double through Settings and seeing it become Connected only after the test request, uploading files and a refused upload, workflows A, B, C, E and F from the console pages, the command palette and keyboard focus, the navigation drawer and dialogs at 390 px, and the landing page with reduced motion. Each script is a sequence of clicks and assertions run with pnpm shot; they are a record of what was checked, not part of pnpm test, and they expect a fresh database in the order listed in that folder's README.

Real provider check

OPENAI_API_KEY=... pnpm smoke:provider openai <model-id>
ANTHROPIC_API_KEY=... pnpm smoke:provider anthropic <model-id>
PCMD_LOCAL_BASE_URL=http://localhost:11434/v1 pnpm smoke:provider local <model-id>

One text request and one tool-calling request to a real provider, through the adapters the worker uses. This is the only check that involves a real model. It needs a key and costs a few tokens.

Results for this version

Recorded on the machine this version was built on (Windows 11, Node 24.16, PostgreSQL 17.10 from embedded-postgres).

Check	Result
pnpm typecheck	Passed: packages/shared, apps/server (source and tests), apps/web
pnpm test	Passed: 117 tests in 9 files, 11 s, against PostgreSQL 17.10
pnpm e2e:local	Passed: 54/54 checks (workflows A–F, cancellation, connection test, key encryption, log redaction)
pnpm build	Passed: server bundle (tsup) and console (Next.js 15.5, 30 static pages)
pnpm smoke:provider anthropic claude-sonnet-5-5	Passed on 2026-09-30 with the owner's key: text request answered "ready" (1.8 s), tool-calling probe called connection_probe (1.2 s), model list returned 13 models. OpenAI and local endpoints: not run, no key or server available
Real task on Anthropic (claude-sonnet-5-5)	Completed on 2026-09-30 through the console: read a text file, ran analyze_csv, wrote briefing.md; 5 model turns, 23 s, 32,044 tokens as reported by the provider
Production at poietescmd.xyz (console on Vercel, backend on the VPS)	First-run setup, the Anthropic connection test for both models, two uploads and a real task (claude-sonnet-5-5, 5 turns, 20 s, 30,473 tokens, briefing.md written) completed through the public console on 2026-09-30
docker compose up (server image, deploy/ layout)	Run on 2026-09-30 on an Ubuntu 26.04 VPS: the server image built from the Dockerfile, postgres, api, worker and caddy came up healthy on the first attempt; the API is served at https://api.poietescmd.xyz. The bundled web service and the console image have not been built there (the console runs on Vercel)