Testing — Contracts (L1)
Testing domain — L1
1. Shared testkit
posthaste-testkit is the dev-only library ([dev-dependencies]) that
standardizes lower-layer test support across crates. It is the reference
implementation of this contract.
| Surface | Responsibility | State |
|---|---|---|
Harness | Disposable config repo + SQLite store + MailService on a temp root; save_account, seed_messages / seed_messages_typed helpers. Exposes service and store for direct driving. | realized |
StalwartFixture | Disposable real Stalwart on free loopback ports, seeded via tools/dev/stalwart; jmap_transport / imap_transport / email accessors; inject(count) SMTP burst; tears down on drop. | realized |
paths | temp_root, free_loopback_port, stalwart_bin, workspace_root. | realized |
| Declarative fixtures | Fixture / FixtureAccount / FixtureMessage typed TOML (accounts + messages with field overrides) loaded via Harness::load_fixture_toml / load_fixture, driving save_account + seed_messages_typed. | realized |
| Mock Gmail IMAP fixture | GmailImapFixture: a stateful, multi-connection mock Gmail IMAP+SMTP server answering the discovery + full-snapshot and CONDSTORE/QRESYNC-delta sync command sets; its imap_transport() wires an ImapSmtp account at it so a seeded X-GM-LABELS INBOX message lands in the store, and vanish_inbox_and_deliver drives a CHANGEDSINCE/VANISHED delta. Variants (start_condstore_only, start_generic_uidplus, start_generic_without_uidplus) exercise capability-degraded paths. Omits IDLE (no push stream); IDLE-push parity is a later increment. Also shipped as a runnable dev server (src/bin/mock-gmail.rs, POSTHASTE_DEV_GMAIL=1 just dev services) so the dev app can point a Gmail IMAP account at the same fixture. | realized |
TestSecretStore / TempDirGuard | In-memory secret store; scoped temp-dir cleanup. | realized |
Tests that touch a real Stalwart gate on POSTHASTE_STALWART_INTEGRATION=1
and skip otherwise; the fixture panics on spawn failure (an environment, not
test, failure).
2. Verification ladder
| Rung | Driver | Proves | State |
|---|---|---|---|
| 1. Unit/integration | cargo / nextest against posthaste-testkit::Harness | domain, store, sync, API logic | realized |
| 2. Frontend logic | bun test (apps/client/frontend) | facade behavior, cache invalidation, routing | realized |
| 3. Live-provider parity | Stalwart-gated cargo tests (StalwartFixture, apps/client/backend/tests) | real JMAP/IMAP projections through the app’s real sync path | realized |
| 4. Agent-driven full app | headless driver + declarative fixtures | multi-step convergence, live-query bugs | planned |
[::state partial plan=docs/eph/PLAN-L2-testkit-roadmap#roadmap]
3. Coverage model
Each behavior boundary must have focused tests. Tests are named by the behavior
they prove, use Arrange/Act/Assert, and link their spec assertion with a
// spec: docs/...#id comment when one exists.
Provider observation matrix
Each row is at least one focused test unless explicitly out of scope.
| Area | Observation | Expected local outcome |
|---|---|---|
| Gmail IMAP labels | Same message via Sent/Starred/All Mail | One message, multiple ImapMessageLocations, unioned membership, one keyword set |
| Generic IMAP copies | Same RFC Message-ID, different UIDs | Separate messages unless a stable identity proves equality |
| IMAP flags | Remote flag change, no UIDNEXT change | MODSEQ/QRESYNC delta applies keywords; no skip on UID count |
| IMAP expunge | UID disappears from a mailbox | Location removed; deleted only when no membership remains |
| IMAP UIDVALIDITY | UIDVALIDITY changes | Stored locations invalidated by authoritative refresh |
| IMAP VANISHED | QRESYNC returns VANISHED UIDs | Matching locations deleted without pruning unrelated messages |
| JMAP changes | cannotCalculateChanges | Full authoritative snapshot replaces the affected set |
| Local mutation echo | Local keyword/mailbox mutation succeeds remotely | All visible views reflect it before a manual sync |
| Remote convergence | Remote mutation by another client | Push/poll converges every view without stale per-mailbox state |
Store, live-query, API, frontend contracts
- Store reconciliation:
SyncBatchapplication is atomic, account-scoped, projection-safe, event-complete. - Live query update: a mutation or sync delta bumps the store generation and converges every affected query; no missed invalidation, no over-broad refetch.
- API boundary: stable error codes, pagination cursors, the SSE event stream (generation heartbeat), command-response shape.
- Frontend state: user-visible transitions, not component internals.
4. Test quality standard
- Name tests by the behavior they prove.
- Arrange / Act / Assert, visually distinct, one logical assertion per test.
- No incidental implementation assertions.
- Link SPECial assertions:
// spec: docs/testing/L1#provider-observation-matrix. - Red-first: write the smallest failing test, confirm it fails for the right reason, then implement.
- Error-path tests assert the typed error and its context, not “something threw”.
- Prefer real behavior through
StalwartFixtureover mocking; mock only true system boundaries (provider responses for unit tests that must not touch a server).
5. Fixtures and Stalwart
A real Stalwart with real email is the parity substrate. StalwartFixture
manages a disposable instance (free ports, temp data, seeded dev@example.org).
Tests inject mail by pre-seeding the maildir or by live SMTP/LMTP delivery; the
app’s own live sync (JMAP WebSocket push / IMAP IDLE) observes the delta — the
observation path is the system under test and must not be bypassed.
Declarative Fixture TOML (accounts + messages with field overrides) loads via
Harness::load_fixture_toml / load_fixture over the programmatic
save_account + seed_messages_typed helpers. JMAP / provider-state
fixtures remain planned (the loader only supports driver = "mock" today).
Programmatic setup (Harness::save_account, StalwartFixture::start) remains
for cases that need direct control.
[::state partial plan=docs/eph/PLAN-L2-testkit-roadmap#roadmap]
6. Forward contracts
Declarative TOML fixtures landed (P3b). The remaining forward contract — the
headless driver for agent-driven full-app scenarios — is specified in
docs/eph/PLAN-L2-testkit-roadmap.md and folds back here as it lands.
[::state planned plan=docs/eph/PLAN-L2-testkit-roadmap#roadmap]
Assertions
| ID | Sev. | Assertion |
|---|---|---|
| shared-testkit-surface | MUST | Lower-layer tests use posthaste-testkit for harness, fixtures, and path helpers rather than ad-hoc copies |
| real-provider-parity | MUST | Parity tests drive a real Stalwart through StalwartFixture and observe via the app’s real sync path, never by injecting into the local store |
| integration-gate | MUST | Real-Stalwart tests gate on POSTHASTE_STALWART_INTEGRATION=1 and skip otherwise |
| provider-observation-contracts | MUST | Provider adapters are tested against the observation matrix before output reaches the store |
| sync-convergence-contracts | MUST | Local and remote mutations converge across every affected mailbox/smart/tag view |
| store-reconciliation-contracts | MUST | Store sync tests prove account scoping, atomicity, event emission, projection refresh, deletion |
| api-boundary-contracts | MUST | API tests prove error codes, pagination, SSE replay, mutation responses |
| frontend-state-contracts | MUST | Frontend tests prove user-visible state transitions, not internals |
| declarative-fixtures | MUST | load_fixture_toml / load_fixture create declared accounts and seed declared messages (with field overrides) so a query reflects the fixture |
| spec-linked-coverage | SHOULD | Tests for SPECial assertions link with a spec: comment |