# Test Strategy

> Automated testing strategy for processkit v1.0.

---

LLMS index: [llms.txt](/processkit/v1.x/llms.txt)

---

> **Alpha.4 documentation review:** This page records design or historical
> planning. For shipped behavior and current gaps, use the
> [issue #135 implementation review](./issue-135-status.md).

The current exploratory strategy is to install processkit into a new
aibox project and try workflows manually. That remains useful as a human
dogfood check, but it is not enough for v1.0. It is not automated, it
makes aibox a hard dependency, and it cannot prove release-gate criteria
repeatably.

## Test Goals

The v1.0 test strategy must prove:

- schema generation is deterministic
- generated schemas validate real and adversarial fixtures correctly
- MCP tools match their Python signatures and JSON Schemas
- writes enforce state machines, guards, validation modes, and event logs
- index reads match canonical files after creates, transitions, and
  migrations
- `query_by_interface` returns complete mixed-kind results
- migration tools preserve data within the RFC gate limits
- pk-doctor catches deliberately invalid entities and every actionable
  finding has an executable or formally recognized disposition
- docs and examples remain buildable
- downstream integrations can consume the published contracts without
  becoming release prerequisites

## Automated Layers

| Layer | Purpose |
|---|---|
| Schema unit tests | Render Jinja + YAML fragments, compare `_generated` output to golden files, and verify merge strategies. |
| Schema contract tests | Validate generated draft-2020-12 schemas against valid and invalid entity fixtures. |
| MCP contract tests | Check every tool signature against its JSON Schema and run typed request/response fixtures. |
| State-machine tests | Exercise valid and invalid transitions, guard failures, terminal states, and emitted events. |
| Index tests | Create and mutate fixture entities, then assert search, relation traversal, backlinks, and interface grouping. |
| Migration tests | Plan and execute v0.x fixture migrations; assert field-loss, orphan, source-hash, alias-resolution, and append-only gates. |
| pk-doctor adversarial tests | Require every expected finding, validate remediation tools against the gateway catalog, execute remediations, and require a clean recheck. |
| Package smoke tests | Install processkit from the local tree or release tarball into a temporary fixture project without aibox. |
| Docs tests | Build Hugo/Docsy locally and verify generated links, private-content exclusion, and GitHub Pages output. |
| Downstream adapter tests | Consumers may run their own integration suites against a signed processkit release; their results are informative and non-blocking for processkit. |

## Fixture Projects

Use local fixture projects under the test tree:

- `empty-project`: no context, used for first install and schema
  generation
- `alpha-project`: small valid corpus covering the alpha ontology slice
- `migration-v0-project`: representative v0.x corpus for migration
  adapters
- `adversarial-project`: invalid frontmatter, bad transitions, broken
  links, malformed bindings, and inconsistent index state
- `remediation-project`: actionable doctor findings with safe fixes,
  archives, migrations, policy exceptions, and external blockers
- `art-project`: a compact first-ART scenario that exercises planning,
  execution, demo, inspect-and-adapt, decisions, risks, and evidence

These fixtures run through plain local repository commands. aibox can consume
the same fixtures in an adapter suite, but the fixtures must not require
aibox or GitHub Actions to exist.

## Alpha Proof

Alpha automation should pass before any alpha tag:

- full schema rebuild from shipped `context/schemas/src/`
- committed `_generated` tree matches the renderer output
- create/read/transition MCP paths work for the alpha slice
- `query_by_interface` works for at least `Record`
- strict and tolerant validation modes are observable
- the alpha fixture migrates from v0.x or maps explicitly
- actionable alpha findings close through the shipped gateway or resolve to
  a recognized non-executable disposition
- docs build locally

Manual dogfood remains useful after this automated baseline, not instead
of it.

## Final Release Proof

The final gate should include:

- all strict 81-gate criteria green
- first-ART validation completed with recorded evidence
- all MCP tools schema-checked
- pk-doctor adversarial and remediation fixtures green after executing their
  declared closure paths
- package smoke tests green from release artifact
- signed archive and native installer verify and install without network
  access or an external project manager
- no known index/schema/migration blocker

This keeps the RFC's first-ART proof while removing the current hard
dependency on manual downstream experimentation.