1 - Maintenance
Maintenance
Internal procedures for building, testing, documenting, and releasing aibox.
The canonical step-by-step release note remains in context/notes/; this page
is the public contributor summary.
Development Checks
cd cli && cargo fmt -- --check
cd cli && cargo clippy --all-targets -- -D warnings
cd cli && cargo test
The helper script wraps the same checks:
./scripts/maintain.sh test
Documentation Site
The public docs live in docs-site/ and use Hugo with the Docsy theme.
git submodule update --init --recursive docs-site/themes/docsy
npm --prefix docs-site ci
./scripts/maintain.sh docs-serve
./scripts/build-docs.sh
./scripts/build-docs.sh is the local verification build. It writes the static
site to docs-site/public/ and prints Hugo render warnings.
The maintenance script also exposes:
./scripts/maintain.sh docs-serve
./scripts/maintain.sh docs-deploy --dry-run
./scripts/maintain.sh docs-deploy
docs-deploy builds the site and pushes the static output to the gh-pages
branch from the local checkout. It does not use GitHub Actions. Use
--dry-run to validate the production build without pushing; use the command
without that flag only when the current checkout is the source that should be
published. The release script runs the same deployment as part of the
container-side release phase.
Use the repository maintenance command for publication rather than
npm run deploy. The maintenance command preserves the project’s local-only
release flow, publishes the already configured /aibox/ site, and ensures
GitHub Pages serves the gh-pages branch.
Published Image
aibox publishes the base Debian image used by generated downstream projects:
./scripts/maintain.sh build-images
./scripts/maintain.sh build-images --no-cache
./scripts/maintain.sh push-images X.Y.Z
Generated project images are built per project by aibox apply. They are not
published from this repository.
Release Boundary
Releases are intentionally split:
| Phase | Where | Command | Purpose |
|---|
| Container side | aibox devcontainer | ./scripts/maintain.sh release X.Y.Z | check dependency/harness state, sync processkit default, bump CLI version, test, audit, build Linux binaries, tag, create GitHub release, deploy docs |
| Host side | macOS host | run-directory command from dist/RELEASE-PROMPT.md | validate immutable inputs, build/smoke Darwin and candidate images without credentials, emit evidence, then publish the fixed manifest |
Both phases run locally. The project deliberately does not use GitHub Actions
for release validation, artifact builds, image publication, or deployment.
Release speed comes from bounded local concurrency, persistent caches, and
reuse of evidence for the exact release commit rather than from moving gates to
a hosted runner.
Do not create GitHub releases by hand with gh release create. The release
script attaches binaries and writes the release notes expected by users.
Container-Side Release
./scripts/maintain.sh release X.Y.Z
This command requires a clean working tree. It may stop after
sync-processkit if a newer processkit release changes the pinned default and
the CLI needs review before release.
The command performs:
- dependency, addon, image, and harness state report in
dist/RELEASE-STATE.md - processkit and host-context aibox doctor runs in
dist/RELEASE-DOCTORS.md;
doctor errors block the release and warnings remain visible for review - processkit release sync check
cli/Cargo.toml and Cargo.lock version bump when needed- format, Clippy, and test checks
- local temporary-workspace E2E and isolated tmux/asciinema probes
cargo auditcargo update --dry-run review for lockfile-resolvable crate updates- Linux release builds for
aarch64-unknown-linux-gnu and
x86_64-unknown-linux-gnu - binary version smoke check
- annotated git tag push
- GitHub release creation with Linux binaries
- Hugo/Docsy docs deployment
dist/RELEASE-PROMPT.md for host-side completion
Independent validation gates run concurrently. The default worker limit is
two; set AIBOX_RELEASE_PARALLELISM to a positive integer that fits the local
machine. Linux release targets build concurrently inside the build gate, and
the version smoke reuses the matching release artifact instead of compiling a
third binary.
Successful gates write local evidence under
dist/release-evidence/vX.Y.Z/<commit>/. Evidence is bound to the exact commit,
Rust toolchain, clean-tree state, release phase, and gate-specific environment.
Audit evidence expires daily, and binary evidence rechecks archive checksums. Set
AIBOX_RELEASE_REUSE_EVIDENCE=0 to force every selected gate to run again.
Container-side timings are summarized in dist/RELEASE-TIMINGS.md; host-side
timings are summarized in dist/RELEASE-HOST-TIMINGS.md. Each summary is a
cumulative view of the append-only timing-events.tsv file beside the
candidate evidence, so retries and resumed release commands retain failed,
passed, and reused gate attempts instead of replacing the earlier timings.
Run ./scripts/maintain.sh release-check-state standalone when you want the
dependency and tool-state report without bumping, tagging, or building. Run
./scripts/maintain.sh release-doctors for the matching diagnostic report.
If a report finding is deferred, create a processkit WorkItem before continuing
the release and mention that WorkItem ID in the release notes or handover.
For cargo update --dry-run, either apply available crate updates in the
release with full validation, or create a WorkItem for the deferred
crate-update pass.
Host-Side Release
The container-side release prepares a checksummed source archive and provenance
record under tmp/host-gates/aibox-release/<run-id>/input/. It writes the
single owner command to dist/RELEASE-PROMPT.md:
./scripts/maintain.sh release-host tmp/host-gates/aibox-release/<run-id>
For an evidence-only rehearsal, append --dry-run. Validation still performs
all candidate builds, probes, cleanup, SBOM generation, vulnerability scanning,
and manifest hashing, but it does not invoke GitHub or GHCR publication. The
validator prints the separate publisher command that can consume the verified
run directory later.
Interactive terminals use the Textual dashboard by default (--ui=auto). Use
--ui=textual to fail if the dashboard cannot start, or --ui=plain for
redirected and captured output. The UI presents a high-level progress bar, a
persistent task list, and a selectable task-filtered log. Space toggles follow,
w toggles wrapping, Ctrl+A/C selects and copies, y copies the selected task
log, End resumes the live tail, and p displays the authoritative evidence
path. The UI is presentation rather than evidence; full output is retained in
evidence/command-results.log.
Content-addressed container layers are reused by default. The Rust registry is
shared in a dedicated credential-free host-gate cache, while compiled artifacts
are isolated by candidate commit so a failed candidate can be retried without a
full two-target rebuild. Use --cold-cache only when deliberately investigating
cache behavior. Cold mode retains downloaded Rust packages but forces downstream
container layers to rebuild.
To retry after a conditional addon or lifecycle check fails, prepare a new run
and name the failed run as its checkpoint source:
NEW_RUN="$(./scripts/maintain.sh release-host-prepare X.Y.Z)"
./scripts/maintain.sh release-host \
--retry-from=tmp/host-gates/aibox-release/<failed-run-id> "${NEW_RUN}"
The retry source must contain byte-identical immutable candidate inputs.
Completed conditional checks are reused only when their candidate-bound
checkpoint checksum is valid. The candidate lifecycle, SBOM, vulnerability
scan, cleanup, manifest assembly, and publication verification remain fresh.
The reviewed entry point accepts only that run-directory path. It rejects
traversal, symlinks, special files, hardlinks, unexpected inputs, unsafe
permissions, checksum drift, and tag/commit mismatches. A previous partial run
is never resumed in place: runtime/ and evidence/ must not exist in the new
run when it starts. Retry imports only validated conditional checkpoints.
The host checkout’s HEAD must match the attested candidate commit, but
unrelated tracked worktree edits do not invalidate the gate because all builds
and probes use the immutable checksummed source archive rather than worktree
contents.
Validation and publication are separate security stages within the one owner
invocation. Candidate-controlled native builds, build scripts, CLI commands,
and the generated runtime smoke receive a fixed environment and run under a
macOS sandbox that denies GitHub configuration, Docker configuration, SSH
material, and Keychain services. Docker uses an empty per-run configuration;
no publication credential, secret, broad mount, or runtime socket is exposed
to the development container or candidate container.
The entry point accepts owner-installed uv from the official standalone path
~/.local/bin/uv or the architecture-native Homebrew prefix. It verifies
ownership and rejects group/world-writable executables instead of searching an
inherited PATH. It lets uv resolve, install when necessary, and run exact
Python 3.14.6 under --no-project, fixed owner cache/managed-Python roots,
and a fully rebuilt environment. The gate invokes python as uv’s command
rather than handing the script to uv, so candidate inline script metadata is
not processed. Candidate project metadata and inherited UV_* settings cannot
select the interpreter or dependencies.
The validation stage builds both Darwin targets, natively smokes the current
architecture, builds the actual candidate foundation/runtime images, exercises
the generated Compose lifecycle and --forget-tmux-state, requires cleanup,
generates a CycloneDX SBOM, and applies the reviewed Grype policy. High or
Critical findings with a listed fixed version block the release; findings with
no listed fix remain explicit non-blocking warnings in the evidence. It also
verifies the checksummed comparison tag, commit, and changed-path list, then
runs affected addon build groups, the LaTeX watcher/preview lifecycle, and the
rootless Podman readiness probe when relevant inputs changed. The readiness
probe verifies the unprivileged binaries, subordinate ID ranges, namespace
helpers, storage/network helpers, and container configuration without granting
the outer development container privileges for nested user-namespace execution. No comparison tag selects
all three surfaces. Every command, selection reason, skip reason, and result is
retained beneath evidence/ with toolchain metadata, image inspection, runtime
logs, hashes, and a release manifest.
The terminal interface streams subprocess output as it is produced. Each
high-level operation reports running, passed, or failed state with elapsed
time, and quiet commands emit a heartbeat every ten seconds. The same
transitions are retained in evidence/steps.log; full argv and output remain
in commands.log and command-results.log, so interactive progress does not
replace auditable evidence.
The terminal groups High/Critical package matches by unique advisory and prints
a bounded summary with severity, affected package names, and disposition. The
complete scanner report remains at evidence/security/vulnerability-scan.json;
counts, grouped advisories, package versions, fix versions, and the blocking or
warning classification are publication-required evidence in
evidence/security/vulnerability-policy.json.
The gate selects the first responsive runtime in this order: the docker CLI
contract exposed by Docker Desktop or OrbStack, then Podman. All image builds,
Compose lifecycles, exec probes, inspection, cleanup, scanning, and publication
use that selected runtime. Docker-compatible builds explicitly enable BuildKit
for features such as COPY --chmod; OrbStack does not need Docker Desktop’s
separate Buildx component.
When Compose and Buildx are installed as Docker CLI plugins, the gate copies
only those owner/root-owned, non-group-writable executables into its empty
per-run DOCKER_CONFIG. It does not copy the owner’s Docker configuration,
registry credentials, contexts, or unrelated plugins.
Before the Darwin build, the gate fetches the exact locked Cargo dependency
graph into a per-run credential-free Cargo home. The actual compilation remains
offline, so a newly locked crate does not require a pre-warmed owner cache and
candidate build scripts do not receive network access.
Only after every gate succeeds does the separate publisher receive normal host
GitHub/GHCR authority. It revalidates the immutable manifest and can upload
only the two Darwin archives plus checksums and push only the fixed aibox
foundation-version, runtime-version, and runtime-latest tags. It cannot build,
run tests, execute candidate code, commit, merge, or accept extra arguments.
Remote asset and image inspection is mandatory.
On failure, keep the run directory as diagnostic evidence. Correct the source,
create a new candidate commit/tag, and prepare a new run ID; do not edit the
old input or selectively reuse its evidence. The owner must review changes to
the gate and publisher before using a changed version.
Verification
After release:
gh release view vX.Y.Z shows all expected binary assets.curl -fsSL https://raw.githubusercontent.com/projectious-work/aibox/main/scripts/install.sh | VERSION=X.Y.Z bash installs the expected version.aibox --version reports X.Y.Z.- For
v0.26.x, docker pull ghcr.io/projectious-work/aibox:base-debian-vX.Y.Z or the matching Podman pull succeeds. - For
v0.27.0+, docker pull ghcr.io/projectious-work/aibox:base-debian-runtime-vX.Y.Z and base-debian-runtime-latest succeed. Foundation images are published as base-debian-foundation-vX.Y.Z; source-hash marker tags are not published. - To remove historical source-hash marker tags from GHCR, first run
./scripts/maintain.sh ghcr-prune-source-tags --repair-mixed and review the mixed-version repair plan. Then run ./scripts/maintain.sh ghcr-prune-source-tags --repair-mixed --execute with read:packages, delete:packages, and Docker Buildx available on the host. - The docs site at
https://projectious-work.github.io/aibox/ reflects the release.
Project Devcontainer
The maintenance script can also operate this repository’s own devcontainer:
./scripts/maintain.sh start
./scripts/maintain.sh status
./scripts/maintain.sh attach
./scripts/maintain.sh stop
Do not confuse .devcontainer/ in this repository with images/. The former
is the environment used to develop aibox. The latter contains image recipes
published for downstream projects.
2 - E2E Test Catalogue
E2E Test Catalogue
aibox deliberately has no container-runtime bridge from its development
container. Tests are placed according to the product behavior they prove, not
according to the historical runner that happened to execute them.
Execution surfaces
| Surface | Where it runs | Contracts |
|---|
| Local | Development container, unique temporary directories | CLI parsing, init/apply, generated files, preservation, configuration, migrations, mocked runtime behavior, terminal rendering |
| Local visual | Development container, per-test tmux socket | Real panes, layouts, keybindings, Yazi/Vim interaction, asciinema output |
| Release host | Owner-controlled macOS gate | Native Darwin binaries and genuine candidate-image build/start/probe/down behavior |
Run the complete local suite with cd cli && cargo test. The compatibility
feature e2e no longer grants access to another machine or runtime. Opt-in
visual matrices run with:
./scripts/maintain.sh test-e2e-visual-status
./scripts/maintain.sh test-e2e-visual-tabs
./scripts/maintain.sh test-e2e-visual-yazi
./scripts/maintain.sh test-e2e-visual
The canonical release and local-E2E gates pass --test-threads=1. The visual
tests share host PTY and tmux scheduling even though each owns a separate
socket; keeping the default serialized prevents slow Yazi/Vim cases from
timing out behind another process-level visual test.
Every process-level visual test removes inherited TMUX and TMUX_PANE, owns
a socket beneath its TempDir, and cleans up only its own server. Tests must
not use global tmux kill-server, broad /tmp/tmux-* removal, or global
pkill; those operations can terminate the developer’s live session.
Removed companion
The former aibox-e2e-testrunner, SSH/SCP runner, nested Podman/systemd/kind
image, passwordless elevation, capabilities, devices, host module mount, and
relaxed security profile were removed for issue #372. No test required a
separate unprivileged clean-room service. The generic Alpine pull/run test was
also removed because it supplied weaker evidence than the actual generated
aibox image lifecycle.
The remaining runtime-only contracts are:
- build the candidate foundation and runtime image from the release source;
- generate a downstream project with the candidate Darwin binary;
- run
aibox apply and Compose up against that locally built image; - verify identity, non-root execution, readiness, tmux/Yazi/status tooling,
and the
--forget-tmux-state attach behavior; - bring down only that run’s Compose project and fail if cleanup fails;
- generate a CycloneDX SBOM and fail the vulnerability gate on high severity.
These contracts inherently need ordinary host Docker and therefore run only in
the macOS release gate. Missing Docker, Apple targets, Syft, Grype, or any other
mandatory prerequisite is a failure, never a passing skip.
Three expensive surfaces remain mandatory when their attested inputs change:
- grouped builds for affected download-based addons;
- a two-revision LaTeX watcher build with byte-identical preview-sidecar output;
- the infrastructure addon’s nested Podman probe, including a true rootless report.
The immutable provenance records the previous version-line tag and commit plus
the exact changed-path list. The gate recomputes that diff before selecting
checks. Base-image, generator, addon-loader, template, or host-gate changes
select every conditional surface; an addon definition selects its group, and
LaTeX or infrastructure changes select their dedicated lifecycle probe. If no
comparison tag exists, all conditional checks run. Every selection and
non-selection is written to evidence. A selected check cannot pass by skipping.
Host run-directory protocol
The container-side release creates immutable input under:
tmp/host-gates/aibox-release/<run-id>/
└── input/
├── provenance.json
├── checksums.sha256
└── source.tar.gz
The owner runs the single command printed in dist/RELEASE-PROMPT.md:
./scripts/maintain.sh release-host tmp/host-gates/aibox-release/<run-id>
Append --dry-run to execute the same builds, runtime probes, cleanup,
security scans, and manifest creation without invoking publication:
./scripts/maintain.sh release-host tmp/host-gates/aibox-release/<run-id> --dry-run
Add --ui=textual to require the interactive dashboard or --ui=plain to
retain line-oriented output. --ui=auto is the default and selects Textual
only for a suitable TTY. The dashboard shows completed high-level tasks in a
progress bar, keeps passed/failed/skipped task rows visible, and lets the
operator filter a bordered log by task. Space toggles follow mode, w toggles
soft wrapping, Ctrl+A selects, Ctrl+C copies only the marked selection, l
selects the last 20 log lines, y copies the current task log, and e copies
the canonical warning/failure Problems bundle. The Problems panel can be
selected to filter the log. End returns to the live tail, and p reveals the
evidence path.
The dashboard is presentation, not evidence. Raw command output remains in
evidence/command-results.log and high-level transitions remain in
evidence/steps.log; use those files for complete copied or attached logs.
The completed run directory can later be published with the exact
release-host-publish.sh command printed by the validator. Do not rerun the
gate without --dry-run; populated runtime/ and evidence/ directories are
intentionally non-resumable.
The entry point accepts that one path plus optional fixed --dry-run,
--cold-cache, --retry-from=<failed-run-dir>, and
--ui=auto|textual|plain flags. It canonicalizes the path,
requires one direct child of the approved root, rejects symlinks, special
files, hardlinks, unexpected files, unsafe permissions, bad checksums, and
tag/commit mismatches, then creates runtime/ and evidence/ itself.
The entry point uses the owner-installed uv binary from a reviewed fixed path
and requires exact Python 3.14.6 with --no-project. It provisions the
hash-locked Textual 8.2.8 environment from scripts/release-host-ui.lock;
candidate PEP 723 and project metadata remain disabled. Its cache and
managed-Python roots are fixed beneath
~/Library/Caches/aibox-host-gates/uv and
~/Library/Application Support/aibox-host-gates/python; candidate
pyproject.toml, inline metadata, uv configuration, and inherited UV_*
variables cannot affect execution. The resolved uv, Python, Textual, lockfile,
and tool paths are recorded in evidence.
Content-addressed container layer reuse is the default. --cold-cache forces
downstream image rebuilding for cache investigations. Rust downloads live in a
dedicated credential-free cache and compiled output is scoped to the candidate
commit. --retry-from accepts only a separate run with byte-identical immutable
inputs and reuses checksummed successful conditional probes; core lifecycle,
security scanning, cleanup, and publication evidence are always regenerated.
Candidate compilation, build scripts, CLI execution, and runtime smoke run
with a fixed sanitized environment and a macOS sandbox that denies access to
GitHub configuration, Docker configuration, SSH material, and Keychain
security services. Docker receives an empty per-run configuration and no
secrets or host mounts. Tool runtime state is fixed under runtime/; no
candidate project configuration controls Python or package resolution.
Only after every required validation succeeds does the gate invoke the small
publisher. The publisher re-verifies the release manifest and hashes, uploads
exactly two Darwin archives plus their two checksum files, and pushes only the
fixed foundation-version, runtime-version, and runtime-latest image tags. It
does not build, test, execute candidate code, modify Git, or accept additional
arguments.
Evidence contains exact commands and results, toolchain/runtime metadata,
changed-path selection reasons, Darwin build and smoke records, image
inspection, generated-runtime and selected conditional-check logs, SBOM,
vulnerability scan, release manifest, and remote publication checks.
Failure and rerun
The gate is fail-closed and does not reuse a partially populated run directory.
On failure, preserve its evidence for diagnosis. After fixing the release
candidate and creating a new tag/commit, run the container-side release again
to prepare a new run ID. Do not edit an old input/, delete selected evidence,
or resume publication from an unverified partial run.
The owner must review the entry point before its first real invocation. Any
later change to the gate or publisher requires a fresh owner review before use.
3 - Version-line porting
Version-line porting
aibox maintains the v0.x and v1.x lines in parallel. A fix landing on either
line must be reconciled with the other line when applicable.
The release gate compares both maintained branches after the recorded
enforcement baselines in .github/version-line-port-baselines.toml. Every
non-merge source commit must have a matching port on the target line or an
explicit not-applicable disposition. This derives obligations from Git history,
so it does not depend on labels, manually created issues, or workflow tokens.
When the equivalent change lands on the other line, add this commit trailer:
Version-Line-Port: ported-from=<full source commit SHA>
The target line’s release gate recognizes and settles the matching obligation.
When a change genuinely cannot or should not cross lines, document the reason
in the commit body and add:
Version-Line-Port: not-applicable
Use not-applicable only for line-specific version metadata, generated release
artifacts, or code that does not exist on the other line. Do not use it to defer
an applicable fix.
Before publishing, run:
scripts/check-version-line-ports.sh check v0
scripts/check-version-line-ports.sh check v1
The release workflow automatically runs the gate for the major version being
published.