Reports and verification

The signed JSON report is what a CertiStack run produces and what an auditor keeps. This page explains where reports go, what is in them, how they are signed, how to manage the signing keys, and how to verify a report, with CertiStack or without it.

Where a report goes

run saves one report per run on the controller, at

<output directory>/<plan_id>/<report_id>.json

The output directory is /var/lib/certistack/reports unless you set --output or CERTISTACK_OUTPUT_DIR. run prints the exact path when it finishes, on a line that contains Controller report saved to:. A failed validation still produces a signed report; a run that never reached the point of creating evidence (for example, an SSH failure, or a node busy with another operation) produces none.

Report files are mode 0600 and directories CertiStack creates are 0700. CertiStack never edits, rotates or deletes a report. The plan's compliance.retention_days is recorded in each report for whoever keeps them, but it does not prune anything: retention is your job. Keep a report together with the public key that verifies it; that pair is what an auditor needs.

Do not edit a report. Changing any byte, even whitespace inside a string, invalidates the signature. To correct a mistake, run the validation again.

The Community Edition emits and verifies signed JSON and nothing more. Rendering a report as an HTML compliance binder or a PDF certificate is an Enterprise Edition feature, done from this same JSON.

What a report contains

A report is one JSON object. Field names are stable within a schema version; report_schema_version names it (currently 1.5).

Top level

Field	Meaning
`report_schema_version`	The report schema. Verifiers ignore fields they do not know, so a newer report still verifies.
`report_id`	A UUID that names the report and its file.
`timestamp`	When the report was built, in RFC 3339 UTC.
`plan_id`, `plan_name`, `environment`	From the plan.
`pbs_fingerprint`, `pbs_namespace`	From the plan's `storage` section, when set there.
`validation_mode`	`full_integrity` for a normal run. `test_skip_integrity` marks the lab-only shortcut that skipped the mapped-image scan; such a report is not production evidence.
`node_fqdn`	The PVE node that ran the worker.
`engine_version`, `git_commit`, `build_date`	The controller build that produced the report.
`frameworks`, `retention_days`	From the plan's `compliance` section. Framework labels declare scope; they are not a certification.
`vm_records`	One record per sandbox VM; see below.
`phase_timings`	Where the run's time went, in seconds: admission, SDN setup, PBS mapping, overlay creation, guest network preparation, VM start, QMP wait, integrity verification, startup grace, probes, soak, teardown, and the total.
`teardown_evidence`	Proof of cleanup; see below.
`rto_seconds`	The recovery time objective achieved. For a pass, time from the start of the run to the last verified probe, excluding teardown. For a failure, elapsed time to the terminal failure.
`all_passed`	The verdict. `true` only when every VM and probe passed and cleanup was verified. A recovery that passed but could not prove its cleanup is reported as `false`.
`failure_reason`	A short, redacted explanation when the run failed before any VM-level record exists (admission, sandbox, mapping or cleanup failures).
`failure_artifact`	A base64 PNG screenshot of the failing VM's display, when one was captured.
`public_key`, `signer_fingerprint`, `signature`	The signing block; see How a report is signed.

`vm_records[]`

Field	Meaning
`vmid`, `source_vmid`, `name`	The sandbox VM, the protected source VM, and the name from the plan.
`verification_status`	`pending`, `booted`, `verified`, `failed` or `incomplete`. `incomplete` means the VM booted but the run ended before its probes finished, so an absent probe can never read as a pass.
`all_passed`	Whether every probe of this VM passed.
`boot_duration`, `boot_duration_seconds`	From the start request to hypervisor readiness (the first in nanoseconds).
`source_snapshot`	The exact PBS snapshot that was restored, with `latest` already resolved.
`source_disks[]`	Per restored disk: `slot`, `archive`, `manifest_reference`, `integrity_status` (`verified`, or `skipped_test_mode`), `image_digest` (SHA-256 of the whole mapped image), and `boot_source` (`local-copy` when the disk was copied before boot).
`source_crypt_mode`, `source_signature_verified`	The weakest PBS crypt mode among the VM's disks (`encrypt`, `sign-only`, `none`), and whether the configured key verified the snapshot manifest.
`source_consistency`, `source_consistency_detail`	How consistent the backup was: `quiesced`, `crash-consistent`, `powered-off` or `unknown`. Read from the backup's own log, which the PBS manifest signature does not cover.
`copied_before_boot`	`true` when the disks were copied to local storage before boot.
`omitted_source_disks`, `excluded_source_disks`, `omitted_nics`	What the restored VM did not get, so a partial restore is never presented as the whole VM.
`restored_hardware`	The virtual hardware the sandbox VM ran with, and whether it came from the backup configuration.
`network_recovery_strategy`, `network_recovery_address`, `network_recovery_seconds`, `network_recovery_files`, `network_recovery_diagnostic`	What guest network recovery did on the disposable overlay, and a live diagnostic when a wire probe failed and `capture_diagnostics` was on.
`probe_results[]`	One entry per probe; see below.
`failure_artifact`	A screenshot for this VM, when it failed.

`probe_results[]`

Field	Meaning
`type`, `target`, `description`	The probe as written in the plan. A VM that failed to boot, or was interrupted, carries a synthetic `recovery` or `validation` entry explaining why.
`origin`	Where the assertion ran: `worker_host` for TCP, HTTP, DNS, LDAP, SQL Server and SMB probes (worker-to-guest reachability through the isolated VNet), `guest_qga` for guest-agent probes (inside the guest), or `hypervisor_qmp` for QMP and screendump probes.
`passed`, `skipped`	`skipped` marks an optional guest-agent probe whose agent was not available.
`duration`	How long the probe took, in nanoseconds.
`detail`, `error`	A response summary, or why the probe failed.

`teardown_evidence`

Field	Meaning
`vm_stopped_and_destroyed`	The sandbox VMs are gone.
`overlay_files_removed`, `loop_devices_unmapped`	The COW overlays removed and the PBS loop devices detached.
`sdn_provisioning`, `sdn_zone_removed`, `sdn_vnet_removed`, `sdn_zone_validated`, `sdn_vnet_validated`, `sdn_cluster_reloaded`	For an ephemeral sandbox, the zone and VNet removed. For a `preprovisioned` one, the zone and VNet that were validated and retained.
`orphan_sweep_clean`	Keeps its historical name. It records that every resource this run recorded as owned was cleaned; it does not mean a host-wide scan happened. `verify-report` labels it "Owned Resources Clean".
`all_cleaned`	Every resource the run created is verified gone.
`journal_recovery_verified`	The controller independently checked the worker's durable journal instead of trusting the worker's claim.
`cleanup_errors`, `initial_cleanup_errors`	What could not be removed, and what a retry had to fix.
`verified_at`	When cleanup was verified.

How a report is signed

Signing happens on the controller, never on the PVE node. The worker returns unsigned evidence; the controller checks the teardown itself and then signs.

The report, without its signature, public_key and signer_fingerprint members, is serialised as canonical JSON under RFC 8785 (JCS): keys sorted, no insignificant whitespace, numbers in their shortest form.
That byte string is signed with the controller's Ed25519 private key.
The hex-encoded signature, the hex-encoded public key, and the key's fingerprint (SHA256: followed by the hex SHA-256 of the raw public key bytes) are written into the report.

Every other member, including ones a future schema adds, is covered by the signature. A report whose JSON repeats a member name at any depth is rejected, because a duplicate could carry a value the signature never covered.

The embedded public_key only says which key claims to have signed. It is never trusted by itself: verification succeeds only against a key you supply.

Keys

Create a key pair

sudo install -d -m 0700 -o "$USER" /secure/certistack
certistack keygen --private /secure/certistack/controller-signing.ed25519 \
  --public /secure/certistack/trusted-signers.pub

The private key stays on the controller. Point the plan's compliance.sign_key_path, or --key or CERTISTACK_SIGN_KEY, at it.

Key file formats

File	Format
Private key	`keygen` writes 128 hexadecimal characters (the 64-byte Ed25519 private key) with mode `0600`. CertiStack also reads a 64-hex-character seed, a PKCS#8 PEM file, or raw bytes. It refuses a key that group or other users can read, a symbolic link, and anything that is not a regular file.
Public key file	One line of 64 hexadecimal characters. `verify-report --public-key` reads it, and it is also a valid one-key keyring.
Keyring	Either a JSON file or a plain list.

A JSON keyring names the people or machines whose reports you trust. The name and node appear in the verification output:

{
  "trusted_signers": [
    {
      "name": "controller-01",
      "public_key": "c7c886ebaa4d7722003b19d141d622d635607097e838e9d1c88fe87424ba76a6",
      "node": "pve-node-01",
      "created_at": "2026-09-30T00:00:00Z"
    }
  ]
}

name and public_key are the fields that matter; fingerprint is computed when absent, and node and created_at are informational. A plain keyring is a text file with one hex public key per line; blank lines and lines starting with # are ignored.

Protect and rotate keys

Keep the private key on the controller only, on encrypted, backed-up storage. Anyone who holds it can sign a report that verifies.
Keep the public keys apart from the reports they verify, in a place the auditor trusts (a signed repository, an internal wiki page that only you can edit, a ticket).
To rotate, create a new pair, add the new public key to the keyring, and point the controller at the new private key. Keep the old public key in the keyring for as long as reports signed with it must verify.
Do not run keygen --force over a key that signed reports you still need: unless you kept its public half, those reports can no longer be verified.
If a private key may have leaked, stop using it, create a new pair, and tell your auditors which reports the old key signed and until when.

Verify a report

certistack verify-report --keyring /secure/certistack/trusted-signers.pub \
  /var/lib/certistack/reports/minimal-single-vm/85a66c2e-439d-4d16-8b78-854395951f5d.json

Give exactly one trust source:

Flag	Trusts
`--keyring <file>`	Any key in a JSON or plain keyring.
`--public-key <file>`	The single key in that file.
`--trusted-key <hex>`	The 64-character hex key you type.

verify-report refuses to run without one, and never looks for keys next to the report or in the working directory.

✓ Audit Certificate Signature VALID
  Trust Source: keyring:/secure/certistack/trusted-signers.pub
✓ Signer identity verified against trusted keyring
✓ Signer Identity: TRUSTED ("controller-01")
  Signer Node: pve-node-01
  Signer Key:  c7c886ebaa4d7722003b19d141d622d635607097e838e9d1c88fe87424ba76a6
  Fingerprint: SHA256:6d364dc4f05ae622f480d2bf20a4b6535e7129731e4b7358bec66ebb91ff0794
  Engine:      v0.1.0 (commit: abcdef123456)
  Schema:      1.5
  Report ID:   85a66c2e-439d-4d16-8b78-854395951f5d
  Plan ID:     minimal-single-vm
  Plan Name:   Minimal single-VM recovery check
  Node:        pve-node-01
  Timestamp:   2026-09-30T22:09:14Z
  Frameworks:  [DORA-Article-12]
  RTO:         42.50 seconds
  All Passed:  true
  Validation:  full_integrity

  Phase Timings Breakdown:
    ...
  Teardown & Safety Evidence:
    - VM Destroyed:          true
    - Overlays Removed:      1 disk(s)
    - Loops Unmapped:        1 device(s)
    - SDN Sandbox Purged:    Zone csSbMin1 / VNet csVnMin1
    - Owned Resources Clean: true
    - All Cleaned:           true (at 2026-09-27T03:10:00Z)

Exit status

Result	Exit
The signature is valid and the signer is trusted	`0`
Unsigned report, invalid or tampered signature, untrusted signer, unreadable or malformed file, no trust source, or a test-mode report without `--allow-test-mode`	`2`

A valid signature does not mean the recovery passed. A correctly signed report of a failed validation verifies with exit status 0 and shows All Passed: false. Always read all_passed and teardown_evidence.all_cleaned, directly or through the JSON summary below.

A machine-readable summary

--format json prints one line of JSON with no credentials, screenshots or guest output, so automation can decide without parsing the human text:

{"report_id":"85a66c2e-439d-4d16-8b78-854395951f5d","plan_id":"minimal-single-vm","validation_mode":"full_integrity","trust_source":"keyring:/secure/certistack/trusted-signers.pub","signature_valid":true,"all_passed":true,"all_cleaned":true,"vm_count":1,"failed_probe_count":0,"incomplete_vm_count":0}

Field	Meaning
`signature_valid`	Always `true` when the command prints a summary; an invalid signature is an error instead.
`all_passed`	The signed verdict. Forced to `false` for a test-mode report you did not allow.
`all_cleaned`	The teardown evidence's `all_cleaned`.
`validation_mode`, `trust_source`	How the run was validated and which trust source was used.
`vm_count`, `failed_probe_count`, `incomplete_vm_count`	Counts across the report.
`failed_probes`	Present only when a probe failed: each with `vmid`, `type`, `target` and `description`.

A failed recovery looks like this, and still exits 0:

{"report_id":"50c88f6e-1195-4419-9106-5de23a30d1bb","plan_id":"minimal-single-vm","validation_mode":"full_integrity","trust_source":"keyring:/secure/certistack/trusted-signers.pub","signature_valid":true,"all_passed":false,"all_cleaned":true,"vm_count":1,"failed_probe_count":1,"incomplete_vm_count":0,"failed_probes":[{"vmid":9001,"type":"tcp","target":"192.0.2.10:22","description":"SSH listener"}]}

all_cleaned: true with all_passed: false is the normal shape of a clean negative result: the recovery failed, and the sandbox was removed. If all_cleaned is false, see Recover from an interrupted run.

Test-mode reports

A report made with the lab-only --skip-integrity switch has validation_mode: test_skip_integrity. verify-report still checks its signature but refuses it as production evidence: it prints a warning, reports all_passed as false and exits 2, unless you pass --allow-test-mode to review it as lab evidence.

Verify without CertiStack

An auditor does not need the CertiStack binary. The signature is a standard Ed25519 signature over an RFC 8785 canonical form, so any toolchain that has both can check it. This script uses Python with the cryptography and rfc8785 packages. The trusted public key is an argument, taken from your own records, never from the report.

python3 -m pip install cryptography rfc8785
python3 verify_report.py report.json c7c886ebaa4d7722003b19d141d622d635607097e838e9d1c88fe87424ba76a6

#!/usr/bin/env python3
"""verify_report.py report.json TRUSTED_PUBLIC_KEY_HEX"""
import json
import sys

import rfc8785
from cryptography.exceptions import InvalidSignature
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PublicKey


def reject_duplicates(pairs):
    names = [name for name, _ in pairs]
    if len(names) != len(set(names)):
        sys.exit("REJECT: a JSON object repeats a member name")
    return dict(pairs)


report_path, trusted_key_hex = sys.argv[1], sys.argv[2].strip().lower()

try:
    with open(report_path, "rb") as handle:
        report = json.load(handle, object_pairs_hook=reject_duplicates)
except ValueError as error:
    sys.exit(f"REJECT: not valid JSON: {error}")

# Trust comes from your own key, never from the report's embedded public_key.
if str(report.get("public_key", "")).lower() != trusted_key_hex:
    sys.exit("REJECT: the report was not signed by the trusted key")
if not report.get("signature"):
    sys.exit("REJECT: the report is unsigned")

signature = bytes.fromhex(report.pop("signature"))
report.pop("public_key")
report.pop("signer_fingerprint", None)

try:
    Ed25519PublicKey.from_public_bytes(bytes.fromhex(trusted_key_hex)).verify(
        signature, rfc8785.dumps(report)
    )
except InvalidSignature:
    sys.exit("REJECT: the signature does not match the report contents")

print("OK: signature valid")
print("all_passed:", report["all_passed"])
print("all_cleaned:", report.get("teardown_evidence", {}).get("all_cleaned"))

Use a real RFC 8785 implementation. Serialising with json.dumps(sort_keys=True) looks equivalent but is not: it writes the float 1.5e-7 as 1.5e-07, which changes the bytes and fails verification of a perfectly good report.

What a valid signature tells you

A valid signature from a key you trust tells you that the report is exactly what that key's holder signed, and so that nothing in it was altered since. It does not tell you that the recovery is good enough for your business, that a framework label in the report has been met, or that the signer's controller was not misconfigured. Framework labels declare scope; they are not a certification. What a run proves, and what it does not, is bounded by the supported recovery contract.

Two things in a report are worth reading every time: validation_mode must be full_integrity, and teardown_evidence.all_cleaned must be true.