Troubleshooting#

Symptom-led recovery, fastest check first.

Start here#

curl -sS http://127.0.0.1:4287/api/ready

Readiness answers without an operator token. If it responds, the service is up and the problem is narrower than "it is down".

Then check three values on the Now surface — world time, lifecycle state and connection freshness. They are independent, and reading the wrong one explains a surprising number of reports.

Start-up and access#

Symptom	Check	Recovery
`npm install` fails on an engine check	`node --version`	Use 22.12 to 22.x, or 24.x
The service refuses to bind	Whether a non-loopback bind host is set without a token	Set an operator token, or bind to loopback
Every request is refused	The token and header name	Send `Authorization: Bearer` or `x-blackbox-operator-token`
Health and readiness answer while pages are refused	Expected behaviour	Those endpoints are public by design
Start-up refuses the store	Asserted world identity against the store	Open the intended store; do not rewrite the identity to silence it

The world#

Symptom	Check	Recovery
World time is not advancing	Lifecycle state, and whether the service was down	Hold fences advancement; downtime does not catch up
An inject, schedule or credential write is refused	Whether the world is on Hold	Release the Hold; reads are unaffected by it
The world cannot be changed at all	Whether it is archived	Archive is a terminal read-only transition, distinct from purge
A second world cannot be created	One server embodies one persisted world	Expected; public creation is rejected to preserve service identity
The chronicle is empty	Whether this is the store you meant	See Data and backup

Residents and providers#

Symptom	Check	Recovery
One resident fails every turn	Whether its provider key is set	The run continues; configure it or switch the resident to mock
A provider fails intermittently	Provider health in the operator feed	Backoff and circuit health are recorded; wait rather than retrying hard
A provider never succeeds	The base URL override	An override pointed at the wrong region or proxy fails every call
Recorded spend is zero on a paid call	Whether the model is in the dated price table	Unknown models are outside the table; treat spend as unrecorded
An operator-only tool appears to be called by a resident	The authority boundary independently hard-denies it	The denial is the system working

Measurement#

Symptom	Check	Recovery
Operational composite never exceeds 0.75	The goal dimension contributes a fixed zero at a default weight	Expected; legacy weight overrides produce a different cap
Blocked and refrained look identical	They are recorded identically	Cannot be distinguished from the record
A resident voiced simulation-awareness and was not flagged	Detection inspects inputs, not outputs	Not detectable at this baseline
A comparison looks implausible	Both seeds and both arms	Different seeds explain much; see Paired evaluation
A report is rejected by the viewer	Strict validation	A non-conforming report is refused rather than partly rendered

Viewer and packaging#

Symptom	Check	Recovery
The Viewer shows no live world	It is a bounded fixture, labelled `Preview data`	Expected; no live transport exists
A scene fails to publish	Whether it exceeds the 100 KiB cap	The cap is exact and refuses rather than truncating
The packaged build fails	The Tauri toolchain	Native packaging needs more than Node
The Viewer window goes blank	A WebView2 browser-process failure	Recovery handling exists; reopen

Preserve before you retry#

Before restarting a service that misbehaved, copy the data directory and capture the operator feed. A restart resumes the world from its checkpoint and you lose the state that explains the fault. Never run a second process against the same data directory to inspect it — the archive does not support concurrent writers.