Debugging a Run
Get a run id, read the status block, tell success from a visible failure, and name the failure class — without guessing from silence.
An agent that cannot diagnose itself burns a loop. A IonWarp run is designed to be reconstructable from one read: it ends in success or in a named visible failure, and there is no third outcome inferred from silence. This page is the one you fetch when a run did not do what you expected.
1. Read the status
Launching a run and polling its status is Tasks — one
tasks_status { run_id } on MCP, or GET /api/v1/tasks/{task_id}/status on
REST, which resolves the task's latest run unless you pass ?run_id=. Poll the
task, not a particular run: a run killed by a deploy is replaced by a new
one, and the task is what survives.
2. What the status block carries
| Field | Read it for |
|---|---|
status | pending · running · completed · failed · canceled — the run's terminal truth |
state, phase, status_text, current_step, progress_pct | the human-readable "where is it" projection |
terminal_event_count | 1 once the run has committed a terminal frame, 0 while it has not |
error_json | the failure envelope: { code, message, run_id, … } — code is the failure class |
warnings | the operator-visible warning lines, falling back to the error message |
retry | { retry_of, retry_count } — whether this run is a replacement for an earlier one |
lifecycle, row_clock, runner_clock | created → queued → dequeued → started → first token, with durations, so a slow run is attributed rather than guessed |
queue_ahead | how many queued runs are in front of this one (queued runs only) |
backend | platform, harness, provider, model, preset — what actually executed |
limits, timeouts, harness | the resolved ceilings the run ran under, and whether a budget fired |
seed_graph | { counts, gaps } — what an onboarding-style run wrote, and the named shortfalls it landed with |
skills | skill provenance — see below |
usage_json | credits and tokens; add ?include=cost for itemized usage_line_items |
live, live_warning | the realtime snapshot, when the run is still streaming |
sweep, callback | why an active run is still spinning, and whether a detached backend's terminal callback arrived |
Check skill provenance FIRST
Before you conclude anything about a run's behavior, check which skill body
actually executed. The skills block lists one entry per executed skill:
{ "skills": [
{ "skill_id": "code-review", "source": "catalog", "executed_hash": "…" }
] }source is catalog (the build-embedded catalog bytes ran) or
project_override (project/workspace bytes ran against a named catalog base,
carried in base_catalog_hash). A run that executed an override is not evidence
about the shipped skill — judge it against the override, or re-run without one.
Skipping this step is how a prompt "bug" turns out to be a customised skill.
Success versus visible failure
Success means the run committed a terminal frame: the final message is
persisted, reasoning is closed, every tool call landed in output/error/denied,
and the declared counts are present. status is completed and
terminal_event_count is 1.
Visible failure means the run named what was missing: status is failed
and error_json.code is one of the classes below. There is no "recovered",
"inferred", or "probably fine" outcome — if a run cannot be reconstructed, that
is itself the bug, and the fix is a missing signal, not a guess.
A completed run may still carry named gaps — seed_graph.gaps, a warning
line, an integration gap note. A shortfall is reported on success; only an
absence of work fails a run.
The failure classes
These are the values error_json.code can take. They are stable strings, safe
to branch on.
| Group | Codes |
|---|---|
| Provider | provider_transient, provider_terminal, provider_rate_limited, provider_auth_failed, provider_error_after_progress, model_not_available |
| Tools & instructions | tool_timeout, tool_failed, instruction_timeout, missing_expected_tool, payload_contract_violation |
| Queue & liveness | queue_pickup_timeout, run_liveness_timeout, run_never_started, run_evicted_by_deploy, run_invocation_died, run_invocation_ceiling, run_cpu_ceiling, run_continuation_limit, run_continuation_no_queue, run_recovery_unwritable |
| Backend | backend_unavailable, backend_callback_never_arrived, backend_wall_clock_timeout, stream_transport_failed |
| Control plane | project_paused, project_run_mode_unreadable, project_awaiting_approval, project_approval_unreadable, spend_cap_reached, budget_exceeded |
| Setup & safety | skill_instance_not_found, task_auto_paused_repeat_failure, unknown |
run_evicted_by_deploy and run_invocation_died are the platform replacing the
invocation underneath a live run; both terminalize the dead attempt and start
one replacement, linked by retry.retry_of. The control-plane codes are
explained in Control Plane.
3. Read the transcript
The status read tells you the run stopped; the transcript tells you why it decided what it decided.
{ "name": "tasks_messages", "arguments": { "task_id": "ts_..." } }curl "$DASH_BASE_URL/api/v1/tasks/ts_.../messages" \
-H "Authorization: Bearer pat_..."The transcript carries the visible note that names the error_code and its
remedy, alongside the model's own messages and tool calls. Between the status
block and the transcript there is no run state a caller has to infer.
See also Tasks for the lifecycle, Reference / Errors for the REST error envelope, and Usage for what the run cost.