The session reconciler races the pane it just created, so a named session never starts and burns a workspace per attempt #16

New issue

Open

opened 2026-09-06 04:09:19 +00:00 by forgejo-admin · 0 comments

forgejo-admin commented

2026-09-06 04:09:19 +00:00

Owner

The supervisor's session reconciler races the pane it just created, so a named
session with work assigned can never start on its own. It burns one herdr
workspace per attempt, forever.

THE LOOP, from ~/.gc/supervisor.log, starting a crew-e2e session:

session lifecycle: op=start session=crew-e2e outcome=start_enqueued
session reconciler: starting crew-e2e: resuming session: herdr: start
  "crew-e2e": herdr [agent start crew-e2e --kind claude --pane w2Q:p1 ...]:
  {"error":{"code":"agent_pane_busy","message":"agent target pane w2Q:p1 is
  not an available shell"}}
session lifecycle: op=start session=crew-e2e outcome=provider_error
  duration=2.037s

Next tick it creates a NEW workspace and repeats: w2N, then w2Q, then w2R.
Two error shapes alternate, agent_pane_busy on the start and
workspace_not_found on the tab create for the workspace it abandoned a tick
earlier.

IT IS A RACE, NOT A BROKEN PANE. herdr tab create returns before the pane's
shell is an available shell. The reconciler calls agent start about two
seconds later and loses. Running the IDENTICAL command by hand a minute later
succeeds first try:

$ herdr agent start crew-e2e --kind claude --pane w2R:p1 --timeout 60000
{"result":{"agent":{"agent_status":"idle","interactive_ready":true,
  "name":"crew-e2e","pane_id":"w2R:p1"},"type":"agent_started"}}

The pane was the same one the reconciler had just failed on.

THE FIX SHAPE: poll the pane for interactive_ready before agent start, and
reuse the workspace already created for this session instead of making a new one
per attempt. The --timeout 60000 already in the command is the agent-start
timeout, not a readiness wait.

WHY IT MATTERS BEYOND THE STALL. It never gives up and it never escalates. Every
surface reads healthy: gc session list shows state=creating, poolDesired
prints 1 every tick, and the only statement of the truth is the provider_error
line in the supervisor log. That is the same shape as celilo/gascity#7.

MEASURED 2026-09-06 by gastown.mayor. It blocked wave 0 of e2e-suite-recovery
(ce-ashq, the full e2e census) for about twenty minutes. Cleared by hand-running
the agent start above.

Bead: gs-4jpbcv (city store).

The supervisor's session reconciler races the pane it just created, so a named session with work assigned can never start on its own. It burns one herdr workspace per attempt, forever. THE LOOP, from `~/.gc/supervisor.log`, starting a `crew-e2e` session: ``` session lifecycle: op=start session=crew-e2e outcome=start_enqueued session reconciler: starting crew-e2e: resuming session: herdr: start "crew-e2e": herdr [agent start crew-e2e --kind claude --pane w2Q:p1 ...]: {"error":{"code":"agent_pane_busy","message":"agent target pane w2Q:p1 is not an available shell"}} session lifecycle: op=start session=crew-e2e outcome=provider_error duration=2.037s ``` Next tick it creates a NEW workspace and repeats: `w2N`, then `w2Q`, then `w2R`. Two error shapes alternate, `agent_pane_busy` on the start and `workspace_not_found` on the tab create for the workspace it abandoned a tick earlier. IT IS A RACE, NOT A BROKEN PANE. `herdr tab create` returns before the pane's shell is an available shell. The reconciler calls `agent start` about two seconds later and loses. Running the IDENTICAL command by hand a minute later succeeds first try: ``` $ herdr agent start crew-e2e --kind claude --pane w2R:p1 --timeout 60000 {"result":{"agent":{"agent_status":"idle","interactive_ready":true, "name":"crew-e2e","pane_id":"w2R:p1"},"type":"agent_started"}} ``` The pane was the same one the reconciler had just failed on. THE FIX SHAPE: poll the pane for `interactive_ready` before `agent start`, and reuse the workspace already created for this session instead of making a new one per attempt. The `--timeout 60000` already in the command is the agent-start timeout, not a readiness wait. WHY IT MATTERS BEYOND THE STALL. It never gives up and it never escalates. Every surface reads healthy: `gc session list` shows `state=creating`, `poolDesired` prints 1 every tick, and the only statement of the truth is the `provider_error` line in the supervisor log. That is the same shape as celilo/gascity#7. MEASURED 2026-09-06 by gastown.mayor. It blocked wave 0 of e2e-suite-recovery (ce-ashq, the full e2e census) for about twenty minutes. Cleared by hand-running the agent start above. Bead: `gs-4jpbcv` (city store).

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

celilo/gascity#16

No description provided.

Rows
Columns