Capacity: how many agents run at once
Fleet runs as many dispatched workers as the machines can carry (#559). The limit is machine load over time, not a fixed count of workers.
The dispatcher
Each pass (every minute), for each node that reports:
- Room is a 15-minute load per vCPU under the threshold, more than 20% of memory available, a
free turn slot (
max_running), and a node that takes work (its card shows no "busy"). - The threshold is
dispatch.max_load_per_cpufor a fixed box (default 0.8) anddispatch.rent_load_per_cpufor a rented one (default: autoscale's--rent-load-per-cpu, 2). - Each node with room takes one new worker a pass. The 15-minute load moves slowly, so the pass after sees only part of the load the last worker added.
- A dispatched worker that is still queued (no node took it yet) uses that room first.
- When no node has room and autoscale may rent another machine under its
--max-spend, the dispatcher starts one worker. That worker waits in the queue, and autoscale rents for it. - An inference profile must also have room. When every profile the workers can use is limited
(on any window) or at its
max_concurrent, pace or usage threshold, the tickets wait. dispatch.max_workers(default 40) is a hard safety ceiling only.
A ticket that waits shows why on its Work card. The server logs a line for each pass that has a ticket waiting:
dispatch: load room=1 rent=false running=7 queued=0 ceiling=40 waiting=3
nodes="elastic-1: load15 0.18/vCPU of 2, mem 81% free, 5/64 turns: room; node-b: load15 0.49/vCPU of 0.8, mem 40% free, 1/1 turns: full; ..."
dispatch.limit_by: max_workers turns the load rule off: then only max_workers limits.
Autoscale
Autoscale rents another machine when every rented machine has been full for --rent-after.
A machine is full when its 5-minute load reaches --rent-load-per-cpu times its vCPUs (or
--rent-load), every slot is taken, or its node refuses work for memory or disk. A node that
refuses work only for its 1-minute load is not full: a short spike does not start the clock.
The live value of --rent-after is 5m. Idle machines are deleted as before. The spend
ceiling (--max-spend) is not changed by any of this.
History
The Machines page shows each machine's 15-minute load per vCPU for the last 24 hours, and a
fleet-wide panel of agents running against fleet load. GET /v1/machines/{id}/history?since=
serves the points: one a minute, kept in memory, so a server restart starts it empty.