Scheduling Research Behind Idle Agent Cost Optimization
Snapshot-restore scheduling cuts idle agent costs by pausing compute between waits.
Snapshot-restore scheduling for idle AI agents stops compute charges during waits by saving execution state to persistent storage, terminating the live process, and restoring it when work resumes. With durable state and sub-second wake latency, agents can pay for storage while inactive without giving up the continuity that per-user agent products require.
Why idle agents are an architectural problem, not a pricing one
An agent waiting for a tool result still occupies a live process unless its infrastructure explicitly releases it. That distinction makes waiting, rather than reasoning, the dominant cost problem in an always-on fleet with long idle periods. Most of the agent’s wall-clock time can pass without useful computation, while the resources holding it remain allocated. The bill follows the process’s lifetime.
The main sources of waiting include tool call latency, human-approval gates, polling for asynchronous results, and retries after failures. Tool calls can accumulate delays before execution starts, while an approval gate leaves the next step dependent on a human response. The agent can stop reasoning during either wait, but it needs to keep its context ready for use. Holding that context in a live process confuses persistence with continuous execution. As concurrent agent headcount grows, paid idle time grows with it, making this architectural choice especially costly when each user gets a separate agent.
How do you stop paying for compute while agents wait?
A live agent process accrues charges at active-process rates even when it performs no useful work. In an always-on deployment, an allocated GPU instance likewise costs money whether or not the agent is reasoning. The infrastructure must stop holding the billable resource before inactivity becomes a saving.
Across a fleet, this mechanism burns far more than one agent’s idle time suggests. Each wait extends the period during which compute remains assigned to work that cannot yet proceed. A separate source of waste adds to the bill when system prompts, tool definitions, and accumulated state are redundantly sent across model calls. That context overhead increases inference spending, but it is not the cause of the idle infrastructure charge. Stopping that charge requires a way to preserve the agent without preserving its running process.
How snapshot-restore scheduling breaks the live-process billing model
When the agent reaches a waiting point, the harness saves its full execution state, including memory, the call stack, and pending continuations, to durable storage. It then terminates the process, removing the idle compute charge. A tool response, human approval, or timer supplies the signal to restore the saved state. Execution resumes at the point where it paused, with storage carrying the cost of continuity during the wait.
This approach belongs to the broader family of durable execution, but the mechanisms are not interchangeable. Temporal uses deterministic event-history replay, Inngest uses step-based memoization, and Rivet Actors use hibernate-and-wake state persistence. These systems demonstrate production approaches to keeping work durable without requiring uninterrupted execution, rather than proving that every durable workflow uses VM snapshots. Microsoft’s Agent Framework, introduced in October 2025 and updated at BUILD 2026, also supports agents scaling to zero while files, disk state, and session identity persist. Full snapshot-restore applies that principle to the execution environment itself.
Why Firecracker micro-VMs are the right substrate for snapshot-restore at fleet scale
A fleet scheduler needs more than a saved application object. It needs an isolation boundary whose full execution state can be serialized and restored quickly, with little memory overhead per agent. Firecracker provides each micro-VM with its own kernel through KVM hardware virtualization. Its small attack surface, memory density, and production hardening make it the default substrate for a dense, multi-tenant agent fleet. SmolVM offers a cross-platform alternative, whereas Firecracker’s scope is Linux/KVM-only.
Firecracker’s snapshot-restore API supports sub-30ms sandbox creation by restoring a ready-state snapshot. The process captures memory and block device state to local NVMe, while a warm pool supports restore latency below sub-50ms. Low overhead lets more agents fit on bare metal, and fast creation lets the scheduler add capacity when it is needed. For example, Maritime deploys each agent in its own Firecracker micro-VM on bare metal, pairing the scheduler with an isolated environment that can preserve kernel state and user processes. GPU access remains a separate constraint. Firecracker does not currently support direct GPU access through VFIO device passthrough, which is under development but has not been merged into the main release.
The 2026 research on using idle time productively rather than just eliminating it
Suspension stops computation that is not needed, but some idle windows give agents time to prepare for the next step. “IdleSpec: Exploiting Idle Time via Speculative Planning for LLM Agents,” by Choi et al., May 2026, arXiv:2605.22154, explores this at the reasoning and orchestration layer. IdleSpec generates plan candidates while the agent waits, then aggregates them once observations become available. Its progressive and recovery candidates accommodate uncertain observations rather than assuming that a speculative branch will be correct. On GAIA and FRAMES with Gemini-2.5-Flash, it surpasses the vanilla baseline by a meaningful margin in average accuracy while keeping end-to-end latency close to the baseline.
“SpecBox: Speculative Sandbox Scheduling for Efficient LLM Agent Serving,” by Zhang et al., July 2026, arXiv:2607.23933, addresses the sandbox infrastructure layer. The paper’s authors are affiliated with Beihang University, the University of Leeds, and the University of Sydney. SpecBox tackles the tension between resource utilization and interactive tail latency through intent-aware sandbox prewarming, stochastic sandbox prefetching from historical traces, and reuse-aware data transmission. Its prototype cuts P99 end-to-end latency by a large amount compared with on-demand sandbox deployment and uses less memory at its peak than permanently reserved deployments.
The papers are complementary because they prepare different parts of the next agent step. IdleSpec prepares reasoning, while SpecBox prepares tool execution environments. Neither makes speculative work free or establishes that every wait should remain active. Instead, they expand scheduling decisions to include both when to suspend work and when preparation makes a wait productive.
How snapshot-restore scheduling changes the unit economics of per-user agent deployment
With a per-user agent model, each customer or workflow has its own persistent, isolated instance. Individual agents may be busy in bursts, but most of the fleet can remain idle most of the time. Under live-process billing, the operator pays to hold that inactive majority in compute. Under snapshot-restore scheduling, most agents take up only storage while idle, and compute spending goes to the agents doing work. The important unit is no longer simply the number of persistent agents, but how many are actively executing.
Maritime implements this distinction through a harness that snapshots the full micro-VM state to persistent storage when an agent pauses on a tool call or approval gate. It terminates the process to remove compute charges and restores the agent in under a second when the signal arrives. The resulting flat-rate pricing rests on the asymmetry between storage and compute costs during inactivity. Persistent disk, kernel state, and session identity are still there for the agent to use after each sleep cycle. That combination makes the scheduling model an enabling primitive for per-user deployment.
What persistent state across sleep cycles requires from the infrastructure layer
Persistence must cover the working environment. Files, credentials, installed packages, and session context all need to survive the transition from running to sleeping and back. Persistent disk lets an agent accumulate a workspace across sessions. The snapshot must also preserve the execution state needed to continue the interrupted task.
The isolation boundary keeps each user’s persistent environment separate from the others. Giving each agent its own kernel and disk separates both execution and stored state at fleet density. Agent memory may also span short-term in-memory structures and long-term vector-indexed storage for cross-session retrieval. The agent must be able to use both tiers after a sleep cycle, though each has a different role.
A human-approval gate makes these requirements concrete. The agent checkpoints its state, sleeps while paying for storage, and resumes with the human’s decision and its prior context intact. Preserving only the decision would not preserve the work that led to the request. Preserving files without the pending continuation would likewise leave part of the task unresolved.
Sub-second wake latency as a hard product requirement, not a performance goal
The central objection to sleeping agents is that savings could come at the expense of responsiveness. For an interactive product, restoring continuity after a long pause in the interface would undermine the point of a persistent agent. Sub-second wake latency is therefore a product requirement that the infrastructure must meet.
A warm pool helps meet this requirement by keeping pre-restored micro-VMs ready to take on an agent’s state. The scheduler loads the agent’s memory and disk snapshot into that ready environment. Sub-50ms VM restore helps meet the sub-second wake budget, but the two measurements cover different parts of the work. The product depends on the complete wake path.
IdleSpec prepares candidate reasoning without requiring speculation to be correct, while SpecBox prewarms MCP-invoked sandboxes to reduce tool-layer startup delays. These techniques complement fast restore. Together, they show how cost reduction and responsiveness can coexist, closing the main objection to suspending agents between tasks.
Building agent-native products on a scheduling-aware infrastructure layer
Application-layer durable execution through Temporal, Inngest, or similar systems is still a powerful alternative, but developers have to write waiting points to fit the framework’s durability model. VM-level snapshot-restore moves the sleep-wake responsibility beneath the application. In Maritime’s infrastructure model, that split lets developers write agent logic while the platform takes care of the persistent execution environment. The developer-facing result is code deployed to a live API endpoint in a single push, with persistent disk, package installation, browser sessions, and a real kernel.
This separation also gives portable agent logic a stable place to run. Google’s A2A protocol and Anthropic’s MCP, adopted by OpenAI and others, address communication between agents and tools, while the infrastructure carries the agent’s isolated state. Protocol support does not remove the need for cost controls. A runaway loop keeps using compute, so sleep scheduling alone cannot stop it. Budget controls and per-agent caps are necessary counterparts to idle-cost optimization.
The architectural gain is a persistent agent that does not require permanent compute allocation. Builders can give each user a workspace and keep the context of an ongoing task without charging for every idle instance as though it were busy. That is what turns snapshot-restore scheduling from an infrastructure technique into a foundation for agent-native products and economically scalable per-user fleets.
