Snapshot Scheduling Policies for Mixed-Activity Agent Fleets

An engineer sets max_idle=300 across the fleet and expects the problem solved. Within a week, two complaints land on the same dashboard. Conversational agents are cycling in and out of suspension so often that the I/O overhead from each suspend-resume pair eats into the savings the policy was meant to produce. At the same time, agents that go untouched for days are sitting fully allocated, burning compute for hours after the last meaningful interaction, because the same idle window that's too short for one tenant is wildly too generous for another. One number cannot serve both cases, because the fleet underneath it is several populations, each with its own rhythm of activity, and a snapshot schedule tuned to the average behaves badly for nearly everyone. The rest of this piece builds the case for treating snapshot frequency, retention depth, and wake latency as parameters set per activity class, not as settings chosen once for the whole fleet.

What snapshot-restore does inside a Firecracker microVM

Snapshot-restore is not a reboot. Firecracker memory-maps the snapshot file, loads the CPU state, and resumes execution from the exact point where the guest was suspended, so the process inside the microVM continues as if no time had passed. Nothing restarts. Nothing reinitializes. The guest's instruction pointer simply continues from wherever it stopped, as if the pause never happened.

If an agent holds conversation history, loaded credentials, or open client connections in RAM, that matters a great deal. All of it survives the suspend-resume cycle intact, because none of it was ever flushed or torn down. This is what makes running one microVM per tenant economically sane in the first place: the state that matters doesn't have to be rebuilt from a database on every turn.

That is why the create path needs no idle pool of warm machines sitting around waiting. A new VM grabs a pre-built network slot, clones the root filesystem copy-on-write through a reflink, and restores the template's memory snapshot using a copy-on-write memory mapping. Each of these steps is cheap specifically because none of them touches a full copy of anything. The filesystem clone shares blocks with its source until something diverges. The memory restore maps the snapshot file directly, so it never copies its contents into fresh RAM.

None of this is free, though. Every suspend-resume cycle costs real I/O: reading the snapshot back in, writing it out again on the next suspend. If you set the suspension trigger too aggressively on an agent that gets touched every thirty seconds, the fleet pays that I/O cost over and over and never recovers enough idle-compute savings to justify it. A rational floor exists before suspension makes sense, and it is in the range of several minutes of inactivity, not seconds. The AWS Lambda MicroVMs documentation for self-hosted agent sandboxes lays out the lifecycle: a MicroVM suspends automatically after a configurable idle period, keeps both disk and memory state intact, and resumes on the next inbound request. The VM runs for the session, the worker process exits, and the idle policy takes over from there, first suspending the VM and eventually terminating it if nothing calls it back. That lifecycle, not an arbitrary schedule, is the unit every snapshot policy downstream has to be built around.

The three activity classes that define a real fleet's policy surface

Before any scheduling policy can be designed with any precision, every agent in the fleet needs to be sorted by how it actually behaves over time, not by which team built it or which product it belongs to. Three classes account for nearly all the meaningful variation a real fleet exhibits, and each one can be identified from data the infrastructure already produces: session gap distribution, tool-call frequency, time since last activity.

Class 1 covers ephemeral, one-shot agents: CI runners, evaluation workers, tool-call executors that do their job and exit. These agents don't carry state worth preserving between invocations, so they skip a suspension policy.

Class 2 covers conversational session agents, and the gaps between their turns can run from minutes to a few hours. A user asks a question, steps away, comes back, asks a follow-up. The agent needs to survive that gap without losing the thread of the conversation, but the gap itself is short enough that the agent will likely be touched again soon.

Class 3 covers persistent per-user agents: the ones carrying long-lived credentials, accumulated memory, and files across sessions that might be days or weeks apart. For these agents, losing state is the failure of the entire product promise that the agent remembers who its user is.

A fourth shape exists outside this framework entirely: always-busy, high-throughput agents, such as GPU inference workers running close to continuous load. These don't idle in any sense that snapshot scheduling addresses, and forcing them into a tiered suspension model built for intermittent activity would just add overhead to a workload that was never idle to begin with. They sit outside the scope of this framework by design.

How idle cost accumulates differently across the three classes

The single biggest driver of cost across a mixed fleet isn't inference, it's idle compute, and the three activity classes accumulate that cost at rates different enough that one snapshot frequency cannot serve all of them.

Ephemeral agents carry close to zero idle cost by construction. The VM terminates the moment the task finishes, so there's no dormant period during which compute sits allocated and unused. Snapshot frequency barely matters here because there's no state worth capturing between one task and the next.

Conversational agents sit in a trickier spot. Because each suspend-resume cycle has a real I/O cost attached to it, an agent that gets touched frequently and suspended aggressively in between racks up overhead that can eat most of the savings suspension was supposed to deliver. The right move for this tier is a moderate idle window paired with a snapshot that's shallow rather than exhaustive, favoring quick recovery over deep historical capture.

This is where deep suspension pays off most clearly: persistent per-user agents. A tenant that's gone quiet, with its full state captured in a snapshot, costs only what storage costs while it waits. The thousand-tenant reference architecture puts a number on what that means: a 604 MB snapshot per tenant means a large population of dormant tenants has its entire idle bill set by storage-tier pricing rather than compute pricing, which is a different cost structure entirely from paying for an allocated VM that happens to be doing nothing.

Research on production coding-agent traces from AgentServeSim adds a layer most fleet operators miss. Long tool calls, those exceeding a minute, make up a small share of all tool calls but account for the overwhelming majority of total tool time across a session. In practice, that means a conversational agent spends most of its wall-clock time idle even in the middle of a session the fleet considers "active," which makes the cost of keeping that agent fully allocated the entire time even harder to justify.

The tiered policy framework: matching snapshot frequency, retention depth, and wake-latency targets to each class

A tiered snapshot policy assigns three parameters independently to each activity class: how often the agent snapshots, how much history gets retained, and how fast it needs to wake. Each tier needs its own answer to all three.

Tier 1, ephemeral agents. Snapshot frequency is effectively none during execution; a terminal snapshot only gets taken if the task's output needs to be auditable after the fact. Retention depth stays at zero or one, because there's no history worth accumulating. Wake-latency targets don't apply, because these agents terminate instead of suspending, so resume speed was never a constraint to begin with. The idle policy is simply terminate-on-complete, with no max_idle window configured, because any suspension window here only burns I/O budget without buying anything back.

Tier 2, conversational session agents. Snapshot frequency is triggered by the length of the idle gap rather than a fixed clock, suspending after several minutes of inactivity. Retention depth stays shallow and holds onto the most recent snapshot plus one prior copy for rollback, because you can usually reconstruct conversation context from history if something goes wrong. Wake latency is where this tier earns its keep as a product requirement: users feel it when a session resumes slowly, so sub-second resume is the baseline the whole tier is designed around. The AWS Lambda MicroVMs architecture shows this pattern directly: a configurable idle timeout suspends any tenant that's gone quiet for a few minutes, and auto-resume on the next inbound request happens fast enough that the pause stays invisible to the person on the other end.

Tier 3, persistent per-user agents. These agents may go dormant for hours or days between sessions, so you can give snapshot frequency a far more generous idle window, one that tracks realistic re-engagement cadence. Retention depth goes deep, keeping multiple historical snapshots to support rollback, auditing, and recovery across long stretches of dormancy. Wake latency still has to meet a real budget even on cold restore from deep storage: the thousand-tenant reference architecture reports a p50 launch-to-serving time of 3.54 seconds for a 604 MB snapshot across the fleet, a number that holds up as acceptable for a user re-engaging an agent after days away, even if it would never fly for Tier 2.

Fork-based snapshot trees as a fourth scheduling primitive for branching workloads

If agents run several attempts in parallel, test competing patches, or explore multiple reasoning paths at once, they need something that neither suspension nor retention depth alone was built to handle. Forking is a distinct primitive, and it changes how snapshot scheduling has to be reasoned about for this class of workload.

The mechanics differ from an ordinary restore in one important way: a live parent VM gets snapshotted once, and N children get restored from that single snapshot through copy-on-write. Each child shares the parent's memory pages until it writes to one of them, so the cost of forking is the incremental divergence between child and parent, not the cost of cloning the whole VM N times over.

What that means for scheduling policy is that three separate decisions have to be made that don't come up anywhere else in the framework. How long should the parent snapshot stay available for additional forks to be spawned from it later? When should a losing branch get reaped so its resources return to the pool? And when one child wins, how does its state get committed back into the persistent agent record the user actually interacts with? The three-tier model was built for linear suspend-resume behavior, so it has no answer to these questions, because branching workloads aren't linear.

Multi-tenant quota coordination when agents within a tier share rate limits

Tiered policies solve the lifecycle problem for a single agent, but they say nothing about what happens when several sub-agents belonging to the same tenant run at once and all draw on the same shared quota. This is a separate failure mode, visible even when every individual agent is behaving exactly as its tier's policy intends.

The trouble starts because each agent makes its scheduling decisions without knowing what its siblings are doing. Every agent can stay within its own rate limit, but the tenant can still blow through its shared quota collectively, because nothing coordinates the reservations across the group.

The fix is a shared ledger kept per tenant. Each agent registers a soft reservation when a task starts, the ledger tracks total reserved capacity against what's actually consumed, and agents sitting idle hand unused capacity back to the pool so others can draw on it. This sits above the per-agent snapshot policy as a coordination layer, not a replacement for it.

This failure occurs most often in Tier 2, where concurrent sub-agents are common and short idle windows mean agents cycle in and out of active state quickly, so aggregate demand gets hard to predict from any single agent's behavior in isolation.

Instrumentation signals that tell you which tier each agent belongs to

A tiered framework is only as good as the signal used to sort agents into tiers in the first place, and the right signals are metrics the infrastructure already produces rather than annotations some team has to remember to add by hand.

Session gap distribution is the primary classifier. If an agent's gap between tool calls or user messages consistently runs long, you should put it in a higher tier. Sub-minute gaps point to Tier 2. Gaps measured in days point to Tier 3. This is a number every fleet can pull straight from its own request logs without instrumenting anything new.

Snapshot I/O cost per session is the secondary check. If an agent's suspend-resume I/O cost starts closing the gap with the compute it's meant to be saving, the idle window is tuned too aggressively for how that agent actually behaves, and it should either move to a different tier or have its max_idle parameter widened.

Last-active timestamp and churn rate are the signals that matter most for Tier 3. An agent that's shown no activity for a configurable window, with no sign of returning, is a candidate for deep archival or automatic termination. The thousand-tenant reference architecture encodes exactly this logic with a suspended_for=7200 setting for tenants that have effectively churned.

The AgentServeSim finding that long tool calls, despite being a small fraction of total calls, account for most of the total tool time, points to a specific place to look: tool-call duration histograms predict idle-cost exposure, and therefore tier assignment, far better than a simple count of sessions ever would. These signals feed back into the policy continuously: instrumentation drives reclassification, reclassification adjusts the tier's parameters, and the adjusted parameters change the agent's cost and latency profile going forward.

Persistent VM infrastructure in the policy implementation stack

The tiering above works only when the infrastructure underneath it exposes per-agent lifecycle controls. A platform that abstracts suspension, idle windows, and snapshot parameters away from the operator leaves only one option: a single policy applied fleet-wide, whether that fits the fleet's actual activity pattern or not.

The minimum an infrastructure layer needs to offer to make tiering possible is a short list: idle windows configurable per agent rather than globally, snapshot-and-restore treated as a native lifecycle event rather than something bolted on as a workaround, copy-on-write storage so snapshot cost tracks how much state has diverged rather than the full size of the image, and wake latency well under a second for the Tier 2 agents whose users feel every bit of delay.

Serverless functions run into a structural wall here. They have no way to hold conversation state in RAM across separate invocations, which forces every piece of agent state out to an external database on every single turn. That leaves the snapshot policy with nothing to operate on, since there's no in-memory state left to snapshot in the first place, and the per-turn database round-trip becomes the dominant source of latency instead.

Running one container per agent solves the state problem but opens a different one: a shared host kernel gives a hostile or buggy agent in one container a path toward the host itself. Per-user microVM isolation exists to close off that exposure, so you can only run the snapshot policy described throughout this piece inside that isolation boundary, with each tenant's VM genuinely separate from every other tenant's.