I have spent enough time watching AI agents work to know their dirty secret: most of an agent's life is not thinking. It is waiting. It waits for a build to finish, for a test suite to run, for a pull request to get reviewed, for a human to approve something. And while it waits, in most agent frameworks, it sits there holding a process, holding memory, and — if you are paying cloud prices — holding your money hostage.
That is why AWS's announcement on October 23, 2026 stopped me mid-scroll. AWS and UC Berkeley have open-sourced Infinity, a research runtime that treats waiting as a solved problem instead of a cost centre. The headline number is wild: 75,000 concurrent agents in the 8 GB of memory of a Raspberry Pi 5. I will be honest with you about what that number really means later in this article, because the honest version is still impressive — just in a different way than the headline suggests.
The problem every agent builder runs into
Let me describe how most agent harnesses work today, because Infinity only makes sense once you see the flaw it fixes.
The standard agent loop is a sequence of blocking steps. The harness calls the model, the model decides to call a tool, the harness invokes the tool, and then it waits. When the tool returns, the loop repeats.
It falls apart the moment agents do real work. A coding agent kicks off a build that takes twenty minutes. A triage agent watches a repository and waits hours for the next pull request. An assistant drafts something and waits until the next morning for a human approval. During all of that waiting, the agent's process stays alive. It holds its memory, its connections, and its place in the queue. It cannot respond to new events that arrive in the meantime, because it is stuck mid-step.
Infinity starts from a different question: what if an agent that is waiting simply stopped existing for a while?
What Infinity actually does
Infinity re-imagines the agent runtime using an old programming idea: suspend and resume, the same technique behind asynchronous functions and coroutines. When an agent calls a tool, Infinity dispatches the call, saves the conversation, and suspends the agent. Nothing keeps running. When the tool result arrives, the runtime resumes the agent from its saved state, right where it left off.
That is the whole trick, and it is a powerful one. A suspended agent has no task, no stack, and no live connection attached to it. What remains is its conversation history — the record of what has happened so far — which is far smaller than a running process.
Infinity is released under the Apache 2.0 open-source licence as part of the Hydro project, a collaboration between AWS and UC Berkeley. Apache 2.0 means you can actually use it in commercial projects without licence anxiety. But a university collaboration also signals what this is: a research project exploring a better architecture, not a polished AWS managed service with an SLA. Keep that distinction in mind.
The 75,000-agent number, read honestly
Now the headline. AWS says Infinity can fit 75,000 concurrent agents in 8 GB of memory — the memory of a Raspberry Pi 5. That sentence has travelled everywhere, and it deserves a slower read than it has been getting.
Here is what the benchmark actually ran, based on AWS's own description and independent analysis of it. AWS ran 80,000 agents on a single thread of a server-class AMD EPYC processor, using in-memory stores and a simulated model. Each agent ran twenty turns: a streamed completion of about 1.2 KB that calls a tool, the tool result coming back through a queue, and a second completion to close the turn. Resident memory grew linearly at roughly 100 KB per idle agent, most of it conversation history, because a suspended agent carries nothing else.
So the honest reading is this: the benchmark measures how cheaply Infinity can remember a suspended agent under those test conditions — about 100 KB each. Divide 8 GB by that rate and you get the Raspberry Pi line. The arithmetic is correct. What it measures is narrower than the headline implies: it is not 75,000 simultaneous model inferences, and it was not run on a Raspberry Pi.
Does that make the number fake? No. Real agents accumulate bigger histories: tool outputs, file contents, diffs, error logs. If an agent's history grows to 1 MB, that same 8 GB holds roughly 8,000 agents, not 75,000 — still a large number, and still far beyond what a process-per-agent design could do on the same hardware. The rate is the real story, not the headcount.
Why the serverless angle changes the bill
The second half of Infinity's pitch is, for me, the more practical one. Infinity is designed to run on standard serverless platforms. When an agent suspends during a tool call, the serverless function can simply end. If the tool call takes hours — or days — nothing needs to run while the agent waits. AWS's reference setup uses Lambda with SQS and Aurora DSQL, but the principle is platform-shaped, not AWS-shaped: waiting costs nothing because nothing is running.
Think about what that unlocks. A triage agent that watches a repository for days costs you nothing between events. An approval workflow where a human takes a weekend to reply does not burn compute all weekend.
For WordPress and web developers, this pattern is familiar in disguise. We already build this way on the web: a PHP request runs, responds, and dies; state lives in the database, not in a process. Infinity applies the same idea to agents. Your agent becomes a state machine with a conversation log, not a pet process you keep alive and worry about.
Agents talking to agents
Infinity includes a few design details that show its authors have thought about real multi-agent systems, not just benchmarks.
First, agent-to-agent communication is built in through two tools: spawn_thread and report_to_parent. A parent agent can spin up child agents for subtasks and hear back from them, which is how most serious agent systems actually end up structured — one coordinator, many workers — even when the framework pretends otherwise.
Second, events in child threads are processed in those threads by default, which keeps the parent's context window from bloating with every detail of every subtask. Context is the most expensive thing an agent consumes; keeping it lean is a cost control and a quality control at the same time.
Third, Infinity uses synthetic tool calls to deliver asynchronous events within the constraints of model APIs. Models expect a turn-taking rhythm — call a tool, get a result — and the real world does not cooperate with that rhythm. Wrapping an incoming event as a synthetic tool call is a neat bridge between the model's world and the real world's world.
What this means for my own work
I am not going to pretend I am migrating anything to Infinity this week. It is a research project, and research projects come with rough edges: sparse documentation, changing APIs, and no one to call when something breaks. But it changes how I think about the next agent system I design, in three concrete ways.
First, I will design agents as suspendable from the start. Even if my runtime does not suspend them yet, keeping agent state in a durable conversation log — rather than trapped in a live process — makes everything downstream easier: retries, debugging, audits, and handoffs between systems.
Second, I will budget agents by idle cost, not active cost. The expensive question is no longer "how much does an agent cost when it works?" but "how much does it cost when it waits?"
Third, I will stop treating long waits as failures. Timeouts exist in most agent systems because waiting is expensive, so we kill agents that wait too long. If waiting becomes free, a three-day approval flow is just a normal Tuesday.
The honest limits
A fair article has to say what Infinity is not, too.
It is a research runtime, not a managed service. There is no Infinity console, no uptime SLA, and no promise that the API you learn this month survives next year. The benchmark used a simulated model, so it says nothing about inference throughput or how Infinity behaves under real model latency and rate limits. State storage is not free either: durable storage for millions of suspended agents, plus the queues and databases to resume them reliably, is a real system to operate.
None of that is a criticism. It is the normal shape of a research release. The right response is to experiment with it on a side project, not to bet a production system on it.
The bigger pattern: everyone is fixing idle agents this week
Infinity did not arrive in a vacuum. In the same week, Google announced Agent Compute Engine, a service that gives each AI agent its own hardware-isolated microVM and pauses idle microVMs via checkpoint and restore, resuming them from snapshots. Different mechanism — full microVMs instead of suspended coroutines — same diagnosis: an agent that is waiting should not be burning resources.
When AWS and Google independently converge on the same problem in the same week, it is a signal worth hearing. The first wave of agent infrastructure optimised for getting agents to work at all. The second wave is optimising for what agents spend most of their time doing: waiting.
Conclusion
Infinity's headline — 75,000 agents in a Raspberry Pi's worth of memory — is an extrapolation, and I have tried to be straight with you about that. But do not let the debunking bury the achievement. A runtime that suspends agents during tool calls, resumes them from saved conversation, runs on serverless platforms where waiting costs nothing, and does it all as Apache 2.0 open source is a meaningful step. It turns the most common agent state — waiting — from a liability into a rounding error.
Frequently asked questions
What is AWS Infinity?
Infinity is an open-source agent runtime released by AWS and UC Berkeley under the Apache 2.0 licence as part of the Hydro project. It suspends agents during tool calls and resumes them when results arrive, instead of keeping a process running while the agent waits.
How many agents can Infinity actually run?
In AWS's benchmark, 80,000 agents ran on a single thread of a server-class AMD EPYC processor with a simulated model, using about 100 KB of memory per idle agent. The "75,000 agents on a Raspberry Pi 5" figure is an extrapolation from that rate to 8 GB of memory, not a test run on a Pi.
Does Infinity work on serverless platforms?
Yes. When an agent suspends, the serverless function can end, so nothing runs — and nothing is billed for compute — while the agent waits, even if the wait lasts days. AWS's reference setup uses Lambda with SQS and Aurora DSQL.
Is Infinity ready for production use?
Not yet in the way a managed AWS service would be. It is a research project: promising architecture, but young tooling, no SLA, and benchmarks based on a simulated model rather than real production traffic. Experiment with it; do not bet a production system on it today.
How is Infinity different from Google's Agent Compute Engine?
Both tackle the same problem — idle agents wasting resources — but differently. Infinity suspends agents to a stored conversation inside a lightweight runtime, while Google's Agent Compute Engine gives each agent a hardware-isolated microVM and pauses it via checkpoint and restore.
Should WordPress and web developers care about Infinity?
Yes, as a design influence even before you use it directly. The suspend-and-resume pattern — state in a durable log, nothing running while waiting — is the same idea the web already uses, and it is how agent features like approval flows and repository watchers become affordable to run.
