Post

Tim Hockin
@thock.in

I am happy to share a little thing we have been cooking up: github.com/agent-substr... Problem statement: How can we effectively host a bazillion mostly-idle agents in a regular-sized Kubernetes cluster? This is *demo-grade* software, but it shows some ideas.

github.comGitHub - agent-substrate/substrate: Agent Substrate: the core systemAgent Substrate: the core system. Contribute to agent-substrate/substrate development by creating an account on GitHub.

5 reposts32 likes

Justin Garrison
@justingarrison.com

I watched the video and read the readme, but I'm having a hard time visualizing what's required to run this (especially for "teleporting") Can you draw me a picture of the infrastructure components?

Tim Hockin
@thock.in

Actors: things that can be suspended/resumed Workers: "stem cell" pods that can resume any actor Proxy: Assign inbound requests for a particular actor (think "chat session" = actor) to available workers When an actor goes idle, suspend to storage & unassign. Most agents are idle most of the time.

Tim Hockin
@thock.in

The effective result is: 1) an agent can run on one worker, be suspended, and be resumed on another worker very quickly 2) we can multiplex many agents onto few workers This is not PhD level insight, of course. It's trying to bring together a set of fairly established ideas into a working model.

Justin Garrison
@justingarrison.com

Agent's are idle most of the time but waiting for replies. I'm assuming when the reply returns from an LLM it would be routed to the correct actor (with memory and filesystem context) Where is the snapshot state stored? CRDs? Is there other requirements like RWX storage? It doesn't look like it

Tim Hockin
@thock.in

One of the interesting discussions to have, and a good reason for going open super early is: Is the process snapshot actually important? Or is the filesystem continuity the real thing people care about. Early feedback is mixed.