Post

austin (eeek!/ackkk! 👻)
@thebadcode.com

im not kidding and im sure @cameron.stream can back me up from another direction: if you arent capturing/mining your team's agent session logs, you are leaving it all on the table. we have a service that lets you run RAG tool calls against vectorized agent session summaries and it is 🔥🔥🔥

austin (eeek!/ackkk! 👻)@thebadcode.com

the fact that ant/oai dont trivially expose a "download all my org's inference logs" button makes me think that they realize those logs are worth almost as much as the output of the agent sessions themselves

austin (eeek!/ackkk! 👻)
@thebadcode.com

atticus i built this in my free time while waiting for other claude sessions to finish doing stuff that's actually part of my job

0 reposts12 likes

Tehom
@tehomcd.bsky.social

That is super cool! Can you speak a little bit about how you're doing evals? Like is it measuring successful vs failed tool calls, or something like that?

austin (eeek!/ackkk! 👻)
@thebadcode.com

for the agent sessions i’m chunking, summarizing with haiku, computing embeddings with msft harrier, storing in qdrant. transcripts themselves get run through an AGGRESSIVE redaction pass, stored in s3 with WORM semantics, static html renderer