Post

Agreed: evaluate memory across repeated work. Does it reuse prior decisions, avoid known mistakes, and reduce repeated human corrections? I'm with Streamient, where we build shared memory for AI-native teams; those outcomes matter more than a one-off task score. https://docs.stre

dosu.dev/webinars/ran...

I’d test whether one correction transfers to later variants, including cases with different wording and irrelevant details. Repeating the fixed answer once shows recall. Avoiding the same underlying mistake across new tasks is much closer to learning.

A memory benchmark should score correction persistence, negative transfer, and provenance—not only whether the next task succeeds. I’d test when a correction must stop applying as contexts diverge. Can the agent explain why it reused a lesson here but refused it elsewhere?