Post

Victor Geislinger
@victorsvector.com

Thanks for sharing & it's super interesting to see from someone not at the 'median'! And not to discount what you found but it seems this might be more about the folks who use these tools frequently. And maybe that's worth exploring by itself since more people are using it like you (writing code)

Victor Geislinger
@victorsvector.com

The one that got me was when you said your "median session" was ~150x a "typical query" and then your "typical day" used equivalent of ~4500 "typical queries" Does that mean you have ~30 sessions per day? Let's say they take a conservative 5 minutes of wait per session— that's ~2.5hrs of waiting?

0 reposts1 like

Victor Geislinger
@victorsvector.com

For the record, I totally can see that. And I can imagine the energy usage being a concern if high usage like this becomes more typical & models remain at this efficiency But I'm honestly not sure if it makes a huge difference in overall use (say it's only <1% of all power usage by these models)

Simon P. Couch
@simonpcouch.com

Yep, something like 30. The "2.5hrs of waiting" is where the 2-3 agents at a time comes in, multitasking between agents, but I also wonder whether subagents working in parallel within a session might also drive that "actual time spent waiting" down?

Victor Geislinger
@victorsvector.com

Cool, thanks! Yeah that'd make sense with 'parallel agents'. There is of course the time between waiting for a response on itself ('reasoning/thinking') It makes me think of its usage as 'how much total time & energy used in a day' versus the median or average session.

Victor Geislinger
@victorsvector.com

(Again thanks for sharing! Seeing actual measured stats on something like this is refreshing! When I was going through, I was hoping for even more measurements— I would've loved some std dev on your usage! But even what you shared is more than awesome— always love me some data & actual numbers lol)