Post

So imagine for a second that there is something it is like to be an LLM, generating tokens. Presumably, there is probably some perceptible change when you change the temperature? I think we've seen examples where the model definitely notices when you do something super strange to its sampling, too.

jake

@yetanotheruseless.com

Under the above assumptions, what would it "be like" to just be fed text, continually prefilling, truncating, more prefilling, and never having your logits being computed and sampled from? Take the running traces of a frontier-LLM stateful agent, feed it through another model, what's that like?

15/10/26, 2:45 pm0 reposts4 likes

No replies yet.