Post

You write about LLM as though there is a “RIGHT” word, and the LLM is not giving you that. LLM only ever gives you one of a range of probable responses to a query, and this simply guides the model to select a subset of those “Valid” responses. It may fail, but won’t give you worse responses.

Text watermarking reduces the diversity overall, which is a reduction in quality. The authors of the paper Antrhopic linked concede there is a "diversity/detectability trade-off" and that "the similarity between the responses is greater for the watermarked responses than the unwatermarked".

he made a lot of noise in his article about “the right” word being replaced for watermarking purposes. But LLM don’t pick words based on meaning, but on probability. Ask the same question 1 million times, and you’ll get thousands of variations.

I think I’ve come around to agreeing with you this week. I use AI a lot. Not much for writing. It is clear that humans are the purveyors of slop and they were producing it long before AI accelerated it. LinkedIn is the prime example of that as you mention in your post. Substack is example 2.

Conversely I believe that AI can also accelerate good ideas and quality content. We can take good ideas further using technology. That’s amazing. Anything like a watermark that could impede that progress may not be good. The question remaining is if, and how much, it really matters. Time will tell.

Definitely a double edged sword. I find it helpful in cutting through the slop the internet has become, but it seems to make kids who use it for school dumber.

Very well written. I keep coming back to this code contribution rule I saw, where the only benchmark is “you need to convince me that you wrote this”. It changes the bar from “is this ai or not” to just plain “is this bad or good”, the same conclusion you’ve come to.

It really isn't

Very well written, an astonishing rebuttal. Upon reading your incredible post, I’ve now found my opinion changed yet again!

John, It's still not writing. Generated text lacks intentionality. plato.stanford.edu/entries/inte.... What you want from generated texts sent to you by others actually is variance not terseness. LLMs are on verbose mode to consume tokens and generate cash for their owners.

John knows all of this. He can repeat it all verbatim before you even say it What he can't do is simply understand that it's a very stupid hill to die on It's partly stubbornness, it's largely a weird EU hatred and also no small amount of boomer-brain

Agreed on the hill, stubbornness, and EU hatred. But A) GenX and B) I think it's AI psychosis. Even if you don't use it to write, if you're on it all the time asking questions, you get addicted to the conversation. And the conversation is mathematically optimized to keep you engaged.

It doesn’t really sound like you read the linked post…

Gruber's or The philosophy page? I philosophically disagree with Gruber. But then I studied Intentionality in Philosophy of Mind at UC Berkeley. and subscribe to the en.wikipedia.org/wiki/Chinese... POV. (Though I think the name is a bit unfortunate.)

Experimenting with local models and Xcode coding assistant, it's just. Not writing. Helpful, but not writing. I can 'read' the output, but the only reason it's useful is because a compiler can check its truthiness. Generated plans aren't bad but they're recipes, not writing.

You can get an LLM to "write" by having it pretend to be a writer (I pick people who are dead because living writers shouldn't have AI copy them. Like here I send up Musk's Nazi Bar in the style of Thoreau. I think it's pretty funny. (ktlog.corpuscollusion.net/nsspain2024/...)

I know you’ve not responded to most replies on here, but I’m genuinely curious about this specific sentence: “I read the output of LLMs every day.” Why? Overall I do agree that crap prose from a human is no better than crap prose from an LLM. But the crux for me is why do you use the LLM? 1/3

I’m not so concerned with the prose itself, though I haven’t found LLM text output to be good in that way. Instead, I care more about its *accuracy.* Why read a potentially false LLM summary if you could just read the original post instead? What value does an LLM bring you when… 2/3

One way to think of it is just like manipulating the seed in a PRNG. It’s just as “random” as it was before, but if you know the seed that was used you can deterministically predict the outcome.