Post

Raphaël Millière
@raphaelmilliere.com

Very cool idea! Some quick thoughts. It looks like the corruption preserves a lot of information (function words, morphology, word order, punctuation, numbers, register) which would strongly constrains the posterior over plausible discourse frames as it were. 1/3

Gary Lupyan@glupyan.bsky.social

arxiv.org/abs/2601.11432 I want to share an astonishing result. LLMs can "translate" Jabberwocky' texts like 'He dwushed a ghanc zawk” & even and even 'In the BLANK BLANK, BLANK BLANK has BLANK over any BLANK BLANK’s BLANK' This has profound consequence for thinking about.. 1/2

arxiv.orgarxiv.org
Raphaël Millière
@raphaelmilliere.com

The LLM has to do something like schema-conditioned infilling: produce a high-probability member of the equivalence class consistent with those constraints. So I'm not sure how unexpected the results are? That's roughly what I'd expect from matching to structurally similar training passages. 2/3

0 reposts9 likes

Raphaël Millière
@raphaelmilliere.com

The main takeaway for me is that structural information in language is far more constraining than intuition suggests. That's very interesting (and I agree that parrot metaphors are misleading) but it seems like a claim about language more than intelligence. 3/3

Roman Feiman
@romanfeiman.bsky.social

Agreed with all of this. One thing LLMs have uncontroversially taught us about language is that "colorless green ideas sleep furiously" really is (surprisingly!) predictably better than ungrammatical variants purely on the basis of (higher-order) correlations between words.