Craig Messner

After a long wait, releasing the SYNTH paper! It’s not pretraining, mid-training or post-training, it’s just training: a fully synthetic single-stage pipeline to train workable reasoning models with unprecedented data efficiency. arxiv.org/abs/2609.378...

Unforeseen but somewhat useful consequence of having a newborn: I'm back on eastern time

Models like Talkie-1930 sound like voices from the past. If they could reliably speak from specified historical vantage points, researchers might also use them to simulate the past. But how reliable are they? Today we release a benchmark answering that question for English contexts 1831-1930.

@404media.co wrote about our new preprint on tell-tale signs of AI-generated stories! Cc @dmimno.bsky.social Paper: arxiv.org/abs/2605.26492 Article: www.404media.co/elias-thorne...

was reading this and enjoying it — and then enjoyed it more when it turned out to be a paper by Sil Hamilton and @dmimno.bsky.social

For fans of Talkie-1930 and all the methodological questions raised by historical models, here's a new entrant into the field, TypewriterLM, trained up to 1913. Corpus, instruction-tuning datasets, and event dataset are released. arxiv.org/abs/2606.02991

excellent analysis of Talkie resobscura.substack.com/p/are-vintag...

Announcing Talkie: a new, open-weight historical LLM! We trained and finetuned a 13B model on a newly-curated dataset of only pre-1930 data. Try it below! with @alecrad.bsky.social and @nicklevin01.bsky.social

At this point tack a sign to my forehead that reads "no, LLMs aren't complete black boxes"

“Pretraining Language Models for Diachronic Linguistic Change Discovery” by @tom-lippincott.bsky.social, @cmessner.bsky.social, & more shows that efficient pretraining techniques produce useful models over corpora too large for easy manual inspection and too small for “typical” LLM approaches: (3/5)

Any leads on research that examines what disjoint exists between human-recoverable and SAE recoverable features out there? (Perhaps after: arxiv.org/html/2506.15..., which features are "naturally" distinguishable by humans but represented densely by SAEs, which end up in "noisy" dense reps.)

Anyone have on instruction tuning for models trained solely on historical data? Turns out texts from 1750 have very few "reddit-like" constructs.