Danny Wood

Danny Wood

@echostatements.bsky.social

AI Research Scientist || PhD in machine learning || Ensembles, probabilistic machine learning, recurrent neural networks || https://echostatements.net

160 followers315 following67 posts

Pinned

Danny Wood
@echostatements.bsky.social

I've added the ability to create Delaunay triangulations to my Voronoi diagram generation Python library. The image generation takes a few seconds but I'm pleased with how easy it is to make pretty and varied outputs with it

Danny Wood
@echostatements.bsky.social

A nice article by Nicholas Carlini on writing great papers. A lot of the advice you can find elsewhere but the sections on collaborations and conclusions both had new insights for me. It also helps that I've really liked the papers of his that I've read nicholas.carlini.com/writing/2026...

nicholas.carlini.com How to win a best paper award (or, an opinionated take on how to do important research) An opinionated perspective on how to do important research that makes a difference (and sometimes win awards).
Danny Wood
@echostatements.bsky.social

A simple command line tool combining a crossword solver and anagram finder. A lot of online crossword solver tools felt unnecessary cumbersome to me, so I've tried to make this as lightweight and simple to use as possible github.com/EchoStatemen...

github.comGitHub - EchoStatements/crossword-tools: Command Line Tools for Solving Cryptic CrosswordsCommand Line Tools for Solving Cryptic Crosswords. Contribute to EchoStatements/crossword-tools development by creating an account on GitHub.

Danny Wood reposted

Katie Mack
@astrokatie.com

Replying to @astrokatie.com

You know that thing where the farther something is, the smaller it looks? That doesn’t quite work in an expanding universe. It’s true for a while but after a certain point, more distant things (of the same size) start to look BIGGER because their light has been traveling since the cosmos was small 🙃

Danny Wood
@echostatements.bsky.social

I've added the ability to create Delaunay triangulations to my Voronoi diagram generation Python library. The image generation takes a few seconds but I'm pleased with how easy it is to make pretty and varied outputs with it

View full thread
Danny Wood
@echostatements.bsky.social

Goodyear: Blimps Unbelievably left-field. In the 20th century, they at least built the blimps themselves, but now their sideline seems to just be being the world's most famous owner of a fleet of airships

Danny Wood
@echostatements.bsky.social

And an honourable mention: Lego: Lego Lego supposedly are the world's largest manufacturer of tyres (at least according to the Guinness*). Sure, tyres aren't their main focus, but that doesn't seem so unusual for a tyre company! * While we're at it: yes, Guinness the brewery/word record authority

Danny Wood
@echostatements.bsky.social

Dunlop: Sports equipment If you're working with rubber anyway, I can see how you get into making tennis balls and how that might spiral into sporting goods more generally Fair enough

Danny Wood
@echostatements.bsky.social

Michelin: Restaurant reviews This clearly got out of hand for them. It started as a more general guide for motorists, (even that is a slightly eccentric thing for a tyre company) but getting your tyres from the world's foremost authority on restaurant quality is odd

Danny Wood
@echostatements.bsky.social

This result was one of the foundational ideas built upon by what has since become by far my most cited paper, so I'm really happy to see a version on arXiv where it can hopefully be preserved for posterity

David Pfau@davidpfau.com

New (sort of) preprint on arXiv today: a generalized bias-variance decomposition for Bregman divergences! arxiv.org/abs/2511.08789

arxiv.orgA Generalized Bias-Variance Decomposition for Bregman DivergencesThe bias-variance decomposition is a central result in statistics and machine learning, but is typically presented only for the squared error. We present a generalization of the bias-variance decompos...
Danny Wood
@echostatements.bsky.social

As David points out, he wasn't the first to discover this result, though he certainly deserves a great deal of credit both for its rediscovery and for (apparently somewhat accidentally) popularising it with a punchy paper/note containing clear and succinct proofs

Danny Wood reposted

Ben Brubaker
@benbenbrubaker.bsky.social

I published a new post on my rarely updated personal blog! It's a sequel of sorts to my Quanta coverage of the Busy Beaver game, focusing on a particularly fearsome Turing machine known by the awesome name Antihydra.

benbrubaker.comWhy Busy Beaver Hunters Fear the AntihydraIn which I explore the biggest barrier in the busy beaver game. What is Antihydra, what is the Collatz conjecture, how are they connected, and what makes them so daunting?
Danny Wood
@echostatements.bsky.social

Since @spmontecarlo.bsky.social shared a post about Crinkled Arcs a couple of weeks ago, I've spent a fair bit of time digging into them This new blog post attempts to collect what I found into an interactive introduction to crinkled arcs and their relationship to Brownian Motion

echostatements.github.ioCrinkled Arcs And Brownian MotionA crinkled arc is a continuous curve that appears as if it is making right-angle turns at every point along its trajectory. Additionally, if you draw a straight line between two recent points and comp...

Danny Wood reposted

Keenan Crane
@keenancrane.bsky.social

“Everyone knows” what an autoencoder is… but there's an important complementary picture missing from most introductory material. In short: we emphasize how autoencoders are implemented—but not always what they represent (and some of the implications of that representation).🧵

Danny Wood
@echostatements.bsky.social

Looking at the code in Hugging Face's library for the new GPT models, I was a bit disappointed by how similar they are to Llama & Mistral models, but there is one cool trick I hadn't seen before: attention sinks. These are a mechanism by which attention heads can say "I don't have anything to add"

View full thread
Danny Wood
@echostatements.bsky.social

Looking at the code in Hugging Face's library for the new GPT models, I was a bit disappointed by how similar they are to Llama & Mistral models, but there is one cool trick I hadn't seen before: attention sinks. These are a mechanism by which attention heads can say "I don't have anything to add"

View full thread
Danny Wood
@echostatements.bsky.social

Looking at the code in Hugging Face's library for the new GPT models, I was a bit disappointed by how similar they are to Llama & Mistral models, but there is one cool trick I hadn't seen before: attention sinks. These are a mechanism by which attention heads can say "I don't have anything to add"

Danny Wood
@echostatements.bsky.social

It looks like they're designed to solve a problem raised in this paper (amongst others), that when an attention head wants to not add additional information, it will give the vast majority of its attention probability mass to non-informative tokens...

arxiv.org/abs/2306.12929

arxiv.orgQuantizable Transformers: Removing Outliers by Helping Attention Heads Do NothingTransformer models have been widely adopted in various domains over the last years, and especially large language models have advanced the field of AI significantly. Due to their size, the capability ...
Danny Wood
@echostatements.bsky.social

But when you have mostly very small values and one or two extremely large ones, this make quantisation much lossier The solution is essentially to add an extra element to the input sequence that the attention heads can "sink" their attention into but is removed from subsequent calculations

Danny Wood
@echostatements.bsky.social

A little etymological fact that I like is that "lemma" and "dilemma" are from the same ancient Greek origin If you translate "lemma" as meaning a proposition, a dilemma is literally having two propositions to consider

Danny Wood
@echostatements.bsky.social

There seems to be a lot of nuance around exactly how to translate "lemma": it can be anything between a premise and or more literally as something like "I take", as in something taken for granted. But "proposition" is nice in that it fits in both contexts