Tom McCoy

🤖🧠NEW PAPER🧠🤖 What is the relationship between neural networks & symbolic systems? Based on evidence from mechanistic interpretability, we argue for a quasi-symbolic view of cognition: Neural nets implement symbols but only approximately Link: www.sciencedirect.com/science/arti... 1/n

For much more, read the paper, in Current Opinion in Behavioral Sciences! www.sciencedirect.com/science/arti... Due to space constraints, we couldn't fit all of the discussion and citations that we ideally would have - very sorry to the many excellent papers we had to leave out! 9/n

🤖🧠NEW PAPER🧠🤖 What is the relationship between neural networks & symbolic systems? Based on evidence from mechanistic interpretability, we argue for a quasi-symbolic view of cognition: Neural nets implement symbols but only approximately Link: www.sciencedirect.com/science/arti... 1/n

Properties that are enabled by being quasi-symbolic rather than precisely symbolic include: 1. Gradient similarity 2. Graceful degradation 3. Softness of constraints 4. Flexibility in learning 5. Handling of partially-symbolic tasks 7/n

🤖🧠NEW PAPER🧠🤖 What is the relationship between neural networks & symbolic systems? Based on evidence from mechanistic interpretability, we argue for a quasi-symbolic view of cognition: Neural nets implement symbols but only approximately Link: www.sciencedirect.com/science/arti... 1/n

On the side of learning, meta-learning enables neural networks to display the rapid learning that is traditionally characteristic of symbolic systems. This rapid learning is supported by compositional structure in parameter space! 5/n

🤖🧠NEW PAPER🧠🤖 What is the relationship between neural networks & symbolic systems? Based on evidence from mechanistic interpretability, we argue for a quasi-symbolic view of cognition: Neural nets implement symbols but only approximately Link: www.sciencedirect.com/science/arti... 1/n

On the side of representations, meaningful symbolic units can often be found inside the vector representations of neural networks as shown by (e.g.) sparse autoencoders, additive analogies, and Tensor Product Representations 3/n

🤖🧠NEW PAPER🧠🤖 What is the relationship between neural networks & symbolic systems? Based on evidence from mechanistic interpretability, we argue for a quasi-symbolic view of cognition: Neural nets implement symbols but only approximately Link: www.sciencedirect.com/science/arti... 1/n

A very nice writeup from Mike Cummings at Yale News on our work finding symbolic structure inside LLM representations! Read it if you want to learn how the vectors inside LLMs are like pointillist paintings! news.yale.edu/2026/10/08/h...

🤖🧠NEW PAPER🧠🤖 (The result of an 8-year project!) LLMs seem very different from symbolic systems. Yet LLMs excel in symbolic domains (e.g., language/code/math). How do they do it? Our finding: LLM representations have implicit symbolic structure! Link in thread ⬇️ 1/n

These results provide a potential explanation for how neural networks can be so successful in seemingly symbolic domains, potentially reconciling longstanding symbolic conceptions of intelligence with the vector-based nature of modern AI. 11/n

🤖🧠NEW PAPER🧠🤖 (The result of an 8-year project!) LLMs seem very different from symbolic systems. Yet LLMs excel in symbolic domains (e.g., language/code/math). How do they do it? Our finding: LLM representations have implicit symbolic structure! Link in thread ⬇️ 1/n

DISCOVER enables us to edit an LLM's internal representations in targeted ways that are then appropriately reflected in changes in the output. The edits that we make would typically be challenging to do in a vector encoding, but they're straightforward w/ a symbolic encoding 9/n

🤖🧠NEW PAPER🧠🤖 (The result of an 8-year project!) LLMs seem very different from symbolic systems. Yet LLMs excel in symbolic domains (e.g., language/code/math). How do they do it? Our finding: LLM representations have implicit symbolic structure! Link in thread ⬇️ 1/n

Given appropriate choices for the type of symbols used by the TPR, the approximation produces high accuracy in all networks we consider: The network continues to produce the right answer when fed the TPR rather than its own actual encoding. 7/n

🤖🧠NEW PAPER🧠🤖 (The result of an 8-year project!) LLMs seem very different from symbolic systems. Yet LLMs excel in symbolic domains (e.g., language/code/math). How do they do it? Our finding: LLM representations have implicit symbolic structure! Link in thread ⬇️ 1/n

We analyze a network by approximating its representations w/ a TPR. We then feed the TPR approximation into the network & see if it still produces the right answer - replacing the network's entire representation-generating process with a simple, closed-form TPR equation. 5/n

🤖🧠NEW PAPER🧠🤖 (The result of an 8-year project!) LLMs seem very different from symbolic systems. Yet LLMs excel in symbolic domains (e.g., language/code/math). How do they do it? Our finding: LLM representations have implicit symbolic structure! Link in thread ⬇️ 1/n

Our hypothesis: The networks succeed on these tasks by implicitly building symbolic structure in vector space. What makes this hypothesis testable is the Tensor Product Representation (TPR) formalism, a proposal for how symbolic structure could be realized in vectors. 3/n

🤖🧠NEW PAPER🧠🤖 (The result of an 8-year project!) LLMs seem very different from symbolic systems. Yet LLMs excel in symbolic domains (e.g., language/code/math). How do they do it? Our finding: LLM representations have implicit symbolic structure! Link in thread ⬇️ 1/n

Since many are starting grad school soon, let me re-share my One Big Tip™️ for research! Research involves many skills - collaborating, writing, presenting, etc. But many of these skills can be unified under a single overarching ability: theory of mind Blog post link in reply

Speaking of poetry hidden in plain sight, everyone should read Anne Cutler's paper on the subject. Absolutely mind-blowing. (link and quote in reply ⬇️)

Quote summarizing the paper: "For if rhythm is so integral a part of our audition, then it ought to be the case that it is hard to overlook; but the most pronounced of rhythms can escape our recognition when they're reproduced in printing in an article or book."

Just found an Easter egg on Wikipedia! On the page for double dactyls (a type of poem), there's a sentence that's formatted as a normal line of text, but it's actually a stanza of double dactyl poetry! And... THE PLOT THICKENS! 1/2