Chris Olah

Chris Olah

@colah.bsky.social

Reverse engineering neural networks at Anthropic. Previously Distill, OpenAI, Google Brain.Personal account.

7.1K followers9 following41 posts

Chris Olah reposted

Nicholas Grossman
@nicholasgrossman.bsky.social

Political violence is bad. It usually begets more political violence. Celebrating political violence is bad. It usually encourages more political violence, against various targets. Campus shootings are bad. They make everyone on campus less safe. It's bad that what I wrote here is controversial.

View full thread
View full thread
Chris Olah
@colah.bsky.social

Of course, I'd be remiss to not mention that many others have made analogies between work in machine learning and biology -- most notable for us is the "bertology" work, which framed it self as studying the biology of the BERT models.

View full thread
Chris Olah
@colah.bsky.social

But we also think it's important for such "biology" results (which are more foreign in style to machine learning) to be treated as worthy of publication independent of methods work (which looks more similar to normal machine learning).

View full thread
Chris Olah
@colah.bsky.social

Finally, you need to believe that a worthy mode of investigation is empirical (rather than theoretical), and a style of empirical research that's more open to the qualitative than purely quantitative. This evokes biology more than physics.

Chris Olah
@colah.bsky.social

But why did the language come up in our paper title? There was actually a further reason, which is that we wanted to separate our "methods" work and what we called our "biology" work (i.e. the empirical research we did using our method).

View full thread
Chris Olah
@colah.bsky.social

At a basic level, one needs to believe deep learning warrants scientific investigation. This doesn't seem very controversial these days, but note that it's already kind of radical. See eg. Herbert Simon's The Sciences of the Artificial.

View full thread
Chris Olah
@colah.bsky.social

One way in which this is important is that the *types of questions* we're interested in are quite bizarre from a traditional machine learning perspective, but natural under the biological frame.

Chris Olah
@colah.bsky.social

I've written multiple papers characterizing (small sets of) individual neurons. Historically, this hasn't seemed like a worthy topic of a paper in ML – I've had to justify it!

View full thread
Chris Olah
@colah.bsky.social

Stepping back, "physics of neural networks" is a whole area of research. Of course, it isn't physics in a classical sense. It's bringing the methods and style of physics to deep learning. We refer to the "biology" of neural networks in a similar spirit!

Chris Olah
@colah.bsky.social

I think there's a deep way in which the scientific aesthetic of biology is very relevant to deep learning and especially interpretability. Biology is to evolution as interpretability is to gradient descent. bsky.app/profile/cola...

Chris Olah@colah.bsky.social

The elegance of ML is the elegance of biology, not the elegance of math or physics. Simple gradient descent creates mind-boggling structure and behavior, just as evolution creates the awe inspiring complexity of nature. x.com/banburismus_...

x.comx.com
Chris Olah
@colah.bsky.social

My colleagues and I have actually been using "biology" quite heavily as a metaphor and handle for several years now, beyond the title of this paper. There are a lot of reasons I think it's useful!

View full thread
Chris Olah
@colah.bsky.social

(This is a cross-post of one of my favorite old twitter threads: x.com/ch402/status... )

x.comChris Olah on X: "The elegance of ML is the elegance of biology, not the elegance of math or physics. Simple gradient descent creates mind-boggling structure and behavior, just as evolution creates the awe inspiring complexity of nature." / XThe elegance of ML is the elegance of biology, not the elegance of math or physics. Simple gradient descent creates mind-boggling structure and behavior, just as evolution creates the awe inspiring complexity of nature.
View full thread
Chris Olah
@colah.bsky.social

I wish people would spend more time looking at the models we create though. It's like we're launching expeditions with complex equipment to reach more and more remote islands and tall mountains... and the biology stops at measuring the size and weight of the animals we find.

View full thread
Chris Olah
@colah.bsky.social

People often complain that modern ML is throwing GPUs at problems without new research ideas. This is like finding evolution ugly because it's just a simple algorithm run for a very long time.

Chris Olah
@colah.bsky.social

The Anthropic Interpretability Team is planning a virtual Q&A to answer Qs about how we plan to make models safer, the role of the team at Anthropic, where we’re headed, and what it’s like to work here! Please let us know if you’d be interested docs.google.com/forms/d/e/1F...

docs.google.comInterest in Attending a Virtual Q&A with Members of the Anthropic Interpretability TeamWe’re planning on doing a virtual Q&A with members of the Anthropic Interpretability team in the near future. We’ll talk about and answer questions about the role of interpretability in making models ...
View full thread
View full thread
Chris Olah
@colah.bsky.social

How does Haiku generate poetry? It turns out that, after the first line, it generates candidates for the rhyming end of the next line and "writes towards them"!

Chris Olah
@colah.bsky.social

In this example, Haiku is very sneaky. It claims to be solving this question, but is actually working backwards so that it comes to the same answer as the human.

Chris Olah
@colah.bsky.social

How does Claude 3.5 Haiku complete the sentence "Fact: the capital of the state containing Dallas is"? We can mechanistically see that Haiku is assembling two different facts ("Dallas is in Texas" and "the capital of Texas is Austin") rather than directly knowing the answer.

You are up to date.