Open Athena

In a new essay, David Hall @dlwh.bsky.social argues that pretraining consumes documents, while posttraining consumes tasks. Open language models could run on Wikipedia and ArXiv. Posttraining has no equivalent commons. David reflects on the choice we get to make: openathena.ai/blog/there-i...

Before ChatGPT, there was AlphaFold winning CASP in 2020. In our latest Meet Our Team, MTS Tim O'Donnell (@timodonnell.bsky.social) talks MarinFold: distilling structure prediction into a vanilla LLM architecture, built fully in the open. Plus: what a single drop of blood remembers. 🔗 bit.ly/oa-tim

Marin's 535B-parameter hero run was originally planned as a 360B model. A new blog from Rafal Wojdyla, MTS, explains how a change in sharding strategy, Expert Parallelism, made room for a ~50% larger model at the same ~23B active parameters, and what it costs. 🔗 openathena.ai/blog/expert-...

We’re excited to announce that we've been selected as one of ARIA's new Activation Partners. We’re looking forward to helping researchers use frontier AI capabilities on the scientific questions that matter most. openathena.ai/blog/open-at...

We've only been observing the ocean directly for a few decades, and mostly at the surface. However, climate projections need to cover decades or centuries. Closing that gap is a machine learning problem, and it's the one Jesse Rusak @jder.bsky.social works on. Meet Jesse, MTS: bit.ly/oa-jesse

18.71 billion documents. 25.25 trillion tokens. 152 data sources, deduplicated, decontaminated, and sorted into 200 buckets before a single training run. A new blog from Will Held @williamheld.com on Datakit, the Marin team's pretraining data pipeline: openathena.ai/blog/marin-d...

A year ago, Marin had one FTE. Recently, the ten-person team launched a 535 billion parameter MoE model. In the Marin 535B-A23B launch note, David Hall @dlwh.bsky.social notes it's going "almost boringly well." Thank you to our colleagues, partners, and advisors for making this possible. bit.ly/535b

With the generous support of The Jen-Hsun and Lori Huang Foundation, we have launched the largest live-streamed pretraining run in history, and Marin’s largest model yet: a fully open source 535B total parameter MoE model, with 18T tokens of data. Learn more at: openathena.ai/blog/huang-f...

Alex Merose @al.merose.com was headed into design until a friend showed him a demo: audio windows shrinking until noise resolved into sound, tuned by ear—a preprocessing step for a brain-computer interface. He is now MTS at OA, working primarily on Samudra with NYU & MIT. Meet Alex: bit.ly/oa-alex

We are thrilled to collaborate with Cornell's Jingjing Zhai, Edgar Marroquin, Matt Pennell, Ed Buckler, and team on PlantCAD2, recently published in Cell Genomics. OA staff members @eczech0.bsky.social and Betsy Cannon are paper co-authors and contributors. Read at: www.cell.com/cell-genomic...

A decade ago, progress in NLP meant encoding a language's structure into the model. @williamheld.com, MTA at OA, came up in that tradition and now works on Marin, our open LLM. Read about the bitter lesson, open development tradeoffs, and language roots & quirks at www.openathena.ai/blog/meet-ou...

Excited to share MarinDNA, a 1B gLM that rivals Evo 2 40B on variant effect prediction while being 2,330x faster. With @eczech0.bsky.social, we built around a standard Transformer so we could reuse LLM infra and methods while focusing on data curation and scaling. openathena.ai/blog/marin-d... 🧵

Simulating ocean climate takes a supercomputer 4,600+ CPU cores to produce 12 simulated years per day (SYPD). Samudra 2 produces 4,800 SYPD on 1 GPU at the same resolution. In a new blog, @al.merose.com reports on Samudra, a neural ocean emulator built in collaboration with NYU & MIT: bit.ly/oa-ss

David Siegel, founder and chairman of Open Athena, writes in Fortune on the importance of open source. David likens AI to the libraries of the future, just as important for learning and dissemination of knowledge as libraries have been. Read the full article at: openathena.ai/blog/david-s...

Research software engineers used to be "builders of pipes." Now those pipes are generated on demand, and the work shifts to figuring out which ideas are worth exploring. VP of Engineering Yael Elmatad discusses the RSE role in the age of agentic tooling in a new blog: openathena.ai/blog/researc...

Betsy Cannon is a member of technical staff, where she leads material science projects RHOAR-Net, with Princeton's Rosen Research Group, and MarinMat. Read her story on learning DFT, how AI changes shared research infrastructure, and what pottery taught her about building by hand: bit.ly/oa-betsy

In 16th-century Italy, mathematicians kept their discoveries secret, saving them to deploy as weapons in public "math duels" for prestige and patronage. In a new blog post, David Hall argues that frontier AI has its own culture of secrecy that we must address. openathena.ai/blog/open-de...

In a new blog, Russell Power explains how the Marin team nearly doubled its sustained TPU usage by creating a custom global scheduler: Iris. Iris searches every region where Marin has compute, places each job wherever capacity appears, and moves data along as needed. 🔗 openathena.ai/blog/cluster...

At a recent panel on driving the technological future forward, our COO and CSO Jared Crooks explained the importance of keeping AI development open, why ethics should be embedded in tech, and more. Watch or read the discussion at the link