Post

ModernBERT is officially released on Transformers v4.48.0. You no longer need to install from git to use. If you are plugging ModernBERT into an existing encoder finetuning pipeline, try increasing the learning rate. We've found that ModernBERT tends to prefer a higher LR than older models.

Benjamin Warner

@benjaminwarner.dev

What's ModernBERT? It's a drop-in replacement for existing BERT models, but smarter, faster, and supports longer context. Check out our announcement post for more details: huggingface.co/blog/modernb...

huggingface.coFinally, a Replacement for BERT: Introducing ModernBERTWe’re on a journey to advance and democratize artificial intelligence through open source and open science.

10/1/25, 6:28 pm0 reposts2 likes

No replies yet.