Post

Jeremy Howard
@howard.fm

I'll get straight to the point. We trained 2 new models. Like BERT, but modern. ModernBERT. Not some hypey GenAI thing, but a proper workhorse model, for retrieval, classification, etc. Real practical stuff. It's much faster, more accurate, longer context, and more useful. 🧵

Jeremy Howard
@howard.fm

We haven't compared it directly, although generally OpenAI's embeddings seem to trail the better open models, so I expect it would be an improvement.

1 repost3 likes

Phillip Carter
@phillipcarter.dev

Great! Looking forward to trying this out sometime. Since we're an AWS shop we'll probably consolidate on Bedrock, and although it has an embedding service, I'd love to use that opportunity to pick something that's actually best for our needs.

Phillip Carter
@phillipcarter.dev

That said, I really do have to hand it to OpenAI for making their models easy to use + comprehensive enough to the point where we could use them in production since Spring of last year, with there still being few replacements for their offerings