Post

We launched some updates to the Discover feed today. This likely won't affect existing users as much, but for new users it's a big improvement. Basically: the ML team managed to build an improved topic classifier, which means Discover is better at responding to selected interests.

The classifier uses BLIP2 embeddings with training on labels which we manually created (yep). The ML team has the exact numbers but anecdotally it seems to have jumped from 40%ish accuracy to more like 90%. I'm surprised at how much more consistent it is.

This affects new users more because they haven't followed or liked anything yet. As you do both, you get more specific postes. When you're new, we don't know much about you, so we use these broad topic classifiers.

4 reposts104 likes

There's still a lot to be done, but I've promised to share details as we develop -- both for users and for other devs -- and I'm gonna deliver on that promise! I think once we feel like we're at our total quality targets, we'll share more detailed writeups on how it works

Classification isn't always cut-and-dried, there's subjectivity - which leads to bias. On Twitter back in the day, topics like "news", "politics", and "tech" were mostly guys and mostly white. Posts by and women and people of color were more likely to be considered "entertainment".

I’d really like to stop seeing so many posts in Discover that I already saw in my following feed from both people I already follow and posts people I’m following are reposting. Please.

If you go to Explore you should have a tool at the top for editing your interests. (If you dismissed it, then check somewhere in your settings I forget where) The interests are 1:1 to the topics right now

Ah, I see. So you rely on those classifications to match content to users for cold-start. How about established users? Do you not use the classifications and instead use the raw content embeddings and assign them to clusters?

Just curious why we can’t upload our own animated gifs, and like… have them animate. Can’t be a technical thing, y’all are too smart for that. So maybe a storage volume issue, or something contractual with Giphy that’s limiting implementation? 🤷🏼‍♀️ Just idle curiosity.