Post

Not to pick on Sam here, but this is a useful point to highlight some of the impossibility of trust & safety work. Meta actually used to have a program *exactly* like this, BUT it became a central feature of Frances Haugen's Facebook Files as an example of Meta being evil. I'll explain...

Sam G@samthegeek.net

IMO the big platforms should have people who keep an eye on all the big political accounts (regardless of alignment) because having them trip automated blocks is… bad.

67 reposts243 likes

What a lot of people have trouble wrapping their mind around is the sheer scale of the messaging. Facebook has some 3 billion active users. If each of them post just once per day, about 100M posts per day, or over 1100 posts per second!

I’m friends with @mergesort.me who used to work on it at Twitter! I know the platforms used to have these folks but AFAIK they’ve moved away from it…

AI companies could show their usefulness by creating a bot that publishes each social media platforms algorithm logic to the world and monitors for any manipulation of political messages….. that would surely increase my positive feelings towards them….

Publishing the algorithms would make it easier for people to game the algorithms to manipulate political messages. There is an entire industry built up around that as-is, when they have to rely mostly on guesswork.

I am definitely willing to be wrong… BUT my point is that if tech companies want regard from the public then their new solutions should solve some of the downsides they introduced.. And at a minimum transparency and accountability for the harms their tools cause ..

Correct. I worked on Search at Google, especially closely with [anti]Webspam folks, and I remember how often there was a public cry to "release the algorithms!" and so on. That was understandable but so very very very misguided for the exact reason you mentioned.

I am definitely willing to be wrong… BUT my point is that if tech companies want regard from the public then their new solutions should solve some of the downsides they introduced.. And at a minimum transparency and accountability for the harms their tools cause ..

For every difficult issue, there is someone on social media with a solution that is simple, obvious, and wrong.

Content moderation at scale is impossible to do well. One part of that is that first pass moderation is often automated in some form, because the scale is way more than most people can wrap their heads around. Even if you could get it right 99% of the time, a lot of mistakes would be made.

So, Meta launched an "Xcheck" program, which took all the high profile accounts (including the "big political accounts") that Sam mentions, and put them into a special category that required a secondary human screening before taking action on them. At a first pass, this makes some sense...

Meta knew they got really bad press when their automated tripwires pulled down big name accounts. So they added in this secondary feature to kick enforcement actions to human review on those high profile accounts. But, when Frances Haugen leaked this to Jeff Horwitz, guess how it was covered?

It seems like this shouldn't have been a thing whose existence could have been "leaked". That the Oversight Board had to recommend increased transparency after the fact... I mean, human review for some slice of accounts is a reasonable policy. Why not just announce that policy upon implementation?