I could see it function very well as an aid in moderation but not any type of solution like most things with AI is today.

In the case of Lemmy and other defederated social media platforms there’s going to be the usual cost hindrance and then the ethical side of it with excessive electricity usage and training data.

Disregarding that, as most know and everyone should know, AIs are not to be considered reliable or accurate ever. They will falsely flag and give false positives to potential comments and posts and images.

However, having an AI aggregate a list of potential bad comments and posts, then have a user manually checking the results, could help with moderation efficiency. Because how many users actually report comments and posts? How many do mods actually miss out on? There’s a lot of content and limited time.

removed by AI moderator

AI is extremely fallible and often makes mistakes, so no.

Ai moderation would lead to every comment being prompt injection

Moderation is work. Trolls traumatize. Humans powertrip.

Correct.

All this could be resolved via AI.

Incorrect, for all the same reasons that facial recognition in ‘AI’ is unethical. All theftboxes adversarial networks are built by humans, most of whom in the ‘AI’ space come standard-equipped with built in racist biases. You see it all the time in facial recognition algorithms that couldn’t tell the difference between a hundred Black people if you ran 'em all side by side. The same thing would happen with AI moderators; they will more likely than not moderate to right-wing white sensibilities, over-target and powertrip on ethnic minorities, and only really contribute to the general ‘apartheid-supporter’ vibe that most of the western internet has.

tl;dr please stop going to bat for theftboxes and the techbro STEMlords who build them.

“Confidently Incorrect” describes terrible moderators and AI.

AI makes shit up all the time. I use it in my automod to report stuff to me, but never to take actions on its own.

I’m just curious how this would differ from automatic moderating tools we already have. I know moderating actually can be a traumatic job due to stuff like gore and CSEM, but we already have automatic filters in place for that stuff, and things still slip through the cracks. Can we train an AI to recognize it when it hasn’t already been put into a filter? And if so, wouldn’t it hit false positives and require an appeal system, which could still be used to traumatize people?

I’ve read about people being automatically banned by AI for saying something along the lines of “I hate burritos” because it had the word “hate” in it, so the AI judged their comment as hate speech and auto-banned them even though they were talking about food or a videogame character. AI is not very good at reading context and the “A” in “AI” is an important detail here.

It’s absurd to give any amount of power over people, however trivial, to a thing which is incapable of thought.

If you’re referring to the data models we have now (as in, not AGI), it’s a solid no for a whole host of reasons.

As it is, it is not intelligent. It is capable of structuring immense datasets and identifying patterns throughout said datasets, but it is incapable of comprehending them at a conceptual level. Even if it can mimic the verbal patterns of context, nuance, humour, sarcasm, irony and even coded speech, it is not capable of understanding any of them. It is not an intelligence as we know and understand it, it’s just a really, really complex math equation.

As it is, all AI is still primarily run by a human consciousness. It cannot decide for itself what to do, it has to be pre-programmed. This means that any biases the human programming said AI might have will be transferred to the program itself given the immensity of data it is meant to process, so you’re right back at human fallibility. At best, contemporary AI is to manual moderation what a chainsaw is to chopping down trees with an axe - just an implement to aid humans in doing exactly what they did before, but maybe faster. That’s it.

Aren’t there already some automated mod tools working to delete CSAM and shit? That’s a form of AI.

But all moderation problems you identify (work, biases) would not fully go away with AI moderation. Someone has to build and manage those tools (work) and train them on how to moderate (incorporating their biases as they do so).

Yes, as soon as we actually invent AI.
The Large Language Models we have now aren’t really it. When we have programs which can come to a well reasoned decision and actually explain the logic of said decision, then we’ll start having something approaching AI. For now, it’s just a well directed random number generator.

AI is stupid

More “accurate” or otherwise, moderating is community engagement. We cultivate our communities by posting relevant content and removing what we find unacceptable. What are we doing if we are not doing both? Allowing a computer to sort the former and the latter? No thank you.