After having been a moderator for a number of communities over the years—BBCs in the 90s, niche forums during the 00s & 10s, Reddit boards in the 20s—I’ve come to believe that running shadow services where disruptive individuals can comingle may be the best way to handle this.
The advent of LLMs really opens the door to shunting off these “community members “ who’d rather contribute in misanthropic ways for the lulz than either leave or not contribute at all. They can take part in an interactive echo chamber that gives just as well as they can. You don’t even need a powerful model so the overall costs to the community are probably lower than the alternatives of trying to coexist with community-arsonists.
I spent years trying to find ways to bring people productively “into the fold” but eventually realized that it is futile in some cases because there’s zero value to the individual or the community to find a middle ground. They want to see things burn, and the community simply wants them out.
If they don't know they're being quarantined, there's no reason for them to find a way around their block. They'll eventually tucker themselves out and move on to whatever their internet equivalent of burning ants with a magnifying glass is.
I used to run a small MMORPG. I only had to ban a player once but when I did they made it their life's mission to get around it and spam profanities. They went through every proxy in existence and I had to resort to banning whole IP ranges to keep them out.
If they believe they’re still trolling people they will keep doing so and won’t make an alternative account and will keep harassing bots instead of real people.
A slur is rarely the problem. Trolls can (sometimes) add flavor to a community and keep things interesting. Mostly they can be an annoyance.
I’m specifically referring to people who have seemingly made it their sole purpose to create as much indiscriminate damage as possible.
You can ban them, block routes for them to attempt to Sybil themselves back to having accounts, etc. but even with great moderation tools and systems, it’s extremely difficult to set up a strong enough set of controls which don’t adversely impact everyone else who you want to have participate in the community.
Yes, a shadow environment is dystopian. It’s not my nature to want to even consider using one.
But we’re talking about privately run communities which also deserve to exist to serve their purposes.
So given the choice between anarchy which drives away people who contribute to make the community what it is and a shadow option for those actively working against its interests, I’ll consider the community first.
You may have misinterpreted my comment. I’m not suggesting you use LLMs as moderators. I’m talking about using LLMs as participant “members” of this shadow board to interact with someone whose account was flagged by a human moderator.
In some cases, I’m sure you are right. That said, the online communities which currently employ shadowban systems continue to use them. That tells me there’s a value greater than their implementation and operation costs.
There is no perfect tech solution to a human problem. But in my opinion, having access to a partial mitigation is better than no mitigation.
this comment immediately replying to "You're talking about using a machine to detect social undesirables then quarantining them in the matrix." raises interesting questions about the nature and purpose of dangbot
The advent of LLMs really opens the door to shunting off these “community members “ who’d rather contribute in misanthropic ways for the lulz than either leave or not contribute at all. They can take part in an interactive echo chamber that gives just as well as they can. You don’t even need a powerful model so the overall costs to the community are probably lower than the alternatives of trying to coexist with community-arsonists.
I spent years trying to find ways to bring people productively “into the fold” but eventually realized that it is futile in some cases because there’s zero value to the individual or the community to find a middle ground. They want to see things burn, and the community simply wants them out.