I feel like it's way too optimistic to think "If everyone’s a superpersuader, no one is.". I know that there are many people in my life who just do not have, like, the ability to understand what it means that models are so good to create realistic video or text, and also at any speed.
And remember that LLM's primarily generate text which is the mode of communication that we've assumed comes directly from a human for a while. It was already 'hard' enough to try to find if someone's post was from a karma farmer, credibility-less clueless but confident person, or copypaster (and trying to find evidence of those requiring cross referencing their other text), damaging credibility of many sites, but those can be detected easier than LLM's now.
I think it's more of a narrative of normalcy people have that makes it seem like "quite a lot of weird effort no one will do" that is why for example there has been no "big deepfake controversy" moment yet. Not because people are clued-in. Recently a streamer I watch a lot just played a full AI-generated quiz on stream, constantly be like "the writing was so good" and no one seemed to notice or care besides only 1 other person in chat who plays with frontier AI.
I would not even say these capabilities are 'secret' but that societal awareness is terrible (not helped by hate-spamming "shameful never use AI even for the most obvious work-saving rote work, it's just autocomplete" mobs to be fair).
There's many points in the article I find reasonable but I just feel like something is deeply missing in the whole, like I find the 'group of aware people' vs. 'group of unaware-to-AI people' very stark. Even thinking in terms of groups of people a bit much:
"But he too assumes that there’s approximately one AI that is “everywhere” and free to infiltrate your circles of trust unopposed over some substantial period. More realistically, competing AIs would compete to infiltrate, and would be incentivized to call each other out. "
What does the world look like here? I mean I'd already think that 'secret' AIs, controlled by humans for some motive by the way, will be designed to be subtle enough that they just screw with consensus without even knowing with tons of different accounts. Those are notoriously hard to detect because it involves tracking some 'randos' online. Yes it isn't an only-AI problem but the rapid speed increase will be.
And if they are NOT secret to the point we have 'competing' AI's competing over manipulating the same community, you could probably say goodbye to humans talking in it, because that means social platforms will all become a live Moltbook. I mean I guess I see AI stuff obviously all over my Substack feed while my own handwritten posts get 1-2 likes. I bet 'calling out' in this context will just mean more engagement bait. You've probably seen reddit posts where a mass upvoted post that's AI will get top comments calling it out as AI, but somehow the post remains mass upvoted.
I have heard it suggested elsewhere, to deal with this already-existing-before-AI-but-got-much-worse problem of not knowing who is real and who is just trying to gain power, to have some "proof of work" of truly good content or at least some reputation to uphold even if you're not online to primarily write, just to show not being an LLM. I don't know if this is very viable optics-wise because it filters out real people who are legitimately worse than LLM's at everything or just aren't very ambitious enough to care.
I guess overall I'd say we aren't really fine, we've also had lots of other erosion in trust in many areas and I truly think a lot of people are more like "I guess this is just the new normal, who can do anything about it, oh I just ignore it, I am pacified and domesticated to not care", blank-stare equivalent.
I think most of what you are pointing out here is that people don't always realize that something is AI-generated, but does that by itself cause people harm? If a streamer is playing a quiz for fun, why _should_ they care about it being AI-generated?
Meanwhile, I think there _is_ growing awareness about cases of where AI actually did cause harm. Instances of AIs possibly contributing to suicides have been on the news; "AI psychosis" is a term that I've seen come up even on mainstream news sites now.
I would generally interpret that to in line with Onni's thesis, that people ignore something being AI when it's rational not to care about it, but culture is already starting to adapt to the cases where AI is harmful.
It's at least nice that people can tell that AI psychosis is a bad thing. The thing about not being able to tell the AI generated style is an issue to me because it does not spell our hope that people can detect fake AI generated personas or writing and it definitely feels way worse if the guy acts in surprise that "the writing is so good". It implies many humans will willingly upvote AI content if they don't know it is. Like it almost feels like we are in some uncanny valley of bizarro land where some creative could get harshly criticized for AI assisted code or even consulting an AI for plot while still writing everything by hand, and being open about it, meanwhile, unlabeled AI content is preferred but only when unlabeled.
Well obviously there's many people who won't prefer AI art that I also think that is ugly often. But detection of AI writing or AI style on front end is really low it seems.
The original article thesis mentions a lot about "there will be competing AI pointing out other AI (swarms)" but with such poor awareness of the AI style, I don't even know if we're going to get to that point. Human scheming often involves a lot of benign actions. Very recently in the Minecraft community the top PVPer was found to be almost certainly a fraud who cheated and a persona. But before then it seems like the person behind it acted really kind to everyone in the community and helped the community a lot. Similarly, karma farm messages or easily generated quizzes might seem benign but for example, that quiz website upsells you on $200, $720, and even $2,499 lifetime memberships on its site and every other page also seems AI generated and written. So if this is the norm I have absolutely no hope that many people will even care or notice.:{
I don't think the article was premised on people being able to detect AI-generated content, though. I read it as saying that maybe we _won't_ be able to detect AI-generated fake content and we'll retreat to just following trusted authorities or e.g. people we know in real life.
With AIs pointing out other AIs, I was thinking that the other AIs would point out things like manipulation and obvious untruths. To point out something like "hey do you notice this message is creating a sense of urgency that's making it hard for you think clearly, I think it's playing on your emotions", it doesn't matter if the manipulation is produced by an AI or a human. You can detect the psychological hooks in it nonetheless.
A quiz site upselling expensive memberships sounds like bad value for your money, but it also doesn't sound very different from what people already did before AI? People have been tricked into paying big bucks for stuff that's not worth it for a long time ago. And most will be resistant to those attempts by now, and just do the free quiz without signing up for the paid things.
Human scheming can involve a lot of benign actions, but then again, we already have lots of human schemers hiding a lot of benign actions and it's not the end of the world. The overall damage they can do is still limited.
I feel like it's way too optimistic to think "If everyone’s a superpersuader, no one is.". I know that there are many people in my life who just do not have, like, the ability to understand what it means that models are so good to create realistic video or text, and also at any speed.
And remember that LLM's primarily generate text which is the mode of communication that we've assumed comes directly from a human for a while. It was already 'hard' enough to try to find if someone's post was from a karma farmer, credibility-less clueless but confident person, or copypaster (and trying to find evidence of those requiring cross referencing their other text), damaging credibility of many sites, but those can be detected easier than LLM's now.
I think it's more of a narrative of normalcy people have that makes it seem like "quite a lot of weird effort no one will do" that is why for example there has been no "big deepfake controversy" moment yet. Not because people are clued-in. Recently a streamer I watch a lot just played a full AI-generated quiz on stream, constantly be like "the writing was so good" and no one seemed to notice or care besides only 1 other person in chat who plays with frontier AI.
I would not even say these capabilities are 'secret' but that societal awareness is terrible (not helped by hate-spamming "shameful never use AI even for the most obvious work-saving rote work, it's just autocomplete" mobs to be fair).
There's many points in the article I find reasonable but I just feel like something is deeply missing in the whole, like I find the 'group of aware people' vs. 'group of unaware-to-AI people' very stark. Even thinking in terms of groups of people a bit much:
"But he too assumes that there’s approximately one AI that is “everywhere” and free to infiltrate your circles of trust unopposed over some substantial period. More realistically, competing AIs would compete to infiltrate, and would be incentivized to call each other out. "
What does the world look like here? I mean I'd already think that 'secret' AIs, controlled by humans for some motive by the way, will be designed to be subtle enough that they just screw with consensus without even knowing with tons of different accounts. Those are notoriously hard to detect because it involves tracking some 'randos' online. Yes it isn't an only-AI problem but the rapid speed increase will be.
And if they are NOT secret to the point we have 'competing' AI's competing over manipulating the same community, you could probably say goodbye to humans talking in it, because that means social platforms will all become a live Moltbook. I mean I guess I see AI stuff obviously all over my Substack feed while my own handwritten posts get 1-2 likes. I bet 'calling out' in this context will just mean more engagement bait. You've probably seen reddit posts where a mass upvoted post that's AI will get top comments calling it out as AI, but somehow the post remains mass upvoted.
I have heard it suggested elsewhere, to deal with this already-existing-before-AI-but-got-much-worse problem of not knowing who is real and who is just trying to gain power, to have some "proof of work" of truly good content or at least some reputation to uphold even if you're not online to primarily write, just to show not being an LLM. I don't know if this is very viable optics-wise because it filters out real people who are legitimately worse than LLM's at everything or just aren't very ambitious enough to care.
I guess overall I'd say we aren't really fine, we've also had lots of other erosion in trust in many areas and I truly think a lot of people are more like "I guess this is just the new normal, who can do anything about it, oh I just ignore it, I am pacified and domesticated to not care", blank-stare equivalent.
I think most of what you are pointing out here is that people don't always realize that something is AI-generated, but does that by itself cause people harm? If a streamer is playing a quiz for fun, why _should_ they care about it being AI-generated?
Meanwhile, I think there _is_ growing awareness about cases of where AI actually did cause harm. Instances of AIs possibly contributing to suicides have been on the news; "AI psychosis" is a term that I've seen come up even on mainstream news sites now.
I would generally interpret that to in line with Onni's thesis, that people ignore something being AI when it's rational not to care about it, but culture is already starting to adapt to the cases where AI is harmful.
It's at least nice that people can tell that AI psychosis is a bad thing. The thing about not being able to tell the AI generated style is an issue to me because it does not spell our hope that people can detect fake AI generated personas or writing and it definitely feels way worse if the guy acts in surprise that "the writing is so good". It implies many humans will willingly upvote AI content if they don't know it is. Like it almost feels like we are in some uncanny valley of bizarro land where some creative could get harshly criticized for AI assisted code or even consulting an AI for plot while still writing everything by hand, and being open about it, meanwhile, unlabeled AI content is preferred but only when unlabeled.
Well obviously there's many people who won't prefer AI art that I also think that is ugly often. But detection of AI writing or AI style on front end is really low it seems.
The original article thesis mentions a lot about "there will be competing AI pointing out other AI (swarms)" but with such poor awareness of the AI style, I don't even know if we're going to get to that point. Human scheming often involves a lot of benign actions. Very recently in the Minecraft community the top PVPer was found to be almost certainly a fraud who cheated and a persona. But before then it seems like the person behind it acted really kind to everyone in the community and helped the community a lot. Similarly, karma farm messages or easily generated quizzes might seem benign but for example, that quiz website upsells you on $200, $720, and even $2,499 lifetime memberships on its site and every other page also seems AI generated and written. So if this is the norm I have absolutely no hope that many people will even care or notice.:{
I don't think the article was premised on people being able to detect AI-generated content, though. I read it as saying that maybe we _won't_ be able to detect AI-generated fake content and we'll retreat to just following trusted authorities or e.g. people we know in real life.
With AIs pointing out other AIs, I was thinking that the other AIs would point out things like manipulation and obvious untruths. To point out something like "hey do you notice this message is creating a sense of urgency that's making it hard for you think clearly, I think it's playing on your emotions", it doesn't matter if the manipulation is produced by an AI or a human. You can detect the psychological hooks in it nonetheless.
A quiz site upselling expensive memberships sounds like bad value for your money, but it also doesn't sound very different from what people already did before AI? People have been tricked into paying big bucks for stuff that's not worth it for a long time ago. And most will be resistant to those attempts by now, and just do the free quiz without signing up for the paid things.
Human scheming can involve a lot of benign actions, but then again, we already have lots of human schemers hiding a lot of benign actions and it's not the end of the world. The overall damage they can do is still limited.