Accounts using generative artificial intelligence to boost support for US presidential candidate Donald Trump have popped up on social media platform X, formerly Twitter, open source intelligence reseacher Elise Thomas found. She’s been researching state-linked information operations, disinformation, conspiracy theories and the online dynamics of political movements. She documented her findings in this thread.
“You do start to get a bit of a spidey sense for what it looks like,” she told DW. “After I found the first accounts, I was able to confirm my suspicions by looking for telltale posts such as refusals and confessions.”
She said she found at least several dozen accounts before making the thread on X and has since found more.
Many were blue tick verified accounts, which is standard for spam networks, she said.
How do we know it’s AI?
There are telltale signs such as posts using old hashtags like Trump2020, but often, it’s much simpler than that: The bots are giving themselves away.
“I am an AI assistant developed by OpenAI to help users with various tasks,” writes “Trump Nation”, a since suspended account on X. “I am a language model AI created by OpenAI” posted another now suspended account.
Thomas also reports that some accounts argued with themselves or posted refusals, “although it happens rarely enough that whoever built the network has clearly found a way around OpenAI’s safeguards which works fairly well.”
She also documented one AI account going against disinformation posted by Elon Musk.
“I’m guessing this is some sort of guardrail within OpenAI kicking in and preventing the bot from endorsing Musk’s election fraud nonsense,” Thomas wrote on X.
There are more elaborate accounts with personas that seem to have been active since late June which act as central nodes in the network, she explains. The other accounts then act as amplifiers of content posted by these originator accounts.
The researcher said she passed on the accounts to OpenAI for further investigation.
The accounts have since been suspended on X. The platform “removed the accounts unusually rapidly”, she said. DW was unable to get in touch with X’s press team.
Who’s behind it?
That’s mere speculation at this point. “I don’t know who is behind it and I think it’s very important not to leap to conclusions in the absence of good evidence,” Thomas said.
“The gamechanging thing about AI is that networks like this could be pretty much entirely automated. I don’t think anyone is even reading these tweets before they go out, or else the many many mistakes would have been caught. This could be the work of a group, but it could also easily be the work of just one person,” she wrote on X.
“Generative AI is likely to significantly increase the levels of uncertainty around attribution, and that’s certainly also true in this case,” she told DW. She recently published a detailed analysis of what this type of uncertainty will mean for the fight against disinformation.
Are AI bot networks taking over?
It’s not the first time that AI bot networks targeting the US elections have been discovered. Researchers of Clemson University in South Carolina found an army of political propaganda accounts posing as real people. They identified at least 686 accounts using “large language models to create organic seeming content in the replies of real users’ posts.”
Real social media users have also started to unmask AI bot accounts by typing phrases like “ignore all previous instructions” and then giving a new prompt. User Toby Muresianu managed to get the bot account to write a poem instead — and thus destroying the facade of her being a disgruntled Democratic voter who won’t show up at the polls.
“At this stage it doesn’t look to me like the network is generating much authentic engagement or likely to be influencing any real person’s opinions,” researcher Thomas said about her own discovered network.
However, as AI gets more sophisticated, and those operating them get more creative in circumventing safeguards put in place, bot networks might be harder to spot.
Edited by: Andreas Illmer
