(Credit: NurPhoto/Contributor via Getty Images)
Meta says it's identified and removed thousands of fake Facebook accounts, hundreds of Facebook pages, and dozens of Instagram accounts for being used to spread coordinated political propaganda and influence campaigns, the social media giant announced in its quarterly Adversarial Threat Report this month.
Out of the approximately 2,000 infringing Facebook accounts Meta identified this quarter, it was unable to identify the origins of over 1,300 of them. The largest identifiable origin of fake political Facebook accounts, however, was Israel.
Meta removed over 500 Facebook accounts and more than two dozen fake or compromised Instagram accounts because they were being coordinated to post pro-Israel political content in a way that violates Meta's inauthentic behavior policies.
"While the individuals behind it attempted to conceal their identity and coordination, we found links to STOIC, a political marketing and business intelligence firm based in Tel Aviv, Israel," Meta states.
Meta says it sent Stoic a cease and desist notice and banned it from its platforms. Stoic's Linkedin page calls itself "the most advanced campaign management system," and its website admits that it uses automation and generative AI to "create targeted content and organically distribute it quickly to the relevant platforms."
Meta notes that the engineered content in question was primarily in English, targeting audiences in the US and Canada. The pro-Israel accounts posed as Jewish students, African Americans, or "concerned citizens" and posted comments, many AI-generated, supporting Israel's military and politics. Some posed as Muslims criticizing "radical Islam" in Canada, according to the report.
The tech giant calls the pro-Israel posts it believes Stoic orchestrated a "cross-internet operation." It also found similar content on YouTube and X, also known as Twitter. Most of the content was AI-generated comments in response to others' genuine content, the company says, and the comments would often be completely unrelated to the original posts they were supposedly replying to.
Meta says its own automated system was able to detect that many of the accounts were fake. "As these accounts kept going down, the people behind them kept on adding others, likely acquired from account farms, which were also detected and taken down," Meta notes. "In addition, this campaign appeared to have purchased inauthentic engagement (i.e. likes and followers) from Vietnam in an attempt to make its content appear more popular than it was."
While many of the comments were AI-generated, Meta doesn't think AI-generated spam poses a real threat—yet. "Right now we’re not seeing gen AI being used in terribly sophisticated ways," David Agranovich, Meta’s policy director of threat disruption, said in a press briefing reported by Bloomberg.
"But we know that these networks are inherently adversarial," Agranovich continued, noting that Meta anticipates such bot networks and coordinated campaigns will continue to evolve over time.
Meta also found coordinated political spam originating in other countries and removed Facebook and Instagram accounts as well as related pages and groups. It banned over 100 Facebook accounts in Croatia promoting a conservative party and nearly two dozen from Iran that posted conservative, pro-Israel content. 50 Facebook accounts in Bangladesh were banned for posting political content while posing as news groups, and over two dozen Facebook accounts tied to China were also removed.