(Credit: Shutterstock/Primakov)
Meta is experimenting with a new feature that uses machine learning, an application of AI, to detect images with nudity sent via direct message (DM) on Instagram. The offending images will then be blurred out and marked with the label "Photo may contain nudity," according to a Meta post published Thursday.
Meta users will also see pop-up warning messages when sending images containing nudity with the new feature. Instagram plans to warn users that unsending a photo doesn't mean someone hasn't already seen it, and that any images sent could become screenshots or sent to others.
Instagram users who try to forward or reshare nudity-flagged images within the app will also get a pop-up warning asking them to "be responsible" and "be respectful" before forwarding the sensitive images. It will still allow them to share the images, though.
When Instagram users receive photos believed to contain nudity, the app will blur out the image and tell users via a pop-up message: "Don't feel pressured to respond." The pop-up also reminds users that they can block the person they're DM-ing, and tells users they can stop any conversations that make them "uncomfortable."
Meta is testing the features because it believes blurring out such images, which can still be viewed, sent, and forwarded in Instagram DMs, could help protect Instagram users under 18. Meta says it also wants to prevent "financial sextortion," which the FBI describes as any scenario where a criminal or predator manipulates users into getting them to send sexually explicit images, which are then used against them unless a ransom is paid.
"We’ve spent years working closely with experts, including those experienced in fighting these crimes, to understand the tactics scammers use to find and extort victims online, so we can develop effective ways to help stop them," Meta's post states.
These new Instagram features will be enabled for minors by default, and Meta also says adults should turn the feature on, too, when it goes live.
Meta says its AI nudity-detection tool for Instagram's end-to-end encrypted DMs will exist on users' devices, so the tech giant won't be able to see the flagged images unless they are reported, according to the post.
Instagram already has a number of privacy features in place to make it harder for predators or scammers to reach its users. Private accounts and restricting who can message you via your Instagram settings are a few ways to do this, and Instagram already blurs images sent to your DM "Requests" folder. Instagram also automatically hides some of these requests even further in its menus, putting questionable DMs in a "Hidden" folder within "Requests." Teenagers' Instagram accounts can also be "supervised" via Meta's "Family Center."
While some might see Instagram's latest features as tech giant-imposed nannying, others may appreciate the social media platform's efforts to blur out unwanted naked selfies sent to their inboxes. Parents who aren't always able to see what their kids are up to at school may also feel more comfortable with them using the online social media platform unsupervised.
"Companies have a responsibility to ensure the protection of minors who use their platforms. Meta’s proposed device-side safety measures within its encrypted environment is encouraging. We are hopeful these new measures will increase reporting by minors and curb the circulation of online child exploitation," the National Center for Missing & Exploited Children's SVP John Shehan said in a statement shared on Meta's post.
Meta's new proposed Instagram features come a few months after US Congress grilled Meta CEO Mark Zuckerberg as well as CEOs from Twitter, Snap, TikTok, and Discord over their child safety policies—or a perceived lack thereof. Before the hearing, Meta announced more Instagram policies intended to protect teenagers from seeing content relating to eating disorders or suicide. It announced even more Instagram pop-ups and features for teens later that month.
Meta has continued to roll out updates to both Instagram and Facebook Messenger's messaging services. Last month, Instagram added DM chat editing and read receipts, and this week said that it's adding true 4K image sharing and the ability to share files up to 100MB on Messenger.