Elon Musk’s X said it will hire 100 full-time staffers to crack down on child sexual exploitation after AI-generated photos of Taylor Swift went viral on the social network last week.
The San Francisco-based company formerly known as Twitter announced on Friday that it will build a “trust and safety center” in Austin, Texas where it will hire “in-house agents” who will be tasked with enforcing the site’s content and safety rules.
“X does not have a line of business focused on children, but it’s important that we make these investments to keep stopping offenders from using our platform for any distribution or engagement with CSE content,” Joe Benarroch, head of business operations at X, said.
Musk, who acquired the site then known as Twitter for $44 billion in late 2022 with the aim of allowing unfettered speech, has been criticized for slashing headcount from the company’s trust and safety operations.
The X boss came under fire in recent weeks for the proliferation of antisemitic and neo-Nazi content on the platform — prompting some advertisers to flee the site.
X in recent days was compelled to block some searches for Swift as pornographic deepfake images of the singer have circulated online.
Attempts to search for her name without quote marks on the site Monday resulted in an error message and a prompt for users to retry their search, which added, “Don’t fret — it’s not your fault.”
However, putting quote marks around her name allowed posts to appear that mentioned her name.
Sexually explicit and abusive fake images of Swift began circulating widely last week on X, making her the most famous victim of a scourge that tech platforms and anti-abuse groups have struggled to fix.
“This is a temporary action and done with an abundance of caution as we prioritize safety on this issue,” Benarroch, head of business operations at X, said in a statement.
Unlike more conventional doctored images that have troubled celebrities in the past, the Swift images appear to have been created using an artificial intelligence image-generator that can instantly create new images from a written prompt.
After the images began spreading online, the singer’s devoted fan base of “Swifties” quickly mobilized, launching a counteroffensive on X and a #ProtectTaylorSwift hashtag to flood it with more positive images of the pop star.
Some said they were reporting accounts that were sharing the deepfakes.
The deepfake-detecting group Reality Defender said it tracked a deluge of nonconsensual pornographic material depicting Swift, particularly on X, formerly known as Twitter.
Some images also made their way to Meta-owned Facebook and other social media platforms.
The researchers found at least a couple dozen unique AI-generated images.
The most widely shared were football-related, showing a painted or bloodied Swift that objectified her and in some cases inflicted violent harm on her deepfake persona.
The Swift images first emerged from an ongoing campaign (called Lewd DALL-E, which I think we should avoid) that began last year on fringe platforms to produce sexually explicit AI-generated images of celebrity women, said Ben Decker of the threat intelligence group Memetica.
One of the Swift images that went viral last week appeared online as early as Jan. 6, he said.
Most commercial AI image-generators have safeguards to prevent abuse, but commenters on anonymous message boards discussed tactics for how to circumvent the moderation, especially on Microsoft Designer’s text-to-image tool, Decker said.
“It’s part of a longstanding, adversarial relationship between trolls and platforms,” Decker said.
“As long as platforms exist, trolls are going to try to disrupt them. And as long as trolls exist, platforms are going to be disrupted. So the question really becomes, how many more times is this going to happen before there is any serious change?”
X’s move to reduce searches of Swift is likely a stopgap measure.
“When you’re not sure where everything is and you can’t guarantee that everything has been taken down, the simplest thing you can do is limit people’s ability to search for it,” he said.
Researchers have said the number of explicit deepfakes have grown in the past few years, as the technology used to produce such images has become more accessible and easier to use.
In 2019, a report released by the AI firm DeepTrace Labs showed these images were overwhelmingly weaponized against women.
Most of the victims, it said, were Hollywood actors and South Korean K-pop singers.
In the European Union, separate pieces of new legislation include provisions for deepfakes.
The Digital Services Act, which took effect last year, requires online platforms to take measures to curb the risk of spreading content that breaches “fundamental rights” like privacy, such as “non-consensual” images or deepfake porn.
The 27-nation bloc’s Artificial Intelligence Act, which still awaits final approvals, will require companies that create deepfakes with AI systems to also inform users that the content is artificial or manipulated.
With Post Wires
Credit: Source link
