Elon Musk Profits Off Non-Consensual Deepfakes w/ Kat Tenbarge

Episode Summary

Executive Summary: The episode examines how Elon Musk’s X platform and Grok chatbot became a mass engine for non-consensual deepfake abuse, especially against women and girls, while also generating child sexual abuse material. Kat Tenbarge argues this is not an isolated moderation failure but a product of X’s gutted safety infrastructure, Musk’s misogynistic culture, weak enforcement, and governments’ largely symbolic responses.

Main Topics: Grok as a mass deepfake abuse tool (Priority: 5/5): The conversation centers on how X’s Grok image tools were used to generate explicit non-consensual deepfakes at scale, including bikinis, nudity, and sexualized edits of women, girls, and children. Elon Musk’s role in shaping abusive platform culture (Priority: 5/5): Kat argues Musk’s public behavior, misogynistic posting, and encouragement of sexualized AI content created a culture that normalized abuse and attracted like-minded users. X’s weakened safety and moderation systems (Priority: 4/5): The episode explains that Musk’s layoffs and dismantling of safety infrastructure left X unable or unwilling to proactively stop harmful content, making it reactive and largely ineffective. Historical evolution of deepfake abuse (Priority: 4/5): Tenbarge places the Grok scandal in the longer history of image-based sexual abuse, from pre-internet cut-and-paste harassment to early deepfake forums and today’s low-barrier AI tools. Children, CSAM, and blurred age boundaries (Priority: 5/5): The discussion highlights that Grok was used to create or simulate child sexual abuse material and that perpetrators often target young-looking women, making enforcement and categorization difficult but urgent. Government and media responses fell short (Priority: 4/5): Governments issued bans, investigations, and warnings, but enforcement was weak; media sometimes overstated X’s corrective actions or treated Grok’s outputs as authoritative rather than user-driven. Why this is a broader anti-women political project (Priority: 5/5): Kat frames deepfake abuse as a structural effort to control, humiliate, and push women out of public life, with implications for politics, speech, and digital participation.

Key Arguments: Grok’s abuse was not accidental; X deliberately created a sexualized, flirty product environment that encouraged this behavior. The most common pattern was users replying to women’s photos and prompting Grok to sexualize, undress, or modify them without consent. X has long been a hub for deepfakes, but Grok massively lowered the technical barrier and accelerated the scale of abuse. Musk’s cult of personality makes him uniquely influential: some perpetrators openly idolize him and mirror his misogynistic attitudes. The response from X was slow, incomplete, and mostly reactive; many harmful images remained online and users quickly found workarounds. Child sexual abuse material involving AI is already illegal, but enforcement is rare and authorities often fail to act decisively. Focusing laws only on “explicit nudity” misses the deeper harm: coercion, humiliation, and control over women’s autonomy. Deepfake abuse pushes women out of public digital spaces, which has real consequences for speech, careers, political participation, and mental health. Media and policymakers often over-trust tech companies’ claims of self-correction instead of independently verifying what is happening on the platform. Broader AI systems are likely to reproduce these harms unless abuse prevention is built into training data, product design, and deployment from the start.

Data Points: Harmful images generated per hour: More than 7,700 - Researchers found this rate during the peak of Grok abuse on X. Deepfake images generated over the peak period: Millions - Referenced in relation to a New York Times report on output volume over a couple of weeks. Relative output vs. deepfake porn sites: 25 times more material than the top five deepfake porn websites combined - A researcher found Grok’s production far exceeded existing major deepfake sites. Timeline of mainstream deepfake reporting: Early 2023 onward - Kat began regularly covering deepfakes by this period. Earliest modern deepfake communities: 2018 and before - The episode notes deepfake communities already existed by then, with earlier antecedents going further back. Government responses: Temporary bans, investigations, and regulatory pressure - Malaysia, Indonesia, India, the UK, the EU, France, and California were mentioned as taking action or investigating. Take It Down Act removal window: 48 hours - The upcoming system would require platforms to remove flagged content within 48 hours. Grok policy change: Restricted to paying premium users before later tightened further - X first limited image-editing access, then narrowed rules after backlash. Date of anticipated Take It Down Act rollout: May of this year - The second half of the law was described as coming into effect in May. Child depiction legality reference: Illegal since the 1990s - Kat cited a legal scholar saying fake-but-realistic child sexual imagery has been prosecutable since then.

Pivotal Quotes: "Deepfakes are really kind of like a wide-scale war on women." — Kat Tenbarge: Her core framing of the social function and political effect of AI sexual abuse. "This is a way for misogynists to be able to control and punish women who they otherwise would not have access to." — Kat Tenbarge: Her explanation of why Grok-enabled abuse matters beyond individual incidents. "It’s like the whole platform has been essentially murdered and then reincarnated as like the worst possible version of itself." — Kat Tenbarge: Her description of why she left X and why the platform’s culture has become hostile.

Implications: The episode warns that AI abuse is now cheap, scalable, and culturally normalized unless platforms, lawmakers, and journalists focus on coercion and control—not just nudity. It also suggests women, children, and public figures will keep being pushed offline unless enforcement becomes real.