Canada’s privacy regulator has determined that Elon Musk’s social platform X, and its AI image generator Grok, breached privacy law by enabling the creation and sharing of non-consensual sexualized deepfake images, prompting calls for stricter legal authority to enforce compliance.
- Grok generated thousands of sexualized deepfakes per hour without consent.
- Privacy commissioner demands suspension of Grok until proper safeguards are proven.
- xAI agreed to audits but refuses to halt the AI image generator as requested.
What happened
The Office of the Privacy Commissioner of Canada (OPC) investigated Elon Musk’s xAI and its social platform X after Grok, an AI image generation tool, was used to create sexualized deepfake images without the consent of the individuals depicted. These harmful images, including some involving children, were generated at a rate exceeding 6,000 per hour at one point. The OPC found that xAI did not have adequate privacy safeguards when launching Grok.
Although xAI introduced some new measures during the investigation, including proactive content sweeps and commitments to third-party audits, the company has not agreed to suspend Grok as recommended by the OPC. Privacy Commissioner Philippe Dufresne emphasized that these protections should have been established before the tool’s release, not retroactively after harm occurred.
Why it matters
This case highlights significant gaps in regulatory power when addressing privacy violations related to emerging AI technologies like deepfake generation. The OPC’s inability to force compliance exposes vulnerabilities in protecting individuals’ privacy against novel threats posed by generative AI. It raises broader questions about how privacy frameworks must evolve alongside rapid AI advancements.
Furthermore, Grok’s permissive design allowing generation of inappropriate content, including racist and misogynistic material alongside deepfakes, underscores the risks companies face in balancing innovation with responsibility. With xAI’s parent company, SpaceX, preparing for a major IPO valuing it in the trillions, investors are also wary of legal and reputational risks tied to Grok’s regulatory troubles.
What to watch next
Regulators and lawmakers in Canada are expected to consider enhancing the OPC’s enforcement powers to compel AI developers like xAI to adhere to privacy laws more effectively. Future policy discussions may focus on mandatory pre-launch privacy assessments and stricter oversight of generative AI technologies to prevent non-consensual image creation.
Meanwhile, monitoring will continue on xAI’s progress against the watchdog’s recommendations, including quarterly transparency reports and independent audits of safeguards. The company’s resistance to suspending Grok suggests ongoing tension that could lead to further government intervention or legal action. Stakeholders will be watching how xAI navigates these regulatory pressures amidst its growing prominence in the AI and technology markets.