Privacy

"One of the most far-reaching recommendations concerns how platforms deliver information.

"The Manhattan court order requiring OpenAI to hand over 20 million anonymized ChatGPT conversations to the New York Times and other publishers as part of a copyright lawsuit is being viewed as a turning point in data privacy, and not in a good way.

"n October 2025, Stanford University’s Institute for Human-Centered AI released a study that should have caused riots in the streets. Led by Dr. Jennifer King, the research team examined the privacy policies of six frontier AI companies—Amazon, Anthropic, Google, Meta, Microsoft, and OpenAI—analyzing twenty-eight documents per company using California Consumer Privacy Act frameworks. Their finding was unambiguous: every single company employs users’ chat data by default to train their models.

"Google was accused in a lawsuit of using its Gemini AI assistant to unlawfully track the private communications of users of its Gmail, instant messaging and videoconference programs.

"Allowing the government to access such unfiltered exchanges and treat them as legal confessions would have a massive chilling effect. If every private thought experiment can later be weaponized in court, users of A.I. will censor themselves, undermining some of the most valuable uses of these systems. It will destroy the candid relationship that makes A.I. useful for mental health and legal and financial problem-solving, turning a potentially powerful tool for self-discovery and self-representation into a potential legal liability.

"Ahead of the new school year, a handful of tech companies issued grim warnings to American educators embracing artificial intelligence in the classroom: Chatbots, they said, could endanger students and lead to self-harm. Vigilance was paramount. “The risks of students using AI can literally be deadly,” one company cautioned. Another noted: “Student lives depend on it.”

"The web was designed without the concept of personal identity at all, and without any tracking system built in. It was designed for anybody to be able to create what they want, and even for anybody to be able to make their own web browser. Not long after its invention, people came up with ideas like cookies and made different systems for logging in, and then big companies started coming in and realized that if they could control the browser, they'd control all the users and the ways of making money. Ever since, there's been a long series of battles over privacy versus monetization, but there's been some small protection for users, who benefitted from those smart original design choices back at the birth of the web.

"Over the last year, federal agents were struggling to uncover the identity of a darkweb child exploitation site, with little success. Then a possible avenue opened up, thanks to the suspect’s use of ChatGPT.

"This work on differential privacy has led to a new open-weight Google model called VaultGemma. The model uses differential privacy to reduce the possibility of memorization, which could change how Google builds privacy into its future AI agents. For now, though, the company's first differential privacy model is an experiment.

"To put it bluntly, the path currently being taken towards agentic AI leads to an elimination of privacy and security at the application layer. It will not be possible for apps like Signal—the messaging app whose foundation I run—to continue to provide strong privacy guarantees, built on robust and openly validated encryption, if device-makers and OS developers insist on puncturing the metaphoric blood-brain barrier between apps and the OS. Feeding your sensitive Signal messages into an undifferentiated data slurry connected to cloud servers in service of their AI-agent aspirations is a dangerous abdication of responsibility.

"Claude’s update relies on a striking pop-up with a large, black "Accept" button. The data sharing toggle is tucked away, switched on by default, and framed positively ("You can help..."). A faint "Not now" button and hard-to-find instructions on changing the setting later complete the manipulative design.

"Anthropic will start training its AI models on user data, including new chat transcripts and coding sessions, unless users choose to opt out. It’s also extending its data retention policy to five years — again, for users that don’t choose to opt out.

"In recent months, AI has dominated headlines, with Donald Trump’s AI plan, the phased rollout of the EU’s AI Act, and a fresh wave of corporate lobbying to stall new rules.

"Meta has recently started using the personal data of Europeans for AI training. Contrary to its GDPR obligations, Meta hasn’t asked for consent in advance. Instead, the company claims to have a ‘legitimate interest’ outweighing the fundamental right to privacy. A key argument in favour of such a ‘legitimate interest’ is the reasonable expectations of users. This begs the question: do people want this to happen? To find out more, noyb has commissioned the Gallup Institute to conduct a study among 1,000 Meta users in Germany. The results are clear: While almost 75% of users heard of Meta’s plans, only 7% actually want their data to be used for AI training. This also means that at least 68 million people never heard about the change."

"While OpenAI scrambled to de-index conversations from Google today, they forgot the internet's most basic rule—nothing truly disappears. Over 100.000 ChatGPT chats are still in Archive.org, although with a twist. The chats aren't just links or fragments. They're complete conversations, frozen in time, containing similar “confessions” we exposed yesterday. Users shared these chats publicly - not by default, but only by clicking Share.