Back to All
Ai/ml
Blog

The Power Paradox — When AI Needs Your Data, Who Do You Trust?

Listen
The Power Paradox — When AI Needs Your Data, Who Do You Trust?
4:18

 

 This week Google hosts its annual Google I/O conference, which is always jam-packed with announcements. This year's keynote was, like almost every major tech conference lately, heavily indexed on AI. Gemini 3.5, Gemini Omni, Project Astra, the hits just kept coming. 

The announcement that I found particularly interesting is Google Spark, an AI agent platform that lives inside your life. Spark is Google's answer to OpenClaw, powered by Gemini, and connected to your entire Google service footprint. Spark can read your emails, scan your calendar, track your spending, know your schedule, your friends, your travel plans. Gemini Spark is AI that watches everything you do, 24/7, and acts on your behalf.

The Verge's Emma Roth published a piece exploring Google's announcements, and cut right to the heart of the matter: this AI future demands trust, and personal data is the currency.

This issue goes well beyond Google. It's a fundamental trade-off facing AI adoption. The more an AI knows about you, the more powerful and capable it becomes. Context is everything. An AI that knows your calendar can schedule meetings intelligently. One that knows your email can draft responses in your voice. One that has access to your files can surface exactly what you need, when you need it.

But... doesn't all of this data sharing feel a bit creepy? Handing your employer, or a tech company, or both, a window into your entire digital footprint gives even the biggest AI enthusiasts a little bit of the ick.

As a technologist, I have high standards for access to my information, and I gravitate toward services that respect my privacy. Apple has long been a strident advocate for customer privacy, implementing differential privacy to allow their systems to improve contextual intelligence in aggregate without requiring invasive data sharing.

Instead of uploading your typing patterns, for example, a model trains on aggregated, anonymized data that preserves privacy while still improving the experience through personalization.

It's not a perfect solution, and Apple's critics rightly point out that they still collect plenty of data through other means. But the philosophy is great: you can have powerful AI that respects your privacy if you're willing to invest in the harder engineering problem of doing AI differently. But differential privacy doesn't scale to the deep personalization of a platform like Google Spark.

Google's track record here is a bit different, given that they are fundamentally in the business of advertising. Personal data is a goldmine for ad targeting, and no one on the planet is better at targeting than Google.

Don't misunderstand me. Google has an incredible ability to understand its users habits to provide them better service, not just to sell more ads. I have paid for YouTube Premium for years now, initially opting out of sharing my viewing habits and history. I mostly discovered content through subscriptions and search.

Eventually I made the choice to share more information with Google, and it immediately was able to help me discover new, highly relevant content that I enjoy. Again, tradeoffs.

I don't think there's a single right answer here. For some use cases, I'm comfortable giving AI systems significant access to my data. I run an AI assistant on my home network that has broad access to my files and systems because I've evaluated the risks, I control the infrastructure, and I have visibility into the implementation. That's a calculated decision and a fair exchange.

But the default settings matter. The opt-out versus opt-in matters. And the question of whether AI companies are earning the trust they're asking for through transparent practices, strong security, and genuine respect for user privacy matters enormously.

As someone who builds with AI and uses it extensively in both personal and professional contexts, I don't think we can afford to punt on this question. The industry is moving fast toward AI that knows everything about us. We need to be intentional, as builders and as users, about what kind of relationship we're willing to have with these systems, and what kind of trust we're willing to extend.

Jonathan LaCour avatar

2 minutes read