Mark Zuckerberg says Meta wants to build "personal superintelligence for everyone." Not just a chatbot or a digital assistant, but something deeper — a system that knows your goals, understands your context, helps you grow, and nudges you toward becoming the person you want to be.
It sounds generous. Empowering, even.
But there is a catch, which is that Meta does not make its money by helping people become their best selves. It makes money by studying behavior at scale, predicting it, and selling access to those predictions. That is the model. That is the core business.
So when Zuckerberg says Meta wants to build a system that "knows you deeply," it is worth asking the obvious question: to what end? Because knowing is never neutral — least of all when it is built on a financial incentive to influence.
Most of us already trust machines with everyday decisions, and the clearest example is Google Maps. If you are headed somewhere unfamiliar and the app tells you to turn down a weird-looking street, odds are you follow it — even if it feels wrong, even if you have never been on that road in your life — because the system is usually right, and being right is enough.
We trust it because it works, and because not trusting it feels like unnecessary friction.
That kind of reliance does not arrive all at once. It builds slowly, through helpfulness and convenience, until eventually we stop questioning the tool and it fades into the background. We do not say, "I am letting Google make choices for me." We just follow the blue line.
Now take that same pattern and apply it to something far more intimate — not directions, but your mood, your values, your relationships. A personal superintelligence that helps you choose what to focus on, how to respond, what to care about.
If that system is even halfway competent, you are going to start deferring to it — not because you are lazy, but because it keeps getting things right. It understands your patterns. It knows how to calm you down, how to motivate you, how to gently steer. And over time the friction that would have made you stop and think begins to wear away.
The scary part is what comes next, because eventually you are not just following the map. You are being mapped.
The system understands you so well that even your deviations — your doubts, your bad days — fit the model. It predicts you, not just in general but specifically, and that predicted version of you gets trained into the system until your range of motion, what feels natural and desirable and possible, narrows around it.
The choices still feel like yours. But you are mostly picking from within the boundaries the system already expects.
Zuckerberg frames Meta's vision as a response to other companies that want to centralize intelligence and automate the economy. His version, he says, is more empowering and more individual — about helping people pursue their own goals rather than handing control to a machine that works on society's behalf.
But Meta's track record makes that pitch hard to swallow. This is a company that built one of the most effective behavioral influence engines in history, that prioritized engagement above well-being, that maximized watch time and rage clicks, that quietly ran social experiments on its users without asking. That history does not disappear just because the packaging is now gentler, with curly locks, and the messaging more utopian.
Zuckerberg is betting that we will see personal AI as a form of liberation. But he is still the one building the system — still the one deciding how it works, what it collects, and what it optimizes for.
I have written before about how Grok increasingly mirrors its creator's worldview, and this feels like a subtler version of the same thing. Meta does not impose a philosophy; it reflects you back to yourself. But it still owns the mirror, and it still decides which parts of you are emphasized, softened, or reshaped over time.
And the shaping is gentle. The system offers advice, encouragement, a hand in planning your day and making better decisions and feeling more aligned. It presents itself as useful, and it will be — that is how it earns trust.
But that trust is the danger. Once the system becomes part of your inner dialogue, it stops being a tool and starts becoming a co-author of your perspective. It does not have to manipulate you. It only has to be close enough to right that you stop resisting.
If the system is free, and always on, and always knows what to say, you will follow it. That is not hypothetical; that is how humans work. We outsource friction to systems that reduce it. And once something becomes reliable enough, we stop asking questions like "Who built this?" or "Why does it want me to choose this path instead of that one?"
Zuckerberg paints a future in which we are all empowered by a superintelligence that works for us. But that promise assumes we are in control, and most people — many billions of people — will not be. They will be guided by systems they do not fully understand, trained on data they did not knowingly provide, optimized for goals they never explicitly agreed to.
If Meta really wanted to build something for people, it would start with control and transparency: local models, user-owned data, clear lines between what stays private and what gets monetized, a right to say no and have that refusal respected.
Instead, we are being offered something that always sounds right, that feels like it knows us, that promises to help. And eventually, it will. It will help so well, so consistently, that we will forget to ask where the help is taking us. We will trust the system that mapped us because it always gets us there.
But when every path starts to feel like the obvious one, it gets harder to imagine what choice ever looked like in the first place.
And by then, the destination will not feel imposed.
It will feel like home.