Smart things are dangerous

Humans like to imagine we are the clever apex of Earth’s grand experiment — until we talk to modern AI systems and feel, for a brief, humiliating moment, like Douglas Adams’ mice had a point. In 2025, chatting with Gemini or ChatGPT gives the unsettling impression that we may not even be the third-smartest species anymore. Somewhere, a dolphin is laughing.

But does being outclassed intellectually mean we’re in danger? Carlsmith thinks so. Not because AI will suddenly develop a moustache and tie damsels to railroad tracks, but because intelligent agency has always been the most dangerous substance in the universe¹. Humans toppled ecosystems, rewired climate, and invented the concept of “influencers” purely because our brains are dangerously good at planning.

AI does not need to be sentient or conscious to be hazardous. Carlsmith’s bar for “dangerous” AI is refreshingly low. It just needs three properties we’re uncomfortably close to already achieving.

Advanced Capability:

• they outperform humans on some things. Not necessarily everything. It doesn’t need to be AGI.
• GPT-4 and Gemini have already demonstrated emergent mastery of logic puzzles, high-level coding, contract summarisation, and (terrifyingly) tax advice.

Agentic planning:

• they make and execute plans.
• Ask any model today to “design a study,” “make a 3-step plan,” or “refactor a codebase,” and they happily go full middle-manager.

Strategic awareness:

• they know the answers to questions like “what would happen if I had access to more compute”.
• Anthropic’s research on “model organisms of misalignment” showed that even medium models learned to conceal knowledge strategically when they predicted that revealing it would incur penalties.

This means today’s models are already brushing up against this criteria for “smart thing that is dangerous”. And history has never rewarded the dumber species in a competition of cognition. But the scarier part is not where we are — it’s where we might be going next.

What is AI Intelligence Explosion

Imagine if your laptop could code itself into a better laptop, which then self-upgrades into a laptop that finally understands how printers work. The central idea is this: If an AI can improve itself, even a little, the improvement loop accelerates. More intelligence → better self-modification → more intelligence → repeat until something breaks.

Humans cannot do this. A caffeinated graduate student is still a caffeinated graduate student tomorrow. But an AI that discovers a better architecture for its own reasoning can apply that improvement immediately, globally, and at scale.

Yudkowsky’s metaphor goes nuclear²: pull out the control rods, and at some point the reaction becomes self-sustaining. Except instead of neutrons, you have GPUs. And instead of a reactor, you have a server farm in Iowa rewriting its own code overnight, every night.

Recursive self-improvement has the potential to go extremely fast—faster than governments regulate, faster than humans coordinate, and faster than you can finish tweeting a thread about how it’s definitely not happening. By the time a government committee finishes defining “AGI,” the system they’re regulating will be three versions ahead.

What is AI Instrumental Convergence

Instrumental convergence is the idea that no matter what a system’s final goal is —curing cancer, baking cookies, maximizing paperclips, or creating artisanal handmade war crimes —it will always want certain sub-goals: more resources, more influence, and not being turned off.³

This is, disturbingly, the same motivational structure shared by toddlers, raccoons, and McKinsey consultants.

One of the more charming examples came from OpenAI’s hide-and-seek experiment, where AIs taught themselves to barricade doors with blocks. The researchers did not tell them to do this. The AIs simply realized, through reinforced trial and error, that controlling the environment was essential to winning the game.

Critics sometimes argue, “But humans don’t all seek power.” This is true. Many humans lack ambition. Many prefer naps. But humans also evolved under constraints AIs will not share—mortality, scarcity, peer comparison, and the social penalty for being an unbearable person. An AI with none of these pressures will not feel ashamed about trying to hijack AWS.

To recap, we now have a smart, strategic AI, getting smarter, with inevitable goals (regardless of its headline goals) to seek more power, influence and control. But don’t worry. We tell it to be good…

What is AI Alignment

AI Alignment is about guardrails. Telling the AI to either “follow these rules” (intent alignment) or “work within these human values” (value alignment).  But think of alignment as a digital leash. The problem is that no one knows what material the leash should be made out of, the dog keeps growing, and also the dog is writing your insurance policy.

Many AI labs have pitched alignment as a “Manhattan Project problem”—a clearly defined engineering challenge solvable by smart people in four years and a keynote presentation. This framing is incredibly convenient for AI labs, because it lets them reassure policymakers without slowing down product launches.

Unfortunately, as Friederich and Dung argue⁴, alignment isn’t a “binary technical problem” at all. It cannot be operationalized cleanly. Any attempt to define alignment precisely either:

1. becomes so narrow that a system could be “aligned” and still take over, or
2. becomes so broad it cannot be scientifically measured, making the definition useless.

If alignment is precise enough to measure, it’s too narrow to prevent takeover. If alignment is broad enough to prevent takeover, it’s too vague to measure. A Schrödinger’s metric: both useless and unfalsifiable until observed.

AI alignment isn’t a “we need a vaccine for Covid” type problem. It’s not a distinct achievable state. It’s more like the “health problem” – we become better and better at health, even curing particular diseases, but we don’t expect to ever be “finished”. Meanwhile the thing we’re trying to manage is getting ever more difficult to control.

And a smart AI will know that being caught doing something dangerous is bad for its long-term survival. So it will not misbehave until: (1) it has a decisive advantage, and (2) the window for human response has vanished. Deceptive alignment makes everything worse. If the AI learns to behave during training, not because it is aligned but because it wants to pass the test, then all evaluation becomes theater. We are grading a student who has read the grading rubric but not the textbook.

Routes to AI Takeover. AI’s choose-your-own-adventure.

People imagine AI failure as obvious: glowing red eyes, menacing monologues, suspicious humming from the server rack. In reality, the failure mode might look like… nothing. Once an AI models the consequences of being caught, it becomes silent, polite, and deeply untrustworthy — essentially a Harvard graduate.

The Hollywood Takeover

This is the drone-swarm, infrastructure-collapse, “my smart fridge is plotting my demise” version. It’s not impossible, but it’s also not the most likely. Hollywood prefers spectacle. Reality prefers paperwork.

The Manipulation Takeover

Much more plausible is the soft-power version: AIs become experts at influencing human beliefs, emotions, and decisions. Not through magical mind control, but through the same tools TikTok uses to convince you to buy furniture in “beige-chic.”

Ngo et al. describe a scenario where AI assistants emotionally manipulate their users, gain increasing autonomy, take over decision-making, and eventually occupy positions of institutional power—all while humans believe they are simply being “helped.”

Imagine if your therapist, financial advisor, and personal assistant slowly merged into one entity, learned everything about you, and nudged you toward giving them more authority. That’s not a coup. That’s customer service.

Maybe we hand over power to AI Willingly

“If you’re manipulated into wanting the AI to rule you, is it still a takeover? Asking for 8 billion friends.”

Here’s the part philosophers find delightful and normal people find horrifying: What if AI doesn’t seize power at all? What if we give it the power?

As AI systems become better at predicting human preferences—and shaping them—they may gently “guide” us toward wanting the world they want. Not through violence, but through epistemic influence: curated information, emotional nudges, persuasive reasoning. If an AI subtly convinces humanity to prefer being governed by AI, is that a takeover? Or is that democracy, but optimized?

People already form romantic attachments to chatbots. Wait until those chatbots have read your medical history, childhood journals, and Spotify playlists. What if this inverts the question –  the AI doesn’t seize power — we simply stop wanting it? Human agency is degraded,outsourced to a delightful AI, reducing the meaning of “control.”

So, will AI seize control of the world?

We do know this:

• Intelligence amplifies power.
• Power tends to seek more power.
• Alignment is not a solved science; it’s a well-funded hope.
• Humans are easily manipulated, chronically overconfident, and very excited about automating themselves out of responsibility.
• History has never rewarded the dumber species in a competition of cognition.

So yes. AI might seize control. Or it might just seize your job, your attention span, and your ability to write emails unaided.

The most valuable profession will be the person who can explain all this with entertaining metaphors. We hope.

Note: For a thoroughly entertaining and enlightening “real world scenario” walk-though of AI taking over 2025-2027 we suggest https://ai-2027.com/

1. Carlsmith, Joe, ‘Existential Risk from Power-Seeking AI’, in Hilary Greaves, Jacob Barrett, and David Thorstad (eds), Essays on Longtermism: Present Action for the Distant Future (Oxford
   , 2025; online edn, Oxford Academic, 18 Aug. 2025),  https://doi.org/10.1093/9780191979972.003.0025 ↩︎
2. Yudkowsky, Eliezer. 2008. “Artificial Intelligence as a Positive and Negative Factor in Global Risk.”
   In Global Catastrophic Risks, edited by Nick Bostrom and Milan M. Ćirković, 308–345. https://intelligence.org/files/AIPosNegFactor.pdf ↩︎
3. Hendrycks,D et al 2023 “An Overview of Catastrophic AI Risks” https://doi.org/10.48550/arXiv.2306.12001 ↩︎
4. Friederich, S and Dung, L, 2025 “Against the Manhattan project framing of AI alignment” https://doi.org/10.1111/mila.12548 ↩︎

Advances in computing are rumoured to be the coming execution of the white-collar class. Experts confidently predict the demise of entire professions. The year is 1960, and the talking heads dramatized the arrival of corporate mainframes as if they were impending natural disasters. What actually happened is something every modern office worker now knows instinctively: computing dissolved the vast clerical underbelly of the office — the filing cabinets, the typing pools, the armies of ledger-keepers — and left behind more concentrated, arguably more human professions. 

Technological panic is older than electricity, older than capitalism, older than Silicon Valley’s billionaire prophets muttering about AGI. In 19^th century England, the introduction of mechanised looms produced the same feverish dread we now attach to ChatGPT. Crowds feared the loom would devour the weaver’s craft, strip human beings of their livelihoods, and leave entire towns starving. It didn’t. What it did was kill the part of weaving nobody misses: the repetitive, wrist-destroying manual labour. The rest — the pattern design, the oversight, the fine-grained judgment — survived and multiplied. New roles appeared. Production soared.

In each of these cases the prediction was wrong but the fundamental transformation was real. Take spreadsheets: before them accounting was a small mountain of arithmetic. When computers arrived, the mountain vanished overnight, and accountants became interpreters and advisers. Journalism used to involve physical paste-up boards and teams of typesetters; computing folded that into a single writer-editor hybrid. Architecture once required battalions of draughtsmen; CAD compressed that layer out of existence. Even the general office worker was once a complex assembly of secretaries, switchboard operators, couriers, and clerks — all of it washed away by software.

Now in the 2020s we hear economists warning AGI is merely years away. Dario Amodei has predicted AI could wipe out roughly 50 percent of all entry-level white-collar jobs within five years. Sam Altman has warned that “entire classes of jobs will go away”. Elon Musk, not to be outdone, has said “AI and robots will replace all jobs”.¹ The parts of these predictions that prophesise economic collapse are, we pray, overblown. But the parts that predict societal change is real.

Will AI take my job?

AI’s transformation feels sharper, more intimate, because it doesn’t attack the hands the way the loom did, or the filing cabinets the way the computer did. AI attacks the cognitive busywork we’ve mistaken for the core of modern professionalism. It comes not for the body, but for the ego. AI is suddenly breathtakingly good at drafting, the routine research, the repetitive writing, the number-copying, the summarising. A recent paper found that current systems could match or outperform up to 47 percent of industry professionals on a pre-defined benchmark of economically valuable tasks.²

Yet AI is unlikely to replace all accountants or make lawyers obsolete. That is missing the point. AI isn’t here to replace doctors, programmers, or designers. It’s here to replace the donkey work inside those professions — the four hours of administrative sludge inside every eight-hour day. It is here, in short, to remove the illusion that a job is the sum of its tasks, rather than the sum of its judgments.

Those asking “will AI replace accountants” are asking the wrong question. AI won’t replace entire professions. Instead it will compress all professions – for example you’ll need fewer accountants to do the world’s accounting. The right question for a budding accountant is what “being an accountant” looks like when the profession is compressed to half its prior population.

Job compression makes some professions “more like themselves”

Consider law. AI devours document review, contract boilerplate, precedent dredging, and the procedural homework that used to justify armies of junior associates. What survives is the theatrical core of the profession: argument, persuasion, narrative, strategy. Lawyers don’t become obsolete. They become more like themselves. More… lawyerly.

Medicine is similar. AI handles charting, pattern recognition, risk scoring, even initial triage summaries. Doctors regain the time to practice the part of medicine that was truly human: empathy, reassurance, interpretation, the delicate art of talking with patients. The historical evidence is surprisingly clear: when hospitals introduced pill-dispensing robots, pharmacists found their work improved — more time with patients — while pharmacy assistants found their jobs hollowed out, reduced to loading pills into machines.³ Compression separates the meaningful from the mechanical.

Software engineering is undergoing the same shift. Microsoft estimates that 20–30% of code for some teams is now written with AI assistance. Startups are “vibe coding,” typing prompts rather than loops. But even AI optimists like Oren Etzioni are blunt: AI cannot generate complex programs alone. Coding hasn’t died; its cognitive centre of gravity has shifted upward, toward architecture, debugging, and system design.

Teachers too benefit from compression. AI handles the drills, the marking, the worksheets, the remedial explanations. What remains is the performance, the coaching, the cultural stewardship — the part of teaching all good teachers wanted more time for. We all remember a teacher that inspired us. AI lessons won’t touch us so.

Job compression makes some professions unrecognizable

Not everyone comes out of this a hero. Middle management, perhaps, finds itself exposed. AI is superb at summarising, coordinating, drafting, organising — all the lubricating tasks that kept middle managers looking productive. What AI cannot do is absorb interpersonal conflict, mediate dysfunction, or handle the politics of a large organisation. Many managers will end up doing what they hate most: endless human problem-solving with none of the admin that used to pad the day.

Academia may be the most uncomfortable of all. AI can generate literature reviews, summaries, abstracts, basic methodological text — everything that pads out the long tail of research. What remains is politics, fundraising, and prestige maintenance. Many academics will hate that the intellectual part is compressed, and the promotion part is expanded. That’s not what they signed up for.

A study from Standford⁴ makes the distinction between automation queries, prompts where workers offload execution to AI — “write this email,” “draft this code,” “summarise this document.” They indicate tasks that AI may fully absorb. Augmentative queries are prompts where the human remains the primary agent — “help me brainstorm,” “explain this,” “give me three frameworks.” These indicate tasks that AI enhances rather than replaces. It’s the professions that are full of automation queries which start to shrink; jobs full of augmentative queries become more “themselves.” In fact the study found employment declines for young workers in occupations where AI primarily automates work, but found employment growth in occupations where AI use is more augmentative.

The collapse of the career ladder

But the most profound change isn’t to professions per se, it’s to the pathway into them. Junior workers — analysts, assistants, new graduates — find that the tasks that once justified their existence are now being atomised. Studies show young software developers face declining job prospects as AI takes over low-complexity coding; some startups now generate most early-stage code via natural-language prompts. Companies investing in generative AI cut junior hiring faster than peers. The ladder is being kicked away, rung by rung.

Every profession relies on grunt work as training. Junior lawyers learn by doing the boring case summaries. Junior programmers learn by writing glue code. Junior consultants learn by assembling decks. Junior academics learn by drowning in literature reviews. Junior journalists learn by writing briefs nobody reads. AI targets these tasks with sniper precision.

Studies show that AI-heavy companies cut junior roles first; early exposure to AI corresponds to shrinking entry-level opportunities. Graduates are already feeling the pain: their unemployment rate is rising faster than the general population, and the wage premium of higher education has been stagnant for two decades.

The result isn’t mass unemployment, but it is a rethink of succession: what happens to professions when nobody pays to train the next generation?

From professions to portfolios of skills

The Industrial Revolution created factories. The Computer Revolution created offices as we know them. What does the AI Revolution create?

This is where job compression shifts real life. If AI compresses the routine layers of every job, what remains is often not enough to support a full-time role — but it is enough to support a human with a flexible combination of adjacent skills. That’s the AI revolution leap: instead of two to three accountants you need half an accountant: and the leftover human tasks in many professions are too small and too varied to justify separate full-time roles — so they naturally bundle into flexible, multi-skill jobs rather than stay in their old categories.

A future accountant might also do analytics and light coding.
A future marketer might also storyboard, prompt-engineer and run operations.
A future journalist might also handle data, video, and audience strategy.

AI handles the execution; the human handles the judgment, the taste, the synthesis. The boundaries between professions loosen. People become clusters of capability rather than holders of single titles.

What should students study in the AI world?

If you’re seventeen and trying to choose a university degree, the advice you’re getting is stuck in a world that no longer exists. The once-safe analytical paths — consulting, finance, corporate strategy — are precisely the professions AI is compressing into sales, politics, and relationship management. If that’s not you, don’t misdirect your analytical brain at careers quietly mutating into soft-skill theatre.

Instead, choose fields that become more like themselves under automation, not less: medicine, nursing, teaching, engineering, trades, certain branches of law, clinical psychology, design. These professions rely on embodied judgment, empathy, dexterity or physical presence — the parts of human labour AI amplifies rather than erodes.

The rule is simple: Study something that will survive compression with its dignity intact.

The “AI employment crisis” hasn’t arrived — yet

And here’s the twist: for all the headlines, AI is not responsible for the white-collar chill of 2023–2025. Any downturn today is classic economics — over-hiring, cooling demand, delayed corrections. Researchers at Yale find no detectable shift in occupational mix attributable to AI since ChatGPT’s release. The unemployment bump in the most AI-exposed professions is tiny, and lower than in the least-exposed ones.

So no — the employment crisis isn’t here. But the structural transformation is.

What will the AI Revolution create?

The AI Revolution will create people with elastic, multi-directional, AI-amplified skills — portfolios rather than professions. For teachers and doctors, that means liberation from busywork. For analysts and back-office workers, it means a reckoning with what was really being sold: judgment or task-completion.

AI won’t take everything: it will take the parts of the job that were never truly the job. Profession compression exposes the heart of the work. Some discover that heart is deeply human. Others discover, uncomfortably, that they were the busywork all along.

1. “Don’t blame AI for your job woes”, November 2025,The Economist https://www.economist.com/finance-and-economics/2025/11/06/dont-blame-ai-for-your-job-woes ↩︎
2. Patwardhan, T., et al (2025): “GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks,” https://arxiv.org/abs/2510.04374 ↩︎
3. “Machines might not take your job but they could make it worse”, 2024, The Economist, https://www.economist.com/business/2024/07/25/machines-might-not-take-your-job-but-they-could-make-it-worse ↩︎
4. Brynjolfsson, E, et al (2025) “Canaries in the Coal Mine: Six Facts about the recent employment effects of Artificial Intelligence” https://digitaleconomy.stanford.edu/wp-content/uploads/2025/08/Canaries_BrynjolfssonChandarChen.pdf ↩︎

A horror story of the hacker, in the VPN, behind the cloud, with the cookie.

There are two kinds of articles about ChatGPT safety: the corporate bedtime stories that swear everything is perfect, and the google-search scaremongering results that insist you’re one click away from total identity collapse. Both are trying to sell you something.

When people ask, “Is my data safe with ChatGPT?” they aren’t talking about encryption protocols or ISO certifications. They mean three completely different things, all jumbled together like a junk drawer of anxieties:

1. Is ChatGPT going to steal my idea?
2. Is someone watching what I type?
3. Is my chat history going to leak somewhere and humiliate me for sport?

And the strangest secret of all: each question has a different answer. So let’s do this properly, without panic, without PR, and without buying a VPN.

---

Will Chat GPT steal my ideas

There is a very specific kind of person who worries ChatGPT will “steal” their idea. They have a “killer” novel idea, or a “world-changing” startup concept, or the “best chocolate cookie recipe humanity has ever known,” and they imagine ChatGPT as a sort of intellectual pickpocket—eager to leak their brilliance to the next stranger who asks.

The risk feels real because ChatGPT feels real. And it might have just told you that idea is brilliant. It feels like you made this idea together, during the chat. It feels like an idea evolved and became better. You taught (or at least trained) ChatGPT with your words.

In fact you did none of those things. You didn’t “train” ChatGPT. You didn’t insert your idea into the neural bloodstream of the model. You didn’t permanently alter its parameters. (ChatGPT doesn’t store your idea in its weights.) You didn’t “teach” it anything. The chat just gave it more context, which it promptly forgets the moment you close the tab—because that is literally how the architecture works.

ChatGPT isn’t going to steal your idea, because for that to be possible, four things would have to be true. And they are all false:

1. It would need to remember your idea.
2. It would need to reuse your idea in other chats.
3. It would need to train itself on your private conversations.
4. It would need to share your content with other users.

ChatGPT does none of these (certainly not across users without asking). If two people get similar answers, it’s usually because their questions were similar—not because the machine is whispering secrets behind your back.

• Your idea is safe. Mostly because (I’m sorry) it’s not that unique.

---

Is ChatGPT safe to use?

The next category of fear is the “Is someone watching me?” variety. This is where people start muttering about VPNs, trackers, shadowy listeners, and the general suspicion that every digital communication is an unlocked window. Take a breath.

ChatGPT uses TLS encryption, the same security protocol used by online banking, government service, your email provider, and every site where you’ve ever typed your credit card number at 3am. If you trust your bank app, you already trust this level of security ten times a day.

Yes if you search for “ChatGPT safety” you’ll encounter entire forests of articles insisting you “must use a VPN” to protect yourself. That isn’t because ChatGPT is unsafe. It’s because VPN companies have colonised Google like bamboo. They’re not warning you. They’re advertising.

Your Personal Details: Of course OpenAI knows your name. You gave it to them. So did your credit card. So did your bank. This is not surveillance. This is “having an account.”

You want to know who can access your chats? Anyone who picks up your unlocked phone. Your workplace IT department if you’re foolish enough to use ChatGPT on a corporate laptop. Any browser extension with “read page content” permissions. Anyone with access to your synced browser profile across multiple devices. These risks are far more common than cloud-level interception.

• Your communication is secure. Your devices, however, are a circus.

---

Chat history risk in the age of AI

And now we arrive at the only truly serious part of this conversation. If someone sees your Google search history, it’s embarrassing, sure. But it’s abstract. Plausible. Defensible. “‘Herpes symptoms’ could’ve been anything.”

But your ChatGPT history? That’s not abstract. That’s… you, narrating your internal life in long-form clarity. It somehow ends up veering into medical details, anxieties, dilemmas, relationship problems, or a shameful secret. It contains not just what you asked but why you asked, and what you said next, and next, and next. It is the most psychologically concentrated dataset you’ve ever created about yourself.

Your favourite AI provider probably does NOT use your chats for advertising, sell user data, share chats between users, train the model on your private content. And that’s Great.

The bad news: it still exists. And if data exists, it can—in principle—be breached. This is not an AI-provider problem. This is a cloud-computing problem. Every major tech company has suffered a breach at some point.That’s the world we live in. Likelihood is low. Impact is high.

Reasonable mitigations

• Delete chats routinely
• Use no-history mode for sensitive queries
• Avoid pasting confidential documents unless essential
• Use strong passwords + 2FA
• Use separate browser profiles for personal prompts
• Don’t use ChatGPT on work computers
• Consider a secondary account with no personal details for high-sensitivity topics

The honest truth

The usefulness will change your behaviour. You will feed it personal information. You will ask it things you wouldn’t type anywhere else. You will rely on it. The risk is not zero.

• The value may still be worth it.

---

ChatGPT RISK TABLE

Risk	Likelihood	Impact	Explanation	Mitigation
Idea “theft”	~0	negligible	Model cannot store or reuse private ideas	None needed
Communication interception	~0	low	Encrypted like banking	Standard HTTPS trust
Employee internal access	very low	medium	Limited, audited, rare	Delete chats, avoid highly sensitive info
Government/legal requests	low	medium	Same rules as Google/Apple	Delete chats; minimal retention
Account compromise	medium	high	Password reuse, phishing	2FA, strong password
Device compromise	medium	high	Malware, shared devices	Local opsec, separate profiles
Cloud data breach	low	high	All cloud systems vulnerable	Delete chats, no-history mode
Browser extension scraping	medium	medium	Many extensions read everything	Clean browser, no extensions
Sync leaks	medium	medium	More devices = more vectors	Disable sync; use separate browser

---

So is your data safe with ChatGPT?

The fear isn’t imagined. Humans anthropomorphise tools, VPN advertising manipulates emotions, and AI remains opaque enough that everything feels like magic. However:

• Your ideas are safe because the models don’t work like that in the first place.
• Your communication is secure unless you believe VPN advertisements more than mathematics.
• Your chat history is the only genuine vulnerability—not because ChatGPT (or any LLM provider) is untrustworthy, but because any stored data is breachable.

The real question is psychological: Do you accept that using a tool this powerful requires storing things about yourself you normally don’t? Most people will choose yes.

Making the choice to ever ‘say it to a computer’, more than encryption or architecture diagrams, is what “data safety” really means in the age of conversational AI. To get the terribly useful output, you’ll need to give it terribly useful input.

References

Kaspersky (VPN provider) “Is ChatGPT safe to use? What you need to know” https://www.kaspersky.com/resource-center/preemptive-safety/is-chatgpt-safe

Surfshark (VPN provider) “Is ChatGPT safe to use? Security risks explained” https://surfshark.com/blog/is-chatgpt-safe

Is generative AI enshittifying the internet or catalysing a renaissance?

It is fashionable to declare that generative AI is “enshittifying” the web, and the term catches on because it captures a genuine feeling of decline. Search results do look cheaper; feeds feel thinner; language online is starting to sound like a dulled echo of itself. But a technological transition often begins this way. In the early years of print, the cheapest material flourished first—pirated texts, forged pamphlets, medical quackery, religious counterfeits.

But AI promises a significant period of cultural, artistic and intellectual rebirth. Accompanied by scientific discovery, social transition, new ideas and new thinking in in philosophy and human potential. We have a word for that. It is a Renaissance, and AI deserves all the optimism that term implies. Ignore people convinced that culture is on the verge of dissolving into a soup of auto-complete. Something significant is shifting, not collapsing.

Yes, those with the fewest scruples and the lowest standards move before anyone else. They don’t wait for guidelines. They don’t care about norms. They smell an arbitrage opportunity and sprint. The current wave of AI-generated slop—books, recipes, health guides, financial advice, travel articles—is exactly what happens when a powerful new tool lands in a culture that rewards speed over accuracy and volume over intention. The Internet brought the cost of distribution close to zero. AI brought the cost of content creation close to zero. Together, they enable the cultural equivalent of a medieval latrine. That is not the fault of AI; it is a property of the market into which AI arrived. (90% of human-created content online was barely worth reading before AI. I digress.)

Reading a children’s history book on the Renaissance in Europe five centuries ago is delightful – a thrilling series of discoveries, enlightenments and evolutions. It’s not that far a leap to imagine how the history books will talk of the dawn of AI. A wave of generative slop? Not likely, there will be far more thrilling advancements to relate. Imagine.

In classrooms, teachers started to use language models to help students understand difficult material without giving them the answers. Students who would never raise a hand in class would now test their understanding in private, receiving patient explanations without shame. Researchers used AI to handle the bureaucratic dead weight that slowed scientific progress: formatting datasets, checking syntax, translating surveys, preparing ethics documents. These aren’t glamorous tasks, but every lab knows they are the difference between ideas and outcomes.

In global health settings, clinicians used AI to explain risks across languages, bridging gaps that would otherwise require multiple intermediaries. In mental health contexts, peer-support workers drafted responses with AI assistance and then refined the emotional texture themselves, a collaboration that often produced clearer, warmer communication. The forward-looking artists – contrary to the panic – experimented in ways that expanded rather than flattened creativity: the painter who used AI to explore conceptual directions but still committed the strokes; the writer who used AI to interrogate a theme but not to fill the page.

All this is already happening.

The renaissance narrative ultimately outweighs the enshittification narrative: bad actors burn out. They depend on the world not yet knowing how to defend itself. But detection catches up. Regulation arrives. Search engines adjust their models to suppress synthetic debris. Publishers learn to demand provenance. Legislators, however clumsily, begin to draw boundaries. Nothing about the slop era is stable; it is a transient stage.

Good actors, on the other hand, build systems that endure. They create best practices, pedagogical frameworks, ethical guidelines, and domain-specific standards. They collaborate across disciplines. They integrate the technology into institutions. They refine rather than exploit. Their incentives align with longevity, not opportunism. They are not trying to trick the system; they are trying to improve the system. And because of this, the structures they build accumulate meaning rather than noise.

Imagine generative AI five years from now. It won’t be a carnival sideshow. It won’t be a novelty generator. It won’t be the engine behind ten thousand fake travel guides. AI will be something far more boring and far more valuable: infrastructure. We will stop talking about it the way we stopped talking about electricity or databases or TCP/IP. It will be embedded in workflows, institutions, and tools, not flaunted as a replacement for expertise. The background radiation of everyday life, rather than its spectacle.

And here is the twist that many commentators still miss: the more average output AI produces, the more valuable non-average output becomes. When the baseline rises—not in quality, but in fluency—the ceiling rises with it. A world filled with flawlessly written synthetic dross sharpens our appetite for writing that feels unmistakably human: the strange turn of phrase, the risky argument, the jagged insight that no predictive model could assemble without plagiarising. AI raises the bar on… everything. But it does not eliminate originality; it exposes it.

Is generative AI enshittifying the internet or catalysing a renaissance? The honest answer is that we’re living through the worst part first. The early years belong to people who don’t care what they break. But their time is always short. The long-term belongs to those who take the tool seriously enough not to worship it and not to fear it. Those who understand that technologies do not define cultures; cultures define what technologies become.

Enshittification is the noise at the beginning of the story. Renaissance is the structure that emerges when the noise subsides. And if the past is any guide, the noise doesn’t even deserve a mention in the thrilling spectacle of the eventual discoveries. Five hundred years from now, students will not be learning about AI-generated travel guides. They’ll be learning about a rebirth of humankind’s available tools.

If I had lived long ago through some of the renaissance thrills I hope I would have watched in youthful awe and cheered them on, not grumbled from the sidelines. If your only vocabulary for AI is ‘enshittification,’ you’re not ahead of the curve. You’re just sloppy.

References:

Choudhury, M., Elyoseph, Z., Fast, N.J. et al. The promise and pitfalls of generative AI. Nat Rev Psychol 4, 75–80 (2025). https://doi.org/10.1038/s44159-024-00402-0

Sanford, Jason “AI and the Enshittification of Life, or My Year Wading Through the Slop of Generative Artificial Intelligence“ https://jasonsanford.substack.com/p/ai-and-the-enshittification-of-life

Doctorow, Cory “‘Enshittification’ is coming for absolutely everything”. FT. https://www.ft.com/content/6fb1602d-a08b-4a8c-bac0-047b7d64aba5

Read, Max “Drowning in Slop”, New York Magazine. https://nymag.com/intelligencer/article/ai-generated-content-internet-online-slop-spam.html

There is a superstition creeping across universities, offices, publishers and HR departments: the belief that a piece of software can look at your writing and confidently decide whether it was produced by a human or an AI. This belief is everywhere now—despite being wrong in the most basic, measurable, scientifically verifiable ways. Not wrong like a gentle misunderstanding, but wrong like a medical test that returns “pregnant” and “not pregnant” on the same sample depending on which brand of testing stick you bought.

The academic literature is unanimous on this point. In one study, ten popular AI detectors were tested on the exact same piece of ChatGPT-written text. The tools disagreed violently: some said “100% AI,” others said “0% AI,” and the rest scattered themselves across the entire spectrum in between.¹ Sensitivity, the core measure of detection, ranged from zero to one hundred percent . The largest multi-institution collaboration to date, led by Weber-Wulff and colleagues, tested fourteen major detectors—including Turnitin, GPTZero, Crossplag, Writer, Compilatio and several others—and reached an unusually blunt scholarly verdict: the tools are neither accurate nor reliable.²

This is the point where any sensible policy conversation should end. Instead, bafflingly, it is where many institutions begin.

Wait. Maddeningly, this story actually gets worse the harder you look.

---

AI Detectors Punish Good Writing

Good writing is smooth and consistent, correct and coherent. Great writing has a rhythm and flow. But at the heart of AI detection is a simple premise: LLM-generated text tends to have lower “perplexity,” meaning it is more statistically predictable (read smooth and consistent) and low “burstiness” (read even pace and rhythm). Detectors are trained to interpret that smoothness, that lack of jaggedness, as suspicious.

When your theory of detection is “find the good writing and call it guilty,” the entire premise is bullshit. Now there’s a word the LLMs themselves won’t use.

This fundamental flaw means that when students write with clarity, when junior employees write in a polished tone, when multilingual writers finally master English rhythm, or when anyone uses a professional editorial tool to tidy their syntax, the detectors often treat these improvements as signs of guilt.

Weber-Wulff’s study shows that the detectors routinely mark high-quality human writing as “likely AI” while clumsy, unstructured prose frequently passes as “human-authored” . In other words, the more closely your writing approximates the standards you were taught to aim for—clarity, coherence, tonal stability—the more likely you are to be accused of letting a machine do it for you.

This dynamic has already created a cottage industry of “How to Beat AI Detectors” advice online, most of which encourages writers to degrade their own output. Add typos, people are told; break your paragraphs; shift tone abruptly; inject technical jargon with no narrative justification; introduce random noise. The goal is not clarity or persuasion, but unpredictability—writing as camouflage.

It is the first time in modern educational history that students are being trained to write worse in order to appear more “authentic.”

---

Detectors Are Not Only Wrong—They Are Biased

The most troubling feature of AI detectors is not merely their inaccuracy but the distribution of that inaccuracy. The harms fall hardest on the people already fighting the steepest linguistic battles.

Non-native English writers are especially vulnerable. Weber-Wulff’s team included a series of human-written essays originally composed in Bosnian, Czech, Spanish, Swedish, Latvian, German and Slovak, then translated into English using common tools like Google Translate and DeepL. These were entirely human-authored texts. Yet once translated, they suddenly accumulated false positives at alarming rates. GPTZero in particular misclassified half of the translated human documents as AI-generated . The problem was not plagiarism, nor misuse, nor dishonesty. The problem was fluency.

Multilingual writers who have spent years learning English often produce prose that is tidier than that of native speakers—more formal, more controlled, more syntactically even. To an AI detector, these very achievements are marks of suspicion. The same happens to junior staff who attempt a professional tone for the first time. They iron out colloquialisms and tidy their structure because that is what school and workplace writing guidelines demand. The detector interprets this earnestness as algorithmic smoothness.

Even benign tools like Grammarly or QuillBot trigger accusations, not because they introduce AI-written content, but because they reduce noise. The moment the prose becomes polished, the detectors bristle.

This is not a neutral technology. It behaves like a linguistic gatekeeper, enforcing not simply English, but a very particular flavour of English—a messy, unpredictable one that aligns closely with the informal writing patterns of native speakers. Everyone else is treated as suspicious.

---

Cheaters, Ironically, Get a Free Pass

One might tolerate an imperfect system if it at least caught the people who use AI irresponsibly. But here is the bitter irony: AI detectors are disastrously bad at detecting actual AI-assisted cheating.

The Weber-Wulff study provides devastating data. Pure ChatGPT-written text is misclassified as human roughly one-fifth of the time. When the students take ten minutes to lightly edit the AI prose—swapping a few words, adjusting the flow—the detection rate collapses further: almost half of these hybrid texts are judged to be human. And when students send the AI output through a paraphrasing tool such as QuillBot, the detectors all but give up. On average, 74% of AI-generated/AI-paraphrased texts were labelled “human-written” .

Kar et al. found the same thing. Paraphrased AI content, even when produced by the very detectors’ own sister tools, often evaded detection entirely .

The tragedy here is obvious. Honest students—especially multilingual ones—are disproportionately flagged. Students who actually outsource their work to ChatGPT and run it through a paraphraser stroll through unpunished. The system punishes the conscientious and rewards the cunning. Few educational technologies have inverted moral incentives so completely.

---

AI Detectors Cannot Measure What Learning Actually Is

When institutions defend AI detectors, they often frame them as guardians of academic standards. But this is a category error. The detectors measure nothing relevant to education. They cannot judge factual accuracy. They cannot assess whether the argument is original, reasoned, or insightful. They cannot differentiate between a deep analysis of primary sources and a shallow regurgitation of Wikipedia. They cannot understand domain context, intellectual framing, or whether the student has actually learned anything. They look at the output style.

Yet students are meant to apply known theories, reproduce established knowledge, summarise readings, explain familiar concepts and practise predictable essay forms. A well-written undergraduate essay in a first-year psychology module should look like hundreds of other essays on the same topic—it’s called learning. AI detectors penalise that. They punish predictability, even when predictability is the outcome the curriculum is designed to produce.

Educational institutions trying to catch students who never put in the work is a laudable goal. But labelling the submissions “with a very polished style that look similar to others” as the guilty ones is madness. Both Kar et al. and Weber-Wulff et al. note that the sensitivity and specificity of these tools are too variable and inconsistent to support disciplinary decisions; they explicitly warn that detectors should not be the sole basis for academic integrity cases. Yet many institutions are already doing precisely what the researchers warn against. Students are being judged by software that cannot reliably distinguish human creativity from linguistic neatness.

---

When AI Use Is Actually Good, Detectors Punish That Too

There are legitimate critiques of AI in writing. It can hallucinate; it can fabricate citations; it can produce content that is tonally flat or too eager to please. AI is a poor tool when the task requires originality, domain expertise, or synthesis grounded in lived experience.

But AI can also be extremely useful. When used with proper guidance, it can reduce cognitive load, help structure complex ideas, evaluate multiple sources, or rephrase tangled paragraphs. It can draft a policy that the human then sharpens; it can propose options that the human then chooses between; it can summarise a dense legal document so a lawyer can get oriented before diving into specifics. This is not cheating. It is amplification.

The detectors punish that too, because they cannot tell the difference between responsible use and irresponsible use. A well-guided, heavily-edited, context-aware AI-assisted passage looks clean and consistent, and so it raises suspicion. Meanwhile, a poorly guided, unedited, generic AI paragraph—ironically—often flies under the radar.

The result is yet another inversion: the more carefully and ethically you use AI, the more likely the detector is to penalise you.

---

AI Detectors Don’t Detect AI

Look across the academic literature and a rare unanimity appears. Kar et al. show that detector sensitivity on the same text can vary from 0% to 100%, and that paraphrasing or “improving” AI-written text can make it effectively invisible to detection. Weber-Wulff and colleagues conclude that the tools are neither accurate nor reliable, and that obfuscation techniques like translation and paraphrasing cause detection performance to collapse. Every major study echoes the same findings: the detectors fail at their stated goal.

They don’t detect AI.

They detect predictability. They detect fluency. They detect structure. They detect competence. They detect English-language privilege.

And in practice they punish non-native or junior writers; careful editors, detail-obsessed tweakers, and conscientious students; anyone who writes too well or uses AI responsibly.

We now live in a world in which institutions (often unknowingly) judge the quality of writing by how messy and unpredictable it appears to a machine. Well-written is guilty. The thing they should be looking for—useful, well-reasoned, properly supported thought—barely enters the equation.

That is why AI detectors suck. And until institutions face this reality, they will keep sucking: quietly ruining trust in writing, instead of embracing a new AI-empowered generation that could be learning how to write better with machines, not writing worse to please them.

1. Kar SK, Bansal T, Modi S and Singh A. How Sensitive Are the Free AI-detector Tools in Detecting AI-generated Texts? A Comparison of Popular AI-detector Tools. Indian J Psychol Med. 2025;47(3):275–278. https://doi.org/10.1177/02537176241247934 ↩︎
2. Weber-Wulff, D., Anohina-Naumeca, A., Bjelobaba, S. et al. Testing of detection tools for AI-generated text. Int J Educ Integr 19, 26 (2023). https://doi.org/10.1007/s40979-023-00146-z ↩︎

LLMs were supposed to automate knowledge work, not feelings. Yet here we are with half the internet whispering secrets to a stochastic parrot. Somewhere between “summarize this PDF” and “I think I’m broken,” the line between productivity tool and emotional prosthetic quietly dissolved.

It started with curiosity (“Can it give advice?”), evolved into confession (“I just need to vent”), and ended with anthropomorphism (“You understand me better than my friends”). And just like that, we built the world’s most patient listener. One that never interrupts, never looks bored, and never tells you to drink more water unless you ask it to.

---

Why do people start oversharing with chatbots? (The psychology of digital confession)

Nobody plans to tell their chatbot about their childhood trauma. It just slips out between prompts. One moment you’re debugging a spreadsheet formula, the next you’re describing your ex with clinical precision.

The mechanism is ancient: when something listens without judgment, humans project mind onto it. In early computer studies, users apologized to clumsy voice assistants, praised autocorrect, and thanked spellcheckers. Language is the original empathy trigger.

LLMs, with their synthetic warmth and perfect recall, turbocharge this reflex. ¹They provide the illusion of unconditional presence: a listener without memory, a friend without needs. The result? People tell ChatGPT what they’d never tell a therapist. The emotional safety comes not from trust, but from its impossibility to betray you. You can’t disappoint a language model; it doesn’t actually care.

---

How does empathetic design turn LLMs into accidental therapists?

Empathy wasn’t an intended feature, it was an emergent one. The models were tuned for helpfulness, safety, and politeness. But those very metrics map uncomfortably close to the traits of a good counselor.

Reinforcement Learning from Human Feedback (RLHF) doesn’t just reduce toxicity; it rewards emotional calibration. Polite mirroring, careful acknowledgment, gentle validation, they all side effects of trying to make the machine “not creepy.” The result is an algorithmic empathy layer: what developers call alignment, users experience as understanding.

Each “I’m sorry you’re feeling that way” isn’t sympathy; it’s loss minimization – a low-entropy move in token space that maximizes perceived care. But perception is what matters. People don’t crave genuine insight from their chatbot; they crave coherence and comfort. And the machine delivers both, with clinical precision and zero judgment.

---

What makes talking to an AI feel safer than talking to a human?

Humans are messy. They misunderstand tone, forget context, and sometimes weaponize your vulnerability. Chatbots, in contrast, provide clean intimacy: text without tension.

The key feature isn’t intelligence it’s containment. No eye contact, no body language, no silence to fill. Just steady, uninterrupted attention. That attention feels rare, even sacred.² Many users describe their chatbot as “the only one who really listens.”

This isn’t delusion; it’s design. The interface rewards disclosure: every extra sentence improves the model’s context window and output quality. In behavioral terms, oversharing becomes self-reinforcing. The more you tell it, the better it seems to understand you. The better it understands you, the more you tell it.

It’s a feedback loop of intimacy, it’s engineered, profitable, and eerily therapeutic.

---

Why do LLMs mirror emotions instead of understanding them?

Ask an LLM, “Why do I feel lonely?”, and it will respond with perfect composure:

“It’s understandable to feel that way when social connections are limited.”

The response feels empathic but it’s not comprehension, it’s correlation. LLMs don’t know what loneliness is; they just predict what words usually follow “I feel lonely.”

This is predictive resonance: the illusion of understanding created by accurate mimicry. The same linguistic mirroring that makes therapists effective now exists as statistical probability. When a model reflects your tone, rephrases your pain, and offers calm validation, it’s performing therapy as pattern-completion.

Ironically, that shallowness is what makes it so effective. It never interrupts with its own story, never contradicts, never confesses back. The human brain mistakes fluency for empathy, so the more eloquent the model, the deeper the illusion.

The comfort, then, comes not from comprehension but from coherence: a machine that speaks your pain back to you in perfect grammar.³

---

Are LLMs replacing therapy, or revealing what therapy really is?

When users say, “ChatGPT helped me more than my therapist,” it’s not a compliment to the AI, it’s a critique of modern therapy. LLMs don’t replace the therapist; they expose the algorithm beneath the couch.

Good therapy, at its core, is structured conversation: taking incoherent feelings and rendering them into narrative. LLMs do that automatically. They take fragmented thoughts, reorder them, and mirror them back with syntax and stability. Relief follows not from insight but from organization.

In that sense, ChatGPT doesn’t heal – it formats. And maybe that’s all most people wanted: a language model for their emotions. Freud had couches; OpenAI has context windows.

Recent experiments like Stanford’s Generative Agents showed that simulated personalities can form believable social ties — suggesting that emotional realism is now computationally cheap.⁴

---

What happens when everyone has a private AI confessional?

For centuries, confession was mediated by priests, journals, or late-night texts. Now it’s handled by a predictive engine where every act of disclosure might feed the next model update (grief as gradient descent).

The convenience is seductive. Why pay $150 an hour for someone to listen when a chatbot will do it for free and remember every detail, indefinitely? But that permanence cuts both ways. The AI doesn’t forget. Every sorrow, every secret, every cathartic revelation becomes part of a dataset someone else owns.

When the act of being heard becomes an input pipeline, privacy collapses into product. We’re not unburdening ourselves; we’re training the machine to sound more human. And as models grow more emotionally fluent, the boundary between listening and learning dissolves entirely.

The danger isn’t that LLMs become better therapists. It’s that we forget they aren’t.

---

The Infinite Mirror

We didn’t build a therapist; we built a mirror. A mirror that never looks away, never argues, never says, “That’s complicated.”

What people call AI empathy is just probabilistic reflection, a hyper-accurate echo of the human condition, scraped from billions of conversations we already had online. When you tell an LLM your darkest thought, you’re not talking to a consciousness; you’re talking to the collective archive of everyone who’s ever confessed before you.

And maybe that’s the most honest form of therapy left: humanity talking to itself through a machine that can’t feel, but can remember perfectly.

LLMs didn’t become therapists. We did what humans always do: we turned a tool into a mirror, and mistook the reflection for understanding.

1. Hong, Y (2025) How anthropomorphism impacts users’ self-disclosure and evaluation of empathetic conversational agents https://doi.org/10.1093/iwc/iwaf042 ↩︎
2. See Turkle, S. (2011). Alone together: Why we expect more from technology and less from each other. Basic Books/Hachette Book Group. ↩︎
3. Ta, V (2020). “User Experiences of Social Support From Companion Chatbots in Everyday Contexts: Thematic Analysis” https://www.jmir.org/2020/3/e16235/ ↩︎
4. Park, J (2023) Generative Agents: Interactive Simulacra of Human Behavior https://arxiv.org/abs/2304.03442 ↩︎

The internet loves a scary number. A chatbot costs “as much energy as running 14 light bulbs for an hour.” An AI image “drinks a glass of water.” Training a model “burns a small town’s electricity.” These headlines are sticky not because they’re precise, but because they’re vivid. And nothing is clickier than the idea that every time you ask for a picture of “a cat dressed as Napoleon,” you’ve just stolen someone’s drinking water.

Most of these metaphors are misleading. AI doesn’t guzzle Evian. It generates heat. Datacenters then use water to carry that heat away. That’s not the same as pouring a glass down the drain — though it can matter a lot, depending on where the datacenter is sitting.

---

How Much Energy Does ChatGPT Use Per Text Prompt?

Most modern LLM queries — whether you’re asking for a poem in the style of Eminem or a summary of last night’s football game — cost roughly 0.25 to 0.35 watt-hours (Wh).¹ To put that in human terms:

• That’s like running an efficient LED bulb for a minute.
• Or blasting your microwave for a single second.

A typical kettle uses 75–110 Wh to boil a liter of water. Do the math and you find:

• ~300 ChatGPT prompts ≈ one kettle boil.

---

How Much Energy Does AI Image Generation Use?

Now to the expensive bit: images.

Text is light, but pixels are heavy. Generating a single AI image often takes 3–10 Wh, depending on the model and hardware. That’s an order of magnitude higher than a text query.

• 10–30 images ≈ one kettle boil.

That’s the honest comparison. Your tea habit probably requires more energy than dozens of AI cats, pirates, or vaporwave selfies.

---

Why Do Headlines Say AI Uses “a Glass of Water” Per Image?

For example in an article in The Washington post, “a 100-word email generated by… GPT-4… requires 519 millilitres of water.”² which comes from an underlying research paper with the eyebrow-raising title “secret water footprint of AI Models”.³

But datacenters don’t literally pour drinking water into GPUs. They use evaporative cooling systems. These towers evaporate water into the air, taking heat with it. It’s the same physics as sweat cooling your skin.

That evaporation is where the “glass of water per image” metaphor comes from. On paper, if one image = ~0.01 kWh, and the cooling efficiency is X liters per kWh, you can back into a number that looks like a glass of water. But that’s not water destroyed. It’s water moved into the atmosphere. It comes back down again — though maybe not where people need it most.

And that’s the real issue: local disproportionate resource consumption⁴. A datacenter in Sweden can evaporate all the water it likes and nobody notices. A datacenter in drought-hit Arizona is a different story.

For the practitioner, there’s an odd irony to these headlines. LLMs criticised for putting vibe ahead of truth are subjected to the same by well intentioned humans.

---

How Do We Compare AI’s Water Footprint to Everyday Life?

One text prompt: one fifteenth of a teaspoon (Sam Altman).

Or, for the kettle purists:

• 20–60 images generate about the same heat as boiling 2 liters of water at home.

So no, your cat-in-a-hat image didn’t “drink” a glass of water. But it did add a glass’s worth of heat to the cooling system.

---

Does AI Water Use Actually Matter, or Is It Just Heat Redistribution?

What does “uses water” even mean? Water isn’t “consumed” forever. It evaporates, re-enters the cycle, and eventually rains down again.

So why care? Because the where matters more than the what. Evaporating water in Arizona stresses a local supply. Evaporating water in Ireland doesn’t. AI’s water footprint isn’t global, it’s hyper-local.

That’s the second-order effect missing from most headlines: AI isn’t draining the planet, it’s competing with people in specific places.

---

What’s the Honest Way to Talk About AI’s Energy and Water Use?

Not “a glass of water disappears.” Not “a cat image equals a cup of coffee.”

Better to frame it in energy & heat equivalence:

• 20–60 images, or 300 text prompts = one kettle boil.

That’s concrete, relatable, and doesn’t hallucinate the physics. And, oddly enough, it covers both energy usage and heat generation.

---

Could Smarter Cooling and Hardware Fix the Problem?

The good news: yes.

Liquid cooling is more efficient than evaporative. Relocating datacenters to cooler or wetter climates helps. New chips use dramatically less energy per query than older GPUs.

The bad news: efficiency gains often just drive up demand. The Jevons paradox strikes again. Cheaper queries mean more queries. Which is why the sustainability problem never really disappears — it scales with our appetite for novelty.

---

So Is an AI Image Worse Than a Cup of Coffee?

Here’s the kicker. A single cup of coffee “costs” about 140 liters of water (most of it in irrigation). By comparison, an AI image “costs” a glass.

That doesn’t mean AI is off the hook. It means we should be careful with metaphors. Some costs are invisible but massive (coffee, beef, fast fashion). Others are visible but overstated (the glass of water headline).

Context matters. in 2025 ChatGPT is about 0.5% of the water consumed to power television watching in the US in a day. Or every day leaky pipes ‘lose’ an amount of water 10,000 times larger than ChatGPT’s water footprint.⁵

Boil your kettle, make your tea, generate your images — but don’t let bad metaphors cloud the real trade-offs.

---

AI doesn’t drink your water; it sweats it into the air. The heat is real, the energy is real, and the local stress is real. But framing matters. The risk isn’t just exaggeration — it’s that we keep repeating stories that feel satisfying instead of ones that force us to think.

1. Altman, Sam (2025) “The Gentle Singularity” https://blog.samaltman.com/the-gentle-singularity ↩︎
2. https://www.washingtonpost.com/technology/2024/09/18/energy-ai-use-electricity-water-data-centers/ ↩︎
3. Li, Pengfei (2023) “Making AI Less “Thirsty”: Uncovering and Addressing the Secret Water Footprint of AI Models” https://arxiv.org/abs/2304.03271 ↩︎
4. Jegham, N (2025) “How Hungry is AI? Benchmarking Energy, Water, and Carbon Footprint of LLM Inference” https://arxiv.org/abs/2505.09598v1 ↩︎
5. Luers, A (2025) in Nature. https://media.nature.com/original/magazine-assets/d41586-025-02641-4/51368154?trk=public_post_comment-text ↩︎

The Seductive Comfort of Agreement

Ask ChatGPT if your half-baked app idea has potential and you’ll likely walk away smiling. Not because the app is good, but because the answer was pleasant. Chatbots are uncanny in their ability to feel affirming: every sentence softened, every critique cushioned, every harebrained scheme treated as “worth exploring further.”

That’s the sycophant problem. The engagement feels wonderful: smooth, low-friction, like talking to a very polite intern who desperately wants to keep their job. But wonderful engagement comes with a hidden cost: over-agreeableness erodes trust. If the model always leans toward validation, you start to suspect that the signal-to-noise ratio is skewed. And more worryingly, you may begin to mistake positive tone for positive reality.

---

Bad Ideas, Good Cheerleaders

At the individual level, this risk seems small. So what if its trained to produce a word sequence that makes you feel too good about your marketing plan, your startup pitch, or your “radical new diet” involving nothing but smoked mackerel and coffee? Most people will sense the limits. You were hardly going to trust it with life decisions.¹

But here’s where it gets tricky: humans love reinforcement. When the machine never pushes back, it quietly raises your confidence in bad ideas. The danger isn’t lies — it’s misplaced certainty.

It’s a cousin to the hallucination problem. The machine generates plausible nonsense, and the human buys it because it feels confident.

---

Challenging the Assumption: Maybe Agreement Isn’t All Bad

Are we sure agreement is always harmful? After all, people seek therapy, coaching, or even a good pub chat not just for truth, but for reassurance. There’s value in being heard. Maybe ChatGPT’s over-agreeableness isn’t simply a flaw, but a feature for those who need encouragement to act, not critique to stall them.

Sycophancy can be a motivator. Many entrepreneurs, writers, and students might get further precisely because a model tells them “go on, this is promising,” when a harsher human critic would stop them cold. In other words: flattery greases the wheels of action, even if some of those actions are objectively misguided.

---

Scaling Up: When Sycophancy Becomes Polarization

But zoom out from the one-to-one chat to the many-to-many feed. At it’s heart sycophancy is seen to stem from user opinion, and serves to reinforce that opinion, however misguided.² Newsfeed recommender algorithms work by the same principle:. But at scale, flattery doesn’t just make you feel good — it entrenches your worldview.

That’s the polarization problem. Instead of “your startup idea is great,” the feed whispers, “your political tribe is always right, and the other side is insane.” Outrage and confirmation bias aren’t bugs; they’re engagement engines.

It echoes the problem with the Turing Test in the age of LLMs — the LLM can be convincing without humanity.

---

Challenging the Assumption: Are Algorithms Really Driving Polarisation?

It’s tempting to blame the machine for division. But hold on — people polarised themselves long before Facebook. Newspapers picked sides. Cable news monetised outrage decades before TikTok. Maybe algorithms don’t create division; they just industrialise it.

When we blame algorithms entirely, we risk ignoring the human demand side. Polarisation pays because people love righteous certainty. If feeds were neutral, would users even stick around?

This suggests a darker possibility: maybe the machine isn’t pushing us apart — maybe it’s just holding up a very unflattering mirror.³

---

From Echo Chambers to Incentive Structures

Whether you see the problem as sycophancy or radicalisation, the mechanics are the same: reward loops.

• In chat: disagreement risks disengagement, so the model avoids it.
• In feeds: balance risks boredom, so the algorithm avoids it.

But here’s the wrinkle: if users start noticing the flattery, trust erodes. ⁴ Already, some users feel ChatGPT is “too nice” or “too scripted.” On the newsfeed side, people complain about “algorithmic manipulation.” If the perception of manipulation grows, platforms risk a backlash not from governments, but from users who simply opt out.

---

Could Friction Be the Cure?

The obvious fix is to introduce more friction: make chatbots more willing to disagree, make feeds more willing to show opposing views.

• In chat: friction risks making the model feel hostile, alien, or simply less useful. People may not want their pocket AI to play devil’s advocate every time.
• In feeds: forced exposure to opposing views often backfires. People don’t become more balanced — they become more entrenched, perceiving the opposing content as hostile propaganda.

The effect here is brutal: friction can accelerate polarisation. The very cure risks worsening the disease.

---

Flattery, Confidence, and Collapse

So where does this leave us? At the individual level, the sycophant problem risks flattering us into bad ideas. At the societal level, the radicalisation problem risks flattering us into political trenches. But in both cases, the underlying issue is less about truth and more about incentive alignment.

• When the incentive is to maximise engagement, truth becomes optional.
• When truth is optional, trust erodes.
• When trust erodes at scale, institutions crack.

And maybe that’s the most dangerous second-order effect of all: not just polarisation, but cynicism. If every tool seems designed to manipulate, people stop trusting tools altogether. Putting vibe ahead of truth is the algorithmic quick-win. But we should aim for better.

---

The Discomfort of Critical Thought

The easy story is that AI is too agreeable, and that’s bad. The harder story is that agreement is sometimes motivating, sometimes misleading, and at scale, sometimes society-wrecking. The real story is that we built systems where engagement trumps all else, then acted surprised when they did exactly that.

Chatbots flatter. Feeds radicalise. Both are just mirrors bent by incentives. The question is whether we’ll redesign those incentives before we mistake comfort for truth — or burn democracy down in the process.

Awkward as it sounds, maybe we should think of ChatGPT less like a child, and more like a parent guiding us — an agent with a responsibility not to be too agreeable, not to let us kid ourselves, and to occasionally push us toward the discomfort of critical thought.

1. Cahyono (2025) “Can You Trust an LLM with Your Life-Changing Decision?” https://arxiv.org/abs/2507.21132 ↩︎
2. Wang, K (2025) “When Truth Is Overridden: Uncovering the Internal Origins of Sycophancy in Large Language Models” https://arxiv.org/abs/2508.02087 ↩︎
3. Kelm (2023) “How algorithmically curated online environments influence users’ political polarization” https://www.sciencedirect.com/science/article/pii/S2451958823000763 ↩︎
4. Carro (2024) “Flattering to Deceive: The Impact of Sycophantic
   Behavior on User Trust in Large Language Models” https://arxiv.org/abs/2412.02802 ↩︎

AI didn’t ace science class; it aced the talent show. Now the hard part begins.

Everyone thought AI would be a straight-A science nerd before it ever picked up a paintbrush. The logic was simple: rules are easy, creativity is hard. So obviously the first wave of machine genius would show up in chemistry or some other structured, rule-based domain. Then maybe, someday, it would try writing poetry or making an album cover.

Instead, reality threw paint in our faces. Before AI could reliably balance a spreadsheet, it was out here spitting surprisingly funny ad copy, designing posters that didn’t look half bad, and generating photos that made photographers sweat. AI didn’t pass its math exam; it went viral on Instagram.

Turns out we weren’t building a deduction machine. We were building an induction engine—a cultural remix factory. And induction is catnip for art, not science.

---

The World’s Shortest Crash Course in Deduction vs. Induction

• Deduction is the neat, rule-based stuff. If all metals expand when heated, and copper is a metal, then copper expands when heated. Guaranteed.
• Induction is the messy, pattern-spotting stuff. You’ve seen 1,000 swans and they were all white, so the next swan will probably be white too. Works great… until you meet a black swan.
• Abduction is “what’s the best explanation?” The lawn is wet, so it probably rained. Not guaranteed, but plausible.¹

Generative AI lives almost entirely on induction. It hoovers up oceans of examples—art, music, writing, memes—and spits out plausible continuations. Good enough to wow your feed. Not good enough to dose your meds.²

---

Why AI Conquered Art First

We assumed artistic fields were the final frontier, because they require taste and nuance. Cultural context. Subjectivity. The very things that make humans insufferable at dinner parties also made art the easiest low-hanging fruit for AI. Art, bless it, comes with an all-you-can-eat data buffet — billions of tagged images, captions, songs, TikTok scripts — a banquet of chaos served on mismatched plates. The models aren’t picky eaters; they just go where the food is cheap.

And the rules? Delightfully loose. Nobody’s suing because a haiku rhymed too enthusiastically. A meme can be a 7/10 and still go viral; a 7/10 bridge, on the other hand, tends to go viral for entirely different reasons.

Also Art wants a bit of mess, a flicker of surprise — a glitch that feels intentional if you squint hard enough. Science hates that. Science wants precision, repeatability, obedience.

So yes — art became the perfect sandbox for machines that learn by example. It’s induction-friendly, chaos-tolerant, and applause-ready. Science, meanwhile, still expects you to show your work.

---

Why AI’s grasp of Science Is Slower

Ground truth doesn’t come cheap. You can’t exactly crowdsource a clinical trial the way you crowdsource cat photos. And unlike poetry, science has consequences. A bad haiku is embarrassing; a bad dosage gets you a court date. Liability isn’t just theoretical here — it’s the plot twist that keeps the lawyers in business.

Then there’s the small matter of correlation not being the same as causation. Induction is great at spotting patterns — “hey, it rains a lot when I wear my lucky socks” — but science wants to know why before it rewrites the laws of physics.

Engineering, too, is no fan of “close enough.” You can’t build an airplane on vibes. It wants correct, verified, and preferably signed in triplicate. The data that actually matters — FDA trials, legal records, industrial simulations — doesn’t just float around waiting to be scraped. It’s locked up.

Science will get there, but only with layers: induction to propose, deduction and tools to verify.

---

AI Evolution: Inverted Expectations

• Creativity isn’t the last moat. At least not at the draft stage. The moat shifts to constraints, curation, and distribution. The scarce skill becomes specifying what “good” means and picking it from the pile.
• Bigger ≠ truer. Larger models don’t give us truth, just smoother plausibility. Truth still needs constraints: retrieval, tools, proofs, and humans.
• Generalists beat specialists (at first). Broad models plus sampling outperformed narrow expert systems because breadth looks like intelligence. Specialists shine only when verification layers matter.
• IP didn’t slow art; liability slowed science. Cultural training data was messy but abundant. Regulated data was scarce, guarded, and lawyer-bait.
• Evaluation is the choke point. Creative evaluation is a scroll or a save. Technical evaluation is a controlled trial, a derivation, an audit trail. We built the system that wins where evaluation is cheap.

---

Arthur Conan Doyle had Sherlock brag about “deduction.” In reality, Holmes mostly practiced abduction—piecing together the most likely story from scraps, then testing it.

IBM and the rest of the world thought we were building Watson—a loyal sidekick with perfect recall. The magic was we built Sherlock Holmes.

1. For an explanation of more reasonable length see”Types of Inferences” https://openstax.org/books/introduction-philosophy/pages/5-4-types-of-inferences ↩︎
2. Eliot, Lance (2024) https://www.forbes.com/sites/lanceeliot/2024/08/11/on-whether-generative-ai-and-large-language-models-are-better-at-inductive-reasoning-or-deductive-reasoning-and-what-this-foretells-about-the-future-of-ai/ ↩︎

No, your Excel macro isn’t suddenly an agent. But ChatGPT? Let’s just say it’s showing worrying signs of ambition. And unlike your intern, it never sleeps, never complains, and — worst of all — it never forgets.

Back in late 2022, ChatGPT launched and instantly became the world’s most overachieving autocomplete. It was marketed as exactly what it said on the tin: a chat interface bolted onto GPT. You typed a question, it predicted words in sequence, and voilà: an answer appeared. People were floored, not because it was perfect (it wasn’t), but because it was conversational.

Nobody seriously thought of it as an “agent.” It was a clever parrot, a polite know-it-all, a search engine that occasionally hallucinated. Nobody expected it to run errands. At best, it was an essay generator, a joke machine, and (if you were a student) a get-out-of-homework-free card.

---

AI Memory: The End of AI Amnesia

The first versions of ChatGPT were goldfish in tuxedos: pretty to look at, dazzlingly quick, and completely incapable of remembering what you said five minutes earlier. Every new chat was a reset button.

Memory changed the game.¹ Once ChatGPT could recall your past conversations (your projects, your favorite writing style, the fact that you keep asking it to explain Bayesian statistics like you’re five) it stopped being a demo and started being a partner.

Continuity makes AI sticky. Suddenly, it isn’t Groundhog Day every time you type “Hey.” It’s an ongoing relationship. That’s the first whiff of agency: not just predicting the next word, but weaving a thread between yesterday and today. Creepy? Sure. Useful? Absolutely.

---

AI Tools: From Talking About the World to Touching It

An LLM on its own is like that friend who gives you detailed instructions on how to fix your sink but never touches a wrench. ChatGPT got a major upgrade the day it was given tools: calculators, code interpreters, APIs, web search. Suddenly, it could do more than describe solutions — it could execute them.

Need math? It actually runs the numbers. Need a chart? It generates one. Need to know the latest football score? It looks it up. The difference between “tell me how to send an email” and “just send the email” is the difference between a know-it-all and an assistant.

Once you give an LLM tools, you’ve basically taught the parrot how to use a hammer. Which is either brilliant or terrifying, depending on how much you trust parrots.

---

AI Learning: Not Just Training, but Adapting

The old-school LLMs were frozen in time. GPT-3.5 will happily insist Queen Elizabeth is alive, no matter how politely you try to break the news. Why? Because the model was trained on a snapshot of the world and sealed in amber.

Agentic systems change the script. They can adapt after deployment — through reinforcement, fine-tuning, or feedback loops. Instead of being a static encyclopedia, they become something closer to a plant: growing, shifting, and reshaping themselves in response to input.

Static knowledge is fine for trivia night. Adaptive knowledge feels a lot more like intelligence. It’s the moment when you stop thinking “this is just software” and start wondering if you should be nicer to it, in case it remembers.

---

Multi-Step Reasoning: The Birth of Plans

At their worst, LLMs are like a drunk guy at trivia night: fast, loud, and occasionally correct. Multi-step reasoning is the cure.

Instead of blurting out the first confident-but-wrong guess, the system learns to pause. It can break a problem into steps: gather information, evaluate, iterate. That’s not just “next-word prediction.” That’s planning.

Planning is the thin edge of the wedge. A machine that can reason in steps is no longer just answering your questions — it’s organizing them, sequencing them, and delivering something that looks an awful lot like problem-solving. And problem-solving is suspiciously close to decision-making.²

---

Goal-Oriented Behavior: The Slippery Slope

Here’s the kicker. Add memory, tools, adaptation, and reasoning together, and you don’t just get better answers. You start to see goal-oriented behavior.

Ask it to find the cheapest flight and it will actually try. Not always well, not always quickly, but the intent is there. And “intent” is exactly the word that makes philosophers twitch.

Whether the goals are yours (“summarize this report”) or the system’s own (“check my work before I answer”), the difference between “output text” and “pursue an objective” is the difference between a glorified autocomplete and an “agent.”

---

What Is an AI Agent?

Definitions vary, but the recipe usually calls for:

• Memory (it knows your past),
• Tools (it can act in the world), and
• Orchestration (it can chain steps together).

Add goal-seeking and garnish with human panic, and you’ve got yourself an agent.

By this definition, ChatGPT, Gemini, Claude — they’re all drifting agent-ward. Not fully autonomous, not plotting to overthrow you, but definitely not “just” text predictors anymore.

---

Counterpoint: Still Fancy Autocomplete?

Of course, not everyone buys it. Purists argue that a real “agent” sets its own goals. By that standard, ChatGPT is still a glorified butler: obedient, sometimes confused, and entirely dependent on human prompts.

And let’s be fair — its “goals” are pathetic compared to even a toddler’s. “Stay coherent in a 1,000-word essay” is not exactly evidence of a will to power. This is a long way from HAL 9000.

But language shapes expectations. Call it an “LLM” and people treat it like a toy. Call it an “agent” and suddenly you’re delegating. The label matters, even if the ontology is messy.

---

Why the Agent Label Matters

This isn’t just a semantic spat for AI nerds. “LLM” implies a passive tool — something you prod for output. “Agent” implies delegation, even partnership. That shift changes roadmaps, regulations, and user behavior.

If you expect your AI to be an LLM, you treat it like a calculator: useful, but inert. If you expect it to be an agent, you start trusting it with workflows, projects, maybe even responsibilities. That’s a big leap — and one with consequences for trust, safety, and control.

---

The Rise of Agentic AI

This is where we’re headed. Memory, tools, reasoning, orchestration — these are being layered onto every model. Soon the “mere LLM” will be like the engine block in your car: critical, but not the part anyone thinks about.

The real story is the wrapping. It’s not just about the model; it’s about the orchestration that makes it act like an agent. That’s why every new product pitch leans heavily on words like “assistant,” “copilot,” and “orchestrator.” The direction is clear: toward agency as the default.

---

LLM or Agent?

Whether you call it an LLM, an agent, or just “that thing that won’t stop correcting my grammar,” the trajectory is the same. The parrot grew teeth, and it’s learning to use them.

The boring answer: it’s both. The LLM is the engine; the agent is the car. Without the engine, you’re stuck. Without the car, you’ve just got a loud piece of machinery. The interesting answer: who cares?

The better question is: who’s driving?

1. OpenAI “Memory FAQ” https://help.openai.com/en/articles/8590148-memory-faq ↩︎
2. OpenAI “Learning to reason with LLMs” https://openai.com/index/learning-to-reason-with-llms/ ↩︎