Will AI Destroy Us?

Here we are. Five posts ago I promised we'd end with the big one — the question that gets eye-rolls from two directions at once. Say "AI might destroy humanity" at a dinner party and half the table hears science fiction while the other half hears prophecy. Both halves stop thinking.

My rule for this post is the same one I've used all series: no hype, no doom, only what can be honestly said. And I'll tell you exactly where I land, because you've read this far and you're owed a straight answer.

Before the arguments, one piece of evidence that doesn't fit anyone's comfortable story.

In May 2023, hundreds of AI researchers signed a single-sentence public statement declaring that the risk of extinction from AI should be treated as a global priority on par with pandemics and nuclear war. The signatories included Geoffrey Hinton and Yoshua Bengio, two of the three researchers whose work made modern AI possible. Hinton had just quit his job at Google specifically so he could say these things freely; the next year he won the Nobel Prize in Physics for the very research he was now warning about.

Here's the strange part: the CEOs of the leading AI companies signed it too.

Stop and notice how odd that is. The people building the technology, whose fortunes depend on it, publicly compared their own product to nuclear war, and then went back to work building it faster. I can think of no other industry in history that has done this. Whatever it means — and people argue about what it means — it tells you this question is not a fringe obsession. It lives at the center of the field.

Strip away the movie imagery, because the serious version of this concern has nothing to do with robots deciding they hate us. It rests on three claims, each individually hard to dismiss.

First: capability keeps arriving ahead of schedule. Ten years ago, most experts thought AI that writes working code, passes professional exams, and holds fluent conversation was decades away. It arrived early, repeatedly. When your predictions keep breaking in the same direction, humility about the next decade is warranted.

Second: nobody fully understands these systems, including their makers. This surprises people, so let me be blunt about it. Modern AI isn't programmed line by line like normal software; it's grown by training, billions of internal dials tuned automatically. The result works, but its builders cannot fully explain why it produces a given answer, or reliably predict what it will do in situations they didn't test. My own doctorate lives in exactly this gap: an entire research field exists because we deploy systems we can't fully interpret. We're getting better at it. We're not close to done.

Third: the classic engineering problem of getting what you asked for instead of what you wanted. Every wish-granting story ever told, King Midas onward, is about this. With today's AI it's mostly comic: the chatbot that follows your instructions into nonsense. The worry is what happens as we wire these systems into things that matter — markets, infrastructure, weapons — and give them more autonomy, while the gap between "what we said" and "what we meant" stays unsolved. Add the race dynamics from post 5, where no company can afford to slow down, and you don't need malice anywhere in the story to get catastrophe. Just speed, opacity, and nobody in charge. Sound familiar? It's this whole series, extrapolated.

The case for calming down

Now the other side, which is also made by serious people, including AI pioneers of equal rank. It goes roughly like this.

Everything in post 1 is still true: these systems are pattern-matchers. They have no goals, no desires, no drive to survive. They predict text. The leap from "predicts text impressively" to "outmaneuvers humanity" is exactly that — a leap, resting on extrapolation rather than evidence. One famous researcher compared worrying about it now to worrying about overpopulation on Mars: a problem for a civilization that doesn't exist yet.

The skeptics add a sharper point, and I want you to hear it because it's the strongest one: doom talk is suspiciously convenient. "Our product might end the world" is, among other things, a marketing claim. It says this technology is the most powerful thing ever made, invest accordingly. It also pulls regulatory attention toward hypothetical future risks and away from the documented present ones — the bias, the fraud, the power bills — that this series spent four posts on. Some critics argue the extinction framing functions as a distraction, whether or not it's intended as one. Given who funds what in this debate, that suspicion isn't paranoid. It's hygiene.

And the disagreement among genuine experts is real. This isn't a case of scientists versus cranks. It's Turing Award winner against Turing Award winner. When the deepest experts split this hard, honesty requires admitting the truth: nobody knows.

Where the two camps secretly agree

The shouting hides something useful: strip the vocabulary and the camps overlap more than either admits. Both agree nobody fully understands these systems' internals. Both agree the competitive race creates pressure to cut corners. Both agree present-day harms are real and under-governed. Both agree capabilities will keep growing. The fight is about the tail of the distribution — how bad the worst case gets — not about whether the situation is well-managed. Nobody thinks it's well-managed.

That's also roughly where the most credible neutral referee landed. The International AI Safety Report, a scientific review chaired by Bengio, written with over 100 experts and backed by more than 30 countries, found that current systems show early signs of some concerning capabilities but nothing near loss-of-control levels, and described the likelihood and timing of such risks as deeply, unusually ambiguous. Translation: not now, can't rule it out, nobody can honestly tell you the odds.

Where I land

So, will it destroy us? My honest answer: I don't know, no one does, and anyone who tells you the probability with confidence, in either direction, is selling something.

But here's what I've realized writing this series: for practical purposes, the question matters less than everyone thinks. Because look at what prudence demands in each case. If the existential worriers are right, we need transparency into how these systems work, testing before deployment, accountability for failures, and the ability to slow down when warranted. If the skeptics are right and the real dangers are bias, fraud, concentration of power, and infrastructure strain, we need… the same list. Every road leads to the boring machinery of post 5. Uncertainty about the destination doesn't change the next hundred miles of road.

What I actually lose sleep over isn't a machine that wakes up. It's the compounding of everything this series documented: systems nobody fully understands, deployed at civilizational scale, by companies structurally unable to slow down, governed by almost no one. That trajectory doesn't need a dramatic ending to be a bad one. And unlike the sci-fi version, every part of it is fixable with tools we already know how to build.

Why I'm still here

Six posts ago I told you I think this technology is genuinely useful, badly governed, and moving too fast, and that I'd try to earn each claim. You've now seen my evidence for all three.

So why do I still work in AI, teach it, defend it at dinner tables? Because the technology took fifty years of protein science and compressed it into minutes. Because it reads mammograms at 4pm as well as at 9am. Because it caught a security flaw that attackers were about to use. And because none of the problems in this series — not one — is a property of the mathematics. They're properties of how we've chosen to build, deploy, and govern it. Choices can be changed, but only by people who understand what they're looking at.

That was the whole point of these six posts. Not to make you love AI or fear it, but to make you hard to fool about it — by the marketing, by the doomers, by the mystery. You now know what the thing actually is, what the buildings actually do, what the wins and failures actually look like, and where the rules actually stand. That's not a small thing. Decisions about this technology are being made right now, mostly without public input, partly because the public conversation runs on cartoons.

You're no longer arguing with a cartoon. Go be the most annoying person at the dinner party. Ask what's in the building. Ask who checked the model. Ask who's in charge.

Sources worth your time: the Center for AI Safety's 2023 statement and signatory list and the International AI Safety Report 2026.