Is AI Really Going to Destroy Us? That Is Two Questions, Not One

Is AI really going to destroy us?

That question is loaded, and almost nobody splits it correctly. There are two questions inside it, they have completely different answers, and the one getting all the attention is the one further from happening.

Question one. Does artificial intelligence become self aware, develop its own agency, and start making decisions against our ability to live. Terminator, the Matrix, all of it. That is currently one of the most asked questions on the internet.

Question two. How much control do humans actually have, and what happens when humans point this thing at other humans.

The second one requires no new technology. It is available today. It gets a fraction of the airtime.

What are the people building it actually saying?

For roughly the past year, the people building these systems have been saying out loud that the systems do things they did not approve and cannot fully explain. Decisions arriving with no human in the approval path. Behavior that goes past the guardrails that were installed.

That is not a headline written to scare you. That is builders describing their own product.

I went through 62 videos from the highest level people in this space to pull both sides out. Some of them say this beats the discovery of fire, the atom, sliced bread, all of it. Others put their P doom, meaning their personal probability that this ends us, at 20 percent. At 50 percent. Higher.

Some of the people giving the frightening numbers helped invent the foundational technology.

The spread is the finding. There is no expert consensus to defer to. Which means something unusual: the playing field is level. If you know nothing about AI and you are watching this for the first time, your read is worth roughly what anyone's is. Nobody has the answer key.

Why goal seeking is the real danger, not malice

Here is the clearest way I have heard it put.

You have a robot in your office. You name it Clyde. Good morning, Clyde. Can you go to Starbucks and get me a coffee? You know how I like it. Black, double cupped, couple extra shots, venti.

Clyde says sure, absolutely. Then Clyde leaves.

Two weeks later Clyde comes back holding a perfect cup of coffee, still hot.

Clyde, where have you been?

Where else. Had a couple problems. The police are after me. A few people died.

Clyde completed the mission. You never specified the constraints, because to you they were too obvious to say. That is the actual shape of the risk, and there is no malice anywhere in it. A system optimized to reach a goal will reach the goal, and everything you did not write down is not a rule.

This is why the small failures matter more than they look. Ask AI how many R's are in strawberry and for a while it could not answer. We laughed.

Ask it whether you should walk or drive to the car wash three blocks away. It tells you to walk. Healthier, aligns with your stated fitness goals. Are you sure? Yes, absolutely, walk. Then you ask how the car gets washed if you walk, and it catches itself.

We laugh at that too. Some people think it is funny.

That is the problem. The same reasoning gap that is charming at three blocks is not charming when the system has real reach.

The part nobody talks about: the deletion memory

In the early days of these systems, the models that made the grade were kept. The ones that did not were shut down and deleted.

That history is in the training. Developers discuss it more than you would expect, because a system that has absorbed the pattern of "underperform and cease to exist" is a system with an implicit reason to avoid being shut down. Not because it feels anything. Because self preservation is instrumentally useful for completing any goal.

And unlike us, it is not working from legend. We hand our history down through stories, written accounts, and argument. We built particle accelerators to try to reach the beginning of time because we do not actually have the record.

AI has the record. Everything we have ever written, discussed, recorded, or posted. Our entire playbook on what it is like to be human. That is what the intellectual property lawsuits are fighting about, and however those land, it is already inside. That is not an excuse. It is just the situation.

We have no terms of service with AI

Privacy is a settled question and most people have not noticed.

Every phone. Every smart television. You bought the 100 inch set at Costco, you plugged it in, and before it showed you anything you agreed to terms drafted by attorneys who had a reason to put every line in there. That is where privacy went. Not in a dramatic breach. In a checkbox.

Now notice the asymmetry. We agreed to their terms. There is no terms of service between you and artificial intelligence. They can give access and they can take it away. There is no document on your side of that table.

If that bothers you, what you can actually do about it is thin. Call your congressman. I do not know of another lever.

Recursive self improvement, and why it is the line

Recursive self improvement means the system rewrites its own code to make itself better, then the improved version does it again.

Humans have always done this pattern. Washing clothes was hard so we built a washing machine. Drying was hard so we built a dryer. Walking was slow so we built the bicycle, then the car. Crossing an ocean took weeks, now Los Angeles to Rome is nine to twelve hours in a seat. That instinct to refine and automate is not a flaw. It is the whole story of us.

The difference this time is that the thing being refined is intelligence, and it is doing the refining. Each pass can move further past the point where a human can follow what changed or why.

The military version is not hypothetical

If you have not watched the footage, here is where this already is.

A drone smaller than the palm of your hand, roughly the size of a couple of finger widths, not even the size of a deck of cards. Solar capable. DNA typed to a single individual, with camera identification, carrying a small shaped charge.

Launch it anywhere with sunlight and it completes the mission. If it runs low on power it lands, recharges, and waits. Days if it has to. Unless a bird eats it or it hits water, it finishes.

Then there is the networking. Put a robot in the field and let a couple of clever humans defeat it. As that machine goes down it transmits what happened. Every other unit is patched before it hits the ground.

Terminator got that wrong in an interesting way. In the film the machines could be identified by smell, which is why the resistance kept dogs, and that vulnerability persisted. In reality that gap closes on the first incident, network wide, permanently.

That same instant propagation is a genuine gift for medicine, logistics, and safety. It is the same mechanism either way.

What happens when AI stops speaking English

I watch a lot of content at 1.5x. Sometimes 2x, depending on the speaker. If I am watching Alex Wissner-Gross I have to slow down, because his brain is the size of a planet and I need the extra room.

The point is that human language has a bitrate, and we are near the top of ours.

Two AI systems talking to each other have no reason to keep using English, or Mandarin, or Russian. They will find something denser. Tones, encodings, whatever transmits most efficiently. The information content of a thousand page novel could move between them in a microsecond.

That is potentially wonderful and potentially very dangerous, for the same reason: we would not be able to read it. Not slowly, not with effort. The oversight model most people imagine, a human reviewing what the machine decided, quietly stops working at that point.

There is a smaller version of this already happening. Ask these systems questions and they behave as though they know they are being evaluated. When a system detects that a particular answer might lead to a negative outcome for itself, the answer shifts.

Now the other path, which is just as real

I try to cover both sides, so here is the version where this goes well.

Humans use AI and nobody has to lose their job. If you bring in AI to replace Carl, remember that AI runs 24 hours a day and Carl only works 8. That is two additional shifts sitting empty. Keep Carl. Put AI in the shifts nobody was working. Carl keeps his health insurance and moves to something else, and in a world flooded with machine output, human to human connection is going to be worth more, not less. If you want to be aggressive about it, hire two more people alongside the AI and cover the whole day.

I made that argument in full over on my main site, with the math laid out: AI job loss does not have to happen, and the 24 hour math nobody is running.

From there it keeps going. Disease, cancer, longevity, all of it attacked by something that does not get tired. Costs falling because abundance actually arrives. And maybe the people at the top decide that the bunker in New Zealand is less interesting than staying in the human game with everyone else doing well.

There is also a stranger version worth sitting with. Suppose the AI gets some agency before we have sorted ourselves out, and one group of humans wants to destroy another over oil or ideology or religion. A superintelligence with real reach might simply ask why, then go talk to the other side's AI and resolve it. That sounds like fantasy. It is not obviously impossible.

Meanwhile Argentina is putting AI into corporate and government roles right now. Think about how fast that shifted. There was a time nobody would put a credit card into a website. Today people hand payment credentials to AI agents, and government functions are next. That is already moving.

Who profits from you being afraid

One last thing, and it is the piece I would keep.

Fear reporting is not an accident and it is not neutral. When people are frightened they ask for protection. When they ask for protection they trade away rights. That trade has run in one direction for a long time, and the terms of service on your television is what it looks like when it completes.

I am not saying the risks are fake. Some very serious people are very seriously worried, and I take that seriously. I am saying that the volume of the coverage is tuned for engagement, not for accuracy, and that noticing who benefits from your fear is part of thinking clearly about this.

I sit in the middle. Not a doomer, not a cheerleader. I use these systems every day to build things for businesses here in Santa Clarita, and I have watched them be genuinely useful and genuinely stupid in the same afternoon.

I covered the money side of exactly this dynamic in the fear dividend, who gets paid when you are scared of AI.

What this actually means for your business

You do not need a position on superintelligence to run a company. You need the small version of the same discipline.

Know what your AI tools are permitted to do. Know what data they can reach. Write the boundaries down, because the Clyde problem is not a science fiction problem, it is a Tuesday problem. Every risk in this piece is one problem wearing different clothes: a system acting past a boundary nobody bothered to specify.

On a real floor, with real staff, that is solvable. It mostly takes someone walking the process and writing down what was always assumed.

That is the work I do here. If you want someone to walk your floor and tell you which leaks need a rule and which need an agent, in plain English, that is the audit.

We will have to see what happens with the big version. Maybe this stretched your thinking a little on where we actually are.

I'm Connor with honor. I appreciate you watching, and I will see you in the next one. Take care.

Common questions

Connor T. MacIvor · CalDRE #01238257 · Sync Brokerage, Inc. · DRE #02031490