What You Must Never Let The Machine Decide

A line of argument is gaining ground that goes like this. Humans are biased and self interested, so we should let AI settle our hardest disputes, because it has no stake in the outcome.

It sounds reasonable and it fails at the first step. I want to walk through why, because the same error shows up in a much smaller and more expensive form inside businesses, and that version is the one that will actually reach you.

Why it fails

Those questions are not unresolved because we lack information. They are unresolved because people want different things.

A dispute about what is fair, or what should be protected, or whose interest wins when two legitimate interests collide, is a values conflict. More analysis does not resolve a values conflict. It just produces a better argued version of whichever value you started with.

So when a machine produces an answer to a values question, it has not been neutral. It has applied a position, one that came from whatever it was trained on and whatever its builders decided. The appearance of neutrality is the dangerous part, because a position wearing the costume of objectivity is much harder to argue with than one that admits what it is.

The tell

Watch what happens when the machine's answer is unwelcome.

The people advocating for machine arbitration are almost always confident about which way it will come out. When it comes out the other way, the enthusiasm for machine arbitration evaporates immediately, which tells you the argument was never really about neutrality.

That is not unique to AI. It is what people do with any referee. But it is worth noticing before you hand something important to one.

The small, expensive version inside a business

Here is where this reaches you.

You will be offered tools that decide things. Which lead gets called first. Which customer gets the discount. Which employee's schedule bends when two people need the same day. Which invoice gets chased.

Some of those are fine to delegate and some are not, and the distinction is not about how sophisticated the tool is. It is about whether the decision trades one person's interest against another's.

The test I use

If you would have to explain the decision to the person it affected, you own it.

Routing a call to whoever is free does not require an explanation. Nobody asks. Give it to the machine.

Deciding which of two employees loses the holiday they both requested does require an explanation, and the explanation cannot be that the system chose. That answer destroys something you cannot rebuild cheaply, because what the person heard is that nobody was willing to be responsible for it.

Same for customers. A scheduling algorithm is fine. An algorithm that decides which customer gets accommodated when you cannot serve both is a decision with your name on it, and outsourcing it does not remove your name, it just removes your involvement.

What you actually lose by delegating too much

Authority, in the original sense. Being the one who is answerable.

A business where the owner can explain every decision that mattered is a business people trust. A business where the answer is increasingly the system does that has quietly transferred its authority to a vendor, and customers feel that shift long before they can articulate it.

This is the whole asset for a local operation. It is the reason somebody calls the person in Valencia instead of the national number. Handing away the decisions that constitute it, one convenient tool at a time, is the most likely way to lose it.

Where I land

Delegate the decisions that have one agreed measure and no aggrieved party. Keep the ones where somebody could reasonably ask why.

That line is not about trusting machines more or less. It will not move much as the tools improve, because it was never about capability. It is about who is willing to stand behind the outcome, and that has always been a question about people.

Common questions

Connor T. MacIvor · CalDRE #01238257 · Sync Brokerage, Inc. · DRE #02031490