by Barry Goldman
My late father-in-law was an old-school family physician. He didn’t believe a computer would ever be as good a diagnostician as a human being. He understood that – in principle at least – a computer could read all the medical literature in every language and remember everything it read. He understood a computer would never over-weight its recent “experience” the way a person might. He knew a computer would not be subject to confirmation bias or groupthink or the rest of the errors to which the human mind is susceptible. But he still believed a good human doctor would always be a better diagnostician than a machine. He believed his decades of experience gave him a special sense that could not be captured in zeroes and ones.
I don’t know what Old Doc Silk would say now that AI can pass the bar exam and write symphonies, solve complex math problems, read x-rays, and write code. I suspect he would change his mind. Nevertheless, I find myself taking his position with regard to the work I do. I don’t think there will ever be a satisfactory AI labor arbitrator or AI judge.
The distinction turns on what we are trying to accomplish. Let’s start with an easy case. We know what an AI chess player is trying to accomplish. It is trying to win chess games. If it can beat its opponents, it’s a good chess player. And if it can beat all its opponents, it’s a champion chess player. Beating all its opponents just is what a chess champion does. The same is true of an AI that reads radiology scans. We know what a tumor is, and we can tell when a tumor has been correctly identified. If we have an AI that correctly identifies more tumors than all the other tumor detectors available (without falsely identifying things that are not tumors), then it is the best tumor detector. There is nothing else to it.
But what would a champion labor arbitrator or a champion judge do better than its competitors? If the answer is it would dispense justice better, that’s fine. But then we need to know what justice is. And that’s the problem we started with. Identifying justice is what we need judges and arbitrators for in the first place.
My point is we can’t start the process with a question, we need to start with an assertion. If you want your image recognition program to identify hats, first you have to train it with a large number of known hats. You tell it, “These 10,000 images are hats. Now what about this image, is it a hat?” Then you “reward” it when it guesses correctly and “punish” it when it guesses wrong. After a million trials it gets good at identifying hats. Or tumors, or winning chess positions.
We can tell the radiology AI what the necessary and sufficient conditions are to identify something as a tumor and then send it out to find tumors in the wild. We could ask the AI judge to do that if we knew the necessary and sufficient conditions to identify a decision as a just one, but we don’t. We can’t even agree which cases are the properly-decided ones. See Roe v. Wade and Dobbs.
This is the lesson we try to teach philosophy students when we introduce the Lifeboat Game. Suppose we have six people trapped in a leaky lifeboat. All six will die if we don’t throw one of them overboard. Who will it be? We’ve got a old person who lived an exemplary life, a young person with great potential, a concert violinist, a brilliant scientist etc., etc. You get the picture. The instructor sets the scene and opens the discussion. There is no limit to the amount of time you can eat up playing this game. Why? Because there is no agreed-upon criterion. We can’t answer the question unless we can establish what we’re trying to accomplish. And to establish what we’re trying to accomplish we need to know, as Kurt Vonnegut liked to ask, “What are people for?”
Back to justice. What is it to do the right thing? I happen to know because my book club just read Justice: What’s the Right Thing To Do? by Michael Sandel. It turns out it depends on who you ask. The utilitarians think the right thing is the one that maximizes happiness for the greatest number. The Kantians point out that that would permit you to torture an innocent child if it would make enough other people happy. They say to do the right thing you have to obey the categorical imperative. The utilitarians point out that obeying the categorical imperative would require you to tell the truth to the ax murderer who asks where your mother is hiding.
The problem with both utilitarianism and Kantian ethics is, at least in some cases, they violate our moral intuitions. So maybe the true test of ethical behavior is moral intuition. But we know that’s not right either. Moral intuitions are all over the map. The trolley problem has been discussed enough so we know almost everyone would flip the switch to kill one person rather than five, but far fewer would push the fat guy off the bridge to accomplish the same thing, and no one can explain why that makes moral sense.
Paul Slovic’s work on psychic numbing and the identifiable victim effect provide incontrovertible evidence of the incoherence of our moral intuitions. See, e.g., The more who die the less we care.
So what is to be done? I want to propose an answer to that question. It comes from Kurt Gödel’s idea of incompleteness in mathematics. Gödel said no logical system complex enough to describe arithmetic can be both complete and consistent. I say no ethical system complex enough to be interesting can be complete and consistent. Any system of ethical decision making will produce some results that conflict with moral reasoning. Utilitarianism sometimes produces monstrous results; Kantian ethics sometimes produces monstrous results; and even the moral intuition we use to determine which results are monstrous sometimes produces monstrous results. The conclusion is that we should abandon the project of producing a complete and consistent moral theory. There is no such thing.
And that is why we can’t build an AI judge. You can build a complete and consistent set of rules for an AI radiologist or protein-folder or Go player because we have a clear understanding of what those things are supposed to do. You cannot build such a set of rules for an AI judge because we don’t. That’s why Holmes said, “The life of the law has not been logic, it has been experience.”
That takes us to the recent refusals of various grand juries around the country to hand down indictments sought by the Department of Justice. As everyone knows, the process is so one-sided a prosecutor can get a grand jury to indict a ham sandwich. And yet this Justice Department has repeatedly been met with refusals to indict. I find this very encouraging. I’m proud of my fellow citizens.
But wait. I am not proud of my fellow citizens when they do the same thing and come out the other way. I’m thinking of jury nullification and the all-white juries throughout American history who failed to convict the white murderers of black people despite overwhelming evidence of guilt. They were following their consciences and doing what they thought was right. So the principle, “Follow your conscience. Do what you know is right” is only a good principle if your conscience is like mine.
And that takes us back to Gödel. Ethics (and in my view judicial decision making is simply applied ethics) is too complicated to be contained in a complete and consistent system of axioms. Despite all our fancy new technology, the process cannot be automated. We still have to do it the hard way.
Unless I’m deluding myself. And that takes us back to Old Doc Silk. His specialty was internal medicine. He didn’t think a good internist could ever be replaced by a machine. But he didn’t feel that way about other specialties. “Heart surgery,” he used to say, “is just plumbing.”
***
Enjoying the content on 3QD? Help keep us going by donating now.