3 Comments
User's avatar
⭠ Return to thread
Mingus Khan's avatar

Perhaps the scariest thing about AI is the inscrutability of its decision making process. It's developers do not know how it arrives at some of the conclusions that it does. What we do know is that it's decisions are based on sometimes tens of thousands of parameters. And as we entrust AI with more and more decision making, this will lead to us living in a world that we cannot understand.

Today if you go to a bank and apply for a loan, you might be denied. The banker might tell you your credit score or income is too low. That sucks, but you know why you were denied and what you can do to change it. But if AI is in charge of determining whether you get a loan or not? You will never truly understand why you were denied. But what goes on under the hood might go something like this:

You applied on a Friday and you are on your second marriage and your first name is longer than 6 characters and you own a motorcycle and together these factors make it 1.1% more likely you'll default and that just placed you outside the score needed for approval.

AI doesn't think like a human at all. It just finds correlations between data, and it doesn't have a theory of causation. And there are some extremely weird correlations out there in the real world. Because of this we could very easily end up in a world where decisions impacting our lives are seemingly arbitrary and capricious because they're made based on criteria that seem to have no relevance to the issue at hand and yet are still statistically meaningful.

That to me is the real danger of AI.

Expand full comment
BillVT's avatar

Lawyers will have a field day when they challenge some of these real world decisions in court. Imagine defense counsel having to explain the technology-based decision making process to a typical American jury. Imagine the nightmare of having to advise potential users of such technology on avoiding a wave of legal quagmires. Then again, I would bet there is already a small army of elite lawyers working on these issues, who are being paid top dollar to figure it all out.

Expand full comment
R Mercer's avatar

We don't know how humans arrive at a lot of their decisions, either. We tell stories about how they did, but in the end they are only stories. WE DON'T KNOW.

I have a student that does something (usually stupid and bad). I ask them why they did it. They often lie to me with some made up reason (rationalization after the fact) when the reality (which some of them will admit) is: they don't know. The mind is not less complex than an AI (it is probably more complex) and we have the additional factor that the mind will lie to itself and it will lie to other minds and the mind is largely unaware of all the things it uses to decide and do things.

We probably have a better idea of what is going on inside an AI than we do of what is going on inside our own heads. We did not build our minds, we DID build the AI. It isn't unexplainable it is just very very complex.

The same thing is true (in a sense) of the human mind, While we did not build it, we can (to an extent) analyze it. The problem is that it is a human mind and a lot of things get in the way of that analysis. Preconceptions, morality, metaphysical beliefs, etc.

Expand full comment