The Structural Boundary of AI Creativity: Why It Cannot Think Like a Human

    ┌──────────────────────────┐     ┌──────────────────────────┐
    │  WHAT AI CAN DO          │     │  WHAT AI CANNOT DO       │
    │                          │     │                          │
    │  ▓▓ Risk                 │     │  ░░ Unknown unknowns     │
    │  ▓▓ Known unknowns       │  ═╌> │  ░░ Transformative      │
    │  ▓▓ Combinational        │     │  ░░ creativity           │
    │  ▓▓ Exploratory          │     │                          │
    │                          │     │  The place where art     │
    │  (predictable, bounded,  │     │  lives and thrives --    │
    │   within distribution)   │     │  that territory belongs  │
    │                          │     │  to us                   │
    └──────────────────────────┘     └──────────────────────────┘

I'm reading Radical Uncertainty right now, by John Kay and Mervyn King. It's mostly a book about economics, but there's a single claim threaded through it that I can't stop thinking about -- and that, I think, also explains why AI struggles to make art that actually moves us.

The claim is this. AI systems are brilliant at working with risk -- situations where we know the variables, even if we don't know the outcome. They're often good at handling known unknowns -- situations where we know we don't fully understand the system, but we can describe the space of possibilities well enough to navigate it. What they cannot do, by construction, is navigate the unknown unknowns -- the things we cannot imagine.

And it is in that unimaginable space that genuine novelty lives.

That sentence does a lot of work, so let me slow down and unpack it.

First, the strong version of the AI case

I want to start by saying what these tools are actually good at, because the rest of this piece won't land if I sound dismissive. I've been genuinely impressed with AI since GPT-3. I used the API before anything else, near the end of 2022, sending it tasks to outline and explain complex topics like behavioral economics -- and it did it fairly accurately. That was the moment.

In my own work, AI has been genuinely useful in two ways. First, for coding -- which has bled directly into data science, where I can now run statistical analyses and R scripts for regression tests that I could not have written myself. I don't know code syntax well. It has never stuck for me. But AI bridges that gap. The two ways most people meaningfully use AI, I think, are these: to expand and multiply expertise they already have, or to bridge a skill gap they don't. Both are real. Both are useful.

So no -- this is not a doom take. AI is an amazing technology. Literally world-shifting. It is a tool. What I want to argue is that there is a boundary around what it can do, and that the boundary is not arbitrary. It's structural.

Risk, uncertainty, and the things we cannot imagine

In 1921, an economist named Frank Knight drew a distinction that turned out to be enormously important. Risk is what you face when the dice have six sides and you just don't know which face is up. Uncertainty is what you face when you don't even know you're playing with dice -- or what dice are, or whether dice is the right metaphor. Risk is measurable. Uncertainty, by Knight's definition, is not.

John Maynard Keynes made the same point a few years later in his Treatise on Probability. About some questions, he wrote, "there is no scientific basis to form any calculable probability whatever. We simply do not know."

The phrase that actually made this idea pop culturally came in 2002, when Donald Rumsfeld -- yes, that Rumsfeld -- gave a press briefing and said:

"There are known knowns; there are things we know we know. We also know there are known unknowns; that is to say, we know there are some things we do not know. But there are also unknown unknowns -- the ones we don't know we don't know."

He was being mocked at the time. But the framework is real, and Kay and King spend Radical Uncertainty arguing that economists and engineers keep ignoring it -- with predictable results.

A simple example: if I had described a smartphone to Milton Friedman in 1976, he wouldn't have understood what I was talking about. Not just that it was improbable. He literally could not have speculated intelligently on whether such a thing would ever be invented, because the device required concepts -- touchscreens, app stores, lithium-ion batteries, mobile broadband -- that hadn't been invented yet. The smartphone wasn't improbable. It was unimaginable.

That's what an unknown unknown is. And that's the territory we need to look at when we ask what AI can and cannot make.

What AI does in a deterministic world

The technical reality is straightforward, even if the marketing sometimes obscures it. These systems are incredibly sophisticated pattern completers. They are trained on massive amounts of data. They are very good at finding the most likely completion of a pattern, drawing on the distributions in that training data.

That is not a dismissal. It's what makes them useful.

It also means they work in a world of known distributions. Structured things -- code, pixels, timeline comparisons, behavioral scatter plots -- anything where the distribution is either fixed or has enough historical examples for the model to learn from, is where AI excels. That world is much larger, more significant, and more generally helpful than AI's critics admit.

But anything that involves truly novel experience will elude these systems. Because they have no frame of reference for it. They predict points on a graph, so they cannot anticipate true chaos. Human behavior on an individual level is a great example. In aggregate, over time, in a tested area, AI can make decent predictions. But what will this one person do? No way. For that reason, AI cannot accurately predict world events, or what will be popular, or the edge cases in markets.

Unknown unknowns require adapting on the fly, and that typically involves making mental associations -- often disparate or creatively connected ones -- in unique ways. That's a totally human capability.

The three modes of creativity, and where AI stops

There's a useful framework here from a researcher named Margaret Boden. She distinguished three kinds of creativity, and they map neatly onto what AI can and can't do.

Combinational creativity is novel combinations of familiar ideas. Apple's iPhone -- a phone and an iPod in one device. It's clever, but nothing about it requires us to think differently about what a phone is.

Exploratory creativity is new moves inside an existing conceptual space. Uber, for instance -- the taxi space already existed, but using civilian vehicles and civilian drivers was a new move inside it. Same category, new corner explored.

Transformational creativity is something different. It changes the conceptual space itself. After transformational creativity, the old rules don't quite make sense anymore. Amazon is a good example. Retail shopping was already a thing, and ordering remotely had been established -- through catalogs, then websites. But Amazon made the online marketplace the default way people shop. The category of "where you buy things" shifted.

Here's the structural claim. AI is good at combinational creativity, and sometimes good at exploratory creativity. It is, by construction, incapable of transformational creativity. Because transforming the conceptual space requires sensing that a new space exists -- and you cannot sense what you cannot imagine.

The strongest objection, and the right answer

The strongest objection to all this is fair: humans also recombine. The adage "everything's been done" exists for a reason. Every artist is working with material that came from somewhere else.

But there's a real difference between the two cases, and it has to do with how the recombination happens. When AI recombines -- even in creative mode, with the temperature parameter cranked up -- it is doing so in a deterministic fashion. It is choosing the most unlikely combination from a predetermined data set. Random-from-determined.

When humans recombine, we use mental associations that have threads that may not make sense to anyone else, much less an AI. These associations are built from layers of cognition -- imagination, experience, environmental stimuli, previous associations -- that AI does not possess. And we can be surprised by what we make. That surprise is the unknown unknown manifesting itself in the work.

AI cannot surprise itself.

The part where art happens

When I was a teenager, I had just been broken up with by my first love. I was at the library with a friend, and he told me to check out a CD -- I didn't even know you could check out CDs. It was a folk singer named Ani DiFranco.

Her songs helped me through that time of sadness. I discovered so many things. That I wasn't alone. That there were words for how I felt. That I could choose to take the power I had given my ex back.

I didn't experience that as recombination. I experienced it as discovery. As something surfacing from a space I hadn't known existed. That's what art does at its best -- it brings you into contact with something you didn't know you were missing.

You cannot recombine your way to a discovery about a space you didn't know existed.

Where this leaves us

AI is an incredible tool. A fantastic piece of technology with the capability to make so many lives easier. I think that's what it's best at -- making human lives easier, the way all good technology does.

But humans remain the creators of fantastical and surprising things that will give rise to new technologies that continue to wow us with their abilities.

So my honest recommendation is this: go use AI. Use the chatbots. Use the APIs. Build awesome things with it. Just remember to always use it as it was intended -- as a tool.

The unknown is where art lives and thrives. That territory belongs to us.