Navigated to Judge AI based on Output, Not Mechanism - Transcript

Judge AI based on Output, Not Mechanism

Episode Transcript

S1

I mean, have you listened to something that I think captures extraordinarily well?

Why?

Arguments that AI don't understand anything and can't possibly understand anything are completely misguided and empty.

This is a blues version of Without Me by Eminem.

It's from the 1950s, which means it's not real.

And he's also never done a blues version of Without Me, to my knowledge.

And so it's AI generated and it's objectively a stunning piece of music, and it's quite different from the original.

So let's listen to it.

S2

Guess who's been back again?

Shady's back.

Tell a friend.

Guess who's back?

Guess who's back?

Guess who's back.

Guess who's back.

Guess who's back.

Guess who's back.

Guess who's back.

Guess who's back.

Guess who's back.

Guess who's back.

No no no no no no no no.

I've created a monster.

Cause nobody wants to see my shoes no more.

They won't shake I'm chopped liver.

Well, if you want shady.

This is what I give you.

A little bit of me mixed with some hard liquor.

Some vodka that'll jumpstart my heart quicker than a shark.

When I get shot at the hospital by the doctor.

When I'm not cooperating, when I'm rocking the table while it's operating.

Hey, you waited this long to stop debating cause I'm back.

I'm on the.

I know that you got a job, Miss Cheney, but your husband's heart problems complicating.

So the FCC won't let me be.

Or let me be me.

So let me see.

They try to shove me down on MTV, but it feels so empty without me.

S1

So every time I listen to that, I feel compelled to move.

I think if music makes you dance and feel things, it is real.

If AI models and scaffolding can be assembled into a product that can replace human workers, it's intelligent, i.e.

it has the ability to understand, pursue, and accomplish goals.

If a technology can perform a task and produce an output that requires understanding, it understands.

So in this frame, understanding is the ability of an actor to interpret a given task and desired outcome well enough to create an acceptable result.

AI can clearly do that now across so many domains.

It's true that if you break open a neural net or a human brain and start poking at it with a stick or a scalpel or an electron microscope.

There is no place to point to and say this is understanding, or here is the intelligence, but it is there in both human brains and in neural nets, because we see the outputs that prove that it's there.

We should stop wasting cycles on does it understand or is it intelligent or it can't be intelligent, because all these behaviors in both animals and technology are the result of emergent functionality.

And the core issue here is that we still lack transparency into emergence itself, not just for tech, not just for llms, not just for AI, but for humans and other animals as well.

So let's not confuse that opacity of emergence itself, which is a universal human problem in curiosity, with a specific implementation of that emergence, opacity and a new intelligence stack judge capabilities by their ground truth outputs.

In other words, in your lexicon, did the creation of that output require understanding and or intelligence if it were a human doing it?

And if so, then did a non-human actually produce that?

Did a non-human technology produce that same thing that if you saw it from someone else, it would have required intelligence?

Then guess what that is?

Intelligence.

Intelligence was used to produce the output.

We can use the output itself, and the fact that we have defined it as requiring intelligence to say that anything that could have produced it had intelligence itself.

I think this framing helps clarify the whole situation a little bit because we can start from ground truth, which is what we already know and accept as being the product of intelligence, right?

If you hear a song like this, if you see a work output from an AI digital worker or something, and you say, well, if a human would have made that, I would have thought it was a good product.

I would have thought this definitely required intelligence.

That statement there we can use as ground truth.

And then from there, it's a quick step to say anything that can produce that then also has that intelligence.

And notice that this is completely separate from being able to explain how it got it.

We just have to remind ourselves we don't know how we got ours either.

We have no idea how.

When you look at a spongy pink brain, how you can store memories in there, how you can have ideas, how you can have thoughts.

We have no idea where inside of that brain any of this stuff is actually performed or stored.

Now, in humans, we are not tempted to say, well, since I can't find it, we are clearly not doing understanding.

We are not doing intelligence.

Those things are not there because I cannot find them by looking at the substrate.

We're not tempted to say that with humans, and we're not tempted to say it, because we can actually look at the outputs of ourselves doing those exact things.

So why are we making this mistake with a different type of intelligence?

Why are we looking at outputs that we would judge as being intelligent or requiring intelligence to make and saying, well, because I can't find where it was made or how it was made, it must not be intelligence.

It just doesn't make sense.

And hopefully this frame will help you have the conversation with yourself or with others.

We'll see you in the next one.

Never lose your place, on any device

Create a free account to sync, back up, and get personal recommendations.