When you save that picture of your cute dog or cat on your computer, you expect it to be there later when you want to show it off or send it to your favorite podcaster.

Side note, Thanks for all the pet pictures.

We love them and keep them coming.

But how can you be sure that the information you store in your computer will be there uncorrupted when you come back for it?

Is our digital information infrastructure robust or fragile.

Since this is a physics podcast, you might be wondering if there are physics reasons that your data might not be safe.

There are in this case.

Unfortunately, it's not aliens.

I'd love if it were aliens.

But the concern is radiation from space.

We know that it can hurt our bodies and rewrite our DNA, which are essentially biological hard drives.

Can it also erupt our computers cosmic ray bitflips?

What does that mean for data centers in space or puppies in space or pictures of cats on Mars?

Oh my?

Welcome to Daniel and Kelly's Extraordinary Digital Universe.

Hello.

I'm Kelly water Smith.

I study parasites and space, and if something goes wrong with my computer, I have no clue how to deal with it.

Hi, I'm Daniel.

I'm a particle physicist and I'm technically an experimental list but really all of my sciences on the computer.

Now, you told me that you were thinking about going into computer science before you decided to go into particle physics.

So are you the person that everyone goes to when they have computer problems or now in the era of like MacBooks, is it just like two hard for you to troubleshoot?

Or yeah?

Are you?

Are you the go to person?

I'm definitely the tech support person in my family.

You know, why won't this thing load?

Or I want to print from here?

Why can't I?

For some reason, everybody comes to me, even though like computer science undergraduate degrees is mostly about like red black trees and sorting and whatever, and you don't really get expertise in that kind of stuff.

Nothing useful, nothing useful exactly.

But you have very recently become aware of how fragile computers are.

Sure, Yes, I'm coming to you from Zex's chromebook because I have with in one year, broken my computer screen for the second time and it is in the shop again.

And have you managed to blame this on like weird particles from space.

No, no, what, just a lot of clumsy members in the Wienersmith household.

Fifty percent of the time it was my fault.

The other fifty percent was my adorable children.

Well, maybe today's episode is going to give somebody over there excuse I didn't break that computer screen, even if my footprint is on it.

It was cosmic rays.

In this case, it was their teeth marks.

But yes, maybe maybe they can blame a really amazing combination of particle rays that came in just the right pattern.

Well, today's topic really connects with a few of my interests because, of course I like thinking about cosmic rays.

They are particles from space, and everything I do is on the computer.

Plus, this is a concept that appears a lot in popular science, cosmic rays flipping bits on computers, And I thought, let's make sure everybody out there really understands how that works and how serious a problem it is, especially with all this talk of sending data centers into space.

Does it occur a lot in pop science?

I can't think of a single example.

Wow, maybe I read the wrong pop science.

Or maybe I read the wrong pop science.

Can you name five examples?

No?

I'm just kidding.

Name one.

I'm winny h.

You put me on the spot, and I cannot name a single one right now off the top of my head.

But I have the feeling like this is something that I hear a lot in popular science, or that people in general know a little bit about, but my benefit from understanding, like the real physics that underlies all of this stuff, so that in the future, when there is an onslaught of popular science articles after we launched the first data centers into space, everyone will be well equipped.

Well, but I mean, this is a thing that could be important for our lives personally, So like, not only could this be important if we put data centers in space, but for example, one of our listeners, Simon, sent us this article where there was a plane that was flying and all of a sudden it dropped really fast, and nobody died.

Some people did get hurt.

But the thought was that some part of the computer on the plane got hit by a cosmic ray.

It flipped a bit, which Daniel's going to explain, and that sort of messed up the control of the plane and the plane dropped really quick and some people got hurt.

And so this is like a thing that could impact our lives, and at some point I have another space related story that I'll tell.

So you know, this isn't just a thing that might be important in the future.

It's a thing that matters.

Now.

Hey, that sounds a lot to me like a popular science article about a cosmic ray.

Bitflip, I thought you said a sci fi thing.

Oh are you too popular?

No play the tape, we said, POPSI this is a concept that appears a lot in popular science.

Does it occur a lot in pop science?

Pop science?

Popular science?

Pop science, popular science, popular science articles?

Oh?

I thought you said sci fi?

All right?

All right, anyway, Daniel and I aren't communicating well today, But that's all right.

We're gonna move forward.

I'll give you the win on that one, all right.

So let's let's lay some groundwork here.

But before we explain how everything works.

I was wondering what people knew about how cosmic rays could mess up those pictures of your puppies.

So I went out there to ask our volunteers their thoughts, without any googling, of course, whether cosmic rays can corrupt computers.

Here's what people had to say.

Perhaps theris magnetosphere is protecting us from that issue.

Cosmic rays are nasty.

Of course, Cosmic rays can corrupt computer data.

Cosmic rays definitely can flip bits in computer systems.

I'm sure cosmic rays can cause problems or another.

I can with some dish drawers.

I'm not sure.

Cosmic rays can absolutely corrupt computer data by flipping essential bits, even when we build in redundancy.

I think there have even been stories of car accidents attributed to cosmic ray disruptions.

I would think so because the higher frequency spectrum could penetrate I wouldn't imagine across the computer barriers.

Cosmic rays and computers do not get along.

I think cosmic rays messing with our computers all the time, and that's why we have checksums and stuff.

Yes, I know for certain in space, electronics that are exposed to cosmic rays do have a risk of that bits getting flipped.

Absolutely, cosmic rays can corrupt data.

Satellites can be corrupted by cosmic rays, so they have to be hardened and redundant.

I'm not sure about on the surface of the Earth, though, a cosmic ray flipped a bit inside the game as the player was playing and basically cheated for them.

They would be able to alter the bit in a data stream.

So yes, great answers.

As always, I think when someone says can X mess up y, my answer is usually yeah, probably as like I'm negative.

But I don't think I really knew that this was something to worry about until I was researching a city on Mars and we came across some stories about like space radiation messing up computers.

Yeah.

People think of their computers as robust, like you put a one in memory somewhere, it's going to be there.

It's not like somethuzz if you a paper that can get overwritten or destroyed or whatever.

But in reality, these are physical systems and they live in the universe, and the universe is not always a friendly place to store to your data.

Yeah, and you're not supposed to bite the screen either.

So let's start with understanding how computers store data because I still find this like slightly magical, which which is kind of great.

So how are computers storing information?

Yeah, it's a great question, and fundamentally computer store information using quantum mechanics.

Right, you could store information on just like a long tape, like the way a Turing machine does.

You're like writing ones and zeros and principle, it can be anything, but you want it to be compact, so you could have like lots and lots of ones and zeros and not have it take up a whole room.

And you want it to be fast so you can read it out really quickly and you're not waiting an hour for that picture of your puppies to load.

And so we make it small and fast by using atomic systems.

And at its core, all these atomic systems are built on semiconductors, which is why you hear about silicon so much.

Silicon is a semiconductor.

Well, what does that mean semiconductors, Well, semiconductors sit between insulators which don't conduct electricity and conductors like metals, which conduct a lot of electricity.

And understand why that is.

You have to understand something about the atomic structure here, and we're used to thinking about atoms by themselves.

You have like energy levels, and a silicon atom has like where electrons can be.

There's like a ladder of them, right, and they can absorb and emit photons.

This kind of stuff is like a very discrete set of energy levels, and that's true for atoms.

And if you have, like a gas of silicon, it's going to emit photons at certain energy levels and absorb at those energy levels, and that's all cool.

So that's atomic physics.

Is atomic physics another word for chemistry, because it kind of sounds like it's another word for chemistry.

It's the foundation of all chemistry.

Yet everything in chemistry comes out of those energy levels.

And then it gets more complicated because you have atoms binding with each other and sharing electrons.

And if you take that too, you get things like solids where you have a whole lattice of atoms, and now you don't have to solve these problems just for one electron.

You have to think about what happens to an electron that's shared among many, many atoms.

And so instead of thinking about a lattice of silicon, atoms is like each a bunch of individual atoms with their own energy levels.

These things overlap because they're so close, and so they spread these sharp atomic energy levels into bands.

So we talked about bands of electron energy levels.

So you shouldn't think about electron in a piece of silicon is like this one belongs to that atom, or belongs to this atom, or belongs to this other atom.

It moves smoothly right the ions the nuclei are bound together, they're sort of like fixed in a lattice, and the electrons are all moving around, and there's different energy levels there, which roughly get grouped into two different kinds of bands.

There's the valence bands.

Those are the low energy ones where the electrons are mostly local and bound to a specific ion.

But then there's above that, there's the conduction band, are usually empty and the electrons can flow around.

So if you have, for example, a metal, then there's a very small gap between the valiance band and the conduction band, and the electrons that fill up the valance band can jump up to the conduction band like getting on the highway really easily, and flowing all around the metal.

So if you put an electric field near a conductor, the electrons will jump up to the conduction band and flow all around.

If you have a big gap there, like no entrances to your freeway, then the electrons are all stuck in their local area, and even if you put an electric field over it, they're really not going to move.

So that's how to understand an insulator and a conductor or something we call a metal.

Usually a semiconductor like silicon, is something where that gap is sort of like in between.

It's not really big, it's not really small.

All that silicon lithography helps you do some fancy local chemistry to make a silicon dioxide or silicon nitride that changes its conductivity, which means you can now separate components and dielectrics and whatever you need to build circuits.

And so you can design really really small electrical circuits just by doping the silicon in various ways.

Okay, so the reason you want to use silicon is because it's helpful to have something that's in between, because you well, so, like, why wouldn't you just want to have a full insulator or a full conductor on your chip as opposed to just using silicon everywhere and then tinkering with it.

Tinkering sounds possibly more complicated than just putting conductors in some places.

Yeah, you could construct it out of just conductors wrapped in insulators, and that's basically how you build circuits on your bench, Right, you have like copper wires wrapped in rubber, right, and that works, But it's really hard to do that at a micro scale.

To manufacture that stuff, and to do it cheaply and at high volume.

It's easier to take silicon, which can be tweaked in either direction.

It turns out to be a lot easier to tweak it than to just like build it from scratch, okay.

And so to tweak it, you just like flick some germanium on it, and now you're good.

Yeah.

Essentially we have a whole episode about how silicon lithography works, and people should dig into the details there.

There's a lot of nuance that's missed in this summary here, but roughly, that's why you want to start with silicon, because you can easily change it to be a conductor or an insulator.

I'll go check that out.

My memory is just the pits lately, Okay.

So now we've got our semiconductors.

Yeah, and so you can use that to build all sorts of microscopic bits of your circuits out of silicon.

But keep that in mind for later when we're talking about what happens when a cosmic ray tears through a piece of silicon.

So, now you have the silicon, and then you want to build a computer, how do you actually use it to store memory?

So your computer has several different kinds of memory depending on your need.

So the memory you might think about are things like RAM.

Right, These are like things your computer remembers when it's turned on, but if your turn off, the computer is empty.

So like loading a program into memory so that your computer can run it.

Right.

So this is you know, random access memory, and these days it's called d RAM dynamic random access memory.

And random access just means that you can like get a piece of informmation from anywhere.

You don't have to like read it in order, like a book.

You can just have an index.

You can say, tell me what's stored here, tell me what's stored there.

Song.

As you know the index, you can look it up.

Okay, So I've got a semiconductor which allows me to move electrons around.

And to build circuits.

And so now the question is how do you use little electrical circuits to store information?

And so DRAM does this by using a capacitor connected to a transistor.

So a capacitor physically, if you build a large one, is just like two surfaces insulated from each other that are separated, and so you can have like a charge stored across them.

You can put a bunch of electrons on one side and then you have like a voltage across the capacitor.

And so capacitor is very useful for all sorts of things circuits.

But in this case you can just ask like, well, is there voltage on it?

If so, I'm going to call that a one.

If there's no voltage across it, I'm going to call that a zero.

You just need some sort of microscopic state which you can control and you can read out, and then you assign the meaning of one to one of the states and zero to the other state.

Okay, but so what could actually be happening in the capacitor is there's a lot of different ways that you could have more charge on one side and less charge on the other, but it all gets sort of summarized down to one in zero.

Yeah, exactly, because this is digital logic.

In the end, everything is analog, like the universe is mostly analog, but we assign digital meaning to it.

And as these things get smaller and faster than the amount of charge to capacitor holds gets really really small, so that it's quick to read out, it's quick to assign, and it's easy to make small.

So in this case, it's like a few tens of thousands of electrons hold the charge.

And you can imagine more complicated systems.

If you don't want to use binary logic, you want to use trinary, you could have like two different threashs.

You could say zero is anywhere from zero to ten thousand electrons, and a one is ten thousand to twenty thousand electrons, and anything above twenty thousand I'm gonna call it two if you have trinary logic, but we use binary.

So it's like zero or somewhere above ten thousand electrons or ten to thirty fempto coulums.

And so there's a capacitor that holds it and then a transistor which allows you access to it.

And so that's the basis of DRAM.

And this requires constant power.

When you turn off the computer, that charge dissipates and it's gone, which is why when you turn off your computer, it doesn't remember anymore what was in its memory.

Okay, so you've got the semiconductor, and those electrons are jumping up into the highway and they're getting moved around in the capacitor so that they can store up these ones in these zeros.

And it's critical there that you have conductors, Like the plates on the capacity are made of conductors and insulators between the plates so that the chargers don't just flow across.

Okay, all right, so you want them to move and then you want them to stay once you've.

Moved there, exactly, Okay, exactly, But these things don't work great.

Like there's a leakage there because these things are super duper small and the insulator isn't perfect, and so every sixty four milliseconds or so, it just like leaks away.

So your computer has to constantly refresh your RAM every sixty milliseconds or so, it rewrites the number onto it.

It's not very stable.

That doesn't sound like a great system.

I mean, I mean it seems to work great.

I love my computer when it's not in the shop.

It's kind of amazing that it works as well as it does.

You don't just like call up a picture and it's like totally corrupted.

Yeah.

Then there's another kind of memory, which is in your CPU.

So your computer basically has like a hard drive to store information.

And then there's the CPU, the central processing unit that does all the actual calculations.

Then it has access to the main memory that we just talked about, the RAM, But when it's doing the calculations, it slurps stuff from the RAM into a special super fast memory called the CASH, which is close to the CPU so that it doesn't have to go all the way to memory which turns out to be a little bit slow.

So you have these hierarchies of memory in your computer which are smaller and faster or bigger and slower.

So SRAM is the fastest, smallest memory.

It's like right there next to the CPU.

It's like if you're trying to do your household budget, you don't keep all of the information right in front of you.

You have like a sheet of paper or a spreadsheet with like the critical details right in front of you that you're calculating right now.

The CPU's CASH is essentially like its.

Little worksheet and sm SRAM stands for is that slow random access memory?

Is that what it stands for.

It stands for a static random access memory, okay, as opposed to dynamic, which is the kind you have for your most of your memory.

And this is cool because it's built by a pair of logic gates usually, so it's built using not gates.

Sont gits are something where if you take in a one, you output a zero, if you take in a zero, you output a one.

So it's like a basic logic gate in the same categories like and gates you know that require two ones to output a one, or an ore gate that requires either of the inputs to be one to output a one.

Oh yes, of course.

And these again are built on top of transistors which are made of silicon, So you can tie a few transistors together to make a nand gate or an and gate or a knot gate.

All these basic digital logics are built on top of the same fundamental technology, which are silicon transistors which have these various levels of doping so they can do what they need to do.

And so the way SRAM works is you have two knock gates looped on each other.

It's kind of silly, but it works really amazingly well.

Like, say I have two knock gates.

If the first one has an output value of one, and then you feed that into the second one, the second one is going to have an output value of zero, right, feed the output of the second one back into the first one, and it's going to keep the first one having an output value of one because it's input with zero.

And so it's just sort of like if you have two of these things in series and then looped on each other, they support each other, and there's two stable states here.

Either the first one is one and the second one is zero, or the first one is zero and the second one is one, and either way they just constantly support each other, confirming each other.

Okay, I'm following, but I feel like the big picture here for anyone who is maybe having a little trouble following, is that in all of these cases, what's happening is we are very carefully keeping track of where electrons are going, and they need to be going in particular places and staying there.

And so you have your DRAM, which is a capacity and a transistor.

You have your s RAM, which has this pair of knock gates which you're keeping your information.

And these are more robust than your DRAM because they don't leak the same way.

But these transistors are super duper small and so like a little bit of tweaking can corrupt them.

And then the last kind of memory in your computer is the hard drive.

This is the most robust, the thing where if you turn it off, it will stay there.

You can put your picture of your puppy on your hard drive and turn your computer off and on it's still there.

And this is also the slowest kind of memory and the biggest and these days this uses flash technology, which essentially like an insulated box and you try to trap some electrons in there, or you let the electrons out, and it's insulated so the electrons can't leave.

And essentially you say, if the electrons are trapped in there, then it's going to be a one.

If the electrons are not trapped, it's going to be a zero.

And so this is like a really big energy barrier that's very robust and it's very hard to pertur, but it's kind of slow, which is why you use it for your biggest, slowest memory, like your hard drives.

Does your computer start with all the electrons that it needs and just move them around or does it pull electrons out of the environment when it needs them?

Why are you Oh, great question?

No, I love this question because it touches on like, you know, electrons versus electricity, right, and mostly electricity is the motion of electrons, So you're not like running out of electrons in a sense.

But people confuse these two things and they like to talk about electrons when they mean electricity.

And remember that these numbers of electrons are super due per tiny, right, Like the number of electrons in silicon is like, you know, ten to the twenty nine per gram or some crazy number.

So if you're talking about circuits that need like ten thousand electrons, it's really just you're never going to run out of electrons.

So electrons are everywhere.

We're all just living in a vast ocean of electrons.

It's about controlling where they are and where they aren't and where they go.

Okay, you don't need to recharge your electrons, but it sounds to me like a really fun scam we can sell people, like let's top up your electrons.

All right, Well, Daniel and I are going to think of how we can monetize electrons.

See if you can hear are commercial for it during the break, and we'll be back in a moment.

We're back.

Daniel just gave us a great explanation for how computers store information, and now we're going to hear about how cosmic rays can mess that all up.

So, Daniel, what is a cosmic ray.

I'm sure physicists have this absolutely nailed down.

Cosmic rays are basically just particles from space.

Right.

Rays is just like a generic name for a particle, or sort of an old fashioned name, and cosmic just means it comes from space somewhere, and the Earth is constantly being hit by particles from space.

Because space is not empty.

You might think of it as like a grand vacuum, but really it's filled with particles, just much lower density than here on Earth.

So in our Solar system, for example, there's a massive producer of cosmic rays in the Sun.

It pumps out protons and electrons and photons and all sorts of stuff, and when the protons and electrons hit the Earth, we consider those cosmic rays.

And Jupiter also puts out a lot of radio contributes to cosmic rays.

The center of our galaxy is a massive source of radiation because there's a lot of crazy stuff going on in there.

It's hot, it's dense, it's nasty.

Things around a black hole also emit cosmic rays because there's a lot of energy there.

And so basically a lot of things in the universe are emitting cosmic rays and they go from fairly low energy all the way up to like the craziest highest energy things ever seen in the universe, which are what which are created by things we don't know.

So, for example, the large hadron collide, which is like the pinnacle of energy the humanity's ever created.

We accelerate things to have like thirteen terra electron bolts.

That's thirteen trillion electron bolts of energy.

But out in space we see things that have a million times as much energy.

Whoa yeah wow, And we don't know what causes that.

Like we know that the center of the galaxy emits some but not that high energy.

We know supernovas creates some but not that high energy.

We know if you take particles from supernovas whiz them around um black hole, that gives them a little bit more energy, but still not that high energy.

So we see these particles from space that have crazy high energy, and nothing we know in the universe can make particles that high energy.

It's like a huge mystery.

Maybe it's a glitch in the simulation.

Maybe it's alien particle physicists shooting their beams at us.

Maybe there's information there we don't know.

Okay, go ahead and mark that spot on your dKu Bingo cards.

Friends, I knew that was coming.

Aliens, Yeah, yep.

And so they impact life on Earth here because they hit us, but they hit the atmosphere, and the atmosphere is a great shield from cosmic rays.

But it doesn't just like delete them.

What happens when a particle hits the top of the atmosphere is it collides into the particles in the atmosphere.

And so you go from having one very energetic particle to two that have half as much energy, to four with a quarter as much energy, to a trillions of particles, each with a small slice of the energy, but still enough to make it all the way down to the star of the Earth.

And so we have a constant flux of cosmic rays on the surface of the Earth.

It's like one hundred and fifty muons per meter squared wow per second at sea level.

So like every fingernail you have has hundreds of muons passing through it every second.

They're everywhere.

So you know what I'm wondering, do I need to worry about that?

Well, it contributes to you know, changes in your DNA because cosmic grays carry energy and if they hit your DNA, they can change essentially the information stored there in great analogy to what happens when a cosmic gray hits a computer.

Right, this is how biology stores information, but it's not totally robust.

You know, I'm not an expert in biology, but my understanding is that cosmic ray can like either hit the DNA directly or it can ionize water, which makes free radicals that react with the DNA.

And in the end you can have something which was originally like a cytosine and now behaves like thiamine, and that gets read out differently by your molecular machine.

Now you're building a different protein and like, oops, now you have cancer, or oops, now you have like laser rays out of your eyeballs.

Right, all sorts of mutations, And so I think this is a big part of our evolutionary history that cosmic creates contribute to changes in our DNA.

At least, it's my favorite explanation for how me and Katrina have an athletic.

Sun Wait, wait, when I got skin cancer, you mean I could have gotten lasers out of my eyes because that would have been way cooler.

Anyway, sunscreen and broad rimmed hats are your friends friends.

Yes, exactly.

And in that case, I think mostly skin cancer is caused by ultraviolet radiation and not by like more deeply penetrating muons or neutrinos.

But it's the same principle, you know, particles in that case, photons from space corrupting the information.

And so think of these things as just high energy particles from space to come down through the atmosphere.

Okay, so you have these particles from space that carry a lot of energy.

Mostly it's muons.

They're also electrons in there.

They don't penetrate as deeply.

Most the electrons are blocked by stuff like skin or you like a small sheet of metal.

Muons have more mass and so they don't interact as much and so they can go deeper into an object.

They are also neutrons.

Neutrons are created in these showers, and because they're neutral, they pass through a lot of material.

They're penetrating as well.

Neutrons are less common.

They're like one hundred times less dense than muons, but they have a lot more mass to them, so they can really do a lot more damage.

So you have these particles from space, and we told you that we build all of our digital infrastructure out of semiconductors, which rely on like the delicate balance between a conductor and an insulator based on how exactly the electron bands are arranged, and essentially build everything out of these building blocks.

Well, what happens when a cosmic ray passes through a lattice of silicon.

It can smash right into one of those atoms, breaking up the nucleus, or it can push that atom, or it can just ionize a bunch of silicon.

The silicon atoms that it kicked off can like flow through the lattice, ionizing it.

And so essentially, if you build your whole digital infrastructure out of this stuff, it's like taking a sharpie and just like scrawling a ride across the page.

Okay, so it's not like it's so I'm thinking about it as a highway now with cars moving along it, the cars being the electrons and the silicon being the highway.

It's not that the cars are getting moved around.

It's that you're essentially like blowing up the highway and now the cars like they don't know where to go anymore.

Yeah, essentially you're blowing up the highway.

And think about for example, your capacitor, right, your basic unit of memory in your computer.

What's going to happen if you suddenly deposit a whole bunch of charge, or you like tear through that and destroy the insulator so the charge leaks out.

You're going to go from a zero to a one, or you're going to go from a one to a zero.

Right, You've built this careful thing out of these bricks and then you have to suddenly just tear right through it, and you still have the machinery around it that's going to try to interpret what's going on the one or a zero.

Right, Your transistor that's next to the conductor is gonna be like, oh, excuse me, by the way, this doesn't seem to be like it's behaving itself.

It's just gonna be like, is there charge zero?

Is there no charge one?

That sounds bad?

Yeah, And the same is true for your s RAM for example.

This was built out of knot gates, and these again are built out of transistors, and transistors have various components of silicon that are dope in various ways so they can do their logic.

If you have a cosmic ray plow through that, it's going to deposit a bunch of charge.

It's going to change the way those gates operate in the same way.

Now, hard drives, which are the most robust and the most difficult to change, have that like insulated box.

So this requires like stronger radiation.

But essentially you want to have like charge in there or not charge in there.

If you have a cosmic ray that comes in and tears through it and essentially deposits a bunch of charge, you're gonna go from a zero to a one.

Or if it breaks the containment and lets the charge out, then you're gonna go from a one to a zero.

So it's possible for every one of these components in your computer to be susceptible to cause and graze, very similar to how it happens with DNA.

Okay, so that all sounds bad, but can we get a little bit more concrete about, Like does that mean your computer is just gonna like one hundred percent crash every time this happens, or like what's the range of outcomes here?

So it depends on where it hits, right, it might hit your computer and it just hits a place where like nothing is being used anyway, Like you have an empty part of your hard drive and it goes from zero to one.

Whatever it was noise, it doesn't really matter.

Or maybe you were about to store a picture of your puppy there and so it goes from zero to one and then you overwrite your puppy on top of it, so it doesn't matter.

Right, So those are the best case scenarios.

You've overwritten your garbage with a puppy.

Oh good, I thought you were gonna say, overrode my puppy, and I was gonna get really upset.

No, that's a disaster.

Yeah, meal is very cute.

Yeah.

So in the sort of next stage of like problematic thing, it's like something goes wrong and it's crucial and your computer crashes.

You know, like your computer follows some instruction in its memory and the instruction isn't what it was supposed to be because it got changed by a cosmic ray and it does this instead of doing that, and it doesn't work and it gets stuck and boom, your computer has crashed.

That's actually a good outcome because you notice it and it stops you from using it, and then you like restart your computer and you're.

Fine, so your computer can recover.

Yeah, if your computer crashes, you just reboot it, right, Like it happens all the time, especially if you use Windows.

Oh ouch, but isn't your silicon chip still like toast because it got hit.

No, No, you could just refresh.

It's like it will change it from zero to one.

But it's not like it's destroyed it, right, Okay.

Usually these things, you know, are robust.

It's about the charge that's stored in that circuit, and so they can like release it or right into it accidentally.

But you restart all this stuff, You're gonna be fine.

It's not permanent damage.

If it happens occasionally.

We'll talk later about what happens to electronics inside the large hadron lighter and why that needs to be much more radiation hard.

But you know, if it happens once or twice, you're fine.

You have like enough silicon there for these things to still work.

Okay, So then what is a bad result.

The bad result is when it happens silently, like you have stored some data on your computer and the bit gets flipped and you don't notice, and later you read that in you do some analysis and you're like maybe you're doing some science or you know, your puppy's eye color has changed or something.

And this is when it's dangerous, when it's a silent mistake.

Right.

Or for example, you're storing voting results from some precinct in Virginia and like, oops, a bit flips and somebody now has more or fewer votes.

Right, This is when we really rely on this stuff.

So not knowing that it's happened and having it happen silently in a way that doesn't cause any errors, that's the worst case scenario.

Okay, this is getting scary.

How often does this happen?

So there's bad news and then there's good news.

Okay, So bad news is that it's not that rare, you know, because there are cosmic rays everywhere, So you should expect like a few bit flips per year per gigabyte of RAM, right, and I have like thirty gigabytes of RAM on my computer, and so I'm expecting like dozens of times a year a bit it's going to be flipped in my RAM.

Oh yikes, exactly, And so that seems like an issue.

But of course the nerds know this and they have built protection for us.

And sound like we're just running out there totally exposed.

If you have a computer in an environment where you absolutely need it to be robust, then there are clever systems to detect these errors and correct them.

I find myself wondering, was there an era when we had computers but we didn't know this and there were like disasters because we didn't know this.

Yeah.

One of the ways this was originally discovered is that electronics near nuclear testing in the fifties had anomalies.

People were like, whoa, what's going on with our electronics?

And then they realized, of course that the radiation from the blast was interfering with the electronics.

It was like in the seventies that the detailed physics involved was understood and like demonstrated in particle beams, and so it was really like in the eighties people understood what's going on here, and can we build protection?

Oh wow?

Okay, well I'm glad we've got protection now, all right, So tell me how the protection works, please.

So the simplest and the most expensive way to protect your data is just to have copies of it, right, Like, if you have really sensitive data stored on a hard drive.

Hard drives used to be much less stable.

You could just like lose a drive, like the head dropped on it.

Back when there were magnetic spinning discs, and so we had these things called like raidar rays where you just had like everything was stored in duplicate or in triplicate even and in triplicate is cool because if one of them gets corrupted, you have two other copies, and so you can vote.

You're like, is this bit supposed to be a zero or a one?

And if two of them agree, then you know, then that's the way you go.

I think that's how like the flight systems work on the Space Shuttle, right, they all vote, the three independent systems or something like that, and they all vote, so the you always have two the degree.

So that's the most complicated and expensive way.

And that's quite expensive because now you're like tripling the cost of your memory.

If you want to store a gigabyte, you really have to have three gigabytes of memory in there just to make it protected, Yanks.

And so since that's so expensive, people came up with cheaper, faster, more clever ways, things like checksums.

Right, Like if you send a message across the internet and you want to make sure the message arrived uncorrupted, then none of it got like garbled along the way.

You can also send a checksum, which is like the output of a little calculation you do on the data.

As simple way to think of a checksums to be like take the whole data set and treat it like a number.

Was it even or odd?

And that way, if I send you a message, and I also send you by the way it should have been even, then you can check to make sure it's even.

That doesn't catch all possible errors, like if you flip two bits, for example, But it's one simple way to say, like were there any mistakes?

So in modern computers they do something which is more sophisticated than just a simple like is it even or odd?

They have these parody checkers that take like groups of four data words.

A word is like a bunch of bits altogether.

And you have parity bits which check how many events there are and how many odds there are.

And if you overlap these things in a clever way, and you have this voting system, you can have like one parody bit checks three words and another parody bit checks three other words.

And so through some clever algorithms and clever math, you can detect any bitflip in any of those words without having complete duplicates.

So this is like ten percent of the data cost.

You have a gigabyte of memory, you build a gigabyte plus ten percent, and you have enough data storage to detect if there were any bits flipped in your core gigabyte.

But we didn't have this clever and cheap version during the Shuttle era, and that's why the Shuttle was so expensive.

I don't know if the memory costs contributed a lot to the Shuttle.

And many computer systems have this already.

So for example, your hard drive very likely has error correction on it.

Your RAM probably has error correction on it.

CPU caches usually do not, because this is the s RAM stuff because it has to be super duper fast.

It really will slow down your computer if you have error correction on that CPU, like the worksheet that the CPU is using.

But if you have a really important computers, like you're operating a data center and you need to provide really robust calculations for your consumers, or you're running something in space, then you need to have error correction on your caches as well.

Although I was reading that often in space applications they take consumer PCs and they just turn off the CPU cache because they'd rather be robust than fast, and so they're like, this part is not robust in space, so let's just turn it off.

All right, Well, let's take a break, and when we get back, Kelly will tell a fun story about how galactic cosmic rays messed up computers in space.

All right, we're back.

We're on the International Space Station and an alarm has just gone off.

Samantha Christafaretti, who's an Italian astronaut, was amazing at her ability to memorize all of the different alarms on the station.

And even though this is the first time she's ever heard this alarm, she immediately knows what it is and she yells ammonia leak.

And so she and Terry Vertz have memorized the protocol for ammonia leaks.

And so what they do is they run over towards the Russian segment and they close the airlock for the American side.

And so what happens if there's an ammonia leak is that the coolant system for keeping the American side of the International Space Station is leaking.

So ammonia is toxic and so they can't stay over there anymore, So they close off their side so that the toxic stuff can't get out.

Then they're supposed to get naked.

Then they're supposed to go through to the Russian side, and then they're supposed to close the hatch and hope that the Russians have extra undies.

And I saw Daniel look confused, and so, Daniel, are you wondering why they closed to the second hatch.

Why did they close the second hatch, Kelly, just to.

Make sure they didn't get any more moonia on the other side.

But you're probably wondering actually why they were supposed to get naked.

Right, yes, exactly.

I was having fun imagining it though.

Okay, well, they apparently were not having fun imagining what it would be like because they skipped that step.

But they were supposed to take the clothes off because the concern was ammonia might stick to their clothes and they might end up bringing ammonia over to the Russian side.

And the Russians use Glai call to cool their side of the International Space Station so they don't have this toxic stuff.

So ammonia sticks to close but not to like skin.

I mean it.

Probably it could stick to everything, but the goal is to bring as little as possible over to the Russian side.

Do you ever think the engineers are just like, let's add getting naked to this protocol and see if the astronauts will really do it.

Yeah, well, so you know, you've got to imagine that at some point they were like, do I really want to see my middle aged colleague naked and zero gravity?

And I think, and they must have both decided no, And so they went over to the Russian side fully dressed.

They locked the second airlock, and then they got permission to go back, and so they go back, and the alarm goes off again, and they go through the whole process the second time, and yet again they decide to not take their clothes off.

And I believe they ended up saying they decided to not take their clothes off because they couldn't smell ammonia.

Ammonia has this like distinct rotten egg smell.

And it turned out that they were right.

It was a false alarm.

Was this the universe trying to get them naked?

Is that what was happening?

It could be, could be, But the best guess that folks came up with afterwards for what happened is that galactic cosmic radiation had hit their computers and had inadvertently turned on the ammonia leak alarm, and there wasn't actually an ammonia leak.

They never were able to find an ammonia leak, but it, you know, they started doing this emergency protocol and they had to eventually do it twice before they were able to fix their computers.

Well, so we're sort of talking about these data centers today, and I guess one thing I want to point out is that even though the International Space Station is to a large extent protected by Earth's strong magnetosphere, which sends a lot of these charge particles, you know, to the poles and protects the International Space Station and the equipment and people within it, you still get hit sometimes, and so you still have to worry about the equipment.

Where are people thinking of putting these data centers?

Are they within the protection of the magnetic field or out past it?

I don't think they want to go very far because the further you go, the harder it is to access and to repair.

And I'm not on record is thinking that any of this is a good idea, and you know, the cooling is are the radiation is tricky because you don't have the benefit of the Earth's sphere, even if you still have some of the magnetosphere.

And so I think it's a crazy idea pushed by tech bros who think it's gonna be fun and sci fi.

And you know, I don't know about that.

We'll have a whole another episode digging into the economics of data centers in space.

But yeah, out there in space there is more radiation, and so you need more protection and you need like shielding for these things, or you need to make extra error correction, or you need to use technologies that are more radiation hard it actually on Earth.

Sometimes I'm frustrated by the amount of error correction.

There are scenarios where it would be better for physics, even if it's worse for computers and for Amazon or whatever, to have less error correction.

Because all of these things are evidence of cosmic rays.

And if you're curious about cosmic rays, where do they hit, what's going on?

Then one of the biggest challenge is seeing them.

The really high energy cosmic grays are really rare.

It's like one per square kilometer per century, either in a lot of square kilometers or a lot of centuries.

And these massive data centers that are being built out are basically huge cosmic ray detectors.

And imagine if you could use all of Google and Amazon's data centers to observe cosmic rays from space.

It would be incredible, right.

I actually looked into this and connected with folks at Google to see if we could use it.

But their air correction is so good they don't even store it.

They don't log it, and be like, by the way, we flipped a bit here and we corrected it, They just correct it and move on.

How hard would it be for them to collect that data?

Oh?

Man, it would require a change in pretty low level operating system stuff, unfortunately.

And you know, for a while they were going to be willing to share the data with us if it was like something they already had, they could just ship to us and we could analyze it.

And that was so excited.

But then it turns out it's not storage and getting them to do anything just for fundamental physics is frustratingly difficult.

Google.

I thought you were trying to not be evil or did they like do away with that a long time ago.

Yeah, I don't know.

I mean, it's not their obligation to fund basic research.

But it's frustrating to me because it means that these cosmic grays are there and they're actually observed by our technology, and then that information is just thrown away.

You know, one person's error is another person's gold maybe Nobel Prize gold.

Right, So that's frustrating.

But there are other times when we think cosmic grays could have affected things on Earth.

The voting example I mentioned earlier was not a hypothetical.

There was an election in two thousand and three in Brussels where one candidate had a number of votes that was suspiciously off by four ninety six.

They're like, that's a weird number, Like, how did this number just appear in the computers?

It's off by this specific power of two, right, fenty ninety six is the power of two, So it could be caused by a single bitflip.

And you know they went through and nobody tinkered with it.

There's no evidence that anybody like infected it or hacked it, or there was a copyer or anything.

They think probably was a bitflip that caused this weird result in an election in Brussels, and so it's certainly happened, right, It's something we need to be aware of.

I mean we need to know about.

Did it change the results of the election?

No, they figured out I think they knew how many votes have been cast in total, and so they knew that there was an excess and affected it.

And I don't think it flipped the answer either way.

So it's just sort of weird and mixed people suspicious, and you know, it highlights something else, which is that our whole digital infrastructure is sensitive to what's going on in space.

We recently had a really nice cosmic light show because there was an extra amount of solar radiation which led to northern lights all over the place on Earth.

But that's extra solar radiation and that can come at any time.

In the eighteen fifties, there was a really spectacular coronal mass ejection, an eruption of charged particles from the Sun which basically like a huge loop of plasma which heads towards the Earth, which is just high energy cosmic rayse that's what plasma is, right, and it can be like billions of tons of materials.

Most of these things missed the Earth because they're it's pretty small and far from the sun, but when it washes over the Earth, it can devastate our digital infrastructure more than just like corrupting a single bit on an airplane or in a voting machine.

This in the eighteen fifties, like set telegraph networks on fire, like you know, sparks were flying.

Yeah, it's crazy.

And if that happened today, like our digital infrastructure is not protected from that kind of incursion, you wouldn't be able to like shop for stuff online for days and days while people fixed it.

I mean, can you imagine, like if something took out all of our satellites, how absolutely pned we would be Yeah, like nope, credit cards, I wouldn't have starlink Internet, there'd be no dKu anymore everyone.

Can you imagine dot civilization crumbles?

Yes?

Right, yep, I agree.

And so in particle physics, we've thought about this a lot because we're operating computers in a very high radiation environment constantly, like we're creating that radiation.

When you smash two particles together, you generate huge numbers of protons and neutrons and muons and all kinds of stuff, And then we have silicon near the collision in order to measure these things to detect these particles in exactly that way.

But if you run it for a while, eventually your silicon gets trashed, you know, so every few years we have to pull the thing out and send in new silicon basically, And so people are doing stuff like exploring whether we can use diamond instead.

You know, we talk about silicon as a semiconductor, but it's not the only semi conductor out there.

You can build these chips out of diamond, which makes them more radiation hard which is very expensive because there isn't a huge consumer diamond lithography industry the way there is for silicon.

So particle physics tends to make its advances by like piggybacking on huge trillion dollar consumer trends, and that hasn't happened for particle physics that we don't have, you know, diamond chips yet, but it's something that people are looking at that we might have to do in the future in order to protect our computers from cosmic grades or radiation more generally.

Can we take a us step back to like so if we'd had another Carrington event type of thing, happen.

Is there something we could have done to protect all of our satellites from an event like that?

Like, should we be encasing all of our satellites in diamonds?

Is what I'm asking.

It would make them so glittery and beautiful, that's right.

Well, essentially what you need is mass, Like either you should build the chips themselves out of diamond to make them radiation hard or is you just shield them, in which case you need mass, and mass, as you know, is expensive to put in space, and that's why it's so difficult to have things like computing in space because it needs the thing which is expensive to put.

So I think it would look awesome, but I don't think we should bling up all of our satellites.

Okay, So for the foreseeable future, we're probably not going to do anything about the fact that if we had a Carrington event type of thing happen again, we would be in a lot of trouble.

Yeah, that would be a massive investment in infrastructure, and I don't see anybody doing it, though I think it would be a good idea.

But until then, we do need to be aware that cosmic rays are everywhere.

They are streaming through your room right now, and they are streaming through your computer.

And sometimes they can flip a bit a zero to one.

They can plow their way through a bunch of silicon and change the way those delicate electronics work.

Most of the time this doesn't affect your computer, or it does it in an obvious way, or it's caught by error correction and fixed.

But sometimes it isn't.

And so next time your computer goes hey wire, you could blame it on your kids, or you could blame it on particles from space.

Or the next time you get that super awesome science result, you should question yourself even more.

That anxiety should keep you up all night, because maybe it was a cosmic ray and you were wrong.

Maybe when you counted parasites you're off by one.

All right, Well, thanks for taking this cosmic journey with us into how silicon electronics work and how they are vulnerable to particles from space.

As always, we are grateful for your shared curiosity.

Thanks everyone, have a good one.

Daniel and Kelly's Extraordinary Universe is produced by iHeartRadio.

We would love to hear from you.

We really would.

We want to know what questions you have about this Extraordinary Universe.

We want to know your thoughts on recent shows, suggestions for future shows.

If you contact us, we will get back to you.

We really mean it.

We answer every message.

Email us at Questions at Danielankelly.

Dot org, or you can find us on social media.

We have accounts on x, Instagram, Blue Sky and on all of those platforms.

You can find us at d and kuniverse.

Don't be shy, write to us