Welcome.

Welcome back to the Deep Dive, your go to shortcut for becoming incredibly well informed.

Yeah, yeah, without drowning in research papers.

That's the goal.

Today we're pulling back the curtain on something pretty crucial, but maybe a bit misunderstood in software development.

End to end tests.

These aren't just some techie detail, they're big reason why the apps you use daily actually work like they're supposed to.

Yeah, absolutely.

And understanding them well, it gives you a peek into why some software feels solid and other stuff feels, well, flaky.

It really does.

And when we say end to end, we really mean simulating like a complete user journey start to finish through the whole system.

Not just tiny pieces then.

No, not at all.

These tests are designed to give you that big picture view, making sure everything hooks together and works seamlessly from the moment you tap that icon.

Really.

OK.

Right.

So let's unpack this.

Our mission for this deep dive?

Figure out what these tests are, why they're so, so important, and then maybe get into some of the surprising complexities.

The trade-offs too.

Exactly the trade-offs.

We'll use some real world examples to make it stick.

Get ready for a few aha moments hopefully.

Sounds good.

So First things first, what are end to end tests?

You might hear other names right, like system tests.

System tests, yeah.

Or sometimes interface tests.

The key thing is where they sit in that, you know, testing period people talk about.

Right at the top.

Yeah, the very tip.

You've got tons of tiny unit tests at the bottom checking little bits of code.

Super fast, those ones.

Then fewer integration tests checking out pieces work together right?

And then way fewer of these end to end tests.

And that position tells you something.

It's about scope, but also about how resource intensive they are.

Meaning meaning you just don't run as many of them compared to the other.

OK.

And their core purpose you said simulating a user.

Exactly.

Think about how you use an app.

You log in, maybe click around, fill out a form, hit submit.

You expect something to happen.

An end to end test tries to automate that entire sequence, the whole flow.

So it's not just.

Does this button look right?

No, it's does clicking this button actually lead to the right data being saved and the right screen showing up?

It checks the whole chain reaction through all the system's layers.

It's the ultimate Does this thing actually work for a real person, Jack?

You got it.

That's the essence of.

It and here's that critical point you mentioned, the resource thing.

These are the most resource intensive automated tests.

By far.

And that's not just a technicality, right?

It has real consequences.

Absolutely.

It means they take way more effort to write.

You often need special tools, complex setups.

And they take the longest to actually run.

Oh yeah, much longer.

Think about that factory analogy.

Checking 1 bolt?

That's a unit test.

Quick checking if the whole car drives off the assembly line perfectly after visiting every single station, that's your end to end test.

Takes way more time.

More.

OK, that makes sense.

Let's make it even more concrete case study one testing a web system.

People often use something called Selenium.

Selenium is a big one.

Yeah, very common for web testing.

So if you're building, say, an online store or social media site, you need to know the whole process works.

Signing up, finding a product, buying it.

All those critical user paths and Selenium lets you build tests that act like, well, like robots using the website.

Like a very precise, maybe slightly literal robot user.

Pretty much these robots, these automated scripts do what a human would.

They open the browser.

Navigate Pages.

Fill in forms, click the buttons, wait for the page to load and then check if the result is what they expected.

So it's like scripting out.

OK computer, pretend you're a user, go here, do this, click that.

Now tell me if it worked.

That's exactly it.

It's how you programmatically verify that whole journey.

We saw an example, right?

How about a Google search?

Yeah, a simple but good one.

The test simulates someone using Firefox going to Google, typing software into the search bar, hitting enter.

Standard stuff.

Right, but then the crucial part, the test checks if the title of the search results page is exactly what it should be.

So even that simple action tests a lot the browser, the website search function getting the results back.

The whole chain for that specific task, but.

It sounds straightforward, but you mentioned challenges.

Yes, it's fascinating because while the idea is intuitive, actually building and maintaining these tests is significantly harder than unit tests, Even harder than integration tests sometimes.

Why is that?

Well, think about it.

A unit test runs in a nice, clean, predictable environment.

Just code talking to code.

Right, isolated.

An end to end test for a web app.

It's interacting with a real browser over the real Internet.

With a complex dynamic web page.

It's messier.

Unpredictable.

Can be the source material highlights a few things.

First, the tool itself, like the Selenium API, the commands you use to control the browser.

It's just more complex.

OK, you're telling it how to deal with visual layouts, things loading at different speeds, mouse hovers.

It's not just simple commands.

It's like giving instructions in a busy changing room versus a quiet library.

Gotcha.

What else?

Second, they have to handle what are called interface events, like maybe a button only appears after a three second animation finishes.

Right, a human would just wait.

Exactly.

But the test script, if you don't explicitly tell it to wait for that button, it might try to click too early, fail, and you get a false alarm, a test failure that isn't a real bug.

So you have to program in patients basically.

You have to anticipate those delays, those dynamic changes, time outs, elements appearing slowly.

It needs to handle that.

OK.

And the third challenge you mentioned fragility.

Yes, this is a big one.

They are much more susceptible to breaking because of tiny, sometimes purely cosmetic changes in the user interface.

Like what the example given was Google changing the internal name of its search input field maybe from queue to search query?

And that breaks the test.

Instantly, because the test script is looking for an element named Q and it's not there anymore, the whole test fails, even though the search functionality itself might be perfectly fine.

Wow, so even moving a button slightly it could?

Break it or changing its ID or its color if the test was checking that.

For some reason this makes them brittle.

You make a small UI tweak and suddenly you have failing end to end tests to fix.

That's a maintenance headache.

I can imagine I heard about a team chasing a failure that was just caused by a random pop up ad blocking a button sometimes.

Exactly that kind of real world messiness.

It's those unpredictable quirks that make E2E web testing tough.

OK, so with all that, the complexity, the fragility, the maintenance, are they actually worth the hassle?

That's the $1,000,000 question, isn't it?

And the answer is generally yes, but with caveats.

It comes down to the alternative, which is doing all that testing manually.

Imagine a person having to click through every single possible user journey on multiple browsers after every single tiny code change.

That sounds awful, yeah.

Impossible to keep up.

With it's incredibly slow, error prone and just doesn't scale.

So compared to that, automated end to end tests, despite their flaws, are still hugely valuable.

So it's a trade off.

They're expensive and brittle, but the alternative is worse for core functionality checks.

Precisely.

The insight here is you use them strategically.

You don't try to test every single obscure corner case with an E to E test.

That would cripple you.

You focus them.

You focus them on the most critical user paths.

The login flow, the checkout process, the main post, a message feature.

Let them be your final high level check that the most important stuff hangs together.

That final sanity check before you release.

Exactly, when you successfully use an app online, chances are an E to E test ran that exact same journey automatically to make sure it worked before it got to you.

That's their real impact, providing that confidence in the core experience.

OK, but what about systems without a visual front end?

Like behind the scenes stuff.

Great question.

Yes, E2E tests are still very relevant and it actually highlights different aspects.

Let's take her second case study, testing a compiler.

A compiler like the software that turns programming code into something a computer can run.

How does that have an end to end journey?

It does, and interestingly, the E2E tests here tend to be conceptually simpler than for web systems.

Simpler.

Why?

Because a compiler's interface isn't a graphical web page with buttons and animations, it's typically much simpler.

It takes an input.

File the source code.

Right, and it produces an output.

File the runnable program or maybe error messages.

Exactly.

It's a file in, file out process.

Much more predictable, less prone to those visual shifts that break web tests.

So how do you test that end to end well?

You create a collection of test programs, little pieces of code written in the language the compiler is supposed to understand.

OK, and these test programs are specifically designed to use different features of the language.

Loops, functions, weird edge cases.

To really exercise the compiler.

Precisely, and for each test program you define exactly what input it should receive when run, and crucially, what output you expect it to produce.

And the output format should be simple.

Ideally, yeah, like a list of strings or numbers.

Something easy for the test script to automatically check against the actual output.

Got it.

So what are the steps in the test itself?

Pretty straightforward mirroring what the compiler does.

First, the test calls the compiler to compile the test program P.

Does it build correctly?

That's step one.

Step 2, if it compiles, the test runs the resulting program using that predefined input data.

And finally, Step 3.

The test captures the actual output from the running program and compares it against the expected output you defined earlier.

And if they match, the test passes.

Correct.

And that whole sequence compile, run, verify output is an end to end test for the compiler because it uses all its major parts, parsing the code, optimizing it, generating the machine code and ensuring that code runs correctly.

It touches everything.

The whole pipeline.

But there had to be challenges here too, right?

It sounds simpler, but maybe not easy.

Oh, definitely challenges.

The big one here compared to unit tests is pinpointing why a test failed.

OK, so the test fails.

Right, it tells you say program X did not produce the expected output.

You know something is wrong in the compiler, but where?

Because the test covered the whole process.

Exactly.

Was the bug in the parser, the optimizer, the code generator?

The end to end test itself often doesn't tell you, it just says the final result is wrong.

So debugging is harder.

It's like knowing the car failed inspection but not knowing if it was the brakes, emissions or headlights without further checks.

That's a great analogy.

You have to do more investigation, maybe run more focused tests to trace that high level failure back to the specific buggy function within the compiler's code.

Unit tests are much better at pinpointing the exact location of a bug.

That makes sense.

Any other hidden challenges for compilers?

Well, actually creating all those test programs can be a huge task in itself.

Think about a complex programming language.

You need test cases that cover every single feature, every weird interaction between features, every possible syntax error it should catch.

That requires deep language expertise and a lot of careful work just to create the inputs and expected outputs.

Just building the test suite is a major project.

It really can be.

Meticulously defining the correct output for hundreds or thousands of intricate test programs is non trivial.

So wrapping this up then, yeah, end to end tests, whether for a flashy website or a background compiler, are about simulating that complete journey.

That complete user interaction or system process, yeah.

They're powerful, they're comprehensive, and they're really vital for making sure the whole system, not just the parts, actually works as intended.

They connect the dots.

Absolutely.

And yes, they are resource intensive.

They take more effort, they run slower, they can be brittle, especially with UIS, but their ability to validate that entire system flow makes them pretty much indispensable in modern software development.

They provide that final crucial layer of confidence.

The yes, this actually works for a user check.

Couldn't set it better.

They really are essential for the reliable software we often take for granted.

Well, thanks for joining us on this deep dive into end to end testing.

We hope this gives you a clearer picture of why these tests matter so much.