Welcome back to the Deep dive.

Today we're moving past the usual suspects, you know, unit integration tests.

We're going deeper into some other really crucial testing types in software development.

Think of it getting the inside scoop.

Our guide for this is Software Engineering a Modern Approach by Marco Tulio Valente.

Great book packed with useful stuff.

And our goal simple, quickly pull out the key ideas about these less common tests so you're up to speed fast.

OK, let's impact this.

So when people talk testing techniques, black box and white box usually come up first.

Seems basic, but what is the real difference?

And why should we, you know, care?

Yeah, good starting point.

The core difference, really it just comes down to what you know about the codes insides when you're testing it.

So black box testing, you're intentionally only looking at the outside the interface, like what's the function name, what parameters does it take?

What's it supposed to give back any errors it might throw?

You don't see the actual code lines.

It's often called functional testing too because you're just checking if it does what it's supposed to do functionally.

Think about testing a car with black box.

You just use the steering wheel, the pedals, the lights.

You wouldn't look under the hood at all.

Just does it drive?

Does it stop?

OK, so exterior view only what it does not how so white boxes pop in the hood?

Exactly.

Yeah, white box testing is the opposite.

Your tests use information about the internal code structure.

You are looking under the hood.

You're looking at the source code, maybe thinking about the different paths through the logic, the decision points, the loops, how things connect inside.

Sometimes called structural tests.

Here you might write tests specifically to make sure, say, every single line of code gets run at least once.

Or maybe every if statement gets tested for both true and false conditions.

Using the car analogy, you'd be like the mechanics checking the engine itself, the wiring, making sure each part works right?

It's an internal inspection.

Got it.

Black is outside, white is inside.

Seems clear enough, but what about unit tests?

I feel like I hear conflicting things.

Are they black or white or neither?

That's a really sharp question because unit tests are kind of slippery.

They don't have to be 1 or the other.

It actually depends entirely on how you write them.

OK, yeah.

If you write a unit test just based on the public interface, the method signature, what it promises to do, you're treating that unit like a black box, purely external view, right?

Based on the spec.

Exactly.

But let's say you write a test.

Then you run a code coverage tool and you see oh, this else branch wasn't executed.

If you then write another test specifically to hit that internal branch, now you're using knowledge of the internal structure.

So that makes it white box even though it's still a unit test?

Precisely your motivation for writing that second test came from looking inside.

So the same unit can be tested with both black box and white box approaches, even within unit testing.

That clears things up.

It's the approach, not the test type itself.

And this whole blurring lines thing, it applies to TDD too, right?

Test Driven Development because you write tests first there.

Absolutely.

TDD is a fascinating case study for this.

Kent Beck himself said something really insightful about it.

He basically said, look, TDD tests are written before the code exists, so by definition they have to be black box tests.

Initially you're testing against an interface that isn't implemented yet.

Makes sense.

But then, he added, he often gets the inspiration for the next test by looking at the code he just wrote to make the previous test pass.

Right, which is white box thinking.

Exactly.

That's the hallmark of white box testing.

So what's fascinating here?

Whereas key DD isn't strictly one or the other.

It's this cycle.

You start black box.

What should it do?

Write minimal code, then use white box insight.

How does it work now to figure out the next black box test?

What else should it do?

It really shows these categories aren't rigid boxes in practice, they're more like different ways of looking at the problem.

It's a really practical way to see it.

Not strict rules, but useful perspectives.

OK, so we know how we might look at the code, but what do we actually test it with?

What data?

Because like you said, testing everything.

Forget it.

Even for simple functions, A function taking two integers, testing every combo could take forever.

Literally.

And just picking random inputs that sounds hit or miss.

You might test the same kind of thing over and over and miss something crucial.

Here's where it gets really interesting.

You've hit on a huge challenge.

Since exhaustive testing is out, especially for black box, we need smarter ways to choose our test inputs and a key technique here is equivalence classes.

The idea is simple but powerful.

You look at all the possible inputs and divide them into groups or classes.

Within each group, all the inputs are considered equivalent in the sense that if one of them finds a bug, any other input in that same group probably would too.

OK.

Grouping similar inputs.

Exactly.

And the beauty is you only need to test one representative value from each distinct class.

This massively cuts down the number of tests you need.

Let's use that income tax example from the book.

Remember the salary ranges Range one $1930.99 to $2826.65 gets 7.5% tax.

Range two $2826.66 to $3764.68 gets 22.5%.

Range 4.

Anything above $4064.68 gets 27.5% right?

Four different tax brackets.

So using equivalence classes, you wouldn't test thousands of salaries, You'd identify these 4 ranges as your equivalence classes for valid inputs and you just one salary from each range.

Maybe 2000 dollars, 3000 dollars, $4000 and $5000.

Just four tests to cover the core logic for all valid positive salaries.

That's incredibly efficient compared to trying everything.

Just pick one from each zone.

What about the edges, The exact points where the tax rate changes?

Aren't those risky spots?

Absolutely spot on.

Those edges or boundaries are notorious bug hot spots, and that's why equivalence classes often go hand in hand with another technique, boundary value analysis.

Equivalence classes tell you which groups to test.

Boundary value analysis tells you which specific values, especially around the edges of those groups, are most critical.

The thinking is programmers often make off by 1 errors or handle comparisons incorrectly.

Right at the limits.

Yeah, I.

Can see that like using tech instead of.

Exactly that kind of thing.

So boundary value analysis says for each boundary of an equivalence class, test the boundary value itself, the value just before it, and the value just after it.

Let's take that first salary range again, $1903.99 to $2826.65.

Boundary value analysis would suggest testing these specific values.

For that lower boundary, $1930.98 just below the boundary, $1923.99 right on the boundary.

For the upper boundary, $2826 right on the boundary, $2826.66 just above the boundary.

OK, so you're testing the exact point and also making sure you don't accidentally include values just outside the range or exclude values right on the edge.

Precisely.

You're probing those transition points very carefully.

Mining equivalence classes with boundary value analysis gives you pretty good coverage without needing an impossible number of tests.

It makes total sense, though I guess defining those classes and boundaries isn't always as neat as salary ranges, right?

Like for a text field or something more complex.

That's true, it requires more thought sometimes for text, boundaries might be empty string, minimum length, maximum length, strings with special characters, etcetera.

But the core idea of partitioning inputs and checking edges still applies.

Right.

OK, so we've COVID testing from the inside, testing from the outside, picking smart inputs, but ultimately software has to work for the customer.

Let's talk about acceptance tests.

These are done by users, right?

Exactly.

This is where the rubber really meets the road.

Acceptance tests are all about the customer deciding if the software is, well, acceptable, doesn't meet their needs, will they sign off on it?

This decision determines if it ships to production or needs more work.

In Agile, you often hear that a user's story isn't truly done until it passes its acceptance tests, which are usually defined and run by the product owner or a customer proxy.

So what makes these tests different besides who runs them?

They sound like they might be manual.

Yes, that's one key difference.

Acceptance tests are typically manual tests, real people interacting with the system customers or their representatives, not usually automated scripts written by devs.

The 2nd and maybe even more important difference connects back to something we mentioned earlier, verification versus validation.

Right verification is did we build it?

Right validation is did we build the right thing?

You got it.

Most of the tests we discussed earlier, unit integration, even system tests based on specs, are primarily verification.

They check if the software matches the blueprint.

Acceptance tests are fundamentally about validation.

They check if the software actually solves the customer's real problem and meets their actual needs, which might not have been perfectly captured in the initial specs.

It's the ultimate reality check.

That's a crucial distinction.

Building the wrong thing perfectly is still building the wrong thing.

Are there different types of these acceptance tests?

Yes, there's usually a sequence.

First comes alpha testing.

This happens with real customers, but it's in a controlled environment.

Think maybe at the developer's office or on a specific testing set they control.

It's a small group, close interaction, easy to observe and get quick feedback.

OK, like a review.

Screening.

Kind of, yeah.

If the alpha tests go well, then you might move to beta testing.

Beta tests involve a much larger group of customers using the software in their own environments, on their own hardware, doing their own real tasks.

It's no longer controlled.

This is where you find issues related to diverse setups, unexpected workflows, and just general real world chaos that you can't easily simulate.

Got it.

Alpha is controlled, beta is out in the wild, more or less.

OK, OK, so far it feels like most of these tests, black box, white box, equivalence classes, even acceptance to some extent, are focused on finding functional bugs.

Does the software do what it's supposed to?

But what about everything else?

Like how fast is it?

How easy is it to use?

What happens if something breaks?

So what does this all mean for a complete testing strategy?

We need to test that other stuff too, right?

Absolutely critical point.

Focusing only on functional requirements is a huge mistake.

The non functional stuff, performance, usability, reliability is often just as important for success.

O yes, a complete strategy must include tests for these nonfunctional requirements.

Take erformance tests.

These measure how the system behaves under load.

Think about that ecommerce site getting ready for Black Friday.

They don't just need the buy button to work, they need it to work fast for millions of simultaneous users.

Otherwise, nobody buys anything.

Exactly.

So they'll run load tests, stress tests, pushing it until it breaks to see where the limit is, spike tests, sudden bursts of traffic, all to ensure it performs well under pressure.

OK.

Performance is key.

What else?

You mentioned usability.

Right usability tests.

These focus purely on the user interface and the user experience.

Is it easy to learn?

Efficient to use?

Are the buttons clear?

Is the workflow logical?

This often involves watching real users try to accomplish tasks with the software and seeing where they get stuck or confused.

It's less about code bugs and more about design flaws.

Makes sense.

A functional but unusable system isn't much good.

And the last one, failure.

We test for failure.

We do failure tests, sometimes called resilience or chaos testing.

The idea isn't to prevent all failures, That's impossible in complex systems.

It's about simulating failures to see how the system reacts.

What happens if a key database goes down, or a network connection fails, or an entire server rack loses power?

Yikes, you actually simulate that.

You do, in controlled ways.

Hopefully.

The goal is to ensure the system degrades gracefully, maybe switches to a backup, recovers properly, and doesn't, you know, corrupt all its data or crash entirely.

It's about building resilient systems that can handle the inevitable bumps in the road.

Wow.

OK, so black box, white box, equivalence classes, boundary values, alpha, beta performance, usability, failure tests.

It's quite a landscape beyond just unit and integration.

There's been a really insightful deep dive.

Thanks for joining us today and exploring this wider world of software testing.