Chapter 5: Deployability – Continuous Integration & Delivery

Welcome to Last Minute Lecture.

This free chapter overview is designed to help students review and understand key concepts.

These summaries supplement not replaced the original textbook and may not be redistributed or resold.

For complete coverage, always consult the official text.

Welcome back to the Deep Dive.

Today, we are really diving into something fundamental for modern business speed, specifically how software gets delivered.

Yeah, it's a huge shift.

Remember the bad old days, quarterly releases,

yearly.

Painful weights.

Now, the big players, especially in e -commerce, are talking about, what, hundreds of releases a day.

Exactly.

That old model of bundling everything up and just, you know, hoping for the best.

It's completely obsolete now.

Competition demands constant updates, fixes new features, hitting users almost instantly.

Hundreds a day.

I mean, 20 years ago, that was sounded absolutely nuts.

So how do you do that without everything just falling apart, constantly breaking?

Right.

And the answer, the core architectural enabler, is a quality attribute we call deployability.

That's our focus today.

We're going to unpack how software architecture makes this rapid,

reliable, predictable move to production possible.

Okay, so our mission for you, the listener, is to get under the hood of continuous deployment, the architecture behind it, and, well, the actual patterns and tricks that companies like Netflix and Amazon use for that speed and control.

Yeah.

Speed, predictability, and control.

That's key.

First things first, though.

Terminology.

Continuous deployment CD gets mixed up with continuous delivery all the time.

What's the actual difference?

It's a really critical distinction, and it boils down to automation.

Continuous deployment means the entire process, from code check -in right through to that code running live in production for users.

It's fully automated.

No human hands touch it.

Wow.

Okay.

That sounds bold.

It's the ultimate aim for many.

Continuous delivery, on the other hand, is automated right up until that final step.

There's a human gatekeeper, someone who has to push the button to deploy to production.

Why the manual step then?

Regulations, policy?

Could be either.

Maybe regulatory compliance needs a final sign -off, or it's just internal company policy.

The automation is there, but that last step is manual.

Deployment is the goal, but the level of automation defines the term.

Got it.

Okay.

Let's talk about the engine driving the deployment pipeline.

If we want that full automation, we need this pipeline.

Walk us through it from code check -in to production.

Right.

The deployment pipeline.

It's the sequence of tools, activities, checks,

everything that happens automatically.

It typically moves through four main environments.

Think of them as stages.

Four stages.

Okay.

Lay them out for us.

Stage one,

the development environment.

This is purely local.

It's where the developer writes code, runs their unit tests on just their piece, their module, make sure it works in isolation.

Standard stuff so far.

Then they commit the code.

Exactly.

That commit triggers stage two, the integration environment.

This is where a CI server usually kicks in.

It builds the code, compiles or interprets it, creates an executable for the service.

By service, you mean any deployable chunk of software.

Yes.

Any independently deployable unit.

The goal here in integration is to run tests to prove that this service works correctly with all the other services it needs to interact with.

Does it play nice with its peers?

Okay.

Integration passed.

Where next?

It moves up to the staging environment.

Think of staging as the full dress rehearsal.

It's designed to mirror production as closely as possible just without real users touching it yet.

So what happens in staging?

This is all about system level quality.

Performance tests, security scans, maybe checking software licenses, even user acceptance testing with internal folks or simulators if it's an embedded software.

You're checking.

Can the whole system handle this change under realistic load?

And if it passes staging,

the final leap.

Production environment.

If all checks pass, it gets deployed using specific strategies.

We'll get to things like blue -green or rolling upgrades later and real users start interacting with it.

And of course, it's closely monitored there.

So the pipeline is basically this automated gauntlet.

Code has to prove itself fit at each stage before it gets to the user.

But what about that age -old problem?

It worked on my machine.

Why doesn't that happen as much now?

Oh, yes.

The classic excuse.

It used to happen because the dev environment, staging, production,

they're all slightly different.

Different libraries, OS patches,

configurations.

Way into chaos.

Total chaos.

The game changer was virtualization technology containers, VMs.

This enables what we call environment parity.

Meaning they're identical.

Not necessarily in scale.

You might have fewer servers in staging than production, but identical in fundamental structure.

The OS, the libraries, the network setup, the dependencies, all configured the same way using infrastructure as code tools.

So tests and staging really reflect what will happen in production.

Much, much more reliably.

That predictability is what allows the speed.

Okay.

Speed is great, but you need to know if the pipeline itself is healthy, right?

How do architects measure the quality of this rapid process?

Good question.

There are three key metrics.

First, obvious one,

cycle time.

How fast can you get from an idea or a commit to production?

Hours, minutes.

Speed.

Makes sense.

Second.

Second and absolutely vital for safety.

Traceability.

If something does go wrong in production.

You need to know what caused it instantly.

Exactly.

Traceability is the ability to pinpoint everything that went into that broken release.

The specific code version, all the library versions, the build tools used, the test cases that ran.

Usually this info is stored in an artifact repository.

So you can roll back precisely or fix it fast.

Precisely.

It turns failure investigation from a week -long archaeological dig into something you can potentially identify in minutes.

And the third metric.

Go on.

Repeatability.

This means doing the same action with the same inputs always produces the same output.

If your build script just pulls latest for a library.

It might work today, break tomorrow if latest changes.

Right.

Repeatability means locking down those versions.

Using the exact same artifacts guarantees the same result every time.

No surprises from shifting dependencies.

We've got the pipeline.

Now let's zoom in on the quality attribute it serves.

Deployability.

What does that mean formally?

Formally, deployability means your software can be put into an environment deployed within a predictable, acceptable time frame and effort.

And critically, if that deployment causes problems, maybe violates an SLA.

SLA meaning service level agreement.

Like performance guarantees to the customer.

Exactly.

If you risk violating an SLA, the system must be designed so you can roll back

also within a predictable time and effort.

Predictability is key for both deploying and undeploying essentially.

And the goal is the customer never even notices the deployment happened, right?

No slowdowns, no errors.

That's the idea.

Yes.

Seamless.

So the architect needs to ensure the system has three core traits.

First, it has to be granular.

Which implies you can't really do this with a giant monolith, can you?

You need smaller pieces.

Generally, yes.

You need the option to deploy just a small part or maybe a whole thing, but granularity is crucial.

Second, it must be controllable.

Meaning you can choose that granularity.

You can monitor the deployment as it happens and you have reliable mechanisms to roll back if needed.

And third, it has to be efficient.

Deployment and rollback shouldn't take ages or require heroic effort.

How do we measure this?

You mentioned response measures.

Give us an example.

It needs to be concrete.

Measurable.

Something like...

This all sounds like it requires more than just code.

It feels like a whole philosophy, a way of working.

This brings us to DevOps, right?

Absolutely.

DevOps is the cultural and practical movement that grew up around enabling this kind of deployability.

It's about breaking down the walls between development and operations teams, using shared tools and practices to dramatically shorten that cycle time from commit to user value.

CD is really the engine of DevOps.

And now we hear DevSecOps a lot, too.

Right.

That's about integrating security practices throughout the entire pipeline, not just tacking it on at the end.

Continuous security testing, vulnerability scanning happening automatically as part of the flow.

Makes sense.

Okay.

Culture, pipeline.

Let's get tactical.

What are the specific tools or techniques an architect uses to manage this?

The deployability tactics.

Right.

The tactics.

These are often provided by the CACD infrastructure itself.

We can group them into managing the pipeline and managing the deployed system.

First, pipeline tactics.

Number one, scale rollouts.

Don't push to everyone at once.

Never.

You gradually release the new version to small controlled groups of users, maybe 1 % than 5 % than 20%.

This needs some kind of smart routing mechanism.

It minimizes the blast radius if there's a problem.

And if there is a problem in that small group.

You need tactic number two, rollback.

This has to be automated and reliable.

Modern deployments can involve multiple services, maybe database schema changes.

The rollback needs to undo all related changes cleanly.

Complex.

Can be.

Which leads to tactic three, script deployment commands.

Treat your deployment automation scripts like production code.

They need documentation, code reviews, testing, version control.

Automating complex orchestration steps removes human error.

Okay.

Those manage the release process.

What about managing the system once it's live?

Three key tactics here too.

First,

manage service interactions.

This is crucial when you have, say, new versions of a service running side by side during an upgrade.

You need a way, maybe an API gateway or service mesh to mediate calls so that incompatible versions don't cause errors.

Avoiding chaos during the transition.

Exactly.

Second tactic tackles that environment inconsistency problem again.

Package dependencies.

Bumble everything.

Yes.

Package the service with everything it needs.

Specific library versions, the right OS configuration, maybe helper containers, like Docker containers, Kubernetes pods or VMs.

This ensures what you tested in staging is exactly what runs in production.

Consistency across the board.

Makes sense.

And the last tactic, the ultimate safety net.

The feature toggle or feature flag.

This is brilliant.

It's like a kill switch built into your code for a specific new feature.

So you deploy the code, but the feature is off.

Or you deploy it, turn it on, and then if monitoring shows it's causing problems, maybe high error rates, slow performance, you can flip that one feature off instantly at runtime.

Without doing a full rollback or deploying new code.

Ah, so you decouple deployment from release.

You can deploy code safely, then decide when to activate the feature for users.

Precisely.

It gives you incredible control and reduces risk dramatically.

Found a bug in that new checkout flow five minutes after release?

Toggle it off.

Fix it.

Toggle it back on later.

No massive rollback needed.

Genius.

Okay.

Let's zoom out again to architectural patterns that enable all this.

Two main types you mentioned.

Structuring services and deploying them.

Let's start with structure.

Microservice architecture.

Right.

This is foundational for a lot of high deployability systems.

The idea is you break your system into small independent services.

Yeah.

Crucially, they communicate only through well -defined interfaces.

Usually network messages like REST APIs or asynchronous events.

No shared databases.

No shared memory.

Ideally, no direct sharing that creates tight coupling.

Each service manages its own data.

They're typically a small thing to pizza rule for team size, often stateless and independently deployable.

And the big win for deployability.

Independence.

Teams can choose their own tech stacks.

Different languages, different databases if appropriate.

And crucially, they can deploy their service without coordinating massive releases with dozens of other teams.

This drastically cuts down time to market.

Scaling is also often easier.

But nothing's free.

What are the tradeoffs?

The downsides.

Well, it's a distributed system, which is inherently more complex.

Network calls are slower and less reliable than in -process calls.

Handling transactions that span multiple services is hard think eventual consistency challenges.

And honestly, just keeping track of hundreds or thousands of microservices, understanding how they all fit together.

That operational complexity and maintaining intellectual control is a real challenge.

Okay.

So that's the structure.

Deployment patterns.

You mentioned complete replacement, swapping out all old instances for new ones without downtime.

Two main ways.

Yep.

First is blue -green deployment.

Imagine you have N servers running the current blue version.

You spin up an entirely new, identical set of N servers running the green new version alongside them.

So for a time, you're running double the infrastructure, two N resources.

Exactly.

That's the cost.

Once the green environment is tested and ready, you flip a switch, usually DNS or a load balancer, to direct all user traffic to green.

Blue is kept idle, ready.

And if green has problems?

You just flip the switch back to blue instantly.

Very safe,

minimal downtime risk.

Rollback is super fast.

Okay.

Safe, but potentially expensive.

What's the alternative?

Rolling upgrade.

Right.

Rolling upgrade is more resource efficient.

Instead of doubling up, you replace instances one by one or maybe in small batches.

So if you have N instances, you bring up one new one, take down one old one, bring up another new one, take down another old one, and so on.

Peak resource usage is only N plus one, then?

Much cheaper.

Why not always do this?

Risk.

Because for a period, you have both old and new versions handling live traffic simultaneously.

This introduces two main dangers.

First,

temporal inconsistency, meaning a single user session, maybe involving multiple requests,

could hit an old instance for step one and a new instance for step two.

If the logic or data format changed, that could lead to errors or weird behavior.

I see.

And the second risk?

Interface mismatch.

If the new version has a changed API or expects data differently, requests coming from clients or other services that only understand the old interface might break when they hit a new instance.

So blue -green is safer because it costs more resources.

Rolling upgrade saves resources but carries more risk during the transition.

That's the core trade -off.

You choose based on your risk tolerance and budget.

Okay.

That leads us to the last set of patterns.

Partial replacement, often called testing and production.

First up, canary testing.

Named after the canary in the coal mine, right?

The idea is you deploy the new version to just a tiny subset of real users first.

Maybe you're at 1 % of traffic to the new version, while 99 % stays on the stable old version.

And these might be specific users, internal folks, beta testers.

Could be.

Sometimes it's internal employees, dog fooding.

Sometimes it's opt -in power users.

Sometimes it's just a random small percentage of the general user base.

You monitor that canary group very closely.

The benefit seems obvious.

Find problems with real users, but limit the damage.

Exactly.

You get real -world feedback and data on performance and correctness before exposing everyone.

If the canary release looks good, you gradually increase the percentage.

If not, you roll it back quickly, only affecting that small group.

There's also a variant called dark launching.

Dark launch.

Where you deploy the new code, maybe even have it execute in production, but users don't see the results.

It might be used to test the performance of a new backend system under load,

or validate a new algorithm without affecting the user experience directly.

Sneaky.

Okay.

And the final pattern, often driven by product and marketing teams,

A -B testing.

Right.

A -B testing isn't just about finding bugs.

It's about running experiments.

You present two or more variations of something, maybe a web page layout, a button color, a recommendation algorithm to different segments of your user simultaneously.

Like the famous Google testing 41 shades of blue.

That's a classic example.

Or e -commerce sites testing different checkout flows.

You measure which version, A or B, leads to better business outcomes, more clicks, higher conversion rates, whatever your metric is.

So you're using real user behavior to make data -driven decisions.

Precisely.

It's a powerful way to optimize.

The main cost is developing and managing the multiple variants, knowing one will likely be discarded eventually.

Okay.

So this whole journey through deployability shows it's really the architectural response to needing to move fast safely.

It's not just speed for speed's sake.

It's about predictable, controlled, high -frequency change.

Absolutely.

And notice how almost every tactic and pattern involves trade -offs.

Microservices give speed, but add complexity.

Blue -green is safe, but costly.

Rolling upgrade is cheaper, but riskier.

Feature toggles add code complexity, but give runtime control.

It's always a balancing act.

Which brings us to a really interesting final thought for you, our listeners, to Ponner.

We talked about rollback as essential,

but think about when a nasty bug is found right after deployment.

Under what circumstances might it actually be better, faster, or safer to roll forward to quickly push out another new version with an immediate fix instead of going through the process of rolling back to the previous stable version?

Yeah, that's a tough call.

Sometimes rollback itself is complex or time -consuming, especially if data changes were involved.

If the fix is simple and can be deployed very quickly using that same rapid pipeline, maybe rolling forward is less disruptive overall than trying to undo everything.

It forces you to weigh the risk and time cost of rollback versus the risk and time cost of a rapid forward fix.

A fascinating dilemma in the heat of the moment.

It really underscores that continuous tension between speed, cost, and risk in modern software.

It really does.

Well, that's a perfect place to wrap up.

Thank you so much for joining us on this deep dive into architecture of deployability.

We hope you found it valuable.

Thanks for listening.

We'll catch you on the next deep dive.