Chapter 4: Statistical Interpretation of Entropy

Welcome to Last Minute Lecture.

This free chapter overview is designed to help students review and understand key concepts.

These summaries supplement, not replace, the original textbook and may not be redistributed or resold.

For complete coverage, always consult the official text.

Welcome to the Deep Dive.

Today, we're taking on one of the most abstract and yet most fundamentally important concepts in material science, entropy.

The big one.

It is.

Our mission is to move beyond that macroscopic sort of phenomenological view of classical thermodynamics, you know, the world of pressure, volume, temperature, and dive deep into the microscopic, the statistical realm.

And it's such a necessary shift.

I mean, for centuries when physicists first embraced the first law, internal energy, that made perfect physical, it's just energy storage, right?

Sure, easy to grasp.

But entropy, yes, and the second law was this, this negulous state function.

It told us if a process was possible, but it didn't really tell us why on a physical level.

We just didn't have that tangible atomic understanding of what S actually was.

So our goal today is to bridge that very gap.

We want to actually quantify disorder, taking the foundational work of Boltzmann and Gibbs to give S the physical meaning it deserves.

By linking it to atoms and molecules and probability.

And we can start with maybe the simplest concept, which was a gift to us from Josiah Willard He basically said entropy is just a measure of a system's mixed up -ness.

Mixed up -ness.

Yeah, at the atomic or molecular level.

So a larger value of entropy just means the constituent particles are more distributed or, well, more mixed up.

And you can see this almost immediately in phase changes, right?

If we think about what's called configurational entropy.

The arrangement of particles in space.

Right.

So a perfect crystalline solid has its particles just rigidly vibrating around these fixed sites.

It's minimal mixed up -ness.

Very low configurational entropy.

But then you move to a liquid.

Now the particles are free to wander.

They're moving through this whole communal volume.

That freedom of movement is a huge increase in the number of accessible configurations.

Which means higher entropy.

Much higher.

And then a gas, of course, with the maximum volume for molecular motion.

Well, that has the highest configurational entropy of the three.

It lines up perfectly with what we already know.

That melting and boiling are always entropy increasing processes.

Okay, but here's where we hit a really fascinating contradiction.

One that absolutely requires this deeper statistical view.

Ah, the supercooled liquid.

The supercooled liquid!

Imagine you have an isolated supercooled liquid and it spontaneously transforms into a crystal.

Now we know this is an irreversible process, which means the total entropy of the universe must increase.

It has to.

That's the second law.

But if we only look at the substance itself, the crystal that forms is highly ordered.

It's seemingly less mixed up than the liquid it replaced.

Configurationally, yes.

So how can a spontaneous process decrease the system's configurational entropy while at the same time increasing the total entropy of the universe?

It feels wrong.

It does.

And the crux of the problem, the solution, is that you can't just isolate configurational entropy from thermal entropy.

Okay, so what's going on with the energy here?

The resolution requires us to look at that energy flow.

So when the supercooled liquid freezes, it releases thermal energy.

The enthalpy of freezing.

Exactly.

And because the system is isolated, that released heat immediately increases the temperature of the system's surroundings, or the reservoir.

And this temperature increase drives a massive, massive increase in thermal entropy.

And that increase is greater than the decrease in configurational entropy within the substance itself.

So the net result is an increase.

The thermal part wins out.

The thermal part wins.

It's a critical reminder.

You have to consider all aspects of S.

And this leads us right to the definition of reversible equilibrium.

When does that total entropy change for the universe become zero?

That happens only at one specific point.

The equilibrium melting temperature, at that single temperature, the increase in disorder of the heat reservoir, the part that absorbs the thermal energy, it perfectly balances the decrease in the disorder of the substance.

And so the total entropy of the universe stays constant.

It stays constant.

And that's why the solid and liquid can coexist reversibly at that point.

Okay.

So to really quantify this mixed -upness, we have to make this transition from what we call the macrostate defined by PVT to the microstate.

A macrostate, it's fixed by just a few measurable variables.

But what is it that defines the microscopic complexity?

That's the microstate, or as Boltzmann called it, the complexion.

The complexion, I like that.

Yeah, it's a great term.

And it's the instantaneous snapshot.

It's defined by the specific positions and the momenta of all 10 to the 24 individual atoms.

The difference in scale is just staggering.

And statistical thermodynamics makes one huge defining postulate about this microscopic world.

It does.

It says that the equilibrium state of a system is simply the most probable of all its accessible microstates.

So we're just searching for maximum probability.

We're looking for what makes a state the most probable.

Let's try to illustrate this.

Imagine a simple system, three distinguishable particles, so n equals 3, and they're confined to a fixed total energy, let's say u prime is 3u, where the u is just the energy spacing between levels.

We just want to see how these three particles can distribute themselves across the energy levels.

Alright, so option one.

All three particles could be in level epsilon 1.

The total energy is 3u.

That works.

It does.

And how many ways can you arrange that?

Well,

just one way.

They're all in the same place.

One microstate, omega equals one.

Right.

Now what about another distribution?

Let's call it distribution C.

One particle is in epsilon 0, one is in epsilon 1, and one is in epsilon 2.

Total energy is still 0 plus 1 plus 2, 3u.

Still 3u.

But now,

because the particles are distinguishable, there are three choices for which particle goes into epsilon 0, then two choices left for epsilon 1, and one choice for epsilon 2, which is three factorial six ways, omega is six.

Six microstates versus one.

It's a dramatic difference.

Yeah.

And if we sum up all the possible arrangements for that total energy of 3u, we find there are 10 total microstates, and this one distribution holds six out of the 10.

It has a 60 % probability.

And this simple exercise, it really reveals the heart of the whole statistical approach, even though every single microstate is equally probable.

One in 10 chance for any specific one.

Right, one in 10.

But the macroscopic state that has the most ways the particles can be arranged, the highest omega, that's the state we actually observe as equilibrium.

And for any real system with a mole of atoms, the difference between the most probable omega max and any other omega is so astronomically large.

That they're basically the same number.

Exactly.

We can essentially equate the total accessible microstates omega with its maximum value.

Maximum probability is maximum omega, which corresponds to the state of maximum entropy.

That is the critical link.

That's the link.

Okay, we've established thermal entropy.

Now let's try to quantify that configurational entropy by looking at mixing.

Good idea.

Imagine mixing two different crystals, say A and B, and let's assume there's no energy change when A and B atoms bond.

So it's just about arrangement.

Purely arrangement.

If we start with all the A atoms on the left and all the B atoms on the right, there's only one way to do that.

Omega equals one.

The unmixed state.

The unmixed state.

But as they spontaneously diffuse and mix, the total number of available spatial configurations just skyrockets.

Right.

If we had just, say, four white atoms and four gray atoms, eight sites total, the total number of possible configurations is eight factorial over four factorial times four factorial, which is 70.

70.

So the spontaneous process is driven by the fact that you've gone from one accessible configuration to 70.

And the most probable mixed state, the one that drives us to equilibrium, is the one with the uniform concentration.

Which in this little case is 36 out of those 70 ways.

This statistical drive is precisely why we get that quantitative result for the molar configurational entropy of mixing.

Which is delta S configuration equals minus R times.

The quantity XA log XA plus XB log XB.

And that equation confirms it.

The maximum entropy of mixing happens precisely when the mole fractions are equal.

XA equals XB equals .5.

Perfectly mixed up.

You can see the same thing with spin entropy.

In a paramagnet with no external field, the energy of an upspin or a downspin is the same.

So for eight atoms, you have two to the eight.

256 total microstates.

And they're all equally likely.

All equally likely.

But the macroscopic equilibrium state we actually observe is one with zero magnetization.

M equals zero.

Which happens when you have exactly four upspins and four downspins.

And that one arrangement has 70 ways of happening, making it by far the most probable state.

That's the equilibrium we see.

OK, so let's shift gears a little bit.

So far, we've only talked about isolated systems.

Every microstate had the same energy, so they were all equally probable.

But what happens if the system is not isolated?

If it can exchange energy?

Right, a much more realistic scenario.

If the system can exchange energy, then surely the lower energy in microstates become far more probable.

They do.

And to describe this, we need the Boltzmann distribution.

Now, this requires some pretty intense mathematical heavy lifting.

A bit of a jump.

It's a jump.

You have to maximize the number of arrangements omega, but now you have to do it while sticking to the constraints of fixed total energy and fixed particle number.

So this is where things like Stirling's approximation come in for dealing with huge numbers and the method of undetermined multipliers.

Exactly.

They're abstract mathematical tools, but they lead to a really profound physical understanding.

So we do all this difficult math to find the most probable distribution, nigh, and what pops out.

The relationship that emerges is that the population of any energy level, nigh, decreases exponentially with its energy.

So nigh is proportional to e to the minus beta epsilone.

The Boltzmann distribution.

That's it.

And those constants that emerge from the math, they become the critical bridge back to the macroscopic world.

The summation over all possible states, z equals the sum of e to the minus beta epsilone, is known as the partition function.

And that function isn't just a summing device, is it?

Not at all.

It contains all the thermodynamic information about the system.

It's incredibly powerful.

And most critically, that mathematical multiplier, beta.

Beta turns out to be inversely proportional to the absolute temperature.

Beta is one over kBT, where kB is Boltzmann's constant.

That's the moment.

That definition proves that the statistical approach is describing the exact same physical reality as classical thermodynamics.

And it leads us directly to the foundational length that Boltzmann postulated.

S prime equals kB times the natural log of omega.

S equals kB ln omega.

That elegant equation is the core of statistical thermodynamics.

It connects the measurable thermodynamic state function, S, directly to omega, the number of ways the system can distribute its particles and energy.

If we were to picture that Boltzmann distribution on a graph, we can really see the influence of temperature.

A higher temperature means a smaller beta, and that causes the population of the upper energy levels to become relatively more populated, right?

We're spreading the energy out more.

We are.

You're increasing the average energy, and by doing so, you're increasing the thermal entropy.

We can even explore this with a really simple two -state system, just an epsilon zero and an epsilon one, as the temperature approaches absolute zero.

That exponential term, e to the minus beta epsilon, just Vega, it forces the population of the higher state, n1, towards zero, while the particles just clump down into the lowest energy state.

Disorder vanishes.

It does.

Entropy approaches zero, which is a perfect alignment with the third law of thermodynamics.

And if we take the derivative of the internal energy with respect to temperature, we get the heat capacity, Cv, and the plot of Cv versus T for this simple system shows a really distinct maximum at a certain temperature.

And that maximum happens exactly where the input energy, the heat, is most effective at mixing the occupations of those two levels.

You're creating the maximum amount of disorder for the minimum input of heat.

This link between statistics and observable properties, this was the whole basis for the early modeling of solids.

Oh, absolutely.

The Einstein model, back in 1907, treated a solid as a collection of independent, distinguishable quantum harmonic oscillators.

A good first step.

A very successful first step.

It predicted that Cv goes to 3R at high temperatures, DeUlong and Petit's law, and it goes to zero at low temperatures.

But it was inaccurate at low T.

Because it assumed all the atoms vibrate independently at just one frequency.

Which they don't.

So the Debye model in 1912 gave us the major improvement.

It assumed a continuous spectrum of vibrational frequencies.

It treated the vibrations as collective waves, as phonons in the solid.

And that conceptual change moving from independent oscillators to correlated vibrations that provided an excellent fit to the experimental data.

Especially at low temperatures, where the Debye model correctly predicts the T -cubed law, that Cv is proportional to T -cubed.

Okay, so let's bring this all home.

Let's tie this whole statistical journey back to the second law.

Classical thermodynamics calls uphill heat, flow heat, moving from cold to hot impossible.

Full stop.

Impossible.

What does the statistical view tell us?

Well, statistical thermodynamics connects that macroscopic law to probability.

Irreversible heat transfer from a hot body A to a cold body B is irreversible, because it dramatically increases the product of their microstates, omega times omega B.

So it increases the total entropy.

Right, the reverse flow.

I mean, it's impossible.

It's just highly, highly improbable.

And as the number of particles gets to a macroscopic scale, the probability of seeing heat flow the wrong way, just it approaches absolute zero.

Right.

And we've shown that the total entropy is additive, because the total number of microstates is the product of the individual parts.

Total is thermal plus configurational and so on, because omega total is omega thermal times omega configurational.

So we did it.

We successfully connected the abstract state function S to the physical reality of statistical mixed -upness.

We started this deep dive looking for a physical understanding of entropy, and we found it quantified in that simple yet profound relationship, S equals Kb log omega.

We now know that the thermodynamic equilibrium state is simply the one that allows for the maximum number of configurations.

And here's maybe a provocative thought to leave you with.

The difference between classical thermodynamics calling an event impossible and statistical thermodynamics calling it highly improbable.

That's a fundamental difference.

So when we design and study systems at the nanoscale, where you're dealing with maybe just dozens or hundreds of particles,

the mathematical probability of observing those impossible low probability events, it becomes finite.

It's not zero.

It could happen.

It could happen.

So how might we leverage or maybe guard against these statistically possible deviations when we're designing things like nanotechnologies?

That's a fascinating challenge to consider as you apply these statistical foundations to the real world of materials.

Thank you for diving deep with us today.