In all of physics, there is perhaps no topic more underrated and misunderstood than entropy. The behavior of large collections of particles, such as the universe, a grain of sand, or a tuna salad sandwich, is dictated by two universal laws: one involving energy, the other involving entropy. And yet, while energy is described in great detail throughout any introductory physics textbook, entropy is relegated to about two or three pages, and is usually badly described.
Well, no more! Here's the real story of physics. Here's what really drives the universe. Here's what your physics instructor won't tell you. Here's entropy !!!
The answer is Entropy
So what's the question? Only this: Why do things happen the way they
do, and not in reverse?
For example:
When you drop an egg on the floor, it breaks. But dropping a broken
egg on the floor doesn't cause it to become
whole.
A new deck of cards comes with all the cards
in order. Shuffling them mixes up the cards. Now take a deck of
cards that's mixed up and shuffle them. The
cards do not come back to their original order.
Pick up a can of air freshener and push down
the button. The air freshener spews out of the can and spreads out
around the room. Now try gathering up the
air freshener and putting it back in the can. Doesn't work, does it?
You put ice in a drink to make it colder. You
trust that when you do this, your drink will get colder while the ice
gets warmer, and eventually melts. Pouring
water into a cold drink won't cause the reverse of this process to
occur. That is, when you pour water into a
cold drink, the water doesn't freeze while the rest of the drink gets
warmer.
The shuffling of a deck of cards, the spray of an aerosol can, the flow
of heat from warm things to cold things, the breaking of
an egg; these are all examples of what physicists call " irreversible
processes" . They occur very naturally, but all the king's
horses and all the king's men can't undo them.
Practically everything that goes on in the universe is irreversible.
(In the technical, " physics" sense, that is. Don't read anything
politcal, philosophical, or sociological into that last statement!)
But why are all these processes irreversible?
The answer is entropy!
Entropy and Poker
Here are two poker hands:
For those of you not familiar with poker, John's hand is one of a type
called a "straight flush." (A straight flush consists of five
cards in a row, of the same suit.) As you might guess, it's considered
a very good hand. Dave's got the type of hand which is
sometimes called "junk" by the more refined poker players. (I usually
have stronger language for it.)
John will most likely savor this moment, for it is a rare one indeed.
A player receiving Dave's cards will most likely be
unimpressed, knowing that this hand is as common as dirt. But is it?
The chances of being dealt John's hand out of a thoroughly shuffled
deck of cards is about one in 2.6 million. On the other
hand, the chances of being dealt Dave's hand is........ about one in
2.6 million!
The chances of receiving any particular five cards in a deal from a
shuffled deck of cards is the same. There are about 2.6
million five card hands that you can make out of a standard 52 card
deck, and each of them is equally likely.
So why is John surprised to see his hand, while Dave isn't surprised to see his?
Because John's hand is one of a very select group of hands, called a
straight flush. Out of the 2.6 million possible hands, there
are only 40 straight flushes. Dave has junk. There are over a million
hands that are junk. In other words, there are very few
combinations of five cards that form a straight flush. There are very
many combinations of five cards that result in junk.
Now here's a little terminology we can use to describe the state of
a poker hand. The particular combination of five cards that
make up a hand we call the "microstate" of that hand. If we group all
the possible hands into classes, such as "full house" and
"three of a kind" , we call these classes "macrostates."
For example,
Entropy and a Box of Air
As an example of entropy in action, consider the air in the building
in which you're sitting. You assume the air is evenly
distributed throughout it. If you go to the kitchen for a beverage,
you don't stop to wonder "Gee, will there be air in the kitchen
when I get there?" Why don't we have to worry about this?
The answer (as you've already guessed if you're on the ball) is entropy.
To see why, let's consider a box of air, with only 10 molecules in it.
I've divided the box in half with a dashed line. The molecules in this
box will be zipping around very fast, bouncing off the walls,
off each other, and so forth. As a result, they'll be distributed randomly
throughout the box. Right now, the number of molecules in the left half
of the box is about the same as in the right half. But it doesn't have
to be that way.
Here, 9 of the 10 molecules are on the left side of the box. What are
the odds of that happening? About one in 102. (Where
did that number come from?) On the other hand, the odds of half the
molecules being on each side is about one in 4. We can
express this in terms of microstates and macrostates. The macrostate
with 9 molecules in the left half has 10 microstates (each
of which has a different molecule on the right). The macrostate with
5 molecules on each side has 252 microstates. In other
words, the macrostate with 5 molecules on each side has a higher entropy
than the one with 9 molecules on the left. And since
all microstates are equally likely (each molecule has the same chance
of being on the left or the right) a macrostate with higher
entropy (more microstates) is more likely than one with lower entropy.
In the case of the two macrostates shown above, the
one with the even distribution is about 25 times as likely (252/10)
as the one with the uneven distribution.
As we consider boxes with more particles, the preference for an even
distribution gets much much stronger as the number of
microstates in a macrostate explodes. For example, suppose we have
a box with 1000 molecules. The number of microstates
with 500 molecules on each side is about 10^299 (that is, 1 followed
by 299 zeroes) while the number of microstates with 900
molecules on one side of the box is only about 10^140. So the former
macrostate has a much larger entropy than the latter, and
is about 10^159 times as likely. And if that seems like an astronomical
number, keep in mind that when it comes to the number
of molecules in a gas, 1000 doesn't even begin to describe it. The
number of air molecules in a shoebox is around 10^22. So
when we have a house full of air, we can expect the molecules to distribute
themselves pretty evenly throughout. Such a state
has a much greater entropy, and so is much more likely to occur. So
go ahead wander into the kitchen to get yourself a cold
one. There'll be plenty of air when you get there. Probably.
Entropy (Remember entropy? This is a web page about entropy.) is defined
in terms of these macrostates and microstates.
Specifically, the entropy of a macrostate is equal to the natural logarithm
of the number of microstates in that macrostate, times
a number called Boltzmann's constant. (Ludwig Boltzmann was the first
person to define entropy in this manner.) So the more
microstates in a macrostate, the greater the entropy. A straight flush
has only forty microstates, so it has a low entropy, while
junk has over a million microstates, so it has a high entropy.
Now here's the key point that leads to the second law of thermodynamics.
Every single possible microstate of our system is
equally likely. This is true whether we're talking about John and Dave's
poker hands, or a glass of water, or the interior of a
neutron star. As a result, the most likely macrostate of a system is
the one with the most microstates. There are over 25,000
times as many junk hands as straight flushes, so a junk hand is over
25,000 times as likely. In other words, the most likely
macrostate of a system is the one with the greatest entropy.
So the reason John is surprised to see his cards while Dave is not surprised
is that John's hand is in a very rare macrostate,
while Dave's hand is in a very common macrostate. We expect to see
high entropy hands like Dave's junk (with many
microstates) and not low entropy hands like John's straight flush.
High entropy hands are more common than low entropy
hands, simply because there are more of them. If we buy a new deck
of cards, we can immediately deal two straight flushes
right off the top of the deck. (The cards in a new deck are all in
order.) The entropy of the cards is low. When we shuffle the
deck, the entropy increases. We start dealing lousy, high entropy,
hands. As you learn more about entropy, you'll see that
increasing entropy is the way of the world. In the meantime, next time
you play poker, take the time to savor those lousy hands.
Each one is just as rare as a great one.
Entropy and Energy
Like entropy, energy is one of the most important ideas in all of science.
You're very familiar with the word " energy" from
everyday life, which may be unfortunate when you come across it in
a scientific context such as this. The reason is that
physicists have drawn many ordinary words into their work and given
them precise, technical meanings; words such as
"energy", "force", "pressure", "field" , and many more. When a physicist
uses these words, he probably doesn't quite mean what you think he does.
In the case of energy, the scientific meaning is not far from your everyday
notion. The images one gets from the word; oil, coal,
nuclear power plants, solar panels, and so on; all provide energy in
the scientific sense. One of the many kinds of energy is
called "kinetic energy". It is the energy a moving object has because
of its motion. The kinetic energy of a moving object
depends on its mass (think "weight") and its speed. The molecules in
a glass of water or an ice cube are all in continual motion,
and so each molecule has a tiny amount of kinetic energy. The temperature
of an object is a reflection of the kinetic energy of
the atoms or molecules that make it up. To sum up, fast molecules =
high kinetic energy = high temperature.
For example, water is warmer than ice, which means that the molecules
in water generally have more kinetic energy (and are
thus moving faster) than the ones in ice. When we place an ice cube
in a glass of water, the slow moving molecules in the ice
are bombarded by the faster molecules in the water. In the resulting
collisions, the molecules in the ice usually end up faster
while the molecules in the water end up slower. Hence the ice gets
warmer and the water gets colder! This transfer of energy
from the warmer substance to the colder substance is called "heat".
Now what does all this have to do with entropy? Well, there's only so
much energy in a glass of water. If we start from scratch
and pass out all the energy in very small doses to all the molecules
in a glass of water, how is the energy likely to distribute
itself? Well, just as the air molecules in a box tend to distribute
themselves evenly throughout the box, the kinetic energy in a
glass of water tends to distribute itself evenly among all the molecules
in the water. The many different ways to distribute the
energy in a glass of water are microstates of the system. Macrostates
in which the energy is evenly distributed have the highest
entropy, and so are the most likely.
When we place the ice cube in the water, the energy is unevenly distributed.
The entropy is low. The system continually
changes its state, and since most of the microstates belong to high
entropy macrostates, the entropy naturally increases. It
doesn't have to do this, since all microstates are equally likely,
but an increase in entropy is overwhelmingly likely, as we saw
with the box of gas.
And so heat flows from the warm substance to the cold substance. You
already knew this, but perhaps not from this
perspective.
All this is summed up in the second law of thermodynamics.
The Second Law of Thermodynamics
Well, here it is at last. The principle that gives the universe direction.
Here's the second law of thermodynamics:
(It's a law, so I figured I'd better make it look like one.)
So what does this mean? Really, it's just a summary of all that I've
said so far. The entropy of a macrostate depends on the
number of microstates that make it up. And since all microstates are
equally, a macrostate with more microstates (higher
entropy) is more likely than one with fewer microstates (lower entropy).
The second law says that if a closed system is in a
state with low entropy, it will naturally find a state with high entropy,
if there's one available. (By "available", we mean an
aerosol air freshener will spread out over the entire room only if
you push down the button. If you don't push down the button,
it can't get out of the can.)
I should point out what we mean by a closed system. A closed system
is a system to and from which no energy (especially
heat) and matter (atoms and molecules, etc.) can flow. It's a system
which has no interaction with anything outside it. This is
sort of an idealization, since nothing in the universe can be completely
shielded from its surroundings. On the other hand, the
universe itself is probably a closed system (cosmologists do debate
this, though) and so with that in mind, you can probably
restate the second law, saying that the entropy of the universe never
decreases, and increases whenever possible.
So now that we've condensed the whole discussion down to one sentence
and carved it into stone, we can look at the
implications more deeply. For example, the second law provides the
answer to the question we posed earlier: Why do things
happen the way they do, and not in reverse? The answer is that as things
happen, the entropy of the universe increases. If they
happened in reverse, the entropy would decrease.
Let's have some examples. Entropy increases when...
... heat flows from a hot object to a cold object. (See "Entropy and energy" )
... a gas flows from a container under high
pressure, to a space at lower pressure. (For example, when you spray
something out of an aerosol can, or when you
let air out of a tire.) (See "Entropy and a box of gas" )
... heat is added to a substance. (Heat is
energy. The more energy in a substance, the more ways you can
distribute that energy. More ways = higher
entropy. Think of it this way. Suppose you're passing out one dollar
bills to a group of 10 people. If you only
have $15, you're rather limited in the number of ways to distribute it.
If
you have $1,000,000 for 10 people, the permutations
(microstates) are nearly endless! Now instead of dollars,
think energy.)
... ice melts. (In ice, the water molecules
are all locked into specific locations, while in water, they are free to
wander throughout the liquid. This freedom
means more microstates for water than ice.)
... water evaporates. (When water on a countertop
evaporates, the water molecules are now free to wander
throughout the entire room. Just as with our
box of gas, the more space the molecules can wander through, the
higher the entropy.)
And these are just a few examples, because the entropy of the universe
increases in every process. (At least it never
decreases.) And when the reverse of any of these processes occurs,
entropy decreases.
But wait! This means that if any water ever froze, its entropy would
decrease. And entropy never decreases. So apparently
freezing water is impossible! What gives?
But wait! I've put water in my freezer, and it comes out ice!
But Wait!
Okay. Here's where we're at. We've said that the entropy of the universe
never decreases. (I've made amazingly convincing
arguments for this by now!) We've said as an example, that the entropy
of water increases when it melts, and decreases when it freezes. But we
know that ice happens!
The key here is the word "universe". Entropy can decrease in some things,
provided it increases in others. This is how ice can
freeze. You put water at, say 40 degrees (Fahrenheit) into your 20
degree freezer. Heat flows from the water (which is
warmer) to the rest of the freezer. When heat is added to something,
its entropy increases. So the entropy of your freezer
increases, while the entropy of the water decreases. As for the total
amount of entropy, it increases. (Entropy increases when
heat flows from a warm substance to a cold substance.)
But it gets more interesting. Let's consider the refrigerator that's
making your ice. The basic function of a refrigerator is to take
heat out of its interior, and transfer it out the back into your kitchen
(or wherever your refrigerator is). Notice which way the
heat flows here. From the refrigerator (cold) to your kitchen (warm).
Entropy increases when heat flows from warm to cold. So here, when heat
flows from cold to warm, it must decrease! Another apparent violation of
the second law of thermodynamics.
Well if this process happened by itself, the second law of thermodynamics
would be violated. But it doesn't. Your refrigerator
doesn't work unless you plug it in. When you plug it in, energy is
delivered from some power plant to your refrigerator. This
energy is used to drive a motor inside your fridge, and pump some fluid
around or something. Whatever it is that refrigerators
do when you plug them in. (Hey, I'm a theorist, okay?) But the byproduct
of this is that more heat is produced and pumped into your kitchen. And
more heat means an increase in entropy.
So if we only consider the heat pumped into your kitchen from the food
storage area of your refrigerator, there's a decrease in
the entropy of the kitchen/refrigerator combination. But the increase
in entropy due to the additional heat produced in the
refrigeration process will make up the difference (with room to spare
for any realistic fridge). The second law of
thermodynamics is safe!
Here's another example. At some point you've probably used a small hand
pump to inflate a bicycle tire or a volleyball. When
you do this, you're moving air molecules from a region where air is
relatively sparse, to a container where the air is much
denser. This represents a decrease in entropy. But just as with the
refrigerator, this process can't take place on its own. It
requires work on your part. Work that requires energy. And a byproduct
of this process is, once again, heat. This heat
represents an increase in entropy which more than makes up for the
decrease in entropy due to the rearrangement of the air
molecules, and the second law holds, as always.
So you see, the second law doesn't say that entropy can never decrease
anywhere. It says that the total entropy of the
universe can never decrease. Entropy can decrease somewhere, provided
it increases somewhere else by at least as much.
That's the law.
Entropy and Disorder
Entropy is sometimes referred to as a measure of the amount of "disorder"
in a system. Lots of disorder = high entropy, while
order = low entropy. It's not too hard to see why this association
came about. If water molecules are confined to a drop of
water, that may seem more orderly than if they are scattered all over
the room in the form of water vapor. And if the water
molecules in that drop are arranged in a hexagonal array (ice!), that's
even more orderly. And indeed, the entropy in ice is
lower than in the same amount of water; and the entropy in water is
lower than in the same amount of water vapor.
If all the air in your room is gathered into the same side of the room,
that can seem more "orderly" than if they're scattered all
over the place. If the energy in your kitchen is concentrated in your
bowl of soup, that may seem more orderly than if the
energy is scattered evenly throughout the room so that your soup is
at room temperature. And again, the more orderly states
are the states with the lower entropy.
Similarly, a room with socks strewn all over the floor has more entropy
than a room in which socks are paired up, neatly
folded, and placed in one side of your sock and underwear drawer.*
So there's some justification to associating entropy with disorder.
(This is why I never clean my apartment. I could decrease the entropy in
my apartment, but according to the second law of thermodynamics I'd only
be increasing it somewhere else.) (That's
a little physics humor there.) (Physicists are funny guys, eh?)
But entropy is more than this, as we've seen. We know that on a cold
day, water freezes. It's clear that the water molecules
become more ordered. Its entropy decreases. This means there must be
an increase in entropy somewhere else. This increase
is in the atmosphere (where the heat from the water goes). But the
air doesn't really become more "disordered".
Thinking of entropy as "disorder" can be misleading, mainly because
while order and disorder are important to entropy, they
don't take into account heat, which we've seen is also very important.
Occasionally one hears creationists argue that evolution
violates the second law of thermodynamics. This is a misunderstanding
that arises from thinking of entropy as a measure of
disorder. The claim is that the incredible precision with which the
human body is arranged could not have evolved from less
complex organisms without violating the second law of thermodynamics.
But the same argument could be used just as
effectively to disprove ice! The problem with the argument is that
it doesn't recognize the increase in entropy in the chemical
processes that take place in assembling a human being, because there's
no visible increase in disorder. If we only think of
entropy as disorder, we miss much of what is going on, and we make
the wrong conclusion.
But you wouldn't do this, because now you know all about entropy! (Well,
not all about entropy. There are still a few details,
such as all the mathematics behind this.) I hope this gives you a new
outlook on life. Or at least gives you some useful cocktail
party banter with which to impress members of the opposite sex. Well,
okay, I hope you learned something.
* Well, applying these principles to collections of large objects, such
as socks, is usually not a good idea. One unspoken
assumption that's made in thermodynamics is that all the microstates
we consider can actually be reached from each other. For
example, in the box of gas all the molecules are in motion, so the
system can easily move from one microstate to another. On
the other hand, if your socks are folded in your drawer, you know that
(barring some kind of intervention) they're going to stay
folded in your drawer. They simply lack the kinetic energy (motion)
to make a transition to other microstates. I threw the socks
in this discussion more for amusement than for education.