lectures princeton

8/8/2019 Lectures Princeton

1/175

Physics 301 10-Sep-2004 1-1

Introduction

In this course we will cover selected topics in thermodynamics and statistical mechan-ics. Since we only have twelve weeks, the selection is necessarily limited. You will probably

need to take a graduate course in thermal physics or do studying on your own in order togain a thorough knowledge of the subject.

Classical (or maybe conventional is better) thermodynamics is an approach to ther-mal physics from the large. Statistical mechanics approaches the subject from thesmall. In thermodynamics, one is concerned with things like the pressure, temperature,volume, composition, etc., of various systems. These are macroscopic quantities and inmany cases can be directly observed or felt by our senses. Relations between these quan-tities can be derived without knowing about the microscopic properties of the system.

Statistical mechanics takes explicit account of the fact that all systems are made

of large numbers of atoms or molecules (or other particles). The macroscopic properties(pressure, volume, etc.) of the system are found as averages over the microscopic properties(positions, momenta, etc.) of the particles in the system.

In this course we will tend to focus more on the statistical mechanics rather thanthe thermodynamics approach. I believe this carries over better to modern subjects likecondensed matter physics. In any case, it surely reflects my personal preference!

Some History (mostly taken from Reif)

As it turns out, thermodynamics developed some time before statistical mechanics.The fact that heat is a form of energy was becoming apparent in the late 1700s and early1800s with Joule pretty much establishing the equivalence in the 1840s. The second lawof thermodynamics was recognized by Carnot in the 1820s. Thermodynamics continuedto be developed in the second half of the 19 th century by, among others, Clausius, Kelvinand Gibbs.

Statistical mechanics was developed in the late 19th and early 20th centuries by Clau-sius, Maxwell, Boltzmann, and Gibbs.

I find all of this rather amazing because at the time of the initial development ofthermodynamics, the principle of energy conservation hadnt been firmly established. Sta-tistical mechanics was developed when the existence of atoms and molecules was still beingdebated. The fact that macroscopic properties of systems can be understood in terms ofthe microscopic properties of atoms and molecules helped convince folks of the reality ofatoms and molecules.

Copyright c 2004, Princeton University Physics Department, Edward J. Groth


2/175

Physics 301 10-Sep-2004 1-2

Still more amazing is the fact that the foundations of statistical mechanics were de-veloped before quantum mechanics. Incorporating quantum mechanics did make somechanges, especially in the counting of states, but the basic approach and ideas of statisti-cal mechanics remained valid. I suspect that this is a reflection of both the strength and

weakness of statistical methods. By averaging over many molecules you derive results thatare independent of the detailed properties of individual molecules. The flip side is thatyou cant learn very much about these details with statistical methods.

Some Thermodynamic Concepts

From mechanics, were familiar with concepts such as volume, energy, pressure (forceper unit area), mass, etc. Two new quantities that appear in thermodynamics are tem-perature (T) and entropy (S).

We will find that temperature is related to the amount of energy in a system. Highertemperature means greater internal energy (usually). When two systems are placed incontact, energy in the form of heat flows from the higher temperature system to the lowertemperature system. When the energy stops flowing the systems are in thermal equilibriumwith each other and we say they are at the same temperature. It turns out if two systemsare in thermal equilibrium with a third system, they are also in thermal equilibrium witheach other. (This is sometimes called the zeroth law of thermodynamics.) So the conceptof temperature is well defined. Its even more well defined than that as we will see later inthe course.

Two systems can exchange energy by macroscopic processes, such as compression orexpansion, or by microscopic processes. It is the microscopic process that is called heattransfer. Consider a collision among billiard balls. We think of this as a macroscopicprocess and we can determine the energy transfer involved by making measurements ofa few macroscopic parameters such as the masses and velocity components. If we scaledown by roughly 24 orders of magnitude, we consider a collision between molecules, amicroscopic process. A very large number of collisions occur in any macroscopic timeinterval. A typical molecule in the atmosphere undergoes 1010 collisions per second. Allthese collisions result in the exchange of energy and it is the net macroscopic transfer ofenergy resulting from all the microscopic energy transfers that we call heat.

Recall that the first law of thermodynamics isdU = dQ + dW ,

where dU is the change of (internal) energy of a system, dQ is energy added to the systemvia a heat transfer, and dW is energy added by doing work on the system.

Aside: you will often see the heat and work written as dQ and dW. This is a reminderthat these quantities are not perfect differentials, just small changes. A system (in equi-



3/175

Physics 301 10-Sep-2004 1-3

librium) has a well defined internal energy U(P , V , . . .) which can be differentiated withrespect to P, V, . . ., but there is no such thing as the heat or work content of a system.The heat and work refer to energy transfers during a change to the system.

So the first law really boils down to a statement of energy conservation. You canchange the energy of a system by adding energy microscopically (dQ) or macroscopically(dW).

While were at it, the second law of thermodynamics can be stated in many ways,but one way (without worrying too much about rigor) is: its impossible to turn heatcompletely into work with no other change. So for example, if you build a heat engine(like a power plant) you cant turn all the heat you get (from burning coal) completelyinto electrical energy. You must dump some waste heat. From this law, one can derive theexistence of entropy and the fact that it must always increase. (Or you can define entropy,and state the second law in terms of the increase in entropy).

Entropy

Earlier, we mentioned that temperature is related to internal energy. So, a picturewe might carry around is that as the temperature goes up, the velocities of the randommotions of the molecules increase, they tumble faster, they vibrate with greater amplitude,etc. What kind of picture can we carry around for entropy? Well thats harder, but as thecourse goes along we should develop such a picture.

To start, we might recall that the change in entropy of a system is the heat added tothe system divided by the temperature of the system (all this is for a reversible process,etc.):

dS = dQ/T .

If a dQ > 0 is added to one system, dQ must be added to a second system. To ensurethat entropy increases, T1 < T2; the first system is cooler than the second system. Themolecules in the first system speed up and the molecules in the second system slow down.After the heat is transfered (in a direction which makes entropy increase) the distributionof molecular speeds in the two systems is more nearly the same. The probability that afast molecule is from system 1 has increased while the probability that a fast molecule

is from system 2 has decreased. Similarly, the probability that a slow molecule is fromsystem 2 has increased and the probability a slow molecule is from system 1 has decreased.In other words, as a result of the increase of entropy, the odds have become more even. Soincreasing entropy corresponds to a leveling of the probabilities.

Higher entropy means more uniform probability for the possible states of the systemconsistent with whatever constraints might exist (such as a fixed total energy of the sys-tem). So entropy is related to the number of accessible states of the system and we will



4/175

Physics 301 10-Sep-2004 1-4

find that maximizing the entropy is equivalent to assuming that each accessible state isequally likely.

The first law of thermodynamics can be written as

dU = dQ + dW = T dSpdV or dS = dU/T +pdV/T ,

where weve assumed that the number of particles in the system is constant and the workdone on the system results from pressure acting while the volume changes. Suppose thesystem is an ideal gas. Then the energy depends only on temperature

dU = nCV dT ,

where n is the number of moles and CV is the molar specific heat at constant volume whichwe take to be constant. The equation of state is

pV = nRT or p/T = nR/V ,

where R is the gas constant. We plug these into the first law and obtain

dS = nCVdT

T+ nR

dV

V,

which can be integrated to give

Sf Si = nCV logTfTi

+ nR logVfVi

.

So, we have an expression for the entropy difference between any two states of an ideal

gas. But how can we relate this to whats going on at the microscopic level? (Note, unlessotherwise stated, by log, I mean a natural logarithm, loge.)

First, lets make a distinction between the macroscopic state and the microscopicstate. The macroscopic state is completely specified (at equilibrium!) by a small number ofparameters such as p, V, n, etc. Classically, the microscopic state requires the specificationof the position and velocity of each particle

r1, v1, r2, v2, . . . , rN,vN ,

where N is the number of particles. N is usually a huge number, comparable to Avogadros

number, the number of particles in a mole, N0 = 6.02 1023

. Since there is such a largeratio of microscopic to macroscopic parameters, it must be that many microscopic statesmay produce a given macroscopic state.

How many microscopic states are there? Why do we want to know? The idea is thatthe macroscopic state which is generated by the most microscopic states is the most likely.Suppose we say that

S log g ,



5/175

Physics 301 10-Sep-2004 1-5

where g is the number of microstates corresponding to the macrostate.

This definition has the desirable property that if we have two non-interacting systemswith states g1 and g2, and we bring them together, the entropy is additive.

S = S1 + S2 .

Since the systems are non-interacting, bringing the systems together does not change thestates available to either system, and any microstate of system 1 may be combined withany microstate of system 2 to yield a microstate of the combined system. This means thatthere are a total of g1 g2 states altogether. By defining the entropy with a logarithm, weensure that its additive (at least in this case!).

So lets count states. At first sight, you might think there are an infinite number ofstates because r and v are continuous variables. Well, perhaps if you change them only

slightly, you dont really get a new state.

Example: Ideal Gas Entropy

Consider one mole of ideal gas at STP. Its volume is V = 22.4 L = 2 104 cm3 and itcontains N0 = 6 10

23 molecules. How big is a molecule? Answer: about 1 A = 108 cm.A molecular volume is Vm 1024 cm3. Imagine dividing our total volume V into cells thesize of a molecule. There are M = V /Vm = 2 1028 cells. Lets specify the micro-positionstate by stating which cells have molecules in them. That is, we are going to specify the

positions of the molecules to a molecular diameter. How many states are there? Pick a cellfor the first molecule. This can be done in M ways. Pick a cell for the second molecule.This can be done in M 1 M ways. For the third molecule, there are M 2 Mways. Continue to the Nth molecule for which there are MN M ways to pick a cell.Altogether there are about

g MN

10281024

,

ways to distribute the molecules in the cells. The fact that we get MN rather than abinomial coefficient depends on the fact that M 1028 N 1024. Also, we shouldprobably divide by N! to account for permutations of the molecules in the cells (since wecant distinguish one molecule from another), but leaving this out wont hurt anything at

this point.

As an example, consider a two dimensional gas containing N = 10 molecules andM = 100 cells. The figure shows a couple of the possible position microstates of this gas.



6/175

Physics 301 10-Sep-2004 1-6

There areM!/N!(MN)! = 1.7 1013

distinct states. Our approximation gives 1020 states; the difference is mostly due to ignoringthe 10! in the denominator.

Knowing the number of states, we have

S Nlog M ,

= NlogV

Vm,

= Nlog V volume term in entropy

constant for given amount of gas Nlog Vm .

The Nlog Vm term is a constant for a given amount of gas and disappears in any calculationof the change in entropy, Sf Si. Similarly, the N! correction would also disappear. Soa lot of the (really awful?) approximations we made just dont matter because things likethe size of a molecule drop out as long as we only consider entropy differences.

The Nlog V term is the volume term in the ideal gas entropy. By considering themicrostates in velocity, we would obtain the temperature term (and we will later in theterm!).



7/175

Physics 301 10-Sep-2004 1-7

What Those Large Numbers Mean

The key aspect of all this is the large number of states! Suppose we have a gas inequilibrium in a container of volume 2V. Why doesnt the gas, by chance, wind up in

one-half the container with volume V? How many states are there in each case?

g1 =

V

Vm

N, g2 =

2V

Vm

N.

And,g2g1

= 2N ,

= 2Avogadros Number ,

= 261023

,

= 1021023

,= 1 000 000

21023 zeros

.

Such a state might be legal, but its extremely!!! unlikely. The fact that a system inequilibrium has the maximum possible entropy is nothing more than the fact that thenormal equilibrium state has so many more ways to occur than an obviously weird state,that the weird state just never occurs.

Quantum Mechanics and Counting States

You might be thinking thats it pretty flaky to assert that we need only specify amolecular position to a molecular diameter. Weve shown that as long as its small, theresolution has no effect on our calculation of changes in the entropy, so this is OK forclassical mechanics.

If we consider quantum mechanics, then we find that systems are in definite states.There are many ways to see this. An example is to consider a particle in a box and fit thewave functions in.

Another way is to consider the uncertainty principle,

pxx h/2 .

If the state of the system is specified by a point in the x px diagram (phase space), thenone cant tell the difference between states which are as close or closer than the above. Sowe can divide up this phase space into cells of h/2 and we can specify a state by sayingwhich cells are occupied and which are not.



8/175

Physics 301 10-Sep-2004 1-8

As a numerical example, consider air (N2) at room temperature. mN2 = 28mp =28 1.7 1024 g = 4.8 1023 g. A typical kinetic energy is mv2/2 = 3kT /2 withT = 300 K and k = 1.38 1016 erg/K, then E 6 1014 erg, v 5.1 104 cm/s,

p 2.4 1018 g cm/s. The molecular size is about r 1 A = 108 cm, so

p r = 2.4 1026 g cm2/s > h = 1 1027 erg s .

Thus, at room temperature, one can specify the momentum of a molecule to a rea-sonable fraction of a typical momentum and the position to about the molecular size andstill be consistent with quantum mechanics and the uncertainty principle. That is, roomtemperature air is classical, but not wildly separated from the quantum domain. If weconsider lower temperatures or higher densities, electrons in metals, etc. quantum effectswill be more important.

The ideal gas at STP is a low occupancy system. That is, the probability that anyparticular state is occupied is extremely small. This means that the most likely numberof occupants of a particular state is zero, one occurs very rarely, and we just dont needto worry about two at all. This is the classical limit and corresponds to the Boltzmanndistribution.

If we have higher occupancy systems (denser and/or colder), then states occupiedby two or more particles can become likely. At this point quantum mechanics enters.There are two kinds of particles: integer spin particles called bosons (such as photons orother particles that we associate with waves) and half-integer spin particles called fermions(protons, electrons, particles that we associate with matter). An arbitrary number ofbosons can be placed in a single quantum state. This leads to Bose-Einstein statistics andthe Bose distribution. At most one fermion can be placed in a quantum state. This leadsto Fermi-Dirac statistics and the Fermi distribution.

A lot of what we do this term will be learning about and applying the Boltzmann,Bose, and Fermi distributions!



9/175

Physics 301 13-Sep-2004 2-1

Reading

This week, you should read the first two chapters of K&K.

Entropy and the Number of States

As we discussed last time, in the statistical view, entropy is related to the numberof microstates of a system. In particular, the entropy is the log of the number ofstates that are accessible to the system when it has specified macroscopic parameters(its macrostate).

The fact that entropy always increases is just a reflection of the fact that a systemadjusts its macroscopic parameters, within the allowed constraints, so as to maximize the

number of accessible states and hence the entropy.

So, a large part of statistical mechanics has to do with counting states and anotherlarge part has to do with deriving interesting results from these simple ideas.

Why is the Number of States Maximized?

Good question. We are going to take this is an axiom or postulate. We will notattempt to prove it. However, we can give some plausibility arguments.

First, remember that we are typically dealing with something like Avogadros numberof particles, N0 = 6.02 10

23. As we discussed last time, this makes the probabilitydistributions very sharp. Or put another way, improbable events are very improbable.

The other thing that happens with a large number of particles has to do with therandomness of the interactions. Molecules in a gas are in continual motion and collide witheach other (we will see later in the term, how often). During these collisions, moleculesexchange energy, momentum, angular momentum, etc. The situation in a liquid is similar,one of the differences between a liquid and gas has to do with the distance a moleculetravels between collisions: in a gas, a molecule typically travels many molecular diameters;

in a liquid, the distance between collisions is of the order of a molecular diameter. In asolid, molecules tend to be confined to specific locations, but they oscillate around theselocations and exchange energy, momentum, etc. with their neighbors.

OK, molecules are undergoing collisions and interactions all the time. As a result, thedistribution of molecular positions and speeds is randomized. If you pick a molecule andask things like where is it located, how fast is it going, etc., the answers can only be givenin terms of probabilities and these answers will be the same no matter which molecule you



10/175

Physics 301 13-Sep-2004 2-2

pick. (Provided you pick the same kind of molecule - youll probably get different answersfor a N2 molecule and an Ar atom, but youll get the same answers for two N 2 molecules.)

Sticky point: suppose we assume that the world is described by classical mechanics.

Also suppose we know the interactions between molecules in some isolated system. Supposewe also know all N0 positions ri and momenta pi (and whatever else we might need toknow to specify the system, perhaps the angular momenta of the molecules, etc.). Then inprinciple, the equations of motion can be solved and the solution tells us the exact stateof the system for all future times. That is, there is nothing random about it! How do wereconcile this with the probabilistic view espoused in the preceding paragraphs?

So far as I know, there are reasonable practical answers to this question, but there areno good philosophical answers. The practical answers have to do with the fact that onecant really write down and solve the equations of motion for N0 particles. But we can inprinciple! A somewhat better answer is that we can only know the initial conditions with

some precision, not infinite precision. As we evolve the equations of motion forward, theinitial uncertainties grow and eventually dominate the evolution. This is one of the basicconcepts of chaos which has received a lot of attention in recent years: small changes inthe initial conditions can lead to large changes in the final result. (Have you ever wishedyou could get a 10 day or 30 day weather forecast? Why do they stop with the 5 dayforecast?)

Of course, the fact that we cannot measure infinitely precisely the initial conditions norsolve such a large number of equations does not mean (still assuming classical mechanics)that it couldnt be done in principle. (This is the philosophical side coming again!) Soperhaps there is still nothing random going on. At this point one might notice that itsimpossible to make a totally isolated system, so one expects (small) random perturbationsfrom outside the system. These will disturb the evolution of the system and have essentiallythe same effect as uncertainties in the initial conditions. But, perhaps one just needs toinclude a larger system!

If we recognize that quantum mechanics is required, then we notice that quantummechanics is an inherently probabilistic theory. Also, Im sure youve seen or will see inyour QM course that in general, uncertainties tend to grow with time (the spreading outof a wave packet is a typical example). On the other hand, the system must be describedby a wave function (depending on N0 variables), whose evolution is determined by

Schroedingers equation . . ..

As you can see this kind of discussion can go on forever.

So, as said before, we are going to postulate that a system is equally likely to be inany state that is consistent with the constraints (macroscopic parameters) applied to thesystem.



11/175

Physics 301 13-Sep-2004 2-3

As it happens, there is a recent Physics Todayarticle on exactly this subject: trying togo from the reversibility of classical mechanics to the irreversibility of statistical mechanics.Its by G. M. Zaslavsky and is called, Chaotic Dynamics and the Origin of StatisticalLaws, 1999, vol. 52, no. 8, pt. 1, p. 39. I think you can read this article and get a feel for

the problem even if some of it goes over your head (as some of it goes over my head).

AsideEntropy and Information

In recent times, there has been considerable interest in the information content ofdata streams and what manipulating (computing with) those data streams does to theinformation content. It is found that concepts in information theory are very similarto concepts in thermodynamics. One way out of the in principle problems associatedwith classical entropy is to consider two sources of entropy: a physical entropy and an

information or algorithmic entropy. This goes something like the following: if we hadsome gas and we had the knowledge of each molecules position and momentum, then thephysical entropy would be zero (theres nothing random about the positions and momenta),but the algorithmic entropy of our list of positions and momenta would be large (andequal to the physical entropy of a similar gas whose positions and momenta we hadntdetermined). What is algorithmic entropy? Essentially, the logarithm of the number ofsteps in the algorithm required to reproduce the list.

In 1998, Toby Marriage wrote a JP on this topic. You can find it at

http://physics.princeton.edu/www/jh/juniors fall98.html .

One of our criteria for junior papers is that other juniors should be able to understand thepaper; so I think you might get something out of this paper as well!



12/175

Physics 301 13-Sep-2004 2-4

Macroscopic Parameters

We will be most concerned with systems in equilibrium. Such a system can usuallybe described by a small number of macroscopic parameters. For example, consider a gas.

If the density of the gas is low enough, it can be described quite well by the ideal gas lawwhen its in equilibrium:pV = NkT = nRT ,

where p is the pressure, V is the volume, N is the number of molecules, n is the number ofmoles, k = 1.381016 erg K1 is Boltzmanns constant or the gas constant per molecule,R = 8.31 107 erg mole1 K1 = N0k is the gas constant per mole, and T is the absolutetemperature.

Notice that some parameters depend on how much gas we have and some dont. Forexample, if we replicate our original system, so we have twice as much, then V, N, U

(internal energy), and S (entropy) all double; p, and T stay the same. We are ignoring thecontribution of any surface interactions which we expect to be very small. Can you thinkwhy? Parameters which depend on the size of the system are called extensiveparameters.Parameters that are independent of the size of the system are called intensive parameters.

Note that the gas law is not the whole story. If more than one kind of molecule is inthe gas, we need to specify the numbers of each kind: N1, N2, . . .. Also, the gas law doesnot say anything about the energy of the gas or its entropy. The gas law is an equation ofstate, but it needs to be supplemented by other relations in order that we know everythingthere is to know about the gas (macroscopically, that is!). For systems more complicatedthan a gas, other parameters may be needed.

Another thing to notice is that not all parameters may be specified independently. Forexample, having specified N, T, and V, the pressure is determined. Thus there is a certainminimum number of parameters which specify the system. Any property of the systemmust be a function of these parameters. Furthermore, we can often change variables anduse a different set of parameters. For a single component ideal gas, we might have

p = p(N , V , T ), U = U(N , V , T ), S = S(N , V , T ) .

We might imagine solving for T in terms of N, V, and U, and we can write

p = p(N , V , U ), T = T(N , V , U ), S = S(N , V , U ) .

Anything that depends only on the equilibrium state of the system can be expressed asa function of the parameters chosen. Which parameters are to be used depends on theparticular situation under discussion. For example, if the volume of a system is under ourcontrol, we would likely use that as one of the independent parameters. On the other hand,many processes occur at constant pressure (with the volume adjusting to what it needs to



13/175

Physics 301 13-Sep-2004 2-5

be). In this case, using p rather than V as the independent parameter will probably bemore convenient.

The Temperature

As we remarked, the entropy is the logarithm of the number of microstates accessibleto a system. The number of states must be a function of the same macroscopic parametersthat determine the macrostate of the system. Lets consider a system described by itsinternal energy U, its volume V, and the number of each kind of constituent particle Na,Nb, . . .. For the moment, we ignore the possibility of reactions which can change particlesof one kind into another kind. This means that our expressions will have the same form forNa, Nb, etc., so well just assume a single kind of particle for the time being and assumewe have N of them. Then the number of microstates is

g = g(U,V,N) .

If we have two systems, that we prevent from interacting, then the number of mi-crostates of the combined system is

g(U , V , N, U 1, V1, N1) = g1(U1, V1, N1)g2(U2, V2, N2) ,

withU = U1 + U2 , V = V1 + V2 , N = N1 + N2 .

This is straightforward. Any microstate in system 1 can be paired with any microstate insystem 2, so the total number of microstates is just the product of the number for eachsystem. Also, we have specified the macrostate in terms of extensive parameters, so wecan write the parameters of the combined system as the sum of those for the individualsystems as well as one set of the individual system parameters.

Following K&K, the dimensionless entropy is just

(U , V , N, U 1, V1, N1) = log g(U , V , N, U 1, V1, N1) = log g1g2

= log g1 + log g2 = 1(U1, V1, N1) + 2(U2, V2, N2) .

So far, we havent really done anything. Weve just written down some definitionstwice. We have prevented the two systems from interacting, so nothing exciting can hap-pen. Now lets suppose we allow the systems to exchange energy. In other words, we allowU1 and U2 to vary, but any change in U1 has a compensating change in U2 so that U isconstant. In addition, we prevent changes in volume and numbers of particles, so that V1,V2, N1, and N2 remain constant.



14/175

Physics 301 13-Sep-2004 2-6

Were placing the systems in thermal contact, but preventing changes in volume orparticle number. We know what will happen: energy flows from the hotter system to thecooler system until they come to thermal equilibrium at the same temperature. We knowthis from our intuitive understanding of the second law: heat flows from a hot object to a

cold object.

But, what about our postulate that a system maximizes the number of accessiblemicrostates? In this case, it means that the system adjusts U1 and U2 to maximize theentropy. So,

U1

V,N

= 0 since is maximized

=

1U1

V,N

+

2U1

V,N

=

1U1

V,N

+

2U2

V,N

U2U1

=

1U1

V,N

2U2

V,N

since U1 = U2.

This means 1U1

V,N

=

2U2

V,N

,

after equilibrium has been established.

So at equilibrium, the rate of change of entropy with respect to energy is the same forthe two systems. If we started out with the two systems and we allowed them to exchangeenergy and nothing happened, then we know that /U was already the same. If system1 and system 2 are in equilibrium with respect to energy exchange and we allow system1 to exchange energy with a third system and nothing happens, then 3/U3 must alsohave the same value and nothing will happen if systems 2 and 3 are allowed to exchangeenergy. Thus, /U has properties very similar to those we ascribe to temperature. Infact, we can define the temperature as:

1

=

U

V,N

.

This makes an intensive quantity (its the ratio of two extensive quantities), and it makesthe energy flow in the correct direction.

This can be seen as follows: if the two systems are not in equilibrium when we allowenergy to flow, then the entropy of the combined systems must increase: The increase in



15/175

Physics 301 13-Sep-2004 2-7

entropy after a very small amount of energy has been transferred is

> 0

= 1 + 2

= 11

U1 + 12

U2

=

1

1

1

2

U1 .

So if 1 < 2, U1 > 0, which means energy flows from the high system to the low system.

Finally, if you remember your elementary thermodynamics, recall that dU = T dSpdV which agrees with this definition of temperature.

Units: from our definitions is dimensionless and has the dimensions of energy. Yourecall that temperature T has the dimensions of Kelvins and entropy S has the dimensionsof ergs per Kelvin. As it turns out,

S = k ,

= kT .

Boltzmanns constant is really just a scale factor which converts conventional units to thefundamental units weve defined above.

Its often said that we measure temperature in Kelvins or degrees Celsius or Fahren-heit because the measurement of temperature was established before the development ofthermodynamics which in turn took place before the connection to energy was fully ap-preciated. What would you think if you tuned in to the weather channel and found outthat the high tomorrow was expected to be 4.14 1014 erg or 0.0259 eV??? (If I did thearithmetic correctly, this is 80F.)

Actually, to measure a temperature, we need a thermometer. Thermometers make useof physical properties which vary with temperature. (Thats obvious I suppose!) The trickis to calibrate the thermometers so you get an accurate measure of the thermodynamictemperature, /k. A recent Physics Today article discusses some of the difficulties indefining a good practical scale for /k < 1 Kelvin. (Soulen, Jr., R. J., and Fogle, W. E.,

1997, Physics Today, vol. 50, no. 8, p. 36, Temperature Scales Below 1 Kelvin.)

One other thing to point out here: Youve no doubt noticed the V, N subscripts. Whenyou read a thermodynamics text, youll often find the statement that this a reminder thatV and N are being held fixed in taking the indicated partial derivative. Well, this is true,but since we have a partial derivative, which already means hold everything else fixed, whydo we need an extra reminder? Answer: since there are so many choices of independentvariables, these subscripts are really a reminder of the set of independent variables in use.



16/175

Physics 301 13-Sep-2004 2-8

Note that we can add energy to a gas keeping the volume and number of particles fixed. Inthis case the pressure and temperature rise. Alternatively, we can keep the pressure andnumber of particles fixed. In this case the volume and temperature increase. Furthermore,

U

V,N

=

U

p,N

.

When its obvious from the context which set of independent variables are in use, I willprobably be lazy and omit the subscripts.



17/175

Physics 301 15-Sep-2004 3-1

Pressure

Last lecture, we considered two systems with entropy as a function of internal energy,volume and number of particles,

(U , V , N, U 1, V1, N1) = 1(U1, V1, N1) + 2(U2, V2, N2) .

We allowed them to exchange internal energy (that is, they were placed in thermal con-tact), and by requiring that the entropy be a maximum, we were able to show that thetemperature is

1

=

U

V,N

.

Suppose we continue to consider our two systems, and ask what happens if we allow

them to exchange volume as well as energy? (Were placing them in mechanical as well asthermal contact.) Again, the total entropy must be a maximum with respect to exchangesof energy and exchanges of volume. Working through similar mathematics, we find anexpression for the change in total entropy and insist that it be zero (so the entropy ismaximum) at equilibrium,

0 =

=1U1

U1 +2U2

U2 +1V1

V1 +2V2

V2

= 1

U1

2

U2U1 +

1

V1

2

V2 V1 ,

from which we infer that at equilibrium,

1U1

=2U2

,

which we already knew, and1V1

=2V2

.

This last equation is new, and it must have something to do with the pressure. Why?

Because, once the temperatures are the same, two systems exchange volume only if onesystem can push harder and expand while the other contracts. We define the pressure:

p =

V

U,N

.

We will see later that this definition agrees with the conventional definition of pressure asforce per unit area.



18/175

Physics 301 15-Sep-2004 3-2

Chemical Potential

Well, theres one variable left, guess what were going to do now! Suppose we allowthe two systems to exchange particles as well as energy and volume. Again, we want to

maximize the entropy with respect to changes in all the independent variables and thisleads to,

0 =

=1U1

U1 +2U2

U2 +1V1

V1 +2V2

V2 +1N1

N1 +2N2

N2

=

1U1

2U2

U1 +

1V1

2V2

V1 +

1N1

2N2

N1 .

So, when the systems can exchange particles as well as energy and volume,

1N1

=2N2

.

The fact that these derivatives must be equal in equilibrium allows us to define yet anotherquantity, , the chemical potential

=

N

U,V

.

If two systems are allowed to exchange particles and the chemical potentials are unequal,

there will be a net flow of particles until the chemical potentials are equal. Like temperatureand pressure, chemical potential is an intensive quantity. Unlike temperature and pressure,you probably have not come across chemical potential in your elementary thermodynamics.You can think of it very much like a potential energy per particle. Systems with highchemical potential want to send particles to a system with low potential energy per particle.Note that we can write a change in the entropy of a system, specified in terms of U, V,and N as

d =1

dU +

p

dV

dN ,

or rearranging,dU = d pdV + dN .

Which is the conservation of energy (first law of thermodynamics) written for a systemwhich can absorb energy in the form of heat, which can do mechanical pV work, and whichcan change its energy by changing the number of particles.



19/175

Physics 301 15-Sep-2004 3-3

First Derivatives versus Second Derivatives

You will notice that the quantitites defined by first derivatives are not material specific.For example, whether its nitrogen gas or a block of steel, the rate of change of energy

with entropy (at constant volume and particle number) is the temperature.

Well eventually define Helmholtz and Gibbs free energies and enthalpy (differentindependent variables) and it will always be the case that the first derivatives of thesequantities produce other quantities that are not material specific.

To get to material specific quantities, one must go to second derivatives. For example,suppose we have a block of steel and nitrogen gas inside a container that is thermallyinsulating, fixed in volume, and impermeable. Then at equilibrium,

UsteelVsteel S,N = Unitrogen

Vnitrogen S,N = p .If we make a change to the volume of the container, we might be interested in

p

V=

2U

V2.

This quantity is related to the compressibility of the material. Nitrogen gas is much morecompressible than steel and most of the volume change will be taken up by the gas, notthe steel. In other words, the material specific quantity (second derivative) is different forthe two materials.



20/175

Physics 301 15-Sep-2004 3-4

Probability

Here, we will introduce some basic concepts of probability. To start with, one imaginessome experiment or other process in which several possible outcomes may occur. The

possible outcomes are known, but not definite. For example, tossing a die leads to oneof the 6 numbers 1, 2, 3, 4, 5, 6 turning up, but which number will occur is not knownin advance. Presumably, a set of elementary outcomes can be defined and all possibleoutcomes can be specified by saying which elementary outcomes must occur. For example,the tossing of the die resulting in an even number would be made up of the elementaryevents: the toss is 2 or the toss is 4 or the toss is 6. A set of elementary events is suchthat one and only one event can occur in any repetition of the experiment. For example,the events (1) the toss results in a prime number and (2) the toss gives an even numbercould not both be part of a set of elementary events, because if the number 2 comes up,both events have occurred!

One imagines that a very large number of tosses of the die take place. Furthermore,in each toss, an attempt is made to ensure that there is no memory of the previous toss.(This is another way of saying successive tosses are independent.) Then the probabilityof an event is just the fraction of times it occurs in this large set of experiments, thatis, ne/N, where ne is the number of times event e occurs and N is the total number ofexperiments. In principle, we should take the limit as the number of trials goes to .From this definition it is easy to see that the probabilities of a set of elementary eventsmust satisfy

pi 0 ,

and i

pi = 1 ,

where pi is the probability of event i and i is an index that ranges over the possibleelementary events.

The above definition is intuitive, but gives the sense of a process occurring in time.That is, we throw the same die over and over again and keep track of what happens.Instead, we can imagine a very large number of dice. Each die has been prepared, as nearlyas possible, to be identical. Each die is shaken (randomized) and tossed independently.Again, the probability of an event is the fraction of the total number of trials in which

the event occurs. This collection of identically prepared systems and identically performedtrials is called an ensembleand averages that we calculate with this construction are calledensemble averages.

You are probably thinking that for the die, the probabilities of each of the six elemen-tary events 1, 2, 3, 4, 5, 6 must be 1/6. Well, they could be, but its not necessary! Youveheard of loaded dice, right? All thats really necessary is that each pi be non-negativeand that their sum be 1. Probability theory itself makes no statement about the values of



21/175

Physics 301 15-Sep-2004 3-5

the probabilities. The values must come from somewhere else. In general, we just assignprobabilities to the elementary events. Often we will appeal to symmetry or other argu-ments to assign the probabilities. For example, since a die is a symmetric cube, no facecan be distinguished (mechanically) from any other face and we can plausibly argue that

the probabilities should be equal.

Aside: well, the dots have to be painted on, so the die isnt perfectly symmetric.Presumably, differences in the amount and pattern of paint on each face make a negligibledifference in the mechanical properties of the die (such as cm, moment of inertia, etc.) soits a very good approximation to regard the die as symmetric. However, some dice haverather large indentations for each dot. Ive occasionally wondered if this might make adetectable difference in the probabilities.

In our discussion of the entropy, we postulated that a system is equally likely to bein any microscopic state consistent with the constraints. This amounts to assigning the

probabilities and is basically an appeal to symmetry in the same way that assigning equalprobabilities to each face of a die is an appeal to symmetry!

Averages

Assuming there is some numeric value associated with each elementary event, we cancalculate its average value just by adding up all the values and dividing by the total numberof trialsexactly what you think of as an average. So, if event i produces the value yi,then its average value is

y =1

N(y1 + y1 + y1

n1 times

+ y2 + y2 + y2 n2 times

+ )

=1

N(n1y1 + n2y2 + )

=i

yipi .

Quantities like y, whose value varies across an ensemble, are called random variables.

After the average, we will often be most interested in the variance (often called thesquare of the standard deviation.) This is just the average value of the square of thedeviation from the average.

var(y) = 2y =

(y y)2

,

where y is the standard deviation in y, not the entropy! The standard deviation is ameasure of the spread about the average.



22/175

Physics 301 15-Sep-2004 3-6

Probabilities for Continuous Variables.

Rather than giving one of a finite (or infinite) number of discrete outcomes, an ex-periment might result in the measurement of a random variable which is continuously

distributed over some finite (or infinite) range. In this case we deal with a probabilitydensity rather than discrete probabilities. For example, we might make a measurement ofa continuous variable x. Then the probability that the measurement falls in a small rangedx around the value x is

Prob(x < result < x + dx) = p(x) dx ,

where p(x) is the probability density. Just as for discrete probabilities, the probabilitydensity must satisfy

p(x) 0 ,

and allowed range of x

p(x) dx = 1 .

We can simply define p(x) = 0 when x is outside the allowed range, so the normalizationbecomes +

p(x) dx = 1 .

The average of any function of x, y(x) is defined by

y = +

y(x)p(x) dx .



23/175

Physics 301 15-Sep-2004 3-7

The Binomial Distribution

As an example of working with probabilities, we consider the binomial distribution.We have N trials or N copies of similar systems. Each trial or system has two possible

outcomes or states. We can call these heads or tails (if the experiment is tossing a coin),spin up or spin down (for spin 1/2 systems), etc. We suppose that each trial or system isindependent and we suppose the probability of heads in one trial or spin up in one systemis p and the probability of tails or spin down is 1 p = q. (Lets just call these up anddown, Im getting tired of all these words!)

To completely specify the state of the system, we would have to say which of the Nsystems are up and which are down. Since there are 2 states for each of the N systems,the total number of states is 2N. The probability that a particular state occurs depends onthe number of ups and downs in that state. In particular, the probability of a particularstate with n up spins and N n down spins is

Prob(single state with n up spins) = pnqNn .

Usually, we are not interested in a single state with n up spins, but we are interested in allthe states that have n up spins. We need to know how many there are. There is 1 statewith no up spins. There are N different ways we have exactly one of the N spins up andN 1 down. There are N(N 1)/2 ways to have two spins up. In general, there are

Nn

different states with n up spins. These states are distinct, so the probability of getting anystate with n up spins is just the sum of the probabilities of the individual states. So

Prob(any state with n up spins) = N

npnqNn .

Note that

1 = (p + q)N =

Nn=0

N

n

pnqNn ,

and the probabilities are properly normalized.

To illustrate a trick for computing average values, suppose that when there are n upspins, a measurement of the variable y produces n. What are the mean and variance of y?To calculate the mean, we want to perform the sum,

y =Nn=0

n

N

n

pnqNn .

Consider the binomial expansion

(p + q)N =

Nn=0

N

n

pnqNn ,



24/175


25/175

Physics 301 17-Sep-2004 4-1

ExampleA Spin System

In the last lecture, we discussed the binomial distribution. Now, I would like to adda little physical content by considering a spin system. Actually this will be a model for a

paramagnetic material. This system is also a trivial subcase of the Ising model.

Well consider a large number, N, of identical spin 1/2 systems. As you know, ifyou pick an axis, and measure the component of angular momentum of a spin 1/2 systemalong that axis, you can get only two answers: +h/2 and h/2. If theres charge involved,then theres a magnetic moment, m, parallel or antiparallel to the angular momentum. Iftheres a magnetic field, B, then this defines an axis and the energy m B of the spinsystem in the magnetic field can be either mB if the magnetic moment is parallel to thefield or +mB if the magnetic moment is anti-parallel to the field. To save some writing,let E = mB > 0 so the energy of an individual system is E.

In this model, we are considering only the energies of the magnetic dipoles in anexternal magnetic field. We are ignoring all other interactions and sources of energy. Forexample, we are ignoring magnetic interactions between the individual systems, whichmeans we are dealing with a paramagnetic material, not a ferromagnetic material. Also,we are ignoring diamagnetic effectseffects caused by induced magnetic moments whenthe field is established. Generally, if there is a permanent dipole moment m, paramagneticeffects dominate diamagnetic effects.

Of course, there must be some interactions of our magnets with each other or withthe outside world or there would be no way for them to change their energies and come toequilibrium. What were assuming is that these interactions are there, but just so smallthat we dont need to count them when we add up the energy. (Of course the smaller theyare, the longer it will take for equilibrium to be established. . .)

Our goal here is to work out expressions for the energy, entropy, temperature, in termsof the number of parallel and antiparallel magnetic moments.

If there is no magnetic field, then there is nothing to pick out any direction, andwe expect that any given magnetic moment or spin is equally likely to be parallel orantiparallel to any direction we pick. So the probability of parallel should be the sameas the probability of antiparallel should be 1/2: p = 1 p = q = 1/2. If we turn onthe magnetic field, we expect that more magnets will line up parallel to the field thanantiparallel (p > q) so that the entire system has a lower total energy than it would havewith equal numbers of magnets parallel and antiparallel.

If we didnt know anything about thermal effects, wed say that allthe magnets shouldalign with the field in order to get the lowest total energy. But we do know somethingabout thermal effects. What we know is that these magnets are exchanging energy witheach other and the rest of the world, so a magnet that is parallel to the field, having energy



26/175

Physics 301 17-Sep-2004 4-2

E, might receive energy +2E and align antiparallel to the field with energy +E. It willstay antiparallel until it can give up the energy 2E to a different magnet or to the outsideworld. The strengths of the interactions determine how rapidly equilibrium is approached(a subject we will skip for the time being), but the temperature sets an energy scale and

determines how likely it is that chunks of energy of size 2E are available.

So suppose that n of the magnets are parallel to the field and N n are antiparallel.K&K define the spin excess, as the number parallel minus the number antiparallel,2s = n (N n) = 2n N or n = s + N/2. The energy of the entire system is then

U(n) = nE+ (N n)E = (2nN)E = 2sE .

The entropy is the log of the number of ways our system can have this amount of energyand this is just the binomial coefficient.

(n) = logN

n

= logN!

(N/2 + s)! (N/2 s)! .

To put this in the context of our previous discussion of entropy and energy, note thatwe talked about determining the entropy as a function of energy, volume, and number ofparticles. In this case, the volume doesnt enter and were not changing the number ofparticles (or systems) N. At the moment, we are not writing the entropy as an explicitfunction of the energy. Instead, the two equations above are parametric equations for theentropy and energy.

To find the temperature, we need /U. In our formulation, the entropy and energyare functions of a discrete variable, not a continuous variable. No problem! Well just sendone magnet from parallel to anti-parallel. This will make a change in energy, U, and achange in entropy, and we simply take the ratio as the approximation to the partialderivative. So,

U = U(n 1) U(n) = 2E , = (n 1) (n)

= log

N

n 1 log

N

n

= log

N!(n 1)!(N n + 1)! n! (N n)!N!

= log

n

N n + 1= log

n

N n 1 cant matter if N n N0

= logN/2 + s

N/2 s ,



27/175

Physics 301 17-Sep-2004 4-3

where the last line expresses the result in terms of the spin excess. Throwing away the 1is OK, provided we are not at zero temperature where n = N.

The temperature is then

=U

=

2E

log(N/2 + s)/(N/2 s) .

At this point its convenient to solve for s. We have

N/2 + s

N/2 s = e2E/ ,

and with a little algebra2s

N

= tanhE

.

The plot shows this functionfractional spin excess versus E/. To the left, thermal

energy dominates magnetic energy and the net alignment is small. To the right, magneticenergy dominates thermal energy and the alignment is large. Just what we expected!

Suppose the situation is such that E/ is large. Then the magnets are all aligned. Nowturn off the magnetic field, leaving the magnets aligned. What happens? The system is

no longer in equilibrium. It absorbs energy and entropy from its surroundings, cooling thesurroundings. This technique is actually used in low temperature experiments. Its calledadiabatic demagnetization. Demagnetization refers to removing the external magnetic fieldand adiabatic refers to doing it gently enough to leave the magnets aligned.



28/175

Physics 301 17-Sep-2004 4-4

The Boltzmann Factor

An additional comment on probabilities: When the spin excess is 2s, the probabilitiesof parallel or antiparallel alignment are:

p = 12

+ sN

, q = 12 s

N.

The ratio of the probabilities is

q

p=

1 2s/N1 + 2s/N

= e2E/ .

This is a general result. The relative probability that a system is in two states with anenergy difference E is just

Probability of high energy state

Probability of low energy state

= eE/ = eE/kT .

This is called the Boltzmann factor. As weve already mentioned, this says that energies

< kT are easy to come by, while energies > kT are hard to come by! The temperaturesets the scale of the relevant energies.

The Gaussian Distribution

Weve discussed two discrete probability distributions, the binomial distribution and(in the homework) the Poisson distribution. As an example of a continuous distribution,well consider the Gaussian (or normal) distribution. It is a function of one continuousvariable and occurs throughout the sciences.

The reason the Gaussian distribution is so prevalent is that under very general con-ditions, the distribution of a random variable which is the sum of a large number ofindependent, identically distributed random variables, approaches the Gaussian distribu-tion as the number of random variables in the sum goes to infinity. This result is calledthe central limit theorem and is proven in probability courses.

The distribution depends on two parameters, the mean, , (not the chemical poten-tial!) and the standard deviation, (not the entropy!). The probability density is

p(x) =1

22e(x )

2

22 .

You should be able to show that

122

+

e(x )

2

22 dx = 1 ,



29/175

Physics 301 17-Sep-2004 4-5

x = 122

+

x e(x )

2

22 dx = ,

var(x) =1

22 +

(x

)2 e(x )

2

22 dx = 2 .

Appendix A of K&K might be useful if you have trouble with these integrals. One canalways recenter so that x is measured from and rescale so that x is measured in units of. Then the density takes the dimensionless form,

p(x) =12

ex2/2 .

Sometimes you might need to integrate this density over a finite (rather than infinite)range. Two related functions are of interest, the error function

erf(z) =2

z0

et2 dt = 2 12

2z0

ex2/2 dx ,

and the complementary error function

erfc(z) =2

z

et2 dt = 2 12

2z

ex2/2 dx ,

where the first expression (involving t) is the typical definition, and the second (obtainedby changing variables t = x/

2 rewrites the definition in terms of the Gaussian probability

density. Note that erf(0) = 0, erf() = 1, and erf(z) + erfc(z) = 1.The Gaussian density is just the bell curve, peaked in the middle, with small tails.

The error function gives the probability associated with a range in x at the middle ofthe curve, while the complementary error function gives probabilities associated with thetails of the distribution. In general, you have to look these up in tables, or have a fancycalculator that can generate them. As an example, you might hear someone at a researchtalk say, Ive obtained a marginal two-sigma result. What this means is that the signalthat was detected was only 2 larger than no signal. A noise effect this large or larger willhappen with probability

12

2

ex2/2 dx = 12

erfc 22

= 0.023 .

That is, more than 2 percent of the time, noise will give a 2 result just by chance. Thisis why 2 is marginal.

Were straying a bit from thermal physics, so lets get back on track. One of thereasons for bringing up a Gaussian distribution is that many other distributions approach



30/175

Physics 301 17-Sep-2004 4-6

a Gaussian distribution when large numbers are involved. (The central limit theoremmight have something to do with this!) For example, the binomial distribution. Whenthe numbers are large, we can replace the discrete distribution in n with a continuousdistribution. The advantage is that it is often easier to work with a continuous function.

In particular, the probability of a spin excess, s, is

ps =N!

(N/2 + s)! (N/2 s)!pN

2+sq

N

2s .

We need to do something with the factorials. In K&K, Appendix A, Stirlings approxima-tion is derived. For large N,

N!

2N NNeN .

With this, we have

ps

2N

2(N/2 + s) 2(N/2 s)NN

(N/2 + s)(N/2+s)(N/2 s)(N/2s) pN/2+sqN/2s

=

1

2N(1/2 + s/N) (1/2 s/N)pN/2+sqN/2s

(1/2 + s/N)(N/2+s)(1/2 s/N)(N/2s)

=

1

2N(1/2 + s/N) (1/2 s/N)

pq

pq

pN/2+sqN/2s

(1/2 + s/N)(N/2+s)(1/2 s/N)(N/2s)

= 12Npq

p1/2 + s/N

(N/2+s+1/2)

q1/2 s/N

(N/2s+1/2).

Recall that the variance of the binomial distribution is Npq, so things are starting to lookpromising. Also, we are working under the assumption that we are dealing with largenumbers. This means that s cannot be close to N/2. If it were, then we would have asmall number of aligned, or a small number of anti-aligned magnets. So, in the exponentsin the last line, N/2 s is a large number and we can ignore the 1/2. Then

ps =

1

2Npq

p

1/2 + s/N

(N/2+s)q

1/2 s/N(N/2s)

.

This is a sharply peaked function. We expect the peak to be centered at s = s0 = s =nN/2 = N pN/2 = N(p1/2). We want to expand this function about its maximum.Actually, it will be easier to locate the peak and expand the function, if we work with itslogarithm.

logps = A +

N

2+ s

logp log

1

2+

s

N

+

N

2 s

log q log

1

2 s

N

,



31/175

Physics 301 17-Sep-2004 4-7

where

A =1

2log

1

2Npq

.

To locate the maximum of this function, we take the derivative and set it to 0

d logpsds

= logp log

1

2+

s

N

1 log q + log

1

2 s

N

+ 1 .

We note that this expression is 0 when s/N = p 1/2, just as we expected. So this isthe point about which well expand the logarithm. The next term in a Taylor expansionrequires the second derivative

d2 logpsds2

= 1N/2 + s

1N/2 s

=

1

Np 1

N q=

1

N pq,

where, in the last line, we substituted the value of s at the maximum. We can expand thelogarithm as

logps = A 12

1

N pq(s s0)2 +

where s0 = N(p 1/2) is the value of s at the maximum. Finally, we let 2 = Npq,exponentiate the logarithm, and obtain,

p(s) 122

e(s s0)2/22 ,

where the notation has been changed to indicate a continuous variable rather than a dis-crete variable. You might worry about this last step. In particular, we have a discreteprobability that we just converted into a probability density. In fact, p(s) ds is the prob-ability that that the variable is in the range s s + ds. In the discrete case, the spacingbetween values of s is unity, so we require,

p(s)

(s + 1) s = ps ,which leads to p(s) = ps. Had there been a different spacing there would be a differentfactor relating the discrete and continuous expressions.

All this was a lot of work to demonstrate in some detail that for large N (and not toolarge s), the binomial distribution describing our paramagnetic system goes over to theGaussian distribution. Of course, expanding the logarithm to second order guarantees aGaussian!

In practice, you would not go to all this trouble to do the conversion. The wayyou would actually do the conversion is to notice that large numbers are involved, so the



32/175

Physics 301 17-Sep-2004 4-8

distribution must be Gaussian. Then all you need to know are the mean and variancewhich you calculate from the binomial distribution or however you can. Then you justwrite down the Gaussian distribution with the correct mean and variance.

Returning to our paramagnetic system, we found earlier that the mean value of thespin excess is

s0 =N

2tanh

E

.

We can use the Gaussian approximation provided s is not too large compared to N/2 whichmeans E< . In this case, a little algebra shows that the variance is

2 = N pq = N

1

2sech

E

2.

For given E/, the actual s fluctuates about the mean s0 with a spread proportional toN and a fractional spread proportional to 1/N. A typical system has N N0, so thefractional spread is of order 1012 and the actual s is always very close to s0.

While were at it, its also interesting to apply Stirlings approximation to calculatethe entropy of our paramagnetic system. Recalling Stirlings approximation for large N,

N!

2N NNeN .

Taking the logarithm, we have

log N! 1

2 log2 +1

2 log N + Nlog N N .The first two terms can be ignored in comparison with the last two, so

log N! Nlog N N .

Suppose our spin system has s0 0. Then the entropy is

log N!(N/2)!(N/2)!

Nlog N

N

2(N/2) log(N/2) (N/2)= Nlog NNlog(N/2)= Nlog2

= 4.2 1023 (fundamental units)= 5.8 107 erg K1 (conventional units) ,

where the last two lines assume one mole of magnets.



33/175

Physics 301 20-Sep-2004 5-1

Reading

K&K, Chapter 3. Also, for a little culture, there is a handout which is a one pagearticle from Science, 1997, G. Bertsch, vol. 277, p. 1619. It describes melting in clusters

consisting of 139 atoms! So 1/N 10%, quite a bit larger than the one part in a trillionweve been talking about when there is a mole of particles! One might expect things to beless well defined, and sure enough, these clusters seem to melt over an appreciable rangein temperature (rather than a single T). This may be a case where statistical mechanicsis just on, or just past the edge!

The Boltzmann Factor

Lets imagine we have an object which is isolated from the rest of the world, so itsenergy, entropy, and so on are constant. Furthermore, it has come to equilibrium, so allparts of the object are at the same temperature, pressure, and so on. We imagine dividingthis object into two pieces: a small piece, called the system; and a large piece called theheat bath or heat reservoir (or just bath or reservoir). The system is supposed to besufficiently small that its useful to think of it as being in a single (quantum) state. Asystem might be a single atom or molecule, but it could be a larger entity if such an entitycan be described as being in a single state. The remainder of the object, the bath, issupposed to be very large and might consist of a very large number of similar atoms ormolecules. (In principle, we should let N , where N is the number of molecules inthe bath.) The system and the bath interact so that the system is in a particular stateonly with some probability. We are going to calculate this probability. We want to speak

of the system as having energy E. This means that the interaction between the systemand the bath must be weak in order that we can ascribe a definite energy to the system.

Consider two states of the system, 1 and 2, with energies E1 and E2. When the systemis in state 1, the bath has energy U E1, where U is the total (fixed) energy of the bathplus system. The number of states that correspond to this energy in the system is

g(U E1) 1 = exp

(U E1) 1 .

Where the first factor is the number of states in the bath and the second factor is thenumber of states (just 1) of the system. To have the factor 1 here is the reason that we

insisted the system be in a single state. Similarly, the number of states in the case thatthe system has energy E2 is

g(U E2) 1 = exp

(U E2) 1 .

Our fundamental assumption is that each state (of the system plus bath together)that is compatible with the constraints is equally probable. So, the ratio of the probability



34/175

Physics 301 20-Sep-2004 5-2

that the system is in state 2 to the probability that it is in state 1 is

P(state 2)

P(state 1)=

g(U E2)g(U E1) = e

(U E2) (U E1) .

Now, E is an energy on the scale of an energy of a single molecule while U is an energytypical of a mole. So we can expand in a tailor series around E = 0,

(U E) = (U) EU

+ = (U) E

+ .

Note that The first term on the right is N times larger than the second term, and weexpect that the first omitted term will be another factor of 1/N smaller. Inserting thisexpansion into our expression for the probability ratio, we have

P(state 2)

P(state 1) = e(U) E2/ (U) + E1/ = e(E2 E1)/ .

The probability ratio is an exponential in the energy difference divided by the temperature.As noted when we discussed the model paramagnetic system, this is called the Boltzmannfactor, and weve just shown that this is a general result.

It may seem we got something for nothing: we made a few definitions, did somemathematical hocus-pocus and voila, out pops the Boltzmann factor. A key ingredientis our postulate that all states which satisfy the constraints are equally probable. Thenumber of states goes up with increasing energy U. The rate of increase (in the logarithm)

is measured by the temperature, . When the system is in a state with energy E, itnecessarily received that energy from the heat bath. The more energy the heat bath givesto the system, the fewer states it has left. This is what makes higher energy states ofthe system less probable.

In principle, we should consider an ensemble of identically prepared heat baths andsystems. An ensemble in which the probability follows the Boltzmann form ( exp(E/))is called a canonical ensemble.



35/175


36/175

Physics 301 20-Sep-2004 5-4

Diversion: the Maxwell Velocity Distribution

Consider a low density gas. Generally, the forces between molecules are only largewhen the molecules are within a few molecular diameters of each other. The mean separa-

tion is many molecular diameters. We expect that a molecule in a low density gas satisfiesour definition of a system which is weakly interacting with its heat bath (all the othermolecules in the gas).

We consider only the translational energy of a molecule. The probability that amolecule is in a particular state with energy E = mv2/2 is just

P(E) eE/ = emv2/2 ,

where m is the mass and v is the velocity of the molecule. Now we have a problem. Thisexpressions applies to a particular state. Usually we think of the energy of translationand the velocity as continuous variables. What constitutes a particular state? Lets post-pone the counting of states for a bit and see how far we can get with some reasonableassumptions.

First of all, if we want to treat the energy and velocity as continuous variables, weshould be talking about a probability density, where the probability that the energy lies inthe range E E+ dE is given by p(E) dE. This will be the probability of getting E nomatter how many different states have this E, and this probability must be proportionalto the Boltzmann factor and to the number of states that have this E. In other words,

p(E) dE = CeE/n(E) dE ,where C is a constant whose value is adjusted to ensure that

p(E) dE = 1, and n(E),

usually called the density of states, is the number of distinct states per unit energy withenergy E. That is, weve put our ignorance of the number of states into this function!

We note that the Boltzmann factor, written in terms of velocities, is

P(E) emv2/2 = em(v2x + v

2y + v

2z)/2 .

This strongly suggests that each component of the velocity is normally distributed. Wemight expect this on other grounds as well. For example, if we pick a molecule and measurethe x component of its velocity, we note that vx is a random variable and its value isdetermined by the previous history of collisions the molecule has undergone. These arethe kinds of conditions that give rise to Gaussian distributions although proving it in thiscase might be painful. We expect that vx = 0. Otherwise, our sample of gas would havea non-zero velocity in the x direction and we are considering a sample at rest (of course!).Also, the mean square value ofvx contributes to the energy and we would expect this to be



37/175

Physics 301 20-Sep-2004 5-5

related to the temperature as the Boltzmann distribution suggests. Finally (among thingsthat we expect), there is no reason to prefer one direction over another, so the probabilitydistributions for vx, vy, and vz should be the same. In fact, the probability of any velocitycomponent should be the same which means that the joint probability should depend only

on the magnitude and not on the direction of the velocity.

Putting all this together, we guess that the joint probability that the x, y, and zcomponents of the velocity are in the ranges vx vx + dvx, vy vy + dvy, and vz vz + dvz, is

p(vx, vy, vz) dvx dvy dvz

=

1

2/memv2x/2 dvx

1

2/memv

2y/2 dvy

1

2/memv2z/2 dvz ,

= 1

2/m 32

emv2

/2 dvx dvy dvz .

This is called the Maxwell velocity distribution. Now lets change variables from Cartesiancomponents to spherical polar components, v, , where

vx = v sin cos ,

vy = v sin sin ,

vz = v cos .

This is just like changing coordinates from x, y, z to r, , . The result is

p(v , , ) dvdd =

1

2/m

32

emv2/2v2 sin dv d d .

Just as we expected, the probability is independent of direction, so we can integrate overdirection ( and ) to obtain the probability density for the velocity and any direction.The integral over directions gives 4 and we have,

p(v) dv = 4

1

2/m

32

emv2/2v2 dv .

Lets change variables one more time, back to the energy E = mv2/2,

p(E) dE = 2

1

32

eE/E12 dE .

Comparing this with our earlier expression, we conclude that the number of states per unitenergy, n(E) must be proportional to

E.



38/175

Physics 301 20-Sep-2004 5-6

To recap: knowing that the probability of a state with energy E is proportionalto exp(E/), we argued that it was reasonable to expect the velocity components to benormally distributed. We used symmetry arguments to specify the form of the distributionand we used the probability distribution in energy to set the variance. We made some

changes in variables to convert our guessed probability density in velocity to a probabilitydensity in energy from which we concluded that the number of kinetic energy states perunit energy is E1/2.

Asidethe Gamma Function

I kind of side-stepped the gamma function last week when we were using Stirlingsapproximation. But I think I should at least introduce the function as its necessary forsome of the integrals you might need to do. For example, to verify that p(E) above is

properly normalized, you will have to integrate over the energy and this is most naturallywritten as a gamma function. Define the gamma function by an integral:

(z) =

0

tz1 et dt .

Using integration by parts,

(z) =

0

tz1 et dt ,

=

tz1 et 0

+(z

1)0

tz2 et dt (z

1) ,

= (z 1)

0

tz2 et dt ,

= (z 1)(z 1) .

The gamma function satisfies a recursion relation. It is straightforward to show that(1) = 1 and with the recursion relation, one has

n! = (n + 1) .

This relation can be used to extend the definition of factorial to non-integers!

The next most interesting arguments of the gamma function after the integers are halfintegers. Using the recursion relation, these can be found if one knows

(1/2) =

.

Can you show this?



39/175

Physics 301 20-Sep-2004 5-7

For completeness, I should mention the incomplete gamma functions. (I couldntresist, sorry!) These arise when the range of integration does not include the entire interval[0, ).

The Partition Function

Weve had a diversion and an aside, so maybe its time to get back to thermal physics.Consider the function

Z() =s

eEs/ ,

where the sum extends over all states s. Among other things, this function normalizes theprobabilities,

P(Es) =

eEs/Z() ;

s P(E

s) = 1 .

Z is called the partition function. Weve written it as a function of temperature, but itsalso a function of the energies of all the states which might be functions of macroscopicparameters of the system.

The average energy of a member of the ensemble is

E =s

EsP(Es) .

Consider the derivative of Z with respect to temperature (the energies of the states, Es,do not depend on temperature),

Z

=

s

eEs/ ,

=s

eEs/

Es

,

=s

eEs/ Es2

,

= 12

s

EseEs/ ,

=Z

21

Z

s

EseEs/ ,

=Z

2E ,



40/175

Physics 301 20-Sep-2004 5-8

from which we deduce that

E = 2 log Z

.

Our discussion followed K&K except that we use E rather than and K&K appearto change the definition of U in midstream from the energy of heat bath to the averageenergy of a state of a single system.



41/175

Physics 301 22-Sep-2004 6-1

Entropy and Probabilities

Weve been using the idea that the entropy is the logarithm of the number of statesaccessible to the system. Weve also said that each state is equally likely. At this point, Id

like to make the connection between entropy and probability. This allows one to constructan expression for the entropy of a system that isnt in equilibrium. It should also improveour intuition about the entropy and the partition function.

We might expect that an expression for entropy can be written in terms of the prob-abilities that a system is in a particular state. If there are g states, and the probabilitiesare p1, p2, . . . , pg, then we would like to write

= (p1, p2, . . . , pg) .

One of the things that will guide us in our selection of the function is that the entropy

should be additive (i.e., an extensive parameter). If we have two non-interacting systemswith total numbers of states g1 and g2, entropies 1 and 2, and probabilities p1i and p2j(the first index is the system, the second index is the state), we can also think of it as asingle system with g = g1g2 states, = 1 + 2 and, since any state in system 1 can becombined with any state in 2, the probability of a state in the combined system must be

pij = p1ip2j .

Since the probabilities multiply, while the entropies add, we might expect that theentropy should involve the log of the probability. The first guess might be

1 = i logp1

i wrong .

Since p1i 1, the minus sign is inserted to make the entropy positive. Why doesnt thisexpression work? There are several reasons. First, suppose one has a totally isolatedsystem. Then only states with the exact energy of the system are allowed. Disallowedstates have p1i = 0 and this will lead to problems with the logarithm. In addition, withthe above expression, the entropy is not additive. To fix up the problem with p1i = 0, wemight try multiplying by p1i since in the limit x 0, x log x 0. Does this make theentropy additive? Consider

= i,j

p1ip2j logp1ip2j ,

= i,j

p1ip2j logp1i i,j

p1ip2j logp2j ,

= i

p1i logp1i j

p2j logp2j ,

= 1 + 2 .



42/175

Physics 301 22-Sep-2004 6-2

We used the fact that

ip1i =

j p2j = 1. We adopt the following expression for theentropy in terms of the probabilities.

= i

pi logpi ,

where we can include or omit states with probability 0 without affecting the value of theentropy.

What set of probabilities maximizes the entropy? The answer depends on the condi-tions under which we seek a maximum. Suppose we are dealing with a completely isolatedsystem. Then a state can have non-zero probability only if it has the required energy (andany other conserved quantities). So lets limit our sum to allowed states. (Here, weredoing this for convenience, not because our expression might blow up!) The other thingwe know is that the probabilities of the allowed states sum to 1. The problem we want to

solve is maximizing the entropy under the constraint that the probabilities sum to 1. Howdo we maximize with a constraint? Lagrange multipliers! So we seek to maximize

X(p) = (p) +

1

i

pi

,

= i

pi logpi +

1

i

pi

.

We set the derivative of X with respect to pi to zero,

0 = Xpi

= logpi 1 .

This gives

pi = e( + 1) ,

so the probabilities of all allowed states are the same when the entropy is a maximum. Wealso set the derivative of X with respect to to 0 which recovers the condition that theprobabilities sum to 1. Solving for , we find = log g 1. (g is the number of allowedstates and the number of terms in the sum.) Finally,

= i

1g log 1g = log 1g = log g ,

as we had before.

Now suppose we consider a system which is not isolated, but is in equilibrium thermalcontact with a heat bath so that the average value of its internal energy is U. Again, wesum only over allowed states. This time states with energies other than U are allowed,



43/175

Physics 301 22-Sep-2004 6-3

provided the average turns out to be U. We want to find the probabilities that maximizethe entropy under the constraints that the probabilities sum to 1 and average energy is U.We find the maximum of

X(p) = i

pi logpi + 1

1i

pi

+ 2

Ui

piEi

,

where Ei is the energy of state i. We want

0 =X

pi= logpi 1 1 2Ei .

It follows thatpi = e

1 1 2Ei .

You are asked to show in the homework that 2 = 1/. So, the probabilities wind up witha Boltzmann factor!

Consider an ensemble of systems. The case in which the energy of each system isidentical and equal probabilities are assigned is known as the micro-canonical ensemble.The case in which the energies vary and the probabilities are assigned with Boltzmannfactors is known as the canonical ensemble.

Heat Capacity

In general, the amount of energy added to a system in the form of heat, dQ, and therise in temperature d resulting from this addition of heat are proportional,

dQ = C d ,

where C is constant of proportionality. Why is constant in quotes? Answer: the badnews is that it can depend on just about everything. The good news is that over a smallrange of temperature it doesnt vary too much, so it can be treated as a constant.

One of the things it obviously depends on is the amount of material in the system. To

remove this dependence, one often divides by something related to the amount of materialand then speaks of the specific heat. For example, dividing by the mass of the system givesthe heat capacity per unit mass, c = C/m. Of course, this is only useful if one is dealingwith a homogeneous material. That is you might speak of the specific heat of aluminumand the specific heat of water, but for boiling water in an aluminum pan you would beconcerned with the heat capacity (which you could calculate from the masses of the panand the water and the specific heats from the Handbook of Chemistry and Physics). In thecase of gasses, the amount of material is usually measured in moles and the heat capacity



44/175


45/175

Physics 301 22-Sep-2004 6-5

We use the earlier expression to get the average energy per magnet and multiply by N toget the total energy in the system,

U = N 2log Z

,

= N 2sinh E/

cosh E/

E

,

= N 2 tanh E/E

2,

= N E tanh E/ ,

as before! The heat capacity is U/, so

CV = N E

tanh E/ ,

= N Esech2E/

E

,

= +N

E

sech

E

2.

At high and low temperatures the heat capacity goes to zero because of the 2 dependenceand the exponential dependence, respectively. When the temperature is low, its very hardfor the bath to get together enough energy to flip a magnet, so increasing the temperatureof the bath has little effect on the total energy of the system. When the temperature ishigh, the magnets are 50% aligned and 50% anti-aligned and the system cannot be made

more random by adding energy.



46/175


47/175

Physics 301 22-Sep-2004 6-7

Pressure

Recall in lectures 2 and 3, we maximized the entropy with respect to changes in energy,volume, and particle number and made some definitions and came up with

dU = d pdV + dN ,

with1

=

U

V,N

,p

=

V

U,N

,

=

N

U,V

.

Observe that if and N are constant, then the pressure is

p =

U

V

,N

.

What does it mean to keep and N constant? N is easy, we just keep the same numberof particles. requires a bit more thought. If the entropy doesnt change, the number ofstates doesnt change. If we perform a volume change at constant entropy, we are changingthe volume without changing the microstates of the system. In other words, the energychange produced by a change in volume at constant entropy is purely a mechanical energychange, not a thermal energy change, We know mechanics: the work done (and energysupplied) when the volume is increased by dV is just pdV, where p is the ordinarymechanical pressure, force per unit area. This argument supports our identification of

p = VU,N

,

as the conventional mechanical pressure.


lectures princeton

Documents