entropy and informationpaulcockshott.co.uk/publication-archive/talks... · 2019-03-11 · entropy...

Motivation : Information Work and MoneyUnderstanding entropy

Our Results/Contribution

Entropy and InformationIn science and political economy

Paul Cockshott1

1School of Computer Science,University of Glasgow

October 2, 2012

Author, WPC Information and production



Outline

1 Motivation : Information Work and MoneyWhat is informationSmith and BabbageBut What is Work?

2 Understanding entropyMaxwellBoltzmannShannon

3 Our Results/ContributionInformation can be applied to industrial processesCopying technologiesParallel transmission



Our Results/ContributionBut What is Work?

What is information

How does it relate to wealth and work and energy

How does it help us understand the economy

We will look at Adam Smith, James Watt, Charles Babbage, KarlMarx, James Clerk Maxwell etc

Labor was the first price, the original purchase-moneythat was paid for all things. (Smith)

The cost of any article may be reduced in its ultimateanalysis to the quantity of labour by which it wasproduced. (Babbage)

Economy of time, to this all economy ultimately reducesitself. . . . [E]conomy of time, along with the planneddistribution of labour time among the various branches ofproduction, remains the first economic law on the basisof communal production. (Marx)




From these thinkers we get the common idea that value = worktimeAt about the same time as one Adam Smith was professor of MoralPhilosophy in Glasgow, and was setting out a coherent formulationof the labour theory of value, Dr Black of the department ofNatural Philosophy along with a technician, one James Watt, werelaying the foundations for a proper understanding of heat andtemperature. These two exercises have more in common thanmight be imagined. Reflection upon it, brings out how conceptsfrom engineering science, from the practice of material production,paralleled and became the foundation for classical economics.




Engines

Figure: The steam engines of Newcomen and Watt




Table

Moral Philosophy Natural Philosophy

Price in gold Temperature on anguineas of whisky mercury thermometer of whisky

Specific labour Specific heat ofcontent of gold mercury

Value of whisky Heat content of the whisky

Labour required to Thermal energy ofmake whisky hot whisky.measured in hours measured in foot pounds

or horse-power seconds

Ability to work or Ability to work orlabouring power of horse-power of thedistillery workers distillery engine ( raising barrels ? )




Ideas

Heat → WorkWork → ValueBut animals too can work, does the labour of a horse create valueor only human work.How does human and animal work differ?Since Watt, artificial power has eliminated the horse, but thehuman worker survives. Why?




The concept of Purpose

A spider conducts operations that resemble those of aweaver, and a bee puts to shame many an architect inthe construction of her cells. But what distinguishes theworst of architects from the best of bees is this, that thearchitect raises his structure in the imagination before heerects it in reality. At the end of every labour process weget a result that already existed in the imagination of thelabourer at its commencement. He not only effects achange of form in the material on which he works, but healso realises a purpose of his own that gives the law tohis modus operandi, and to which he must subordinatehis will. (Marx, Cap I pp. 177–8)




Hold on!

Are animals really lacking in purpose?The spider may be so small, and her brain so tiny, that it seemsplausible that blind instinct, rather than the conscious prospect offlies, drives her to spin. But it is doubtful that the same applies tomammals.The horse at the plough may not envisage in advance the corn hehelps to produce, but then he is a slave, bent to the purpose of theploughman. Reduced to a source of mechanical power, overcomingthe dumb resistance the soil, he is readily replaced by a John Deere.Does the wolf stalking its prey not intend to eat it?




Information

Turn to Marx’s human example, an architect, and his argumentlooks even shakier. For do architects ever build things themselves?They may occasionally build their own homes, but in general whatgives them the status of architects is that they don’t get theirhands dirty with anything worse than India Ink.The key to our ability to build complicated structures is that wecan store the information required to build them in an externalpaper and ink plan and then follow the plan.Stored information is what distinguishes our work from animals.




Links

Watt develops the steam engine

Smith studies division of labour

De Prony applied Smith’s division of labour to producing logand trig tables

Babbage studied this, and extracted the general principle ofthe division of mental from manual labour. Babbage and hisfriend, the astronomer, John Herschel, checking manuallycalculated tables, Babbage, finding error after error, wasdriven to exclaim ’I wish to God these calculations had beenexecuted by steam’.

From this came the difference engine which mechanised DeProny’s manual division of labour in the production of tables




Engines

Figure: The difference engine of Babbage, note the sequence controller on the right




Links

1 Babbage needs precision engineering to make difference engine

2 He studies most existing production technologies and developsprecision die casting

3 He writes the On the Economy of Machinery andManufactures,

4 Marx, in London studying political economy, comes accrossBabbage’s work and uses it as the basis for his understandingof automated ( self acting ) factory production.

5 Babbage’s idea that the key to automation is the developmentof automatic sequential control mechanisms, taking controlfrom the worker, gives Marx the key to the alienation oflabour in the automated factory

Smith + Watt → Babbage → Marx → communism




MaxwellShannon

Gas initially in equilibrium. Demon opens door only for fastmolecules to go from A to B, or slow ones from B to A.→ Slow molecules in A, fast in B.� B hotter than A, and can be used for power.Information or intelligence has produced energy.....




MaxwellShannon

Formula for Entropy

Maxwell’s proposed counter-example to the second law wasexplicitly based on atomism. With Boltzmann, entropy is placed onan explicitly atomistic foundation, in terms of an integral overmolecular phase space.

S =−k

ˆf (v) log f (v)dv (1)

where v denotes volume in six-dimensional phase space, f (v) is thefunction that counts the number of molecules present in thatvolume, and k is Boltzmann’s constant.




MaxwellShannon

Shanons equationThe communications engineer Shannon introduces the concept ofentropy as being relevant to sending messages by teletype.

H =−n

∑i=1

pi log2 pi (2)

The mean information content of an ensemble of messages isobtained by weighting the log of the probability of each messageby the probability of that message. He showed that no encoding ofmessages in 1s and 0s could be shorter than this




MaxwellShannon

Chaitin

Chaitin’s algorithmic information theory defines the informationcontent of a number to be the length of the shortest computerprogram capable of generating it. This introduction of numbers isa slight shift of terrain. Shannon talked about the informationcontent of messages. Whereas numbers as such are not messages,all coded messages are numbers. An information theory defined interms of numbers no longer needs the support of a prioriprobabilities.Whereas Shannon’s theory depended upon the a priori probabilityof messages, Chaitin dispenses with this support.




MaxwellShannon

Mandelbrot

Figure: The Mandelbrot set, a complex image generated from a tinyamount of information. In fact it uses the formula z = z2 + c where z is acomplex number.




MaxwellShannon

Links

1 Babbage and Marx thought human work was the source ofvalue - an objectivist theory. Marx: value a crystalization ofhuman energy.

2 20th century economics rejected this in favour of saying thatvalue was something subjective.

3 The problem with the work time theory was that it had noplausible mechanism.

4 It was not till the 20th century that mathematician computerscientists came up with plausible mechanisms for the theory.




MaxwellShannon

Links: Turing, Capek - universality

1 Prague 1921 Capek invents the idea of the Universal Robot.The Universal Robot was an android that could do anyproductive task

2 England 1936 Turing (re-)invents Babbage’s idea of theUniversal Computer – one which can compute anymathematical function. Cycles of the Unversal Turingmachine are the fundamental cost metric for algorithms.

3 London 1938 World’s first science fiction TV broadcastRossum’s Universal Robots by Capek, introduces the notion ofthe Robot to the English.

4 Here is the first key to understanding Babbage and Marx’sideas of value

1 we humans are the only universal robots so far known,2 thus human cycle times are the basic cost metric for any task




MaxwellShannon

Links: Boltzmann, von Neumann, Machover

1 1872 Boltzmann founds entropy on statistical mechanics

2 1938 von Neumann’s Model of General Economic Equilibriumtheoretically applies matrix mechanics to the economy.

3 1945 von Neumann invents the current model of universalcomputer.

4 1984 at UCL recursion theorist Machover shows that Babbageand Marx’s theory of value follows as a necessary consequenceof Boltzmann’s entropy maximisation in chaotic marketeconomy.

5 1998 Shaikh The Empirical Strength of the Labor Theory ofValue applies von Neumann’s maths and a von Neumannmachine to empirically test Machover, and show that Babbageand Marx were right.




Information can be applied to industrial processesCopying technologiesParallel transmission

There are three fundamental ways by which the flow through anyproduction process can be increased.

1 Accelerating the production cycle.

2 Parallelizing production.

3 Eliminating wasted effort.

These are all well studied. What is less understood is the role thatentropy and information play in this.





Paper making

Consider the process of producing a book we see that theproduction process encompasses two opposite phases.First, we have the production of the paper. This is anentropy-reducing process. The blank sheets of paper obviouslyhave low information content with respect to human language, butthey also constitute a low entropy state with respect to the rawmaterial.In a sheet of paper the cellulose fibres are constrained in bothorientation and position. This implies a reduction in the volume ofstate space that the fibres occupy, and thus, from Boltzmann, acorresponding reduction in entropy.





Printing

Next, there is the writing of the text—whether by hand, as in thedistant past, or using a printing press. This is an entropy-increasingprocess. Imagine that the text to be printed exists as binary datain a file on disk, encoded using ASCII or UNICODEClearly the book contains this information, since by sending thebook in the post to someone we enable them to recreate therelevant binary file. Thus, by the equivalence of information andentropy, we have increased the entropy of the book relative to theblank sheets of paper.





2 phases

In the first phase a low-entropy material is created; in the secondphase the entropy of this material is increased in a controlled way.Initially natural information is removed; subsequently anthropic orhuman-created information is added. The natural informationremoved in the first stage is of no interest to us, while that addedin the second stage is dictated by our concerns.





Is this general?

It is easy to see the relevance of information theory to the printingindustry. Its product, after all, contains information in the everydayas well as the technical sense of that word. Does this approachprovide insights into how other production processes function?





Weaving and Spinning

For a rather different example, consider the process of producingcloth. The starting material is wool or cotton fibres in a randomtangled state. This is first carded to bring the fibres into roughalignment, and then simultaneously twisted and drawn to spin thefibres into yarn. In the yarn both the volume and orientation aresharply reduced. Energy is used to reduce the entropy of thecotton. The weaving of the cotton then increases the entropy byallowing two possible orientations of the fibres at right angles toone another (or more if we take into account the differences inpossible weave).





Car production

Other industries that use thin, initially flat materials clearly have alot in common with printing. The manufacture of car body partsfrom sheet steel, or the garment industry, share the pattern ofproducing a low entropy raw material and adding information to it.In pressed steel construction, added information is encoded in theshape of the dies used to form the car doors, roof panels etc.We can quantify it using Chaitin’s algorithmic information theory,as proportional to the length of the numerically controlled machinetool tape that is used to direct the carving of the die.





Replication

If steam powered the industrial revolution, the technologies ofreplication were the key to mass production. The classic exampleof the importance of accurate replication was in the production ofthe Colt revolver in the mid 19th century. Prior to Coltestablishing his factory the gun trade was dominated by handicraftmanufacturing techniques. The different parts of a gun’smechanism were individually made by a gunsmith so that theyfitted accurately together.For parts to be interchangeable they must be made to very precisetolerances. This improvement in accuracy of production involvesthe parts having a lower entropy, occupying a smaller volume ofphase space, than the old hand made parts. Again by theequivalence of information and entropy this means that thestandardized parts embody less information than the hand madeones.





Accuracy and information

Prior to the introduction of numerically controlled machine tools,replicated parts had to be composed of circular and planarelements which could be produced on lathes or milling machines.The limited information content of these can be seen when youconsider that in turning a smooth bore gun barrel one only has tospecify the inner and outer radii and its length. If an axle and abearing are being produced separately to fit together, then onewants the uncertainty in the surface of the bearing, given thesurface of the axle, to be reduced below a certain limit.





Accuracy continued

��

��

��A

B

Figure: When inserting axle A into bearing B we want to minimize theconditional information H(B|A), between B and A.





Binary cross section

000000111111111111111111111111111110000000000001111111111111111111111111111100000000000001111111111111111111111111110000000000000001111111111111111111111111000000000000000001111111111111111111111100000000000000000001111111111111111111110000000000000000000000111111111111111110000000000000000000000000011111111111110000000000000000000000000000000111111100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

Figure: A slice through the axle.





Generator

program circ ;const

b: array [boolean ] of char =( ‘1’ , ‘0’ );c =20;r =18;

vara: array [-c ..c ,-c ..c ] of boolean ;

begin

a←√

ι20 + ι2

1 < r ;write(ba);

end .

We cannot guarantee to have found the shortest such program.1

1The program is in Vector Pascal which is fairly concise .Author, WPC Information and production




Bearing Clearly if the bearing exactly fitted the axle the expandedencoding for a slice through the bearing would be an array similarbut with 1s and 0s swapped round. This can be produced by atrivial change to the program circ, the addition of a singlestatement.write(bnota);replaces the line:write(ba);This must come close to minimizing the conditional entropy of thetwo parts.





Imperfect construction

0000001111111111111111111111111111100000000000011111111111111111111111111111000000000000011111111111111111111111111100000000000000011111111111111111111111110000000000000000011111111111111111111111000000000000000000001111111111111111111000000000000000000000001111111111111111100000000000000000000000000011110111111000000000000000000000000000000000001110000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000

Figure: Part of a pin with a fault on its circumference.





means added code

we would need to add the following lines to the generator of A’ tomake the bitmap for B:ar,1← false;ar,2← false;ar+1,0← false;ar,0← false;ar−1,−1← true;

write(bnota);This contains extra information, to correct the bitmap of A’ togenerate that of B. In pre-industrial production, the extras steps inthe generator program would imply additional steps of filing andgrinding to make parts fit.





Pottery

1 Hand formation

2 Turn on wheel – only have to supply diameter as information

3 Casting – parallel information transmission, the Romansamian ware industry based on such casting is the firstparallelised mass production of consumer goods





Law of reproduction

The algorithmic information in repeated production grows by a lawof the form H(P)≤ H(c) + log n where P is the total productmade up of n repetitions of c . If we look at the process ofreproduction as a whole there are two terms the first given by thecomplexity of the original and a second logarithmic term given bythe number of repetitions.





Law of reproduction - pottery

In the case of the Samian ware pottery there is the original work ofproducing the master or pattern piece which corresponds to H(c),but then the number of copies that could be made growsexponentially with the number of successive steps of copying: ifthe master is used directly to produce the pots then L pots can bemade, where L is the lifetime of the master. If the master producesmoulds which in turn produce the pots then L2 pots can be made,etc. Invert the relationship and we find that the number ofsuccessive steps of copying will be related to the number of potsproduced n as logL(n), a relationship suggested by the predictionsof information theory.





A key advantage of moulding is in the information transmission

On a potters wheel the information relating to shape istransmitted serially by the potter moving his hands upwardsand controling the diameter

I samian ware the information about shape is transmitted inparallel accross the whole surface.

The information content that is transmitted can also be muchhigher. On a turned pot the information consists of only twonumbers for each height from base.





Engines

a b

a. Spinning wheel,b.Compton’s Mule. A multiplicity ofspindles and the moving frame substituted for the stretching

movement of the hand spinner’s arms.





Parallelism elsewhere

There are may other instances where increased productivity hasarisen from switching from serial to parallel transmission ofanthropic information in the 2nd stage of a production process:

1 The shift from manuscripts to printed books.

2 The invention of iron casting replacing the work ofblacksmithing. Look at the streets of Edinburgh New Town,the profusion of iron railings came from the cheap mouldingtechnologies of the Carron ironworks.

3 The shift from Edison phonograph drums to gramophonediscs.

4 The change from wire-wrap backplanes to printed circuitbackplanes in computing.

Productivity advances in chip techology all stem from the parallelimaging processes used in photo-lithography.





Cast Iron

The ability to melt and cast iron as opposed to just forging ironproducts allowed the mass production of iron goods





Sound Recording

The cylindrical phonograph could not be readily produced bystamping from a mould, but the gramophone could be.





Computer Circuits

Mid 1960s computers (left) had to be sequentially wired usingdiscrete wiring. By the early 1980s the equivalent circuit wereprinted (right) opening the way for mass production and thedevelopment of computers as a consumer good.





1

2 1832 Babbage, C.: 1832, The Economy of Machinery andManufactures, London.

3

4 1999 Chaitin, G.: 1999, Information and randomness: Asurvey of algorithmic information theory, The Unknowable,Springer, Signapore.

5

6 1875 Maxwell, J. C.: 1875, The Theory of Heat, London.

7

8 1906 Marx, K., :1867, Capital, 1906 edition, downloadabletext and pdf from Google Books.


entropy and informationpaulcockshott.co.uk/publication-archive/talks... · 2019-03-11 · entropy...

Documents