Springer ThesesRecognizing Outstanding Ph.D. Research

Non-equilibrium Energy Transformation ProcessesTheoretical Description at the Level of Molecular Structures

Viktor Holubec

Springer Theses

Recognizing Outstanding Ph.D. Research

For further volumes:http://www.springer.com/series/8790

Aims and Scope

The series ‘‘Springer Theses’’ brings together a selection of the very best Ph.D.theses from around the world and across the physical sciences. Nominated andendorsed by two recognized specialists, each published volume has been selectedfor its scientific excellence and the high impact of its contents for the pertinentfield of research. For greater accessibility to non-specialists, the published versionsinclude an extended introduction, as well as a foreword by the student’s supervisorexplaining the special relevance of the work for the field. As a whole, the serieswill provide a valuable resource both for newcomers to the research fieldsdescribed, and for other scientists seeking detailed background information onspecial questions. Finally, it provides an accredited documentation of the valuablecontributions made by today’s younger generation of scientists.

Theses are accepted into the series by invited nomination onlyand must fulfill all of the following criteria

• They must be written in good English.• The topic should fall within the confines of Chemistry, Physics, Earth Sciences,

Engineering and related interdisciplinary fields such as Materials, Nanoscience,Chemical Engineering, Complex Systems and Biophysics.

• The work reported in the thesis must represent a significant scientific advance.• If the thesis includes previously published material, permission to reproduce this

must be gained from the respective copyright holder.• They must have been examined and passed during the 12 months prior to

nomination.• Each thesis should include a foreword by the supervisor outlining the signifi-

cance of its content.• The theses should have a clearly defined structure including an introduction

accessible to scientists not expert in that particular field.

Viktor Holubec

Non-equilibrium EnergyTransformation Processes

Theoretical Description at the Levelof Molecular Structures

Doctoral Thesis accepted byCharles University in Prague, Czech Republic

123

AuthorDr. Viktor HolubecFaculty of Mathematics and PhysicsCharles University in PraguePragueCzech Republic

SupervisorProf. Petr ChvostaFaculty of Mathematics and PhysicsCharles University in PraguePragueCzech Republic

ISSN 2190-5053 ISSN 2190-5061 (electronic)ISBN 978-3-319-07090-2 ISBN 978-3-319-07091-9 (eBook)DOI 10.1007/978-3-319-07091-9Springer Cham Heidelberg New York Dordrecht London

Library of Congress Control Number:2014939642

� Springer International Publishing Switzerland 2014This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part ofthe material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission orinformation storage and retrieval, electronic adaptation, computer software, or by similar or dissimilarmethodology now known or hereafter developed. Exempted from this legal reservation are briefexcerpts in connection with reviews or scholarly analysis or material supplied specifically for thepurpose of being entered and executed on a computer system, for exclusive use by the purchaser of thework. Duplication of this publication or parts thereof is permitted only under the provisions ofthe Copyright Law of the Publisher’s location, in its current version, and permission for use mustalways be obtained from Springer. Permissions for use may be obtained through RightsLink at theCopyright Clearance Center. Violations are liable to prosecution under the respective Copyright Law.The use of general descriptive names, registered names, trademarks, service marks, etc. in thispublication does not imply, even in the absence of a specific statement, that such names are exemptfrom the relevant protective laws and regulations and therefore free for general use.While the advice and information in this book are believed to be true and accurate at the date ofpublication, neither the authors nor the editors nor the publisher can accept any legal responsibility forany errors or omissions that may be made. The publisher makes no warranty, express or implied, withrespect to the material contained herein.

Printed on acid-free paper

Springer is part of Springer Science+Business Media (www.springer.com)

Supervisor’s Foreword

Recently developed experimental techniques enable a detailed study of energytransformation processes on microscopic scales. We have now reached anunprecedented understanding of the functioning of molecular motors and newpossibilities are emerging for designing molecular machines. In general, the sys-tems studied in the respective experiments are driven far from thermodynamicequilibrium and are exposed to strong thermal fluctuations because of their smallsize. Both these aspects have to be taken into account in proper theoreticaltreatments. Work and heat quantities need to be considered as stochastic variables.It is just the mean values of these random variables that correspond to the conceptsof work and heat as we know them from classical macroscopic thermodynamics.Problems of determining free energy differences between initial and target states incontrolled processes, and of determining efficiency and optimization of molecularmachines need to be treated on the basis of stochastic thermodynamics. This newperspective offers many surprising possibilities but, at the same time, the mathe-matical methods needed in the theoretical treatments incorporate advancedmethods of stochastic calculus.

In his thesis, Viktor Holubec addresses challenging theoretical problems in thisactive field of current research. Broadly speaking, his main results cover threedomains: (1) Exact analytical solutions of work and heat distributions for iso-thermal nonequilibrium processes in suitable models are obtained, (2) Corre-sponding solutions for cyclic processes involving two different heat reservoirs arefound, (3) Optimization of periodic driving protocols for such cyclic processeswith respect to maximal output power, efficiency, and minimal power fluctuationsis studied.

Viktor’s exact analytical solutions for work and heat distributions are importantfor different reasons. First, they serve as a reference for theoretical investigationsof more complicated extended models, which may be tractable by similar tech-niques or which may yield to some kind of perturbative treatment around thereference model. Second, they allow one to get insight into the structure of the tailsof work distributions, in particular in the regime of work values much smaller thanthe mean work. Generic findings with respect to the tail structure in this regime areuseful for developing improved estimators of free energy differences based onapplication of the Jarzynski equality. Third, the exact solutions provide valuable

v

test cases for simulation procedures of the underlying stochastic processes, givinginformation on the number of stochastic trajectories that need to be generated inorder to obtain faithful results for simulated work and heat distributions.

I personally appreciate Viktor’s strong inclination to struggle for exact solutionsand, at the same time, to check the emerging complicated analytical expressionsthrough extensive numerical simulations. I share with him a satisfactory and awarm feeling that, in all individual cases, complete agreement between the ana-lytical and simulated results have been achieved.

I would like to thank Viktor for many interesting discussions during thepreparation of his thesis and I dare to express the belief that the following reportwill lighten the voyage into the ‘‘Brownian world’’ for other pilgrims.

Prague, April 2014 Prof. Petr Chvosta

vi Supervisor’s Foreword

Abstract

The thesis is devoted to the thermodynamics of externally driven mesoscopicsystems. These systems are so small that the thermodynamic limit ceases to holdand the probabilistic character of the second law cannot be ignored. Thermalforces becomes comparable to other forces acting on the system and they have tobe incorporated in the underlying dynamical law, i.e., in the master equation fordiscrete systems, and in the Fokker–Planck equation for continuous ones. In thefirst part of the thesis we investigate dynamics and energetics of mesoscopicsystems during nonequilibrium isothermal processes. Due to the stochastic natureof the dynamics, the work done on the system by the external forces must betreated as a random variable. We derive an exact analytical form of the workprobability density for several model systems. In particular, the knowledge of theexact formula improves the analysis of experimental data using the recentlydiscovered fluctuation theorems. In the second part of the thesis we study anonequilibrium cyclic process which incorporates two isotherms with differenttemperatures. During the cycle, the system can produce a positive work on theenvironment. We analyze two specific models of such mesoscopic heat enginesand we optimize their performance.

vii

Acknowledgments

In the first place I would like to thank my supervisor, Prof. RNDr. Petr ChvostaCSc., for his guidance and help during our long cooperation and to my colleague,RNDr. Artem Ryabov, for many inspiring discussions. Also, I must thank all othercolleagues who cooperated on writing the common publications. Special thanksshould go to my co-workers from the Universitaet Osnabrueck for their activeinterest in my research work. Further, I feel obliged to thank my colleagues at thedepartment and at the faculty for providing me friendly and inspiring conditionsduring my study. Finally, I would like to thank my family and my friends. Withouttheir support and tolerance the preparation of the manuscript would either beimpossible or I would have no family and friends after its finishing.

ix

Contents

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2 Stochastic Thermodynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172.1 Discrete Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.1.1 Dynamics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182.1.2 Energetics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

2.2 Continuous Systems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242.2.1 Dynamics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242.2.2 Energetics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252.2.3 On the Work Definition . . . . . . . . . . . . . . . . . . . . . . . . 28

2.3 Work Fluctuation Relations . . . . . . . . . . . . . . . . . . . . . . . . . . 312.4 Heat Engines. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

2.4.1 Limit Cycle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362.4.2 Diagrams of the Limit Cycle . . . . . . . . . . . . . . . . . . . . 38

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

3 Discrete State Space Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 473.1 Two-Level System. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

3.1.1 Dynamics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 473.1.2 Energetics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

3.2 Ehrenfest Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 523.2.1 Dynamics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 533.2.2 Energetics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

3.3 Infinite-Level Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 553.3.1 Dynamics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 563.3.2 Energetics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

3.4 Kittel Zipper . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 583.4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 593.4.2 Unzipping in an Extended Kittel Model. . . . . . . . . . . . . 603.4.3 Formalization of the Model . . . . . . . . . . . . . . . . . . . . . 653.4.4 Solution of the Model . . . . . . . . . . . . . . . . . . . . . . . . . 663.4.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

xi

4 Continuous State Space Models . . . . . . . . . . . . . . . . . . . . . . . . . . 794.1 The Two Works: A Toy Model. . . . . . . . . . . . . . . . . . . . . . . . 79

4.1.1 Indirectly Controlled Force . . . . . . . . . . . . . . . . . . . . . 794.1.2 Directly Controlled Force. . . . . . . . . . . . . . . . . . . . . . . 82

4.2 Sliding Parabola . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

5 Heat Engines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 915.1 Two-Level Heat Engine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

5.1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 915.1.2 Description of the Engine and Its Limit Cycle . . . . . . . . 925.1.3 Probability Density for Work and Heat . . . . . . . . . . . . . 965.1.4 Engine Performance . . . . . . . . . . . . . . . . . . . . . . . . . . 975.1.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1015.1.6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

5.2 Diffusive Heat Engine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1065.2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1065.2.2 Description of the Engine and Its Limit Cycle . . . . . . . . 1075.2.3 Thermodynamic Quantities. . . . . . . . . . . . . . . . . . . . . . 1115.2.4 Diagrams of the Limit Cycle . . . . . . . . . . . . . . . . . . . . 1135.2.5 External Driving . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1145.2.6 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124

6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

Appendix A: Work Distributions for Slow and for Fast Processes . . . . 129

Appendix B: Validity of Jarzynski Equalityfor a Unidirectional Process. . . . . . . . . . . . . . . . . . . . . . 133

Appendix C: Simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139

Curriculum Vitae . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151

xii Contents

Abbreviations

x, x(t), X, X(t) Column vectorxT, [x(t)]T Line vectorxi, [x]i ith component of the vector xX; XðtÞ MatrixXij; ½X�ij Element of the matrix X at the ith line and jth columndiagfa; b; . . .; zg Diagonal matrix with a; b; . . .; z on the diagonalAiðtÞ Variable corresponding to the microstates i of the system with

discrete state spaceAðx; tÞ Variables corresponding to the microstates x of the system with

continuous state spaceX Random variableXðtÞ Stochastic processProbfXðtÞ ¼ ag Probability that XðtÞ ¼ a_xðtÞ Time derivative, _xðtÞ � dx=dt½a; b� Closed interval from a to b(a, b) Open interval from a to bjxj Absolute value of xdðzÞ Dirac d-function of the variable zdij Kronecker dexpðzÞ; ez Exponential function of the variable zlogðzÞ Natural logarithm of the variable ztanhðzÞ Hyperbolic tangent of the variable zIm Zð Þ Modified Bessel function of the first kind of the variable z with

the parameterm1F1(a, b; z) Kummer function of the variable z with the parameters a and b

2F1(a, b; z) Gauss hypergeometric function of the variable z with theparameters a, b and c

HðzÞ Unit-step function. HðzÞ equals to 1 for z [ 0and to 0 otherwise

Hða; b; zÞ Hða; b; zÞ ¼ Hðz� aÞ �Hðz� bÞATA Attempt time algorithmFATA First attempt time algorithmFEL Free energy landscape

xiii

FRTA First reaction time algorithmPV diagram Pressure volume diagramRTA Reaction time algorithmWPD Work probability densityWPDs Work probability densities

xiv Abbreviations

Chapter 1Introduction

More than two centuries ago, a desire to understand energy transformation processesin nature had given birth to the classical thermodynamics [1–3]. The famous the-ory which expounded the connection between the concepts of heat and work andintroduced general laws governing the system transformations, in particular, thoseinvolving the exchange of heat, work and matter with an environment. Cornerstonesof this classical theory, the first and the second law of thermodynamics, are so univer-sal that they are successfully exploited in all branches of physics. The first law claimsthat the total energy of a closed system is conserved. The second law introduces thenotion of the total entropy production and says that, during any process, the entropyof the universe can never decrease. The second law, contrary to the first one, hasstatistical character. It is valid with certain (in most cases very large) probability andis exact only in the thermodynamic limit, the limit of infinite number of particlesin the system. Inter alia, the two laws imply fundamental limits on the efficiency ofheat engines and refrigerators which were first revealed by Sadi Carnot [4].

The thermodynamic description of systems in equilibrium was justified by equi-librium statistical mechanics [5, 6]. This theory states that the probability to find asystem in contact with a heat bath in its specific microstate is given by the Boltz-mann factor. In the language of the statistical mechanics the statistical character of thesecond law becomes obvious. The most striking example is the famous Boltzmanndefinition of entropy S = kB log �, where kB stands for the Boltzmann constantand � denotes the number of microstates corresponding to a given macrostate. Themost probable macrostate is compatible with the largest number of microstates �.The corresponding entropy is maximal possible and coincides with that defined in theclassical thermodynamics. Nevertheless, the system may visit also other macrostates,although the probability of these fluctuations rapidly decreases with the number ofparticles in the system. For small deviations from equilibrium, linear response theory[7, 8] allowed to express transport properties of systems under influence of smallexternal fields through equilibrium correlation functions. Examples are the fluctua-tion dissipation theorem [9], which was first formulated by Harry Nyquist [10] andlater proven by Herbert Callen and his coworkers [11], and the Green-Kubo relations

V. Holubec, Non-equilibrium Energy Transformation Processes, Springer Theses, 1DOI: 10.1007/978-3-319-07091-9_1, © Springer International Publishing Switzerland 2014

2 1 Introduction

[12, 13]. On a more phenomenological level, linear irreversible thermodynamicsprovided the so called Onsager reciprocal relations between such transport coeffi-cients and entropy production in terms of forces and fluxes [14–16]. For a long time,no universal exact results beyond this linear response regime were available.

One of the most challenging and exciting fields of science today is a living cellwhich is, by definition, a non-equilibrium system (equilibrium = dead). Within thecell, one permanently observes many non-equilibrium processes such as activity ofmolecular motors [17, 18], ion diffusion through membranes [19], DNA replication[20, 21] etc. These systems are mostly so small that the thermal fluctuations inevitablyplay an important role in their dynamics and, at the same time, they are large enoughthat the quantum effects can be neglected. Such systems are often referred to asmesoscopic systems. An example is a dielectric colloidal particle diffusing in water(heat bath) and driven by an externally controlled potential, which can be for instancerealized by a focused laser beam—the so called optical trap [22, 23]. Assume that wemodulate the potential and detect the position of this driven Brownian particle [24].If we repeat the experiment many times with the same initial state of the particle andwith the same driving, we find out that the particle seldom evolves twice along a giventrajectory. The reason is simple—we can never guarantee that the bath itself is at thebeginning of each realization in the same state. The particle dynamics is characterizedby a stochastic process and, from the many repetitions of the experiment, we obtainan ensemble of possible trajectories together with their probabilities. Along eachtrajectory a different amount of work is done on the particle and, similarly, a differentamount of heat is dissipated into the water. These basic thermodynamic variablesthemselves become stochastic and are described by certain probability densities.

The necessity to describe the thermodynamics of such non-equilibrium processesled to the generalization of classical thermodynamics and equilibrium statisticalmechanics. This generalization was developed during the last two decades [25].Using the new approaches based, both on stochastic dynamics [26–29] and onthermostated Hamiltonian dynamics [30], new general laws applicable to non-equilibrium mesoscopic system were revealed. These exact results, valid in an arbi-trary non-equilibrium regime, generically refer to the probability densities of thermo-dynamic quantities like exchanged heat, applied work or entropy production, whichare, for the strongly non-equilibrium processes, typically non-Gaussian. In this sensethey represent a generalization of the second law of thermodynamics that is validonly for average values of these thermodynamic quantities. In particular these resultsallow to quantify the probability that, during a non-equilibrium process, a negativeamount of entropy is produced. Occasionally such events have been called (transient)violations of the second law. It turns out that the probability of these events becomestypically exponentially small in the relevant system size. Short review of these resultsis given below (for more details see for example the thorough reviews [25, 31, 32]).

First, for a thermostated fluid under an external shear, a remarkable symmetryof the probability distribution for the entropy production in the non-equilibriumsteady state was discovered numerically and justified heuristically by Evans et al.[33]. This result, known as the steady state fluctuation theorem, was first proven byGallavotti and Cohen [34] for a large class of systems using concepts from chaotic

1 Introduction 3

dynamics [30]. Later the theorem was proven also for driven Langevin dynamics [35]and for driven diffusive dynamics [36]. As a variant of the steady state fluctuationtheorem, Evans and Searles [33, 37] also derived a transient fluctuation theoremvalid for relaxation towards the steady state.

Second, Jarzynski proved an elegant equality which allows to express the freeenergy difference between two equilibrium states by a nonlinear average of the ran-dom work required to drive the system, during an arbitrary non-equilibrium (finitetime) isothermal process, from one state to the other [38, 39]. By comparing prob-ability distributions for the work done during the process used by Jarzynski withthat for the work done during the time-reversed one, Crooks found a refinement ofthe Jarzynski equality, the so called Crooks fluctuation theorem [40–42]. These twotheorems, together with a further refinement of the Jarzynski equality, the Hummer-Szabo relation [43], became particularly useful for determining equilibrium freeenergy differences and landscapes of biomolecules [32, 44–48]. Due to their exper-imental importance these relations are the most prominent ones within this classof exact results (some of which were found even much earlier [49, 50] and thenrediscovered) valid for non-equilibrium systems driven by time-dependent forces.Some of them are also used in this thesis and, hence, we review them in more detailin Sect. 2.3. A theorem which applies to transitions between two non-equilibriumsteady states, and, therefore, is analogous to the Jarzynski equality, was derived byHatano and Sasa [51].

Third, for driven Brownian motion, Sekimoto realized that three central conceptsof classical thermodynamics, namely the internal energy, the exchanged heat and theapplied work, can be meaningfully defined on the level of individual trajectories ofthe stochastic process [52, 53]. These stochastic quantities are entering the first lawof thermodynamics, which remains valid separately for each individual trajectoryof the process. These considerations gave birth to what Sekimoto called stochasticenergetics in his monograph [54]. Fourth, Maes and Netocný emphasized that theentropy production in the medium is related to the part of the stochastic action(which determines the probability that the stochastic process in question evolvesalong a specific trajectory) that is odd under the time-reversal [55, 56].

Finally, building systematically on the concepts briefly noticed previously[41, 57], a theory unifying the described results emerged. Seifert realized that oneshould not only make account of the fluctuating entropy produced in the heat bath,but one should also properly assign a stochastic entropy to the system itself [58]. Thislast step allowed to define the key quantities known from classical thermodynamicsalong the individual trajectories and, hence, make them accessible to experimentalmeasurements and numerical simulations. Such approach which considers both thefirst law (energy conservation) and the entropy production along the individual tra-jectories, was called stochastic thermodynamics [59]. A theory with the same namehas been originally introduced by the Brussels school in the mid-eighties [60, 61].It describes non-equilibrium steady states in chemical reaction systems.

Stochastic thermodynamics as considered in this thesis represents a generalizationof classical thermodynamics. It applies to various non-equilibrium phenomena inthe presence of time varying external fields, where in particular the systems are

4 1 Introduction

Fig. 1.1 The horse (the monkey) chasing the carrot (the banana). The position of the bananacorresponds to the steady state of the system determined by the instantaneous value of the drivingparameters. The position of the monkey depicts the immediate state of the system which is attractedto this steady state. Sadly, in the models presented in this thesis the driving parameters are alwaysmodulated, the banana never stops and, hence, the monkey never gets it. In order to calm down theanimal rights activists, let us note that both the monkey and the banana in the figure are plastic.Photo Jana Benešová ©.

in contact with a thermal reservoir. Such phenomena are of vital interest in manyareas of current research [25, 32, 59, 62]. Examples are ageing and rejuvenationeffects in the rheology of soft-matter systems and in the dynamics of spin glasses,the phenomenon of stochastic resonance [63–66], relaxation and transport processesin biological systems such as molecular motors [19, 67–74], ion diffusion throughmembranes, or DNA replication, driven diffusion systems with time-dependent biassuch as colloidal particles, and Nano-engines. Most of these systems are embeddedin an aqueous solution which serves as a heat reservoir.

Let us mention that the behavior of the systems in question, i.e. mesoscopicsystems driven by time-varying forces, can be understood intuitively using the famoushorse-carrot analogy. The position of the horse (depicted in Fig. 1.1 by the monkey)represents the immediate state of the system and the carrot (depicted in Fig. 1.1 bythe banana) reflects the external driving. More precisely, the position of the carrotrepresents the steady state compatible with the current adjustment of the externaldriving. If the driving remains constant in time (the banana does not move) thesystem relaxes towards the steady state (the monkey approaches the banana). Whenthe external forcing varies with time (the banana moves) the system “lacks behind”the steady state (the monkey chases the banana, which constantly escapes). Thisanalogy is precise, except for time scales. If one would not move the banana, anymonkey would catch it in a twinkling of an eye. Although the physical relaxationtowards steady states is exponential, it is the slower the closer the system is to the

1 Introduction 5

steady state. Hence the time needed to reach the steady state is formally infinite. Tobehave like this would be quite unpleasant for any monkey, indeed.

Three types of non-equilibrium situations can be distinguished for the abovedescribed systems [25, 32]. First, one can prepare the system in a non-equilibriuminitial state by placing it into an external potential, and study its relaxation towardsthermal equilibrium (we observe the monkey approaching an immobilized banana;its position represents Gibbs equilibrium state corresponding to the external poten-tial). An example is to study how the exponentially decreasing concentration of gasparticles with height in a homogeneous gravitational field (the so called barometricformula) is gradually reached in time if one starts with a constant particle concentra-tion. Second, general driving can be caused by the action of time-dependent externalforces, fields, flows or unbalanced chemical reactions (the monkey chases a con-stantly moving banana). This setting is the most general one and its special casewhen the external forces are caused by gradients of time-dependent potentials isthe main topic of this thesis. In the following chapters we give several examplesof such processes. Third, in the case of applying time independent forces that arenot caused by gradients of some potential energies the system will eventually reacha non-equilibrium steady state (the monkey approaches an immobilized banana;the banana position represents the non-equilibrium steady state compatible with theexternal constraints). Contrary to the equilibrium states, these non-equilibrium steadystates are generally characterized by non-vanishing currents. An example is a wirethe ends of which are kept on different temperatures. In such situation a constantheat flow throughout the wire is eventually established.

In all these cases, even under strong non-equilibrium conditions, the temper-ature of the heat reservoir remains well-defined. This property together with thetime-scale separation between the observable, typically slow, degrees of freedomof the system and the unobservable fast ones of the thermal bath (and, in the caseof (bio)polymers etc, by fast internal ones of the system) allows for a consistentthermodynamic description [75–77]. The collection of the relevant slow degrees offreedom yields the possible microstates of the system. An example of such collectionof microstates is the set of all spacial conformations of a (bio)polymer. The state ofthe system at a given time is then determined by the probability to find the systemat this time in a specific microstate. Since the system changes its microstate bothdue to the driving and due to the random thermal fluctuations, the system dynamicsis necessarily stochastic. Similarly to the case of a driven colloidal particle, a seriesof measurements performed on the system under the same conditions (except forthe internal coordinates of the bath) would lead to a set of possible trajectories withtheir relative frequencies. By inspecting these trajectories it is therefore possible tomeasure (both experimentally and in numerical simulations) the probability densitiesof the thermodynamic variables defined along the individual trajectories.

This ensemble of the trajectories is fully characterized by (1) the initial state of thesystem, (2) the properties of the thermal noise, and (3) the (possibly time-dependent)external driving. Mathematically, the time-scale separation implies that the underly-ing stochastic dynamics becomes Markovian one, i.e., the future state of the systemdepends solely on the present one (the dynamical law has no memory) [28, 29]. If

6 1 Introduction

the state space of the system is a discrete one the dynamics of the system state can bedescribed by a master equation [26, 27]. For the systems with a continuous state spacethe dynamics of the system microstate may follow a Langevin equation. In this casethe dynamics of the system state (ensemble) is driven by a Fokker-Planck equation[26, 27]. Within such stochastic dynamics, the above-quoted fluctuation theoremscan be derived for any system using a rather unsophisticated mathematics. It is suffi-cient to invoke a conjugate dynamics, typically, but not exclusively, time-reversal, toderive these theorems in a few lines [32]. Going beyond the thermodynamic frame-work, it turns out that many of the fluctuation theorems hold formally true for anykind of Markovian stochastic dynamics [78].

In the present work we always assume that the system dynamics is stochasticand fulfills the above mentioned dynamical laws. Other approaches, which can bealso used to describe the time evolution of the systems in question, are the Hamil-tonian dynamics [79] and the thermostated dynamics [62, 80]. In fact, many of thefluctuation theorems were originally derived within these deterministic frameworks.Although sometimes these descriptions are considered to be more fundamental thanthe stochastic one, for the purposes of this thesis, the stochastic approach providestwo crucial advantages. First, contrary to the deterministic descriptions, it allowsto describe transitions between discrete states like in (bio)chemical reactions withessentially the same conceptual framework used for the systems with a continuousstate space. Second, it allows to focus on the relevant (and measurable) degrees offreedom and ignore the water molecules etc. The possibility of this coarse-grainingenables the analytical study of the dynamics of relatively complex systems like bio-molecules. Using the deterministic approaches this is possible only numerically.Finally, let us note that some of the fluctuation theorems can be formulated also foropen quantum systems [81–83]. In this thesis we do not consider quantum systemsexplicitly, however, the obtained results are directly applicable to open quantumsystems whenever quantum coherences, i.e., the role of the non-diagonal elementsin the density matrix [84], can be neglected. The dynamics of the driven or openquantum system is then equivalent to a classical stochastic one.

Aims of the Thesis

In this thesis we focus on exactly solvable models from the field of stochastic ther-modynamics, the basic mathematical concepts of which we review in Chap. 2. Exactanalytical solutions are very important on different reasons. We will mention onlyfew examples. Firstly, they serve as reference for theoretical investigations of morecomplicated models, which may be tractable by similar techniques or which may betreated by some kind of perturbative treatment around the reference model. Secondly,they allows to better understand and quantify various phenomena. Thirdly, they pro-vide valuable test cases for numerical and simulation procedures of the underlyingstochastic processes and give information on the necessary number of stochastic tra-jectories that need to be generated in order to obtain faithful results. In this thesis we

1 Introduction 7

provide exactly solvable models which can be divided into two groups according tothe time-dependence of the reservoir temperature.

First, in Chaps. 3 and 4, we consider externally driven mesoscopic systems withboth discrete (Chap. 3) and continuous (Chap. 4) state space which, moreover, arein contact with a thermal reservoir at a constant temperature. For various specificmodels we analytically investigate the dynamics and energetics during the emergingisothermal processes. In particular, we calculate the probability density for work, heatand internal energy. Although, from the theoretical point of view, these results arethemselves interesting, they can also help to better utilize the experimental data. Forinstance, as mentioned above, in many experiments the Jarzynski equality is usedin order to extract the equilibrium free energy differences from non-equilibriumstretching experiments [32]. The extracted free energy difference strongly dependson the tail of the work distribution for large negative work values. These work valuescorrespond to highly improbable realizations of the experiment and hence can notbe measured accurately enough [38, 85–87]. Therefore it is important to use thecorrect fit (the Jarzynski estimator) of the measured work distribution in order toaccurately obtain the tail. To this end, some generic behavior in the tail regimeneeds to be assumed and attempts have been made recently in this direction [88–90].Although complicated from the mathematical point of view, the models where thework fluctuations can be treated analytically are often too simple from the physicalpoint of view. Investigations have been conducted for simple spin systems driven bya time-dependent external field [91–96] and for diffusion processes in the presenceof a time-dependent potential [97, 98]. Analytical solutions are known for two-statesystems [93, 94, 96, 99, 100], for systems with one continuous state variable [88, 89,97, 98, 101–105] and for systems composed of Nsub two-level subsystems in the limitof large Nsub [106, 107]. The models studied in this thesis are of similar complexity.More realistic models can be studied using computer simulations, which can alsohelp to find suitable Jarzynski estimators. To this end we present in Appendix C anew Monte Carlo algorithm which can be used for these simulations.

In the second part of the thesis we focus on periodically driven mesoscopic sys-tems which communicate with two heat reservoirs at different temperatures (or witha single reservoir with a time-periodic temperature). During the cyclic process thesystem can perform a positive mean work on the environment. Such stochastic heatengines have been intensively studied during the last decade. The standard thermody-namic variables such as power and efficiency are now defined using the mean valuesof work and heat. The available models can be roughly classified according to thedynamical laws involved. In the case of the classical stochastic heat engines, the statespace can either be discrete or continuous (c.f., for example, [71, 72, 96, 108] andthe references in the thorough review [25]). Examples of the quantum heat enginesare studied, e.g., in [70, 109, 110]. In the background of most of these studies is aneffort to describe possible forms of engines that could work efficiently in the Brown-ian realm of large thermal fluctuations. The heat engines in question usually operateon time-scales smaller than intrinsic relaxation times of their working medium andhence far from quasi-static conditions, when the work fluctuations are not insignif-icant (one example is Fig. 2 in the amazing experimental study [111], where the

8 1 Introduction

measured work fluctuations are depicted). Therefore the common analysis of thestochastic heat engines which only considers the mean values of the thermodynamicvariables and does not reflect fluctuations (one exception is the study [96]) may not bealways sufficient. In this thesis we first, in Sect. 2.4, present a more detailed reviewof the results obtained in the field and also the basic theoretical concepts neededfor the analytical treatment of such stochastic heat engines. Next, in Chap. 5, wepresent two exactly solvable models. In the first one we consider a two-state systemexposed to a simple two-branch isothermal driving. Within this setting we derivethe probability density for the work performed per operational cycle and we discussits properties. In the second example we consider a system with a continuous statespace, the colloidal particle diffusing in an asymmetric log-harmonic potential. Forthis system we impose the driving which consists of two isotherms and two adia-batic branches. This setting, in particular, allows us to found the exact form of theprotocol which yields the maximum output power of the engine and verify the recentgeneral results concerning the corresponding efficiency [108, 112–114] reviewed inSect. 2.4. The main motivation of these studies has been to find a relation for theefficiency at maximum power that would be of similar generality as the Carnot’sformula for maximum efficiency [4]. It is interesting to note that the first attempt inthis respect was made already nearly sixty years ago by Novikov [115] in the fieldof endoreversible thermodynamics. His result was later independently rediscover byCurzon and Ahlborn and is now known as Curzon-Ahlborn efficiency [1, 116]. In theboth models we have also discussed the possibility to minimize power fluctuations.Specifically, we show that the driving favored by the maximum power is not optimalwith respect to the fluctuations. This fact could disadvantage the power maximizingprotocol in applications where sharp non-fluctuating values of power are needed. Tosum up, the main results of the thesis are given in Chaps. 3–5 and also in Apps. A–C.Some of them have already been published. Specifically, the zipper model (Sect. 3.4)have been published in [117], the sliding parabola model in [118], and the two-levelmotor (Sect. 5.1) is described in [96, 119]. Finally, the new algorithm for MonteCarlo simulation of stochastic jump processes (Appendix C) has been published in[120].

References

1. Callen, H. (2006). Thermodynamics and an Introduction to thermostatistics. Student (edn.),Wiley. ISBN 9788126508129. http://books.google.cz/books?id=uOiZB_2y5pIC.

2. Müller, I. (2007). A history of thermodynamics: The doctrine of energy and entropy. London:Springer. ISBN 9783540462279. http://books.google.cz/books?id=u13KiGlz2zcC.

3. Van Wylen, G., Sonntag, R., & Borgnakke, C. (1994). Fundamentals of classical ther-modynamics. Number sv. 1 in Fundamentals of Classical Thermodynamics, Wiley. ISBN9780471593959. http://books.google.de/books?id=IeIeAQAAIAAJ.

4. Carnot, S. (2012). Reflections on the motive power of fire: And other papers on the secondlaw of thermodynamics. Dover books on physics, Dover Publications. ISBN 9780486174549.http://www.google.de/books?id=YdpQAQAAQBAJ.

References 9

5. Gibbs, J. (2010). Elementary principles in statistical mechanics: Developed with especialreference to the rational foundation of thermodynamics. Cambridge Library Collection—Mathematics, Cambridge University Press. ISBN 9781108017022. http://www.google.cz/books?id=7VbC-15f0SkC.

6. Huang, K. (1963). Statistical mechanics. New York: Wiley. http://books.google.cz/books?id=MolRAAAAMAAJ.

7. De Groot, S., & Mazur, P. (2013). Non-equilibrium thermodynamics. Dover Bookson Physics, Dover Publications. ISBN 9780486153506. http://books.google.cz/books?id=mfFyG9jfaMYC.

8. Demirel, Y. (2013). Nonequilibrium thermodynamics: transport and rate processes in physi-cal, chemical and biological systems. Elsevier Science. ISBN 9780444595812. http://books.google.cz/books?id=WSmcAAAAQBAJ.

9. Kubo, R. (1957). Statistical-mechanical theory of irreversible processes. I. general theory andsimple applications to magnetic and conduction problems. Journal of the Physical Societyof Japan, 12(6), 570–586. doi:10.1143/JPSJ.12.570, http://journals.jps.jp/doi/pdf/10.1143/JPSJ.12.570., http://journals.jps.jp/doi/abs/10.1143/JPSJ.12.570.

10. Nyquist, H. (1928). Thermal agitation of electric charge in conductors. Physical Review, 32,110–113. doi:10.1103/PhysRev.32.110, http://link.aps.org/doi/10.1103/PhysRev.32.110.

11. Callen, H. B., & Welton, T. A. (1951). Irreversibility and generalized noise. Physical Review,83, 34–40. doi:10.1103/PhysRev.83.34, http://link.aps.org/doi/10.1103/PhysRev.83.34.

12. Green, M. S. (1954). Markoff random processes and the statistical mechanics of time-dependent phenomena. II. irreversible processes in fluids. The Journal of Chemical Physics,22(3), 398–413. http://dx.doi.org/10.1063/1.1740082, http://scitation.aip.org/content/aip/journal/jcp/22/3/10.1063/1.1740082.

13. Kubo, R. (1966). The fluctuation-dissipation theorem. Reports on Progress in Physics, 29(nr. 1), 255. http://stacks.iop.org/0034-4885/29/i=1/a=306.

14. Kondepudi, D., & Prigogine, I. (1998). Modern thermodynamics: From heat enginesto dissipative structures. Wiley. ISBN 9780471973935. http://books.google.de/books?id=kxcvAQAAIAAJ.

15. Prigogine, I. (1955). Thermodynamics of irreversible processes. Thomas. http://books.google.de/books?id=DEQLPwAACAAJ.

16. Onsager, L. (1931). Reciprocal relations in irreversible processes. I. Physical Review, 37,405–426. doi:10.1103/PhysRev.37.405, http://link.aps.org/doi/10.1103/PhysRev.37.405.

17. Schliwa, M., & Woehlke, G. (2003). Molecular motors. Nature, 422, 759–765. doi:10.1038/nature01601.

18. Schliwa, M. (2006). Molecular motors. Wiley. ISBN 9783527605651. http://books.google.cz/books?id=6PJMfQlIS1kC.

19. Reimann, P. (2002). Brownian motors: Noisy transport far from equilibrium. Physics Reports,361(2–4), 7–265. ISSN 0370–1573, http://dx.doi.org/10.1016/S0370-1573(01)00081-3,http://www.sciencedirect.com/science/article/pii/S0370157301000813.

20. Kornberg, A., & Baker, T. (2005). Dna replication. University Science Books. ISBN9781891389443. http://books.google.cz/books?id=KDsubusF0YsC.

21. Tackett, A. J., Morris, P. D., Dennis, R., et al. (2001). Unwinding of unnatural sub-strates by a DNA helicase. Biochemistry, 40(2), 543–548. doi:10.1021/bi002122+,pMID:11148049, http://pubs.acs.org/doi/pdf/10.1021/bi002122%2B, http://pubs.acs.org/doi/abs/10.1021/bi002122%2B.

22. Neuman, K. C., & Nagy, A. (2008). Single-molecule force spectroscopy: optical tweezers,magnetic tweezers and atomic force microscopy. Nature Methods, 5, 491–505. http://dx.doi.org/10.1038/nmeth.1218.

23. Cohen, A. E. (2005). Control of nanoparticles with Arbitrary two-dimensional force fields.Physical Review Letters, 94, 118102. doi:10.1103/PhysRevLett.94.118102, http://link.aps.org/doi/10.1103/PhysRevLett.94.118102.

24. Blickle, V., Speck, T., Helden, L., et al. (2006). Thermodynamics of a colloidal particle ina time-dependent nonharmonic potential. Physical Review Letters, 96, 070603. doi:10.1103/PhysRevLett.96.070603, http://link.aps.org/doi/10.1103/PhysRevLett.96.070603.

10 1 Introduction

25. Seifert, U. (2012). Stochastic thermodynamics, fluctuation theorems and molecular machines.Reports on Progress in Physics, 75(nr. 12), 126001. http://stacks.iop.org/0034-4885/75/i=12/a=126001.

26. Van Kampen, N. (2011). Stochastic processes in physics and chemistry. North-Holland Per-sonal Library, Elsevier Science. ISBN 9780080475363. http://books.google.cz/books?id=N6II-6HlPxEC.

27. Gillespie, D. T. (1992). Markov processes: An introduction for physical scientist. San Diego:Academic press, Inc.

28. Feller, W. (2008). An introduction to probability theory and its applications, (2nd ed.). Number2 in Wiley publication in mathematical statistics, Wiley. ISBN 9788126518067. http://books.google.de/books?id=OXkg-LvRgjUC.

29. Feller, W. (1950). An introduction to probability theory and its applications.Number 1 in Wiley mathematical statistics series, Wiley. http://books.google.cz/books?id=x9sgAAAAMAAJ.

30. Klages, R. (2007). Microscopic chaos, fractals and transport in nonequilibrium statisticalmechanics. Advanced series in nonlinear dynamics, World Scientific. ISBN 9789812565075.http://books.google.de/books?id=nwpiaeOMKmcC.

31. Sevick, E., Prabhakar, R., Williams, S. R., et al. (2008). Fluctuation theorems. Annual Reviewof Physical Chemistry, 59(1), 603–633. doi:10.1146/annurev.physchem.58.032806.104555,http://www.annualreviews.org/doi/pdf/10.1146/annurev.physchem.58.032806.104555.

32. Ritort, F. (2008). Advances in Chemical Physics, Volume 137, chapter Nonequilibrium Fluctu-ations in Small Systems: From Physics to Biology (pp. 31–123). Wiley. ISBN 9780470238080.doi:10.1002/9780470238080.ch2, http://dx.doi.org/10.1002/9780470238080.ch2.

33. Evans, D. J., Cohen, E. G. D., & Morriss, G. P. (1993). Probability of second law violationsin shearing steady states. Physical Review Letters, 71, 2401–2404. doi:10.1103/PhysRevLett.71.2401, http://link.aps.org/doi/10.1103/PhysRevLett.71.2401.

34. Gallavotti, G., & Cohen, E. G. D. (1995). Dynamical ensembles in nonequilibrium statisti-cal mechanics. Physical Review Letters, 74, 2694–2697. doi:10.1103/PhysRevLett.74.2694,http://link.aps.org/doi/10.1103/PhysRevLett.74.2694.

35. Kurchan, J. (1998). Fluctuation theorem for stochastic dynamics. Journal of Physics A: Math-ematical and General, 31(nr. 16), 3719. http://stacks.iop.org/0305-4470/31/i=16/a=003.

36. Lebowitz, J., & Spohn, H. (1999). A gallavotti-cohen-type symmetry in the largedeviation functional for stochastic dynamics. Journal of Statistical Physics, 95(1–2),333–365. ISSN 0022–4715, doi:10.1023/A:1004589714161, http://dx.doi.org/10.1023/A%3A1004589714161.

37. Evans, D. J., & Searles, D. J. (1994). Equilibrium microstates which generate second lawviolating steady states. Physical Review E, 50, 1645–1648. doi:10.1103/PhysRevE.50.1645,http://link.aps.org/doi/10.1103/PhysRevE.50.1645.

38. Jarzynski, C. (1997). Nonequilibrium equality for free energy differences. Physical ReviewLetters, 78, 2690–2693. doi:10.1103/PhysRevLett.78.2690, http://link.aps.org/doi/10.1103/PhysRevLett.78.2690.

39. Jarzynski, C. (1997). Equilibrium free-energy differences from nonequilibrium measure-ments: A master-equation approach. Physical Review E, 56, 5018–5035. doi:10.1103/PhysRevE.56.5018, http://link.aps.org/doi/10.1103/PhysRevE.56.5018.

40. Crooks, G. (1998). Nonequilibrium measurements of free energy differences for micro-scopically reversible markovian systems. Journal of Statistical Physics, 90(5–6), 1481–1487. ISSN 0022–4715. doi:10.1023/A:1023208217925, http://dx.doi.org/10.1023/A%3A1023208217925.

41. Crooks, G. E. (1999). Entropy production fluctuation theorem and the nonequilibriumwork relation for free energy differences. Physical Review E, 60, 2721–2726. doi:10.1103/PhysRevE.60.2721, http://link.aps.org/doi/10.1103/PhysRevE.60.2721.

42. Crooks, G. E. (2000). Path-ensemble averages in systems driven far from equilibrium. PhysicalReview E, 61, 2361–2366. doi:10.1103/PhysRevE.61.2361, http://link.aps.org/doi/10.1103/PhysRevE.61.2361.

References 11

43. Hummer, G., & Szabo, A. (2001). Free energy reconstruction from nonequilibrium single-molecule pulling experiments. Proceedings of the National Academy of Sciences, 98(7), 3658–3661. doi:10.1073/pnas.071034098, http://www.pnas.org/content/98/7/3658.full.pdf+html,http://www.pnas.org/content/98/7/3658.abstract.

44. Braun, O., Hanke, A., & Seifert, U. (2004). Probing molecular free energy landscapes byperiodic loading. Physical Review Letters, 93, 158105. doi:10.1103/PhysRevLett.93.158105,http://link.aps.org/doi/10.1103/PhysRevLett.93.158105.

45. Collin, D., Ritort, F., Jarzynski, C., et al. (2005). Verification of the crooks fluctuation theoremand recovery of RNA folding free energies. Nature, 473, 231. http://dx.doi.org/10.1038/nature04061M3.

46. Ritort, F. (2004). Poincaré Seminar 2003: Bose-Einstein condensation—entropy, chapter workfluctuations, transient violations of the second law and free-energy recovery methods: Per-spectives in theory and experiments (pp. 63–87). Basel: Birkhäuser. doi:10.1007/978-3-0348-7932-3_9.

47. Mossa, A., Manosas, M., Forns, N., et al. (2009). Dynamic force spectroscopy of DNA hair-pins: I. force kinetics and free energy landscapes. Journal of Statistical Mechanics: Theory andExperiment, 2009(nr. 02), P02060. http://stacks.iop.org/1742-5468/2009/i=02/a=P02060.

48. Ciliberto, S., Joubaud, S., & Petrosyan, A. (2010). Fluctuations in out-of-equilibrium sys-tems: From theory to experiment. Journal of Statistical Mechanics: Theory and Experiment,2010(nr. 12), P12003. http://stacks.iop.org/1742-5468/2010/i=12/a=P12003.

49. Bochkov, G., & Kuzovlev, Y. (1981). Nonlinear fluctuation-dissipation relations and stochas-tic models in nonequilibrium thermodynamics: I. generalized fluctuation-dissipation theo-rem. Physica A: Statistical Mechanics and its Applications, 106(3), 443–479. ISSN 0378–4371, doi:10.1016/0378-4371(81)90122-9, http://www.sciencedirect.com/science/article/pii/0378437181901229.

50. Bochkov, G., & Kuzovlev, Y. (1981). Nonlinear fluctuation-dissipation relations and stochas-tic models in nonequilibrium thermodynamics: II. Kinetic potential and variational princi-ples for nonlinear irreversible processes. Physica A: Statistical Mechanics and its Applica-tions, 106(3), 480–520. ISSN 0378–4371, doi:10.1016/0378-4371(81)90123-0, http://www.sciencedirect.com/science/article/pii/0378437181901230.

51. Hatano, T., & Sasa, S.-I. (2001). Steady-state thermodynamics of Langevin systems. PhysicalReview Letters, 86, 3463–3466. doi:10.1103/PhysRevLett.86.3463, http://link.aps.org/doi/10.1103/PhysRevLett.86.3463.

52. Sekimoto, K. (1997). Kinetic characterization of heat bath and the energetics of thermal ratchetmodels. Journal of the Physical Society of Japan, 66(5), 1234–1237. doi:10.1143/JPSJ.66.1234, http://jpsj.ipap.jp/link?JPSJ/66/1234/.

53. Sekimoto, K. (1998). Langevin equation and thermodynamics. Progress of TheoreticalPhysics Supplement, 130, 17–27. doi:10.1143/PTPS.130.17, http://ptps.oxfordjournals.org/content/130/17.full.pdf+html, http://ptps.oxfordjournals.org/content/130/17.abstract.

54. Sekimoto, K. (2010). Stochastic energetics. Lecture notes in physics, Springer. ISBN9783642054488. http://books.google.cz/books?id=4cyxd7bvZHgC.

55. Maes, C. (2003). On the origin and the use of fluctuation relations for the entropy. SéminairePoincaré, 2, 29–62.

56. Maes, C., & Netocný, K. (2003). Time-reversal and entropy. Journal of Statistical Physics,110(1–2), 269–310. ISSN 0022–4715. doi:10.1023/A:1021026930129, http://dx.doi.org/10.1023/A/%3A1021026930129.

57. Qian, H. (2001). Mesoscopic nonequilibrium thermodynamics of single macromoleculesand dynamic entropy-energy compensation. Physical Review E, 65, 016102. doi:10.1103/PhysRevE.65.016102, http://link.aps.org/doi/10.1103/PhysRevE.65.016102.

58. Seifert, U. (2005). Entropy production along a stochastic trajectory and an integral fluctuationtheorem. Physical Review Letters, 95, 040602. doi:10.1103/PhysRevLett.95.040602, http://link.aps.org/doi/10.1103/PhysRevLett.95.040602.

59. Seifert, U. (2008). Stochastic thermodynamics: principles and perspectives. The EuropeanPhysical Journal B—Condensed Matter and Complex Systems, 64, 423–431. ISSN 1434–6028, doi:10.1140/epjb/e2008-00001-9, http://dx.doi.org/10.1140/epjb/e2008-00001-9.

12 1 Introduction

60. Van Den Broeck, C. (1986). Stochastic thermodynamics. In W. Ebeling & H. Ulbricht (Eds.),Selforganization by nonlinear irreversible processes (Vol. 33 pp. 57–61). Berlin Heidelberg:Springer Series in Synergetics, Springer. ISBN 978-3-642-71006-3. doi:10.1007/978-3-642-71004-9_6, http://dx.doi.org/10.1007/978-3-642-71004-9_6.

61. Mou, C. Y., li Luo, J., & Nicolis, G. (1986). Stochastic thermodynamics of nonequilibriumsteady states in chemical reaction systems. The Journal of Chemical Physics, 84(12), 7011–7017.

62. Evans, D. J., & Searles, D. J. (2002). The fluctuation theorem. Advances inPhysics, 51(7), 529–1585. doi:10.1080/00018730210155133, http://www.tandfonline.com/doi/pdf/10.1080/00018730210155133, http://www.tandfonline.com/doi/abs/10.1080/00018730210155133.

63. Gammaitoni, L., Hänggi, P., Jung, P., et al. (1998). Stochastic resonance. Reviews of Mod-ern Physics, 70, 223–287. doi:10.1103/RevModPhys.70.223, http://link.aps.org/doi/10.1103/RevModPhys.70.223.

64. Chvosta, P., & Reineker, P. (2003). Analysis of stochastic resonances. Physical Review E,68, 066109. doi:10.1103/PhysRevE.68.066109, http://link.aps.org/doi/10.1103/PhysRevE.68.066109.

65. Jung, P., & Hänggi, P. (1990). Resonantly driven Brownian motion: Basic concepts and exactresults. Physical Review A, 41, 2977–2988. doi:10.1103/PhysRevA.41.2977, http://link.aps.org/doi/10.1103/PhysRevA.41.2977.

66. Jung, P., Hänggi, P. (1991). Amplification of small signals via stochastic resonance. PhysicalReview A, 44, 8032–8042. doi:10.1103/PhysRevA.44.8032, http://link.aps.org/doi/10.1103/PhysRevA.44.8032.

67. Astumian, R. D. (1997). Thermodynamics and kinetics of a brownian motor. Sci-ence, 276(5314), 917–922. doi:10.1126/science.276.5314.917, http://www.sciencemag.org/content/276/5314/917.full.pdf, http://www.sciencemag.org/content/276/5314/917.abstract.

68. Astumian, R. D., & Hanggi, P. (2002). Brownian motors. Physics Today, 55(11), 33–39.doi:10.1063/1.1535005. http://link.aip.org/link/?PTO/55/33/1.

69. Hänggi, P., Marchesoni, F., & Nori, F. (2005). Brownian motors. Annalen der Physik, 14(1–3), 51–70. ISSN 1521–3889, doi:10.1002/andp.200410121, http://dx.doi.org/10.1002/andp.200410121.

70. Allahverdyan, A. E., Johal, R. S., & Mahler, G. (2008). Work extremum principle: Structureand function of quantum heat engines. Physical Review E, 77, 041118. doi:10.1103/PhysRevE.77.041118, http://link.aps.org/doi/10.1103/PhysRevE.77.041118.

71. Van den Broeck, C., Kawai, R., & Meurs, P. (2004). Microscopic analysis of a thermal brown-ian motor. Physical Review Letters, 93, 090601. doi:10.1103/PhysRevLett.93.090601, http://link.aps.org/doi/10.1103/PhysRevLett.93.090601.

72. Sekimoto, K., Takagi, F., & Hondou, T. (2000). Carnot’s cycle for small systems: Irreversibil-ity and cost of operations. Physical Review E, 62, 7759–7768. doi:10.1103/PhysRevE.62.7759http://link.aps.org/doi/10.1103/PhysRevE.62.7759.

73. Parrondo, J., & de Cisneros, B. (2002). Energetics of brownian motors: A review. AppliedPhysics A, 75(2), 179–191. ISSN 0947–8396. doi:10.1007/s003390201332, http://dx.doi.org/10.1007/s003390201332.

74. Takagi, F., & Hondou, T. (1999). Thermal noise can facilitate energy conversion by a ratchetsystem. Physical Review E, 60, 4954–4957, doi:10.1103/PhysRevE.60.4954, http://link.aps.org/doi/10.1103/PhysRevE.60.4954.

75. Gunawardena, J. (2014). Time-scale separation—michaelis and menten’s old idea, still bear-ing fruit. FEBS Journal, 281(2), 473–488. ISSN 1742–4658, doi:10.1111/febs.12532, http://dx.doi.org/10.1111/febs.12532.

76. Thomas, P., Grima, R., & Straube, A. V. (2012). Rigorous elimination of fast stochastic vari-ables from the linear noise approximation using projection operators. Physical Review E,86, 041110. doi:10.1103/PhysRevE.86.041110, http://link.aps.org/doi/10.1103/PhysRevE.86.041110.

References 13

77. Berglund, N., & Gentz, B. (2006). Noise-induced Phenomena in slow-fast dynamical systems.Probability and Its Applications, Springer London. ISBN 978-1-84628-186-0. http://www.springer.com/mathematics/probability/book/978-1-84628-038-2.

78. Harris, R. J., & Schütz, G. M. (2007). Fluctuation theorems for stochastic dynamics. Journalof Statistical Mechanics: Theory and Experiment, 2007(nr. 07), P07020. http://stacks.iop.org/1742-5468/2007/i=07/a=P07020.

79. Gaspard, P. (2006). Hamiltonian dynamics, nanosystems, and nonequilibrium statisticalmechanics. Physica A: Statistical Mechanics and its Applications, 369(1), 201–246. ISSN0378–4371, doi:http://dx.doi.org/10.1016/j.physa.2006.04.010, http://www.sciencedirect.com/science/article/pii/S0378437106004055.

80. Evans, D., & Morriss, G. (2008). Statistical mechanics of nonequilibrium liquids. Theoret-ical chemistry, Cambridge University Press. ISBN 9780521857918. http://books.google.cz/books?id=65URS_vPwuQC.

81. Esposito, M., & Mukamel, S. (2006). Fluctuation theorems for quantum master equations.Physical Review E, 73, 046129. doi:10.1103/PhysRevE.73.046129, http://link.aps.org/doi/10.1103/PhysRevE.73.046129.

82. Esposito, M., Harbola, U., & Mukamel, S. (2009). Nonequilibrium fluctuations, fluc-tuation theorems, and counting statistics in quantum systems. Reviews of ModernPhysics, 81, 1665–1702. doi:10.1103/RevModPhys.81.1665http://link.aps.org/doi/10.1103/RevModPhys.81.1665.

83. Campisi, M., Hänggi, P., & Talkner, P. (2011). Colloquium : Quantum fluctuation rela-tions: Foundations and applications. Reviews of Modern Physics, 83, 771–791, doi:10.1103/RevModPhys.83.771, http://link.aps.org/doi/10.1103/RevModPhys.83.771.

84. Cohen-Tannoudji, C., Diu, B., & Laloe, F. (1996). Quantum mechanics. Wiley. ISBN9780471569527. http://books.google.cz/books?id=CjeNnQEACAAJ.

85. Ritort, F., Bustamante, C., & Tinoco, I. (2002). A two-state kinetic model for the unfold-ing of single molecules by mechanical force. Proceedings of the National Academy of Sci-ences, 99(21), 13544–13548. doi:10.1073/pnas.172525099, http://www.pnas.org/content/99/21/13544.full.pdf+html, http://www.pnas.org/content/99/21/13544.abstract.

86. Zuckerman, D. M., & Woolf, T. B. (2002). Theory of a systematic computational error infree energy differences. Physical Review Letters, 89, 180602. doi:10.1103/PhysRevLett.89.180602, http://link.aps.org/doi/10.1103/PhysRevLett.89.180602.

87. Gore, J., Ritort, F., & Bustamante, C. (2003). Bias and error in estimates of equilibrium free-energy differences from nonequilibrium measurements. Proceedings of the National Academyof Sciences, 100(22), 12564–12569. doi:10.1073/pnas.1635159100, http://www.pnas.org/content/100/22/12564.full.pdf+html. http://www.pnas.org/content/100/22/12564.abstract.

88. Engel, A. (2009). Asymptotics of work distributions in nonequilibrium systems. PhysicalReview E, 80, 021120. doi:10.1103/PhysRevE.80.021120, http://link.aps.org/doi/10.1103/PhysRevE.80.021120.

89. Nickelsen, D., & Engel, A. (2011). Asymptotics of work distributions: The pre-exponentialfactor. The European Physical Journal B, 82, 207–218. ISSN 1434–6028, doi:10.1140/epjb/e2011-20133-y, http://dx.doi.org/10.1140/epjb/e2011-20133-y.

90. Palassini, M., Ritort, F. (2011). Improving free-energy estimates from unidirectional workmeasurements: theory and experiment. Physical Review Letters, 107, 060601. doi:10.1103/PhysRevLett.107.060601, http://link.aps.org/doi/10.1103/PhysRevLett.107.060601.

91. Chatelain, C., & Karevski, D. (2006). Probability distributions of the work in the two-dimensional Ising model. Journal of Statistical Mechanics: Theory and Experiment, 2006(nr.06), P06005. http://stacks.iop.org/1742-5468/2006/i=06/a=P06005.

92. Híjar, H., Quintana-H, J., & Sutmann, G. (2007). Non-equilibrium work theorems for the two-dimensional Ising model. Journal of Statistical Mechanics: Theory and Experiment, 2007(nr.04), P04010. http://stacks.iop.org/1742-5468/2007/i=04/a=P04010.

93. Chvosta, P., Reineker, P., & Schulz, M. (2007). Probability distribution of work done on atwo-level system during a nonequilibrium isothermal process. Physical Review E, 75, 041124.doi:10.1103/PhysRevE.75.041124, http://link.aps.org/doi/10.1103/PhysRevE.75.041124.

14 1 Introduction

94. Šubrt, E., & Chvosta, P. (2007). Exact analysis of work fluctuations in two-level systems.Journal of Statistical Mechanics: Theory and Experiment, 2007(nr. 09), P09019. http://stacks.iop.org/1742-5468/2007/i=09/a=P09019.

95. Einax, M., & Maass, P. (2009). Work distributions for Ising chains in a time-dependentmagnetic field. Physical Review E, 80, 020101. doi:10.1103/PhysRevE.80.020101, http://link.aps.org/doi/10.1103/PhysRevE.80.020101.

96. Chvosta, P., Einax, M., Holubec, V., et al. (2010). Energetics and performance of a microscopicheat engine based on exact calculations of work and heat distributions. Journal of StatisticalMechanics: Theory and Experiment, 2010(nr. 03), P03002. http://stacks.iop.org/1742-5468/2010/i=03/a=P03002.

97. Mazonka, O., & Jarzynski, C. (1999). Exactly solvable model illustrating far-from-equilibriumpredictions. eprint arXiv:cond-mat/9912121, December 1999, arXiv:cond-mat/9912121.http://arxiv.org/abs/cond-mat/9912121.

98. Baule, A., & Cohen, E. G. D. (2009). Fluctuation properties of an effective nonlinear systemsubject to Poisson noise. Physical Review E, 79, 030103. doi:10.1103/PhysRevE.79.030103,http://link.aps.org/doi/10.1103/PhysRevE.79.030103.

99. Manosas, M., Mossa, A., Forns, N., et al. (2009). Dynamic force spectroscopy of DNA hair-pins: II. irreversibility and dissipation. Journal of Statistical Mechanics: Theory and Experi-ment, 2009(nr. 02), P02061. http://stacks.iop.org/1742-5468/2009/i=02/a=P02061.

100. Verley, G., Van den Broeck, C., & Esposito, M. (2013). Modulated two-level system: Exactwork statistics. Physical Review E, 88, 032137. doi:10.1103/PhysRevE.88.032137, http://link.aps.org/doi/10.1103/PhysRevE.88.032137.

101. Ryabov, A., Dierl, M., Chvosta, P., et al. (2013). Work distribution in a time-dependentlogarithmic-harmonic potential: exact results and asymptotic analysis. Journal of PhysicsA: Mathematical and Theoretical, 46(nr. 7), 075002. http://stacks.iop.org/1751-8121/46/i=7/a=075002.

102. Speck, T. (2011). Work distribution for the driven harmonic oscillator with time-dependentstrength: Exact solution and slow driving. Journal of Physics A: Mathematical and Theoreti-cal, 44(nr. 30), 305001. http://stacks.iop.org/1751-8121/44/i=30/a=305001.

103. van Zon, R., & Cohen, E. G. D. (2003). Stationary and transient work-fluctuation theoremsfor a dragged Brownian particle. Physical Review E, 67, 046102. doi:10.1103/PhysRevE.67.046102, http://link.aps.org/doi/10.1103/PhysRevE.67.046102.

104. van Zon, R., & Cohen, E. G. D. (2004). Extended heat-fluctuation theorems for a system withdeterministic and stochastic forces. Physical Review E, 69, 056121. doi:10.1103/PhysRevE.69.056121, http://link.aps.org/doi/10.1103/PhysRevE.69.056121.

105. Cohen, E. G. D. (2008). Properties of nonequilibrium steady states: A path integral approach.Journal of Statistical Mechanics: Theory and Experiment, 2008(nr. 07), P07014. http://stacks.iop.org/1742-5468/2008/i=07/a=P07014.

106. Imparato, A., & Peliti, L. (2005). Work-probability distribution in systems driven out ofequilibrium. Physical Review E, 72, 046114. doi:10.1103/PhysRevE.72.046114, http://link.aps.org/doi/10.1103/PhysRevE.72.046114.

107. Ritort, F. (2004). Work and heat fluctuations in two-state systems: A trajectory thermody-namics formalism. Journal of Statistical Mechanics: Theory and Experiment, 2004(nr. 10),P10016. http://stacks.iop.org/1742-5468/2004/i=10/a=P10016.

108. Schmiedl, T., & Seifert, U. (2008). Efficiency at maximum power: An analytically solvablemodel for stochastic heat engines. EPL (Europhysics Letters), 81(nr. 2), 20003. http://stacks.iop.org/0295-5075/81/i=2/a=20003.

109. Henrich, M. J., Rempp, F., & Mahler, G. (2007). Quantum thermodynamic Otto machines:A spin-system approach. The European Physical Journal Special Topics, 151(1), 157–165.ISSN 1951–6355, doi:10.1140/epjst/e2007-00371-8, http://dx.doi.org/10.1140/epjst/e2007-00371-8.

110. Abah, O., Roßnagel, J., Jacob, G., et al. (2012). Single-Ion heat engine at maximum power.Physical Review Letters, 109, 203006. doi:10.1103/PhysRevLett.109.203006http://link.aps.org/doi/10.1103/PhysRevLett.109.203006.

References 15

111. Blickle, V., & Bechinger, C. (2011). Realization of a micrometre-sized stochastic heatengine. Nature Physics, 8(2), 143–146. doi:10.1038/nphys2163, http://dx.doi.org/10.1038/nphys2163.

112. Esposito, M., Kawai, R., Lindenberg, K., et al. (2010). Efficiency at maximum power of low-dissipation carnot engines. Physical Review Letters, 105, 150603. doi:10.1103/PhysRevLett.105.150603, http://link.aps.org/doi/10.1103/PhysRevLett.105.150603.

113. Esposito, M., Lindenberg, K., & Van den Broeck, C. (2009). Universality of efficiency at max-imum power. Physical Review Letters, 102, 130602. doi:10.1103/PhysRevLett.102.130602,http://link.aps.org/doi/10.1103/PhysRevLett.102.130602.

114. Zhan-Chun, T. (2012). Recent advance on the efficiency at maximum power of heat engines.Chinese Physics B, 21(nr. 2), 020513. http://stacks.iop.org/1674-1056/21/i=2/a=020513.

115. Novikov, I. I. (1958). The efficiency of atomic power stations. Journal of Nuclear Energy II,7, 125.

116. Curzon, F. L., & Ahlborn, B. (1975). Efficiency of a Carnot engine at maximum poweroutput. American Journal of Physics, 43(1), 22–24. doi:10.1119/1.10023, http://link.aip.org/link/?AJP/43/22/1.

117. Holubec, V., Chvosta, P., & Maass, P. (2012). Dynamics and energetics for a molecular zippermodel under external driving. Journal of Statistical Mechanics: Theory and Experiment,2012(nr. 11), P11009. http://stacks.iop.org/1742-5468/2012/i=11/a=P11009.

118. Holubec, V., Chvosta, P., & Ryabov, A. (2010). Thermodynamics, chapter four exactly solvableexamples in non-equilibrium thermodynamics of small systems. InTech, 153–176, doi:10.5772/13374. http://www.intechopen.com/books/thermodynamics.

119. Chvosta, P., Holubec, V., Ryabov, A., et al. (2010). Thermodynamics of two-stroke enginebased on periodically driven two-level system. Physica E: Low-dimensional Systems andNanostructures, 42(3), 472–476. ISSN 1386–9477. doi:http://dx.doi.org/10.1016/j.physe.2009.06.031, Proceedings of the International Conference Frontiers of Quantum andMesoscopic Thermodynamics FQMT ’08, http://www.sciencedirect.com/science/article/pii/S1386947709002380.

120. Holubec, V., Chvosta, P., Einax, M., et al. (2011). Attempt time monte carlo: An alterna-tive for simulation of stochastic jump processes with time-dependent transition rates. EPL(Europhysics Letters), 93(nr. 4), 40003. http://stacks.iop.org/0295-5075/93/i=4/a=40003.

Chapter 2Stochastic Thermodynamics

Stochastic thermodynamics is an advancing field with many applications to smallsystems of current interest [1–5]. Its brief history is summarized in the Introduction.In this chapter we review the basic theoretical concepts involved in this domain.Specifically, in Sects. 2.1 and 2.2, we review the dynamical equations for the meso-scopic systems with discrete and with continuous state space, respectively. Further,in these two sections, we present equations which describe fluctuations of work, heatand internal energy of the system during isothermal processes. In Sect. 2.3 we give ashort review of the celebrated work fluctuation relations [1, 6–9], which represent ageneralization of the second law of thermodynamics. Finally, in Sect. 2.4, we reviewthe recent results in the field of stochastic heat engines and present the theory neededto their analytical description.

2.1 Discrete Systems

Let us consider a general N -level system in contact with a thermal reservoir atthe temperature T . We assume that the system is driven by an external agent. LetGi = Ui −kBT log �i denote free energy landscape (FEL) of the system alone, whereUi (�i ) stands for the energies (degeneracies) of the individual levels and kB is theBoltzmann constant. Moreover, let Vi [Y(t)] = Vi (t) denote an additional energydue to the interaction with the external agent. The discrete index i = 1, 2, . . . , Nlabels the individual microstates (levels) available to the system and Y(t) stands for avector of control parameters driven by the external agent. The FEL of the compoundsystem, Fi (t), and the energies of its individual levels, Ei (t), read

Fi (t) = Gi + Vi [Y(t)] = Gi + Vi (t) i = 1, 2, . . . , N , (2.1)

Ei (t) = Ui + Vi [Y(t)] = Ui + Vi (t) i = 1, 2, . . . , N . (2.2)

V. Holubec, Non-equilibrium Energy Transformation Processes, Springer Theses, 17DOI: 10.1007/978-3-319-07091-9_2, © Springer International Publishing Switzerland 2014

18 2 Stochastic Thermodynamics

Note that the external driving can not force the system to follow a specific sequenceof microstates. It just influences, through the level free energies, the transition prob-abilities between the individual microstates.

2.1.1 Dynamics

At an arbitrary fixed time t , the state of the system is specified by the column vectorp(t). The components of this vector [p(t)]i = pi (t), i = 1, . . . , N , are the occupationprobabilities of the individual microstates (levels), i.e., the probabilities to find thesystem at the time t in the microstate i .

In many experimentally important situations, the time evolution of the system canbe described as a Markov process. Such process is governed by the master equation[10]. The transition rates in the equation depend on the temperature of the bathand on the external parameters which influence the FEL of the system. Since thefree energies depend on time, the rates must be time dependent as well. Hence theunderlying Markov process is time inhomogeneous one.

One possible probabilistic approach [11, 12] to the analysis of a continuous timeMarkov process uses its decomposition into a discrete time Markov chain and asystem of random points on the time axis. The transitions between the states of theMarkov chain occur just at random instants. Usually, the time intervals betweenindividual transitions are taken as independent exponentially distributed randomvariables. This decomposition can be used for simple and fast simulations of timeinhomogeneous Markov processes. The details are discussed in Appendix C.

Formally speaking, the time evolution of the system is described by the time-inhomogeneous Markov process D(t), where D(t) = k if the system resides in themicrostate k at the time t . The master equation can be written as

d

dtR(t | t ′) = −νL(t)R(t |t ′), R(t ′ | t ′) = 1, (2.3)

where 1 is the (N × N ) unity matrix, L(t) is the (N × N ) matrix of transition rates,andR(t | t ′) is the (N × N ) matrix of transition probabilities. ν−1 sets the elementarytime scale. In the limit ν → ∞ (ν → 0) the relaxation processes are infinitely fast(slow).

If the transition rates would remain constant, the system should relax towardsthermal equilibrium specified by the instantaneous values of the level free energies(2.1). In order to guarantee this behavior it is often assumed that the transition ratesfulfill the so called (time local) detail balance condition

exp[−βF j (t)]Li j (t) = L ji (t) exp[−βFi (t)], (2.4)

2.1 Discrete Systems 19

where β = 1/(kBT ). The matrix of the transition probabilities, R(t | t ′), is a stochas-tic matrix [13]. Its elements are given by

Ri j (t | t ′) = Prob{

D(t) = i | D(t ′) = j}. (2.5)

This means that the matrix R(t | t ′) evolves an arbitrary column vector of the initialoccupation probabilities, p(t ′), as

p(t) = R(t | t ′) p(t ′). (2.6)

Because this formula is valid for any initial vector p(t ′), it already suggests that thematrix R(t | t ′) fulfills the Chapman-Kolmogorov condition [10, 14]

R(t | t ′) = R(t | t ′′)R(t ′′ | t ′). (2.7)

Here the matrix multiplication on the right hand side amounts for the summationover the intermediate states at the time t ′′. This formula is valid for any intermediatetime t ′′ and reflects the Markov property of the underlying stochastic process D(t).

2.1.2 Energetics

The random internal energy of the system at the time t ,

U(t) = ED(t)(t), (2.8)

changes according to the first law of thermodynamics as

U(t) dt = W(t + dt, t) + Q(t + dt, t)

= [ED(t)(t + dt) − ED(t)(t)] + [ED(t+dt)(t) − ED(t)(t)

], (2.9)

where we use the abbreviation f (t) = d f (t)/dt . The first (the second) term on theright hand side represents the random work (heat) accepted by the system duringthe infinitesimal time interval [t, t + dt] [15]. Differently speaking, if the systemdwells in the level i during the time interval [t, t + dt] then the work done on thesystem by the external agent, W(t + dt, t), equals Ei (t + dt) − Ei (t) and no heatis accepted. On the other hand if the system changes its microstate from j to iduring the infinitesimal time interval [t, t + dt] and the level energies Ei (t), E j (t)remain constant, then the heat accepted by the system from the thermal environment,Q(t + dt, t), is Ei (t) − E j (t) and no work is done. These definitions are furtherdiscussed in detail in Sect. 2.2.3, also see Fig. 2.1.

The random work done on the system by the external agent when the controlparameter is altered from Y(t ′) to Y(t) reads

20 2 Stochastic Thermodynamics

W(t, t ′) =t∫

t ′dt ′′

N⎡

i=1

Ei (t′′)δiD(t ′′), (2.10)

and represents a functional of the stochastic process D(t). Due to the first law of ther-modynamics (2.9), the random heat accepted from the reservoir represents anotherfunctional of the process D(t), specifically

Q(t, t ′) = U(t) − U(t ′) − W(t, t ′). (2.11)

For an analytical treatment of work and heat fluctuations it is useful to introducethe augmented process

{W(t, t ′), D(t)

}[16–18] which describes both the work and

the microstate variable. This augmented process is again a time inhomogeneousMarkov process and its two-time properties are described by the (N × N ) matrixG(w, t | w′, t ′) with the matrix elements

Gi j (w, t | w′, t ′) = 1

σProb

⎣W(t, 0) ∈ (w,w + σ)

D(t) = i

∣∣∣∣

W(t ′, 0) = w′

D(t ′) = j

⎤, (2.12)

where σ → 0. The time evolution of G(w, t | w′, t ′) is given by [16–18]

∂

∂tG(w, t | w′, t ′) = −

⎦∂

∂wE(t) + νL(t)

⎫G(w, t | w′, t ′) (2.13)

with the initial condition G(w, t ′ | w′, t ′) = δ(w − w′)1. Here E(t) is the diagonalmatrix E(t) = diag{E1(t), . . . , EN (t)}. Note that R(t | t ′) = ⎬ ∞

−∞ dwG(w, t | 0, t ′)and that G(w, t | w′, t ′) = G(w − w′, t | 0, t ′). Equation (2.13) represents a hyper-bolic system of N 2 coupled partial differential equations with time-dependent coef-ficients. It can be derived in several ways. For example, as explained in reference[19], one considers at the time t the family of all realizations, which display at thattime the work in the infinitesimal interval (w,w + dw) and, simultaneously, whichoccupy a given microstate. During the infinitesimal time interval (t, t +dt), the num-ber of such paths can change due to two reasons. First, while residing in the givenmicrostate, some paths enter (leave) the set, because the energy levels move and anadditional work has been done. Secondly, some paths can enter (leave) the describedfamily, because they jump out of (into) the specified level. These two contributionscorrespond to the two terms on the right hand side of Eq. (2.13). Another derivation[18] is based on an explicit probabilistic construction of all possible paths and theirrespective probabilities.

The Chapman-Kolmogorov condition for the augmented process assumes theform

G(w, t | w′, t ′) =∞∫

−∞dw′′ G(w, t | w′′, t ′′)G(w′′, t ′′ | w′, t ′). (2.14)

2.1 Discrete Systems 21

Here the matrix multiplication on the right hand side amounts for the summation overthe intermediate states at the time t ′′, and the integration runs over all possible inter-mediate values of the work variable w′′. The equation is valid for any intermediatetime t ′′ ∈ [t ′, t].

Equations (2.3) and (2.13) are exactly solvable only in several cases. Investigationshave been conducted for simple spin systems driven by a time-dependent externalfield [11, 18–25]. As a result, analytical solutions are known for several two-levelsystems [11, 18, 23, 24, 26] and for systems composed of Nsub two-level subsystemsin the limit of large Nsub [19, 25]. A generalization of these results and also someother exactly solvable models are presented in Chap. 3. Further, in Appendix A, wediscuss the limit of infinitely fast (slow) relaxation (ν → 0, ∞). In these limitingcases Eqs. (2.3) and (2.13) can be solved for any model.

The matrixG(w, t | w′, t ′) provides a complete description of the energetics of theprocess D(t). The joint probability density for the internal energy U(t) and the workW(t, t ′) performed on the system during the time interval [t ′, t], given the internalenergy initially was u′, (regardless of the final state of the system at the time t) isgiven by

ξ(u, w, t; u′) =N⎡

i, j=1

δ [u − Ei (t)] δ[u′ − E j (t

′)]

Gi j (w, t | 0, t ′) p j (t′). (2.15)

Here δ(x) stands for the Dirac δ-function and p j (t ′) denotes the initial occupationprobabilities. The function ξ(u, w, t; u′) already yields the probability density forthe work,

ρ(w, t) =∞∫

−∞du

∞∫

−∞du′ ξ(u, w, t; u′). (2.16)

An analogous integration over the variables u′ and w gives the probability densityfor the internal energy U(t),

ς(u, t) =∞∫

−∞du′

∞∫

−∞dw ξ(u, w, t; u′). (2.17)

Furthermore, using the definition (2.11), the function ξ(u, w, t; u′) gives also theprobability density for the heat Q(t, t ′) transferred from the reservoir during thetime interval [t ′, t] [23]

χ(q, t) =∞∫

−∞du

∞∫

−∞du′

∫ ∞

−∞dw δ

[q − ⎭

u − u′ − w)]

ξ(u, w, t; u′). (2.18)

22 2 Stochastic Thermodynamics

The functions (2.16)–(2.18) yield all raw moments of internal energy (2.8), work(2.10) and heat (2.11) by single integration

〈[U(t)]n∝ =∞∫

−∞du un ς(u, t), (2.19)

〈[W(t, t ′)]n∝ =∞∫

−∞dw wn ρ(w, t), (2.20)

〈[Q(t, t ′)]n∝ =∞∫

−∞dq qn χ(q, t). (2.21)

Let us note that these moments can be also calculated directly from the definitions(2.8), (2.10) and (2.11) only using the solution of the master Eq. (2.3). This approach isconvenient for the internal energy where one can use the identity

⎬ ∞−∞ du un ς(u, t) =

∑Ni=1[Ei (t)]n pi (t). For the work and the heat the situation is more complicated and

this approach is useful only for the first few moments. For example, the mean valuesof the internal energy, work and heat, are

U (t) = 〈U(t)∝ =N⎡

i=1

Ei (t)pi (t), (2.22)

W (t, t ′) = 〈W(t, t ′)∝ =N⎡

i=1

t∫

t ′dt ′′ Ei (t

′′)pi (t′′), (2.23)

Q(t, t ′) = 〈Q(t, t ′)∝ =N⎡

i=1

t∫

t ′dt ′′ Ei (t

′′) pi (t′′). (2.24)

In order to calculate the second moments of the work and the heat without theprobability densities (2.16) and (2.18), one needs the two-time correlation function

〈hD(t) fD(t ′)∝C =

⎧⎪⎪⎪⎨

⎪⎪⎪⎩

N∑

i=1

N∑

j=1hi f j Ri j (t | t ′)p j (t ′), t ≥ t ′

N∑

i=1

N∑

j=1fi h j Ri j (t ′ | t)p j (t), t ≤ t ′

. (2.25)

The second raw moments of the work (2.10) and the heat (2.11) are then given by

2.1 Discrete Systems 23

〈[W(t, t ′)]2∝ =t∫

t ′dt ′′

t∫

t ′dt ′′′

⎝∂ED(t ′′)(t ′′)

∂t ′′∂ED(t ′′′)(t ′′′)

∂t ′′′

⎞

C

(2.26)

and

〈[Q(t, t ′)]2∝ = 〈[W(t, t ′)]2∝ +N⎡

i, j=1

[Ei (t) − E j (t′)]2

Ri j (t | t ′)p(t ′)

− 2N⎡

i, j, k=1

[Ei (t) − Ek(t′)]

t∫

t ′dt ′′

∂ED(t ′′)(t ′′)∂t ′′

δD(t ′′) j Ri j (t | t ′′)R jk(t′′ | t ′)pk(t

′),

(2.27)

where ∂ED(t)(t)/∂t ≡ dEi (t)/dt |i=D(t). Obviously, in order to obtain the highermoments of the work and the heat, the higher time correlation functions must bedefined and the resulting expressions become too complicated. Finally, let us definethe variances of internal energy, heat and work which corresponds to the widths ofthe individual probability densities

[�U (t)]2 = 〈[U(t)]2∝ − [U (t)]2, (2.28)

[�W (t, t ′)]2 = 〈[W(t, t ′)]2∝ − [W (t, t ′)]2, (2.29)

[�Q(t, t ′)]2 = 〈[Q(t, t ′)]2∝ − [Q(t, t ′)]2. (2.30)

The entropy of the system at the time t is given by the standard formula [15]

Ss(t) = −kB

N⎡

i=1

pi (t) log [pi (t)] . (2.31)

Its increment during the time interval [t ′, t]

Ss(t, t ′) = Ss(t) − Ss(t′), (2.32)

together with the entropy transferred to the reservoirs during the time interval [t ′, t]

Sr(t, t ′) = − Q(t, t ′)T

, (2.33)

determines the total entropy produced during the time interval [t ′, t]

Stot(t, t ′) = Ss(t, t ′) + Sr(t, t ′) ≥ 0. (2.34)

24 2 Stochastic Thermodynamics

2.2 Continuous Systems

Consider an externally driven one dimensional microscopic system in contact witha thermal reservoir at the temperature T . Let G(x) = U(x) − kBT log �(x) denotethe FEL of the system alone, where U(x) denotes the energy of the x th microstateand �(x) stands for its multiplicity. Moreover, let V [x, Y(t)] = V(x, t) denotean additional energy due to the interaction with the external agent. The continuousindex x ∈ (−∞,∞) labels the individual microstates available to the system andY(t) stands for the vector of the parameters controlled by the external agent. As inthe discrete case, the external driving can not force the system to follow a specificsequence of microstates. It just influences, through the FEL, the transition probabili-ties between the individual microstates. The FEL, F(x, t), and the energy landscape,E(x, t), of the compound system read

F(x, t) = G(x) + V [x, Y(t)] = G(x) + V(x, t), (2.35)

E(x, t) = U(x) + V [x, Y(t)] = U(x) + V(x, t). (2.36)

Two examples of such setting are depicted in Figs. 4.1 and 4.2 and further discussedin Sect. 4.1.

2.2.1 Dynamics

In the present work we assume that the thermal forces can be described as the sum ofthe linear friction force and the Langevin white-noise force. We neglect the inertialforces. Then the equation of motion for the particle position (microstate of the system)is the stochastic differential equation [10, 14, 27]

�d

dtX(t) = − ∂

∂xF(x, t)

∣∣∣∣x=X(t)

+ N(t), (2.37)

with initial condition X(t ′) = x ′. In Eq. (2.37) � stands for the particle mass timesthe viscous friction coefficient, and N(t) represents the delta-correlated white noise,〈N(t)N(t ′)∝ = 2D�2 δ(t − t ′). Here D = kBT/� denotes the diffusion constant.The Fokker-Planck equation corresponding to the stochastic differential Eq. (2.37)is [10, 14, 27]

∂

∂tR(x, t | x ′, t ′) =

⎣D

∂2

∂x2 + 1

�

∂

∂x

⎦∂

∂xF(x, t)

⎫ ⎤R(x, t | x ′, t ′), (2.38)

with the initial condition R(x, t ′ | x ′, t ′) = δ(x − x ′). The conditional probabilitiesR(x, t | x ′, t ′) dx play the same role as the matrix elements (2.5) for the discreteprocess D(t). More precisely, the function R(x, t | x ′, t ′) evolves an arbitrary initial

2.2 Continuous Systems 25

probability distribution p(x, t ′) as

p(x, t) =∞∫

−∞dx ′ R(x, t | x ′, t ′) p(x ′, t ′). (2.39)

The function p(x, t) specifies the state of the system at the time t . Concretely, theprobability to find the system at the time t in a microstate which lies in an infinitesimalneighborhood of the microstate x reads p(x, t) dx , or, mathematically, Prob{X(t) ∈(x, x + dx)} = p(x, t) dx .

This setting represents a continuous analogy of the discrete one described inSect. 2.1. However, it should be noted that the master equation formulation is moregeneral than the continuous one, including the Langevin and Fokker-Planck cases asits special limits [10, 14]. The matrix multiplication (summation over the microstates)in the discrete model is now represented by x-integration. For example the Chapman-Kolmogorov Eq. (2.7) presently assumes the form

R(x, t | x ′, t ′) =∞∫

−∞dx ′′ R(x, t | x ′′, t ′′)R(x ′′, t ′′ | x ′, t ′). (2.40)

2.2.2 Energetics

The random internal energy of the system at the time t ,

U(t) = E[X(t), t], (2.41)

changes according to the first law of thermodynamics (2.11) as

U(t) dt = W(t + dt, t) + Q(t + dt, t)

= {E[X(t), t + dt] − E[X(t), t]} + {E[X(t + dt), t] − E[X(t), t]} . (2.42)

The first (the second) term on the right hand side corresponds to the random work(heat) accepted by the system during the infinitesimal time interval [t, t + dt] [15].Differently speaking, if the system dwells at the position x during the time interval[t, t + dt] then the work done on the system by the eternal agent, W(t + dt, t),equals E(x, t +dt)−E(x, t) and no heat is accepted. On the other hand if the systemchanges its position from x ′ to x during the infinitesimal time interval [t, t + dt] andthe energies E(x, t), E(x ′, t) remain constant, then the heat accepted by the systemfrom the thermal environment, Q(t +dt, t), is E(x, t)−E(x ′, t) and no work is done.

26 2 Stochastic Thermodynamics

As a result the random work satisfies the stochastic differential equation [28]

d

dtW(t, t ′) = ∂

∂tE[X(t), t] ≡ d

dtE(x, t)

∣∣∣∣x=X(t)

(2.43)

with the initial condition W(t ′, t ′) = 0. Although this work definition, which we usein the whole thesis, naturally stems from the stochastic thermodynamic definition(2.10), it recently raised a heated discussion in the literature [29–32] and an alternativework definition was proposed [29]. This question is discussed in detail in Sect. 2.2.3.

The random work done on the system by the external agent when the controlparameter is altered from Y(t ′) to Y(t) reads

W(t, t ′) =t∫

t ′dt ′′

∞∫

−∞dx E(x, t ′′)δ[x − X(t ′′)]. (2.44)

It represents a functional of the process X(t). Due to the first law of thermodynamics(2.42) the random heat accepted from the reservoir corresponds to another functionalof the process X(t). This functional is given by (2.11).

The Fokker-Planck equation corresponding to the stochastic differential equations(2.37) and (2.43) reads [10, 27]

∂

∂tG(x, w, t | x ′, w′, t ′) =

⎣D

∂2

∂x2 + 1

�

∂

∂x

⎦∂

∂xF(x, t)

⎫

− E(x, t)∂

∂w

⎤G(x, w, t | x ′, w′, t ′),

(2.45)

with the initial condition G(x, w, t ′ | x ′, w′, t ′) = δ(x − x ′)δ(w − w′). The condi-tional probabilities G(x, w, t | x ′, w′, t ′) dx dw play similar role for the continuousprocess X(t) as the matrix elements (2.12) for the discrete process D(t).

Specifically, if we define the function ξ(u, w, t; u′) as

ξ(u, w, t; u′) =∞∫

−∞

∞∫

−∞dx dx ′ δ [u − E(x, t)]

× δ[u′ − E(x ′, t ′)

]G(x, w, t | x ′, 0, t ′) p(x ′, t ′) (2.46)

the definitions (2.16)–(2.21) remain valid. In analogy with the discrete model themoments of the random variables in question can be also calculated from the def-initions (2.41), (2.44) and (2.11) only using the solution of Eq. (2.38). One justsubstitutes the continuous variables in the formulas in question for their discreteequivalents and change the summations to the integrations. The raw moments of

2.2 Continuous Systems 27

the internal energy are now given by⎬ ∞−∞ du un ς(u, t) = ⎬ ∞

−∞ dx [E(x, t)]n p(x, t).The mean values of the internal energy, the work and the heat (2.22)–(2.24) now read

U (t) = 〈U(t)∝ =∞∫

−∞dx E(x, t)p(x, t), (2.47)

W (t, t ′) = 〈W(t, t ′)∝ =∞∫

−∞dx

t∫

t ′dt ′′ E(x, t ′′)p(x, t ′′), (2.48)

Q(t, t ′) = 〈Q(t, t ′)∝ =∞∫

−∞dx

t∫

t ′dt ′′ E(x, t ′′) p(x, t ′′). (2.49)

Similarly, the second moment of the work (2.26) is given by

〈[W(t, t ′)]2∝ =t∫

t ′dt ′′

t∫

t ′dt ′′′

⎝∂E[X(t ′′), t ′′]

∂t ′′∂E[X(t ′′′), t ′′′]

∂t ′′′

⎞

C

, (2.50)

with the two-time correlation function defined as

〈h[X(t)] f [X(t ′)]∝C

=

⎧⎪⎪⎨

⎪⎪⎩

∞⎬−∞

dx∞⎬

−∞dx ′ h(x) f (x ′)R(x, t | x ′, t ′)p(x ′, t ′), t ≥ t ′

∞⎬−∞

dx∞⎬

−∞dx ′ f (x) h(x ′)R(x, t ′ | x ′, t)p(x ′, t), t ≤ t ′.

(2.51)

The entropy of the system at the time t is given by the standard formula [15]

Ss(t) = −kB

∞∫

−∞dx p(x, t) log [p(x, t)] . (2.52)

Its increase during the time interval [t ′, t], Ss(t, t ′), the entropy Sr(t, t ′) transferredto the reservoirs during the time interval [t ′, t] and the total entropy produced duringthe time interval [t ′, t], Stot(t, t ′), are given by Eq. (2.32)–(2.34).

As in the discrete case, Eqs. (2.38) and (2.45) can be solved analytically only forfew simple settings. Examples can be found in the works [4, 33–40]. In Chap. 4 wepresent a generic exactly solvable model—the “sliding parabola model”. Anotherexactly solvable model, an extension of the so called “breathing parabola model”, isdiscussed in Sect. 5.2.

28 2 Stochastic Thermodynamics

2.2.3 On the Work Definition

Recently, an extensive criticism of the work definition has been raised (2.44) [29, 32].The objections against the definition presented in [29] are:

• The work (2.44) done during a quasi-static process does not equal the free energychange of the system itself.

• After a gauge transformation F(x, t) → F(x, t)+g(t), E(x, t) → E(x, t)+g(t)one obtains a transformed system which possess the same dynamics as the originalone, however, the work (2.44) done on the transformed system and that done onthe original differ.

• The work should be defined in the traditional mechanical way as (force × dis-placement). Using this definition, the random work done on the system by theexternal agent when the control parameter is altered from Y(t ′) to Y(t) reads1

WE(t, t ′) = −t∫

t ′dt ′′ ∂

∂xV ⎭

x, t ′′)∣∣∣∣x=X(t ′′)

dX(t ′′)dt ′′

, (2.53)

where −∂V(x, t)/∂x = F(x, t) denotes the force applied on the system by theexternal agent. The work (2.53) done during a quasi-static process equals the freeenergy change of the system itself and is gauge invariant.

The papers [30, 31, 41] explain that the controversies caused by the above objec-tions can be easily resolved. To this end we compare the the two work definitions ona specific, simple, example.

The thermodynamic work represents the work done by the external agent onthe compound system system-interaction [30, 42, 43]. As an example consider thesituation depicted in Fig. 2.1. The microscopic system (the black ball) is driven byan external agent (the car). The interaction between the car and the ball is realized bythe spring with zero unloaded length, which contains the interaction energy V(x, t)and acts on the ball (car) by the force F(x, t) [−F(x, t)]. The ball is surroundedby thermal environment and the impacts of its molecules (small arrows) can causethat the ball moves against the force applied. In such case the energy of the impacts[the heat (2.11)] is stored in the spring and the internal energy of the compoundsystem (ball-spring), E(x, t), increases. If the spring moves the ball, its kinetic energyimmediately dissipates into the environment. Part of the energy E(x, t) is transferredas heat (2.11) into the bath. Another way how to increase (decrease) the energyE(x, t)is to drive the car [externally manipulate with the interaction potential V(x, t)].Assume that the impacts of the molecules from the environment cause the ball tostand still. If one drives the car to the left, the spring must be stretched and the work(2.44) is done on the compound system [E(x, t) increases and the petrol is consumed].

1 Note that the generalization of this definition for the discrete models is not very natural, because,in such case, one has to substitute a finite difference for the position derivative in the force definition.

2.2 Continuous Systems 29

F (x,t)

Fig. 2.1 Sketch illustrating the meaning of thermodynamic work (2.44) and that of the mechanicalwork (2.53)

However, if one drives to the right, the spring pulls the car and the compound systemdoes the work (2.44) on the car [E(x, t) decreases and one can store the energy in anaccumulator].

Contrary to the thermodynamic work (2.44), the mechanical work (2.53) describesthe work done by the external agent on the system itself [30, 42, 43]. Consider againthe situation depicted in Fig. 2.1. If the ball moves towards the car, the energy E(x, t)decreases at the expense of the ball kinetic energy which is quickly (immediately)transferred as heat into the bath. The work (2.53) is done on the ball by the spring, itcauses its movement against the friction. Similarly, if the distance between the balland the car decreases the heat is transferred into the spring and the ball (the thermalenvironment via the ball) does the work on the spring.

From the last two paragraphs one can deduce that the work definition (2.53) issimilar to the definition of heat (2.11). Indeed the mechanical work can be rewritten as

WE(t, t ′) = −V [X(t), t

] + V [X(t ′), t ′

] + W(t, t ′) (2.54)

and thus, using Eqs. (2.11) and (2.36), we have WE(t, t ′) = U [X(t)

] − U [X(t ′)

] −Q(t, t ′). For freely diffusing particle [U(x) = 0] the mechanical work thus equalsthe (minus) heat transferred from the compound system to the thermal environment,WE(t, t ′) = −Q(t, t ′). This is not surprising. If one neglects the inertia and setsU(x) = 0, the only agent which protects the system from moving freely (withoutconsuming work) is the thermal environment.

In [44] the thermodynamic work (2.44) is referred to as the inclusive work. Simi-larly, the mechanical work (2.53) was called the exclusive work. The reason for theseterms is the following. During a quasi-static isothermal process the inclusive workis trajectory independent and it equals the change of the Helmholtz free energy F(t)of the system described by the full free energy landscape (2.35), i.e.,

[W(t, t ′)]eq = �F(t, t ′) = F(t) − F(t ′) = − 1

βln

Z(t)

Z(t ′), (2.55)

where F(t) = − ln[Z(t)]/β and the partition function Z(t) reads

Z(t) =∞∫

−∞dx exp [−βF (x, t)]. (2.56)

30 2 Stochastic Thermodynamics

The free energy obtained from the inclusive work thus includes the terms which stemfrom the interaction with the external agent. A general proof is presented for examplein [30] a proof using Eq. (2.13) for discrete systems is presented in Appendix A. Thegauge transformation F(x, t)+g(t) changes the quasi-static work as [W(t, t ′)]eq →[W(t, t ′)]eq +g(t)−g(t ′) and the free energy difference as �F(t, t ′) → �F(t, t ′)+g(t)− g(t ′), indeed. However, the possibility to insert a function g(t) into F(x, t) isnot unphysical—the work can be done on the system without changing its dynamics.For instance the function g(t) may represent an energy input which is the same forall microstates of the system. Consider that the one dimensional microscopic systemfrom Fig. 2.1 is putted into an elevator placed in a gravitational field. If one neglectsthe inertial effects, the energy flow into the system caused by the elevator movementup and down would be described exactly by the gauge therm g(t).

From Eq. (2.54) one can see that the exclusive work done during a quasi-staticisothermal process equals

[WE(t, t ′)

]eq = −V [

X(t), t] + V [

X(t ′), t ′] + �F(t).

Note that, in contrary to the equilibrium inclusive work,[WE(t, t ′)

]eq still depends

on the random initial and final states of the system, X(t) and X(t ′). Nevertheless, itsaverage value equals to the increase of the free energy F0(t, t ′) of the system alone,i.e.,

⎝[WE(t, t ′)

]eq

⎞= �F(t) −

∞∫

−∞dx V(x, t)

exp[−βF(x, t)]Z(t)

+∞∫

−∞dx V(x, t ′)exp[−βF(x, t ′)]

Z(t ′)= F0(t) − F0(t

′) = �F0(t, t ′), (2.57)

where the free energy of the unperturbed system at the time t reads

F0(t) = G⎭〈[X(t)]eq∝). (2.58)

Here 〈[X(t)]eq∝ stands for the mean equilibrium state of the system correspondingto the instantaneous FEL F(x, t). It is given by

〈[X(t)]eq∝ =∞∫

−∞dx x

exp[−βF(x, t)]Z(t)

. (2.59)

The free energy obtained from the exclusive work thus excludes the terms whichstem from the interaction with the external agent and therefore it is also invariantwith respect to the gauge transformation F(x, t) + g(t).

Let us stress that the both equilibrium works can be used for measuring the freeenergy differences between the states of the system alone, �F0(t). The averageequilibrium exclusive work is exactly �F0(t, t ′). The average equilibrium inclusivework yields the free energy difference of the compound system, �F(t, t ′), which

2.2 Continuous Systems 31

translates to �F0(t, t ′) as

�F0(t, t ′) = −V ⎭〈[X(t)]eq∝, t) + V ⎭〈[X(t ′)]eq∝, t ′

) + �F(t, t ′). (2.60)

The formula (2.60) can be used once one knows the correct form of the interactionenergy V (x, t). However, in most experimentally relevant situations this function isknown indeed. In Sect. 4.1 we illustrate the connection between the exclusive andinclusive work in the form of a specific, physically relevant, model.

2.3 Work Fluctuation Relations

Investigation of dynamics and thermodynamics of mesoscopic systems under influ-ence of (external) time dependent forces is of great interest over last two decades[45, 46]. As we mentioned in the Introduction, one of the most interesting theoreticalresult in this field was the discovery of the so called fluctuation theorems [1, 47, 48,48–52]. General relations depicting time-irreversibility of various non-equilibriumstochastic processes occurring in microscopic systems exposed to non-equilibriumconditions, such as entropy production [8, 15, 53–59] and work done on the envi-ronment [15, 44, 48, 60, 61].

In this section, we focus on the theorems involving the two work definitions(2.44) and (2.53) presented in Sect. 2.2.2 The two works represent different randomvariables and hence they possess different probability densities [44]. Fluctuationtheorems for the thermodynamic work (2.44) are studied in detail for last two decades[15, 48, 60, 61]. The most important results in the field are Jarzynski equality [1, 48]

〈exp[−β W(t, t ′)

]∝ = exp[ − β�F(t, t ′)

](2.61)

and Crooks fluctuation theorem [7–9]

ρF(w)

ρR(−w)= exp

{−β[�F(t, t ′) − w

]}. (2.62)

The average on the right-hand side of Eq. (2.61) is taken over the work distribution(2.16) given that the system is initially in equilibrium, i.e., p(x, t ′) = π(x, t ′) =exp

[−βF(x, t ′)]/Z(t ′). In Eq. (2.62) this work distribution is denoted as ρF(w).

By the subscript F (forward) we stress that the externally controlled parameters aredriven from Y(t ′) to Y(t), t ′ < t . ρR(w) in Eq. (2.62) stands for the work probabilitydistribution for the time-reversed process. Here the externally controlled parametersare driven from Y(t) to Y(t ′) and the system starts from the equilibrium state corre-sponding to the FEL of the compound system at the time t , F(x, t). The free energy

2 In this section, we use the notation introduced in Sect. 2.2 for the continuous models, neverthelessthe results remains valid also for the discrete models described in Sect. 2.1, indeed.

32 2 Stochastic Thermodynamics

difference �F(t, t ′) in Eqs. (2.61) and (2.62) is given by (2.55). The only assump-tion needed for validity of Eqs. (2.61) and (2.62) is the microscopic reversibilityof the underlying dynamics. The discrete models (Sect. 2.1) are reversible if thedetailed balance condition (2.4) is fulfilled. The time-reversibility for the continu-ous models is already incorporated in the Langevin Eq. (2.37). The theorems (2.61)and (2.62) represent an elegant refinement of the second law of thermodynamics.For example, using Jensen’s inequality, the well known form of the second law,�F(t, t ′) ≤ W (t, t ′), follows from Eq. (2.61). Note that Eq. (2.61) is a corollary ofEq. (2.62), but not vice versa.

Equivalent fluctuation theorems for the mechanical work (2.53) were discoverednearly three decades earlier [43, 44, 62, 63]

〈exp[−β WE (t, t ′)

]∝ = 1, (2.63)

ρF(w)

ρR(−w)= exp(βw). (2.64)

Here the average in Eq. (2.63) is taken over the probability density for the work (2.53)given that the system starts at the time t ′ from the state π(x) = exp [−βG(x)] /Z0,Z0 = ⎬ ∞

−∞ dx exp [−βG(x)], the thermal equilibrium state corresponding to the FELof the unperturbed system, G(x). This probability density is in Eq. (2.64) denoted byρF. The subscripts F and R have the same meaning as in Eq. (2.62) with the differencethat the system starts the reversed process again from the state π(x).

The theorems for the thermodynamic work (2.44) relate, different from the theo-rems for the mechanical work (2.53), the work done during a non-equilibrium process(easy to measure experimentally) to the increase of the equilibrium free energy ofthe compound system (sometimes hard to measure experimentally), and thus theyreceived much more attention in the literature. The obtained free energy difference�F(t, t ′) can be interesting in itself or can be used for calculating the incrementof the free energy of the unperturbed system, �F0(t, t ′), via Eq. (2.60). Moreover,another generalization of Eq. (2.61) allows the reconstruction of the whole FEL ofthe unperturbed system, G(z), using the work measurements. This result is calledHummer-Szabo relation and reads [64]

Z(t ′) exp⎟β V[z, Y(t)]

⎠〈δ[X(t) − z] exp[−β W(t, t ′)]∝ = exp[−β G(z)]. (2.65)

The knowledge of G(z) allows to calculate the equilibrium free energy differences�F(t, t ′) and �F0(t, t ′) from their definitions (2.55) and (2.58). Here we have tostress that there exist also different, and often more accurate, methods which allowto obtain G(z) etc experimentally. One possibility is to measure the time whichthe system needs to leave the individual microstates (survival probability), see, forexample, [65, 66].

Validity of the equalities (2.61), (2.62) and (2.65) was confirmed in various the-oretical models [23, 33, 67], in many experiments [68–71] and also in numerousnumerical simulations [72, 73]. They can be derived for thermostated reversible

2.3 Work Fluctuation Relations 33

Table 2.1 The values of the arbitrary functions pF(x) and pB(x) that yield the individual relations(2.61), (2.63) and (2.65) from Eq. (2.66)

Equation pF(x) pB(x)

(2.61)exp

[−βF(x, t ′)]

⎬ ∞−∞ dx exp [−βF(x, t ′)]

exp [−βF(x, t)]⎬ ∞−∞ dx exp [−βF(x, t)]

(2.63)exp [−βG(x)]

⎬ ∞−∞ dx exp [−βG(x)]

exp [−βG(x)]⎬ ∞−∞ dx exp [−βG(x)]

(2.65)exp

[−βF(x, t ′)]

⎬ ∞−∞ dx exp [−βF(x, t ′)]

δ[X(t) − z

] exp{−βF [

X(t), t]}

exp [−βF(z, t)]

deterministic systems [74], and also for microscopically reversible stochastic dynam-ics using both the Langevin equation [8], and the Master equation formulation [48].In single-molecule experiments [75], all the relations (2.61), (2.62) and (2.65) areused for reconstructing free energy profiles of biomolecules [68, 69]. Influence of theexperimental errors originating both from the instrument noise and from the uncer-tainty of measurements on the free energy difference estimate was studied in [76, 77].It was found that the obtained free-energy difference is correct if the stochastic errorsare statistically the same for the conjugate forward and reverse protocols [76], cf.the example in Appendix C. The discussion about the correct work definition alsooccurred in the field of fluctuation relations [44, 78]. The result is that the correctwork definition, which fulfills the experimentally useful fluctuation theorems (2.61),(2.62) and (2.65), is the definition of the thermodynamic work (2.44).

In order to close this section let us note that the integral fluctuation theorems(2.61), (2.63) and (2.65) can be immediately derived from the single relation [15]

⎝exp

⎦ −Sr(t, t ′)kB

⎫pB(x)

pF(x)

⎞= 1, (2.66)

where

Sr(t, t ′) = −Q(t, t ′)T

(2.67)

is the (random) entropy transferred from the system into the thermal environment dur-ing the time interval [t ′, t] and the auxiliary function pB(x) fulfills the normalizationcondition

⎬ ∞−∞ dx pB(x) = 1. The average in Eq. (2.66) is taken over all trajectories

of the stochastic process X(t) that depart at the time t ′ with an arbitrary probabilitypF(x) dx from the position that lies in the infinitesimal interval (x, x + dx). Thevalues of the arbitrary functions pB(x) and pF(x) that yield the individual relations(2.61), (2.63) and (2.65) are given in Table 2.1. Even more general formula, whichcontains all the presented fluctuation relations, was derived by Harris [49].

34 2 Stochastic Thermodynamics

2.4 Heat Engines

One of the hot topics in the field of stochastic thermodynamics are stochastic heatengines [23, 28, 79–87]. The variety of models can be roughly classified accordingto the dynamical laws involved. In the case of the classical stochastic heat engines,the state space can either be discrete [23, 86–89] or continuous [80–82, 85–87].Examples of the quantum heat engines are studied, e.g., in [90–93].

In the present work we consider periodically operating classical stochastic heatengines which communicate with two baths at the temperatures T+ and T−. Lettp denote the duration of one operational cycle, Wout ≡ −W (tp, 0) stands for the(mean) work performed by the engine during one period and

Qin =tp∫

0

dtdQ(t, 0)

dt�

⎦dQ(t, 0)

dt

⎫, (2.68)

where the function �(•) equals to 1 for a positive argument and to 0 otherwise,denotes the heat transferred into the engine from the reservoirs per cycle. The engineefficiency is then defined by the standard formula

η = Wout

Qin. (2.69)

Similarly, the (mean) power output of the engine reads

Pout = Wout

tp. (2.70)

The traditional consideration of efficiency of heat engines operating between the twobaths leads to the Carnot’s upper bound3

ηC = 1 − T−T+

. (2.71)

This bound is only achieved under reversible conditions where the state changesrequire infinite time and hence the power output is zero. Real heat engines generate afinite power output (2.70), i.e., they perform finite work Wout during a cycle of a finiteduration tp. The Carnot’s inequality η ≤ ηC was recently generalized by Sinitsyn[94] (see also [95]) who derived a fluctuation theorem which relates the statistics ofthe (random) heat extracted from the hot reservoir during the cycle and that of the(random) work performed by the engine per cycle.

An alternative way to classify the performance of the heat engines is to comparetheir efficiencies at maximum power. On the macroscopic level, the first works on

3 Without loose of generality we assume that T+ > T−.

2.4 Heat Engines 35

this subject were performed by Chambadal [96] and Novikov [97]. They studied, inthe framework of endoreversible thermodynamics, the efficiency of nuclear powerplants and derived the famous formula for the efficiency at maximum power

ηCA = 1 −√

T−T+

. (2.72)

This result have been referred to as the Curzon-Ahlborn efficiency since it was, nearlytwenty years later, independently rediscovered by Curzon and Ahlborn [98]. Afterthe derivation of ηCA, the discussion whether or not it represents the upper bound forthe efficiency at maximum power has been initiated. The result was negative. Theefficiency at maximum power is always model dependent and no universal upperbound was discovered, yet (see, for example, the reviews [87, 99]).

Nevertheless Tu [100] recently realized that large variety of heat engines[82, 89, 98, 100] (see also the reviews [86, 87]) exhibit similar efficiency at maxi-mum power in the case of small difference between the two reservoir temperatures(i.e., ηC is small). Later it was proven [83] that the Taylor expansion of the efficiencyat maximum power

ηP ≈ ηC

2+ η2

C

8+ o

(η3

C

)(2.73)

is general for “strong coupling” models (the term ηC/2) that possess a “left-rightsymmetry” (the term η2

C/8). In the strong coupling models the energy (heat) flow isproportional to the flow that performs the work (matter flow). The left-right symmetrymeans that the heat flow through the system changes its sign when the two heatreservoirs are interchanged. If the second assumption is relaxed, only the term ηC/2of the expansion (2.73) remains generally valid.

The authors of the study [83] also derived a lower and an upper bound for theefficiency at maximum power for a non-equilibrium analogue of the Carnot’s cycle[84], the cycle which consists of two non-equilibrium isotherms (say branches I andIII) and two non-equilibrium adiabatic branches. More precisely, assume that thedurations of the two isothermal branches are t+ and t− and that the total entropiesproduced during those isotherms are S+ and S−, respectively. If these variables fulfillthe relations tp ≈ (t+ + t−), S+ ≈ 1/t+ ≥ 0 and S− ≈ 1/t− ≥ 0, then the efficiencyat maximum power is bounded by the inequalities

ηC

2≤ ηP ≤ ηC

2 − ηC. (2.74)

In Chap. 5 we present two examples of stochastic heat engines [23, 85, 86]. Anengine based on the two-level system, which is introduced in Sect. 3.1.2, is inves-tigated in Sect. 5.1 (see also [23]). An engine based on a particle diffusing in anexternal log-harmonic potential is studied in Sect. 5.2. The both models representexamples of the above mentioned non-equilibrium Carnot’s cycle. The second one

36 2 Stochastic Thermodynamics

demonstrates the validity of the relations (2.73) and (2.74). In the next section wedescribe the generic periodic driving used in the two models.

2.4.1 Limit Cycle

An important part of the driving of a heat engine is represented, except the externallycontrolled parameters Y(t), by the time dependence of the reservoir temperatureT = T (t). Under a periodic driving, Y(t) = Y(t + tp), T (t) = T (t + tp), thestate of the system eventually, after a transient regime, becomes also periodic. Thesystem attains an uniquely defined limit cycle, the operational cycle of the engine. Anexample of such cycle is depicted in Fig. 5.8. As mentioned above, the generic drivingused in the models presented in this thesis is consists of two adiabatic branches andtwo isotherms. Specifically, we assume the following driving:

• Branch I (isothermal): The temperature equals T+ and the control parameterschanges smoothly from Y(0) to Y(t−+ ).

• Branch II (adiabatic): The temperature and the control parameters changes rapidlyfrom T+ to T− and from Y(t−+ ) to Y(t++ ), respectively. We assume that the processis so fast that the state of the system remains unchanged (no heat is exchanged withthe reservoir). The work done on the system W (t++ , t−+ ) equals U (t++ ) − U (t−+ )

and no entropy is produced.• Branch III (isothermal): The temperature equals T− and the control parameters

changes smoothly from Y(t++ ) to Y(t−p ).• Branch IV (adiabatic): The temperature and the control parameters changes rapidly

back from T− to T+ and from Y(t−p ) to Y(tp) = Y(0), respectively. We assumethat the process is so fast that the state of the system remains unchanged (no heatis exchanged with the reservoir). The work done on the system W (tp, t−p ) equalsU (tp) − U (t−p ) and no entropy is produced.

Up to now, both the master Eq. (2.3) and the Fokker-Planck Eq. (2.38) have beenconsidered only for an isothermal process. However, neither in the case of Eq. (2.3)nor in the case of Eq. (2.38) the time dependence of the temperature does not breakthe Markov property of the underlying stochastic dynamics. Therefore one can usethe Chapman-Kolmogorov Eqs. (2.7) and (2.40) to compose the solution of Eqs. (2.3)and (2.38) for the periodic driving above from their solutions for the isotherms I andIII (during the adiabatic branches II and IV the system state does not change). In thenext two paragraphs we execute the proposed procedure for the systems with discreteand with continuous state space, respectively.

2.4.1.1 Discrete Systems

Let R+(t | t ′) = R(t | t ′), t ′, t ∈ (0, t++ ) and R−(t | t ′) = R(t | t ′), t ′, t ∈ (t++ , tp)denote the solution of Eq. (2.3) during the isothermal branches I and III, respectively.

2.4 Heat Engines 37

Using the Chapman-Kolmogorov Eq. (2.7) the solution of the master Eq. (2.3) for theabove described periodic driving reads4

Rp(t | t ′) =⎧⎨

⎩

R+ (t | t ′), t ′, t ∈ [0, t++

],

R− (t | t++ )R+(t++ | t ′), t ′ ∈ [0, t++

],∧ t ∈ [

t++ , tp],

R− (t | t ′), t ′, t ∈ [t++ , tp

].

(2.75)

The matrix Rp(t | t ′) can be used for calculation of the time correlation function(2.25) during the cycle. Further we focus on the matrix Rp(t) ≡ Rp(t | 0), whichevolves any initial state of the system as p(t) = Rp(t) p(0). The system states at theends of the individual periods form a Markov chain and, in order to obtain the periodicstate of the system during the limit cycle, p(t) = p(t + tp), we are interested in itsfixed point behavior limn→∞

[Rp(tp)

]n p(0). If we take the stationary state as theinitial condition, p(0) = pstat, the system revisits this special state after each periodof the driving. Therefore, to find the vector pstat, it suffices to solve the eigenvalueproblem [13]

pstat = Rp(tp) pstat . (2.76)

Similarly, let the matrix G+(w, t | w′) = G(t, w | w′, 0), t ∈ (0, t++ ) and the matrixG−(w, t | w′) = G(t, w | w′, t++ ), t ∈ (t++ , tp) denote the solution of Eq. (2.13)during the isothermal branches I and III, respectively. According to the Chapman-Kolmogorov Eq. (2.14) the solution of Eq. (2.13) for the periodic driving reads

Gp(w, t) =

⎧⎪⎪⎨

⎪⎪⎩

G+(w, t | 0) , t ∈ [0, t++ ] ,∞∫

−∞dw′ G−(w, t | w′)G+(w′, t++ | 0) , t ∈ [t++ , tp]. (2.77)

These results completely describe dynamics and energetics of the engine during thelimit cycle. Specifically, the system state during the cycle reads

p(t) = Rp(t) pstat (2.78)

and the energetics of the engine during the cycle is determined by the function

ξ(u, w, t; u′) =N⎡

i, j=1

δ [u − Ei (t)] δ[u′ − E j (0)

] [Gp(w, t)

]i j pstat

j (2.79)

which enters Eqs. (2.16)–(2.18).

4 It is important to note that the matrix Rp(t) ≡ Rp(t | 0) is, contrary to the driving Y(t), alwayscontinuous, i.e., Rp(t

−+ ) = Rp(t++ ) and Rp(t−p ) = Rp(tp).

38 2 Stochastic Thermodynamics

2.4.1.2 Continuous Systems

The description for the continuous systems is similar. Let R+(x, t | x ′, t ′) = R(x, t |x ′, 0), t ′, t ∈ (0, t++ ) and R−(x, t | x ′, t ′) = R(x, t | x ′, t++ ), t ′, t ∈ (t++ , tp) denotethe solution of Eq. (2.38) during the isothermal branches I and III, respectively. Usingthe Chapman-Kolmogorov Eq. (2.40) the solution of the Fokker-Planck Eq. (2.38) forthe periodic driving above reads

⎧⎪⎪⎪⎪⎨

⎪⎪⎪⎪⎩

R+(x, t | x ′, t ′), t ′, t ∈ [0, t++ ],∞∫

−∞dx ′′ R−(x, t | x ′′, t++ ) R+(x ′′, t++ | x ′, t ′), t ′ ∈ [0, t++ ] ∧ t ∈ [t++ , tp],

R−(x, t | x ′, t ′), t ′, t ∈ [t++ , tp].(2.80)

The eigenvalue problem (2.76) now translates into the integral Eq.

∞∫

−∞dx ′ Rp(x, tp | x ′, 0) pstat(x ′) = pstat(x). (2.81)

The dynamics of the engine is described by the (periodic) state of the system duringthe limit cycle

p(x, t) =∞∫

−∞dx ′ Rp(x, t | x ′, 0) pstat(x ′). (2.82)

The energetics of the engine is determined by the function p(x, t), which entersEqs. (2.48)–(2.49) for the mean values of the work, the heat and the internal energyand by the functions Rp(x, t | x ′, t ′) and pstat(x), which yield the time correlationfunction (2.51) and hence describe the fluctuations of the work and the heat.

2.4.2 Diagrams of the Limit Cycle

In this section we discuss the possibility to depict the limit cycle of an engine in adiagram similar to the pressure volume (PV) diagram known from classical thermo-dynamics. The mean work done per cycle by a discrete system can be rewritten as[cf. Eqs. (2.23) and (2.48)]

2.4 Heat Engines 39

Wout = −W (tp, 0) = −tp∫

0

dtN⎡

i=1

⎦d

dtEi (t)

⎫pi (t)

= −Y(tp)∫

Y(0)

dYN⎡

i=1

⎦d

dYVi (Y)

⎫pi (Y)

= −K⎡

j=1

Y j (tp)∫

Y j (0)

dY j

N⎡

i=1

⎦∂

∂Y jVi (Y1, . . . , YK )

⎫pi (Y)

=K⎡

i=1

Yi (tp)∫

Yi (0)

dYi fi (Yi ), (2.83)

where K stands for the number of components of the external driving Y(t) andpi [Y(t)] = pi (t). The individual terms on the right-hand side of Eq. (2.83) cor-respond to the work done by the individual components of the driving. For thecontinuous models the functions fi (Yi ) are given by (p(x, t) = p[x, Y(t)])

fi (Yi ) = −∞∫

∞dx

⎦∂

∂YiV(x, Y)

⎫p(x, Y)

= −∞∫

∞dx

⎦∂

∂YiV(x, Y1, . . . , Yk)

⎫p(x, Y). (2.84)

Since Y(tp) = Y(0), each of the K contributions on the right-hand side of Eq. (2.83)equals the oriented area enclosed by the parametric plot of the system response,represented by the average fi (Yi ), versus the i th component of the driving, Yi (t).The parameter t runs from 0 to tp. Similar decomposition of work is well knownfrom classical thermodynamics. As an example consider a magnetic gas and let thedriving possess two components, the volume Y1 = V and the magnetic field (spatiallyhomogeneous magnetic flux density) Y2 = B. The thermodynamic work done bythe system per cycle then reads [101]

Wout =V (tp)∫

V (0)

dV p(V ) −B(tp)∫

B(0)

dB I (B), (2.85)

where f1 = p denotes the gas pressure and − f2 = I stands for the component ofthe total magnetic moment of the gas parallel to the external magnetic field. Theparametric plots corresponding to the individual terms on the right-hand side of

40 2 Stochastic Thermodynamics

Eq. (2.83) thus represent an analogy of the well known PV diagrams. In the field ofstochastic thermodynamics such diagrams were used for the first time in the works[23, 102, 103]. Several examples of the diagrams are presented in Chap. 5 (seeFigs. 5.1, 5.4, 5.9 and 5.11).

Important eye-guides in the diagrams can be formed by the so called equilibriumisotherms. The curves corresponding to the values of the functions fi if a given cyclewould be carried out quasi-statically. They are given by

f eqi [Yi (t)] =

N⎡

j=1

{∂V j [Y1(t), . . . , YK (t)]

∂Yi (t)

}exp[β(t)F j (t)]

∑Nj=1 exp[β(t)F j (t)]

(2.86)

and

f eqi [Yi (t)] =

∞∫

−∞dx

[∂V(x, Y1(t), . . . , YK (t))

∂Yi (t)

]exp[β(t)F(x, t)]

⎬ ∞−∞ dx ′ exp[β(t)F(x ′, t)]

(2.87)for the discrete and for the continuous models, respectively. The equilibrium isothermcorresponding to the first isothermal branch is obtained if one takes in Eqs. (2.86) and(2.87) t ∈ (0, t−+ ) and β(t) = 1/(kBT+). Similarly, the equilibrium isotherm corre-sponding to the second isothermal branch is obtained if one assumes t ∈ (t++ , t−p )

and β(t) = 1/(kBT−). In Figs. 5.1, 5.4, 5.9 and 5.11 it can be clearly seen that thenon-equilibrium functions fi are “attracted” to the equilibrium isotherms, which rep-resent the carrot (the banana) in the horse-carrot (banana-monkey) analogy depictedin Fig. 2.44. The more “reversible” the individual non-equilibrium isotherms are thestronger this attraction is (cf. Fig. 5.11).

References

1. Jarzynski, C. (1997). Nonequilibrium equality for free energy differences. Physical ReviewLetters, 78, 2690–2693, doi:10.1103/PhysRevLett.78.2690.

2. Carberry, D. M., Reid, J. C., Wang, G. M., et. al. (2004). Fluctuations and irreversibility: Anexperimental demonstration of a second-law-like theorem using a colloidal particle held inan optical trap. Physical Review Letters, 92, 140601, doi:10.1103/PhysRevLett.92.140601.

3. Hänggi, P., & Marchesoni, F. (2009). Artificial Brownian motors: Controlling transport onthe nanoscale. Reviews of Modern Physics, 81, 387–442, doi:10.1103/RevModPhys.81.387.

4. Speck, T. (2011). Work distribution for the driven harmonic oscillator with time-dependentstrength: Exact solution and slow driving. Journal of Physics A: Mathematical and Theoreti-cal, 44(30), 305001, http://stacks.iop.org/1751-812144/i=30/a=305001.

5. Bressloff, P. C., & Newby, J. M. (2013). Stochastic models of intracellular transport. Reviewsof Modern Physics, 85, 135–196, doi:10.1103/RevModPhys.85.135.

6. Jarzynski, C. (1997). Equilibrium free-energy differences from nonequilibrium measure-ments: A master-equation approach. Physical Review E, 56, 5018–5035, doi:10.1103/PhysRevE.56.5018.

References 41

7. Crooks, G. (1998). Nonequilibrium measurements of free energy differences for microscopi-cally reversible Markovian systems. Journal of Statistical Physics, 90(5–6), 1481–1487. ISSN0022–4715, doi:10.1023/A:1023208217925.

8. Crooks, G. E. (1999). Entropy production fluctuation theorem and the nonequilibriumwork relation for free energy differences. Physical Review E, 60, 2721–2726, doi:10.1103/PhysRevE.60.2721.

9. Crooks, G. E. (2000). Path-ensemble averages in systems driven far from equilibrium. PhysicalReview E, 61, 2361–2366, doi:10.1103/PhysRevE.61.2361.

10. Van Kampen, N. (2011). Stochastic processes in physics and chemistry. North-Holland Per-sonal Library: Elsevier Science. ISBN 9780080475363, http://books.google.cz/books?id=N6II-6HlPxEC.

11. Chvosta, P., Reineker, P., & Schulz, M. (2007). Probability distribution of work done on atwo-level system during a nonequilibrium isothermal process. Physical Review E, 75, 041124,doi:10.1103/PhysRevE.75.041124.

12. Chvosta, P., & Reineker, P. (1999) Dynamics under the influence of semi-Markovnoise. Physica A: Statistical Mechanics and its Applications, 268, 103–120, ISSN 0378–4371. doi:10.1016/S0378-4371(99)00021-7, http://www.sciencedirect.com/science/article/pii/S0378437199000217.

13. Motl, L., & Zahradník, M. (1995). Pestujeme lineární algebru. Karolinum. ISBN9788071841869, http://books.google.cz/books?id=0shyAAAACAAJ.

14. Gillespie, D. T. (1992). Markov processes: An introduction for physical scientist. San Diego:Academic press, Inc.

15. Seifert, U. (2008). Stochastic thermodynamics: Principles and perspectives. The EuropeanPhysical Journal B: Condensed Matter and Complex Systems, 64, 423–431. ISSN 1434–6028,doi:10.1140/epjb/e2008-00001-9.

16. Imparato, A., & Peliti, L. (2005). Work probability distribution in single-molecule exper-iments. EPL (Europhysics Letters), 69(4), 643, http://stacks.iop.org/0295-5075/69/i=4/a=643.

17. Imparato, A., & Peliti, L. (2005) Work distribution and path integrals in general mean-fieldsystems. EPL (Europhysics Letters), 70(6), 740, http://stacks.iop.org/0295-5075/70/i=6/a=740.

18. Šubrt, E., & Chvosta, P. (2007). Exact analysis of work fluctuations in two-level systems.Journal of Statistical Mechanics: Theory and Experiment, 2007(09), P09019. http://stacks.iop.org/1742-5468/2007/i=09/a=P09019.

19. Imparato, A., & Peliti, L. (2005). Work-probability distribution in systems driven out ofequilibrium. Physical Review E, 72, 046114, doi:10.1103/PhysRevE.72.046114.

20. Chatelain, C., & Karevski, D. (2006). Probability distributions of the work in the two-dimensional Ising model. Journal of Statistical Mechanics: Theory and Experiment, 2006(06),P06005, http://stacks.iop.org/1742-5468/2006/i=06/a=P06005.

21. Híjar,H., Quitana-H, J., & Sutmann, G. (2007). Non-equilibrium work theorems for thetwo-dimensional using model. Journal of Statistical Mechanics: Theory and Experiment,2007(04), P04010, http://stacks.iop.org/1742-5468/2007/i=04/a=P04010.

22. Einax, M., & Maass, P. (2009). Work distributions for Ising chains in a time-dependentmagnetic field. Physical Review E, 80, 020101. doi:10.1103/PhysRevE.80.020101, http://link.aps.org/doi/10.1103/PhysRevE.80.020101.

23. Chvosta, P., Einax, M., Holubec, V., et al., et al., (2010). Energetics and performance of amicroscopic heat engine based on exact calculations of work and heat distributions. Jour-nal of Statistical Mechanics, 2010(03), P03002, http://stacks.iop.org/1742-5468/2010i=03/a=P03002.

24. Verley, G., Van den Broeck, C., & Esposito, M. (2013). Modulated two-level system: Exactwork statistics. Physical Review E, 88, 032137, doi:10.1103/PhysRevE.88.032137.

25. Ritort, F. (2004). Work and heat fluctuations in two-state systems. Journal of StatisticalMechanics: Theory and Experiment, 2004(10), P10016, http://stacks.iop.org/1742-5468/2004/i=10/a=P10016.

42 2 Stochastic Thermodynamics

26. Manosas, M., Mossa, A., Forns, N., et al. (2009). Dynamic force spectroscopy of DNAhairpins: II. Irreversibility and dissipation. Journal of Statistical Mechanics: Theory andExperiment, 2009(02), P02061, http://stacks.iop.org/1742-5468/2009/i=02/a=P02061.

27. Risken, H. (1985). The Fokker-Planck equation: Methods of solution and applications. Berlin:Springer.

28. Sekimoto, K. (1997). Kinetic characterization of heat bath and the energetics of thermal ratchetmodels. Journal of the Physical Society of Japan, 66(5), 1234–1237, doi:10.1143/JPSJ.66.1234.

29. Vilar, J. M. G., & Rubi, J. M. (2008). Failure of the Work-Hamiltonian connection for free-energy calculations. Physical Review Letters, 100, 020601, doi:10.1103/PhysRevLett.100.020601.

30. Peliti, L. (2008). On the work-Hammiltonian connection in manipulated systems. Journalof Statistical Mechanics: Theory and Experiment, 2008(05), P05002, http://stacks.iop.org/1742-5468/2008/i=05/a=P005002.

31. Zimanyi, E. N., & Silbey, R. J. (2009). The work-Hamiltonian connection and the usefulnessof the Jarzynski equality for free energy calculations. Journal of Chemical Physics, 130(17),171102, doi:10.1063/1.3132747,0902.3681.

32. Vilar, J. M. G., & Rubi, J. M., (2011). Work-Hamiltonian connection for anisoparamet-ric processes in manipulated microsystems. Journal of Non-Equilibrium Thermodynamics,36, 123–130. doi:10.1515/jnetdy.2011.008, http://www.degruyter.com/view/j/jnet.2011.36.issue-2/jnetdy.2011.008/jnetdy.2011.008.xml.

33. Mazonka, O., & Jarzynski, C. (1999). Exactly solvable model illustrating far-from-equilibriumpredictions. eprint arXiv:cond-mat/9912121, arXiv:cond-mat/9912121, http://arxiv.org/abs/cond-mat/9912121.

34. Baule, A., & Cohen, E. G. D. (2009). Fluctuation properties of an effective nonlinear systemsubject to Poisson noise. Physical Review E, 79, 030103, doi:10.1103/PhysRevE.79.030103.

35. Engel, A. (2009). Asymptotics of work distributions in nonequilibrium systems. PhysicalReview E, 80, 021120. doi:10.1103/PhysRevE.80.021120, http://link.aps.org/doi/10.1103/PhysRevE.80.021120.

36. Ryabov, A., Dierl, M., Chvosta, P., et. al. (2013). Work distribution in a time-dependentlogarithmic-harmonic potential: Exact results and asymptotic analysis. Journal of PhysicsA: Mathematical and Theoretical, 46(7), 075002, http://stacks.iop.org/1751-8121/46/i=7/a=075002.

37. Nickelsen, D., & Engel, A. (2011). Asymptotics of work distributions: the pre-exponentialfactor. The European Physical Journal B, 82(2011), 207–218. ISSN 1434–6028, doi:10.1140/epjb/e2011-20133-y.

38. van Zon, R., & Cohen, E. G. D. (2003). Stationary and transient work-fluctuation theoremsfor a dragged Brownian particle. Physical Review E, 67, 046102, doi:10.1103/PhysRevE.67.046102.

39. van Zon, R., & Cohen, E. G. D. (2004). Extended heat-fluctuation theorems for a system withdeterministic and stochastic forces. Physical Review E, 69, 056121, doi:10.1103/PhysRevE.69.056121.

40. Cohen, E. G. D. (2008). Properties of nonequilibrium steady state: A path integral approach.Journal of Statistical Mechanics: Theory and Experiment, 2008(07), P07014, http://sacks.iop.org/1742-5468/2008/i-07/a=P07014.

41. Campisi, M., Hänggi, P., & Talkner, P. (2011). Colloquium: Quantum fluctuation rela-tions: Foundations and applications. Reviews of Modern Physics, 83, 771–791, doi:10.1103/RevModPhys.83.771.

42. Gibbs, J. (2010). Elementary principles in statistical mechanics: Developed with especial ref-erence to the rational foundation of thermodynamics. Cambridge University Press: CambridgeLibrary Collection—Mathematics. ISBN 9781108017022, http://www.google.cz/books?id=7VbC-15f0SkC.

43. Schurr, J. M., & Fujimoto, B. S. (2003). Equalities for the nonequilibrium work transferredfrom an external potential to a molecular system. Analysis of single-molecule extension

References 43

experiments. The Journal of Physical Chemistry B, 107(50), 14007–14019, doi:10.1021/jp0306803.

44. Jarzynski, C. (2007). Comparison of far-from-equilibrium work relations. Comptes RendusPhysique, 8, 495–506. doi:10.1016/j.crhy.2007.04.010, arXiv:cond-mat/0612305.

45. Bustamante, C., Liphardt, J., & Felix, R. (2005). The nonequilibrium thermodynamics ofsmall systems. Physics Today, 58, 43, http://dx.doi.org/10.1063/1.2012462.

46. Ritort, F. (2006). Single-molecule experiments in biological physics: Methods and applica-tions. Journal of Physics: Condensed Matter, 18(32), R531, http://stacks.iop.org/0953-8984/18/i=32/a=R01.

47. Schuler, S., Speck, T., Tietz, C., et. al. (2005). Experimental test of the fluctuationtheorem fora driven two-level system with time-dependent rates. Physical Review Letters, 94, 180602,doi:10.1103/PhysRevLett.94.180602.

48. Ritort, F. (2008). Nonequilibrium fluctuations in small systems: From physics to biology.In Advances in chemical physics (Vol.137, pp. 31-123). John Wiley & Sons Inc., ISBN9780470238080, doi:10.1002/9780470238080.ch2.

49. Harris, R. J., & Schü, G. M. (2007). Fluctuation theorem for stochastic dynamics. Journalof Statistical Mechanics: Theory and Experiment, 2007(07), P07020, http://stacks.iop.org/1742-5468/2007/i=07/a=P07020.

50. Esposito, M., & Van den Broeck, C. (2010). Three detailed fluctuation theorems. PhysicalReview Letters, 104, 090601, doi:10.1103/PhysRevLett.104.090601.

51. Sagawa, T., & Ueda, M. (2010). Generalized jarzynski equality under nonequilibrium feed-back control. Physical Review Letters, 104, 090602, doi:10.1103/PhysRevLett.104.090602.

52. Sagawa, T., & Ueda, M. (2012). Nonequilibrium thermodynamics of feedback control. Phys-ical Review E, 85, 021104, doi:10.1103/PhysRevE.85.021104.

53. Kurchan, J. (1998). Fluctuation theorem for stochastic dynamics. Journal of Physics A: Math-ematical and General, 31(16), 3719, http://stacks.iop.org/0305-4470/31/i=16/a=003.

54. Searles, D. J., & Evans, D. J. (1999). Fluctuation theorem for stochastic systems. PhysicalReview E, 60, 159–164, doi:10.1103/PhysRevE.60.159.

55. Hatano, T., & Sasa, S.-I. (2001). Steady-state thermodynamics of Langevin systems. PhysicalReview Letters, 86, 3463–3466, doi:10.1103/PhysRevLett.86.3463.

56. Seifert, U. (2005). Entropy production along a stochastic trajectory and an integral fluctuationtheorem. Physical Review Letters, 95, 040602, doi:10.1103/PhysRevLett.95.040602.

57. Esposito, M., & Van den Broeck, C. (2010). Three faces of the second law. I. Master equationformulation. Physical Review E, 82, 011143, doi:10.1103/PhysRevE.82.011143.

58. Van den Broeck, C., & Esposito, M. (2010). Three faces of the second law. II. Fokker-Planckformulation. Physical Review E, 82, 011144, doi:10.1103/PhysRevE.82.011144.

59. García-García, R., Domínguez, D., Lecomte, V., et. al. (2010). Unifying approach for fluctu-ation theorems from joint probability distributions. Physical Review E, 82, 030104, doi:10.1103/PhysRevE.82.030104.

60. Kurchan, J. (2007). Non-equilibrium work relations. Journal of Statistical Mechanics: Theoryand Experiments, 2007(07), P07005, http://stacks.iop.org/1742-5468/2007/i=07/a=P07005.

61. Baule, A., & Cohen, E. G. D. (2009). Steady-state work fluctuations of a dragged particleunder external and thermal noise. Physical Review E, 80, 011110, doi:10.1103/PhysRevE.80.011110.

62. Bochkov, G., & Kuzovlev, Y. (1981). Nonlinear fluctuation-dissipation relations and stochasticmodels in nonequilibrium thermodynamics: I. Generalized fluctuation-dissipation theorem.Physica A: Statistical Mechanics and its Applications, 106(3), 443–479, ISSN 0378–4371,doi:10.1016/0378-4371(81)90122-9.

63. Bochkov, G., & Kuzovlev, Y. (1981). Nonlinear fluctuation-dissipation relations and stochasticmodels in nonequilibrium thermodynamics: II. Kinetic potential and variational principlesfor nonlinear irreversible processes. Physica A: Statistical Mechanics and its Applications,106(3), 480–520, ISSN 0378–4371, doi:10.1016/0378-4371(81)90123-0.

44 2 Stochastic Thermodynamics

64. Hummer, G., & Szabo, A. (2001). Free energy reconstruction from nonequilibrium single-molecule pulling experiments. Proceedings of the National Academy of Sciences, 98(7), 3658–3661. doi:10.1073/pnas.071034098, http://www.pnas.org/content/98/7/3658.full.pdf+html,http://www.pnas.org/content/98/7/3658.abstract.

65. Mossa, A., Manosas, M., Forns, N., et al. (2009). Dynamic forces spectroscopy of DNAhairpins: I. Force Kinetics and free energy landscapes. Journal of Statistical Mechanics:Theory and Experiment, 2009(02), P02060, http://stacks.iop.org/1742-5468/2009/i=02/a=P02060.

66. Nostheide, S., Holubec, V., Chvosta, P., et. al. (2013). Unfolding kinetics of periodic DNAhairpins. arXiv preprint arXiv:1312.4146, arxiv.org/abs/1312.4146.

67. Crooks, G. E., & Jarzynski, C. (2007). Work distribution for the adiabatic compression of adilute and interacting classical gas. Physical Review E, 75, 021116, doi:10.1103/PhysRevE.75.021116.

68. Liphardt, J., Dumont, S., Smith, S. B., et. al. (2002). Equilibrium information fromnonequilibrium measurements in an experimental test of Jarzynski’s equality. Science,296(5574), 1832–1835. doi:10.1126/science.1071152, http://www.sciencemag.org/content/296/5574/1832.full.pdf, http://www.sciencemag.org/content/296/5574/1832.abstract.

69. Collin, D., Ritort, F., Jarzynski, C., et al. (2005). Verification of the crooks fluctuation theoremand recovery of RNA folding free energies. Nature, 473, 231, http://dx.doi.org/10.1038/nature04061M3.

70. Blickle, V., Speck, T., Helden, L., et. al. (2006). Thermodynamics of a colloidal particle ina time-dependent nonharmonic potential. Physical Review Letters, 96, 070603, doi:10.1103/PhysRevLett.96.070603.

71. Gomez-Solano, J. R., Bellon, L., Petrosyan, A. et. al. (2010). Steady-state fluctuation relationsfor systems driven by an external random force. EPL (Europhysics Letters), 89(6), 60003,http://stacks.iop.org/0295-5075/89/i=6/a=60003.

72. Xiao, T. J., Hou, Z., & Xin, H. (2008). Entropy production and fluctuation theorem alonga stochastic limit cycle. The Journal of Chemical Physics, 129(11), 114506. doi:10.1063/1.2978179, http://link.aip.org/link/?JCP/129/114506/1.

73. Holubec, V., Chvosta, P., Einax, M., et al. (2011). Attempt time Monte Carlo: An alterna-tive for simulation of stochastic jump processes with time-dependent transition rates. EPL(Europhysics Letters), 93(4), 40003, http://stacks.iop.org/0295-5075/93/i=4/a=40003.

74. Sevick, E., Prabhakar, R., Williams, S. R., et. al. (2008). Fluctuation theorems. Annual Reviewof Physical Chemistry, 59(1), 603–633. doi:10.1146/annurev.physchem.58.032806.104555,http://www.annualreviews.org/doi/pdf/10.1146/annurev.physchem.58.032806.104555.

75. Manosas, M., & Ritort, F. (2005). Thermodynamic and kinetic aspects of RNA pulling experi-ments. Biophysical Journal, 88(5), 3224–3242, ISSN 0006–3495. doi:10.1529/biophysj.104.045344, http://www.sciencedirect.com/science/article/pii/S0006349505733749.

76. Maragakis, P., Ritort, F., Bustamante, C., et. al. (2008). Bayesian estimates of free energiesfrom nonequilibrium work data in the presence of instrument noise. The Journal of ChemicalPhysics, 129(2), 024102. doi:10.1063/1.2937892, http://link.aip.org/link/?JCP/129/024102/1.

77. Calderon, C. P., Harris, N. C., Kiang, C.-H., et. al. (2009). Quantifying multiscale noise sourcesin single-molecule time series. The Journal of Physical Chemistry B, 113(1), 138–148, doi:10.1021/jp807908c.

78. Alemany, A., Ribezzi, M., Ritort, F. (2011). Recent progress in fluctuation theorems and freeenergy recovery. AIP Conference Proceedings, 1332(1), 96–110. doi:10.1063/1.3569489,http://link.aip.org/link/?APC/1332/96/1.

79. Takagi, F., & Hondou, T. (1999). Thermal noise can facilitate energy conversion by a ratchetsystem. Physical Review E, 60, 4954–4957, doi:10.1103/PhysRevE.60.4954.

80. Sekimoto, K., Takagi, F., & Hondou, T. (2000). Carnot’s cycle for small systems: Irreversibilityand cost of operations. Physical Review E, 62, 7759–7768, doi:10.1103/PhysRevE.62.7759.

81. Van den Broeck, C., Kawai, R., & Meurs, P. (2004). Microscopic analysis of a thermal Brown-ian motor. Physical Review Letters, 93, 090601, doi:10.1103/PhysRevLett.93.090601.

References 45

82. Schmiedl, T., & Seifert, U. (2008). Efficiency at maximum power: An analytically solvablemodel for stochastic heat engines. EPL (Europhysics Letters), 81(2), 20003, http://stacks.iop.org/0295-5075/81/i=2/a=20003.

83. Esposito, M., Lindenberg, K., & Van den Broeck, C. (2009). Universality of Efficiency at Max-imum Power. Physical Review Letters, 102, 130602, doi:10.1103/PhysRevLett.102.130602.

84. Esposito, M., Kawai, R., Lindenberg, K., et. al. (2010). Efficiency at maximum power of low-dissipation carnot engines. Physical Review Letters, 105, 150603, doi:10.1103/PhysRevLett.105.150603.

85. Blickle, V., & Bechinger, C. (2011). Realization of a micrometre-sized stochastic heat engine.Nature Physics, 8(2), 143–146, doi:10.1038/nphys2163.

86. Seifert, U. (2012). Stochastic thermodynamics, fluctuation theorems and molecular machines.Reports on Progress in Physics, 75(12), 126001, http://stacks.iop.org/0034-4885/75/i=12/a=126001.

87. Zhan-Chun, T. (2012). Recent advance on the efficiency at maximum power of heat engines.Chinese Physics B, 21(2), 020513, http://stacks.iop.org/1674-1056/21/i=2/a=020513.

88. Arnaud, J., Chusseau, L., & Philippe, F. (2010). A simple model for Carnot heat engines.American Journal of Physics, 78(1), 106–110. doi:10.1119/1.3247983, http://link.aip.org/link/?AJP/78/106/1.

89. Esposito, M., Kawai, R., Lindenberg, K., et. al. (2010). Quantum-dot carnot engine at maxi-mum power. Physical Review E, 81, 041106, doi:10.1103/PhysRevE.81.041106.

90. Rezek, Y., & Kosloff, R. (2006). Irreversible performance of a quantum harmonic heat engine.New Journal of Physics, 8(5), 83, http://stacks.iop.org/1367-2630/8/i=5/a=083.

91. Henrich, M. J., Rempp, F., & Mahler, G. (2007). Quantum thermodynamic Otto machines:A spin-system approach. The European Physical Journal Special Topics, 151(1), 157–165,ISSN 1951–6355. doi:10.1140/epjst/e2007-00371-8. http://dx.doi.org/10.1140/epjst/e2007-00371-8.

92. Allahverdyan, A. E., Johal, R. S., & Mahler, G. (2008). Work extremum principle: Struc-ture and function of quantum heat engines. Physicak Review E, 77, 041118, doi:10.1103/PhysRevE.77.041118.

93. Abah, O., Roßnagel, J., Jacob, G., et. al. (2012). Single-Ion heat engine at maximum power.Physical Review Letters, 109, 203006, doi:10.1103/PhysRevLett.109.203006.

94. Sinitsyn, N. A. (2011). Fluctuation relation for heat engines. Journal of Physics A: Mathemat-ical and Theoretical, 44(40), 405001, http://stacks.iop.org/1751-8121/44/i=40/a=405001.

95. Lahiri, S., Rana, S., & Jayannavar, A. M. (2012). Fluctuation relations for heat engines intime-periodic steady states. Journal of Physics A: Mathematical and Theoretical, 45(46),465001, http://stacks.iop.org/1751-8121/45/i=46/a=465001.

96. Chambadal, P. (1957). Les Centrales nucléaires. Colin: Collection Armand Colin, http://books.google.cz/books?id=TX8KAAAAMAAJ.

97. Novikov, I. I. (1958). The efficiency of atomic power stations. Journal of Nuclear Energy II,7, 125.

98. Curzon, F. L., & Ahlborn, B. (1975). Efficiency of a Carnot engine at maximum poweroutput. American Journal of Physics, 43(1), 22–24. doi:10.1119/1.10023, http://link.aip.org/link/?AJP/43/22/1.

99. Durmayaz, A., Sogut, O. S., Sahin, B., et. al. (2004). Optimization of thermal systemsbased on finite-time thermodynamics and thermoeconomics. Progress in Energy and Com-bustion Science, 30(2), 175–217, ISSN 0360–1285. doi:10.1016/j.pecs.2003.10.003, http://www.sciencedirect.com/science/article/pii/S0360128503000777.

100. Tu, Z. C.(2008). Efficiency at maximum power of Feynman’s ratchet as a heat engine. Jour-nal of Physics A: Mathematical and Theoretical, 41(31), 312003, http://stacks.iop.org/1751-8121/41/i=31/a=312003.

101. Callen, H. (2006). Thermodynamics & an intro to thermostatistics. (Student ed.), New York:Wiley India Pvt. Limited, ISBN 9788126508129. http://books.google.cz/books?id=uOiZB_2y5pIC.

46 2 Stochastic Thermodynamics

102. Holubec, V. (2009). Nonequilibrium thermodynamics of smallsystems. Diploma Thesis,Charles University in Prague, Faculty of Mathematics and Physics.

103. Chvosta, P., Holubec, V., Ryabov, A., et. al. (2010). Thermodynamics of two-strokeengine based on periodically driven two-level system. Physica E: Low-dimensional Sys-tems and Nanostructures, 42(3), 472–476, ISSN 1386–9477. http://dx.doi.org/10.1016/j.physe.2009.06.031; Proceedings of the International Conference Frontiers of Quantum andMesoscopic Thermodynamics FQMT ’08, http://www.sciencedirect.com/science/article/pii/S1386947709002380.

Chapter 3Discrete State Space Models

3.1 Two-Level System

Consider a two-level system (N = 2) with the general transition rate matrix

L(t) =(

νU(t) −νD(t)−νU(t) νD(t)

). (3.1)

The transition rate from the level 1 to the level 2 equals βL21(t) = −βνU(t). Sim-ilarly, the transitions from the level 2 to the level 1 occur with the rate βL12(t) =−βνD(t). The model is depicted in Fig.3.1.

3.1.1 Dynamics

The master Eq.(2.3) with the general transition rate matrix (3.1) can be solved ana-lytically using the variation of parameters [1]. The result is

R(t | t ′) = 11 − 1

2

(1 −1

−1 1

)

1 − exp

⎡

⎣−t∫

t ′dt ′′δ+(t ′′)

⎤

⎦

⎫⎬

⎭

+ 1

2

(−1 −11 1

) t∫

t ′dt ′′ exp

⎡

⎣−t∫

t ′′dt ′′′δ+(t ′′′)

⎤

⎦ δ−(t ′′), (3.2)

where δ±(t) = β[ νU(t) ± νD(t) ]. This matrix describes dynamics of the system.

V. Holubec, Non-equilibrium Energy Transformation Processes, Springer Theses, 47DOI: 10.1007/978-3-319-07091-9_3, © Springer International Publishing Switzerland 2014

48 3 Discrete State Space Models

Fig. 3.1 The two-level model

3.1.2 Energetics

The master Eq.(2.3) for N = 2 can be solved for arbitrary transition rates. On thecontrary, exact solutions of the corresponding Eq.(2.13), which describes energeticsduring a given process, are known only for few specific settings. In the followingwe give two exactly solvable models [2, 3] and one model which was, up to now,solved only approximately, nevertheless is quite important from the physical pointof view [4]. Without loose of generality we assume for all the three models that thedegeneracy of the two levels, �i , equals 1, i.e., the free energies (2.1) and energies(2.2) coincide. We consider the symmetric, linear form of the driving

E1(t) = F1(t) = −E2(t) = −F2(t) = U

2+ V

2(t − t ′), (3.3)

i.e., the gap between the two energy levels changes with the constant velocity V/2.The matrix E(t) in Eq.(2.13) then assumes the form

E(t) = V

2diag{1,−1}. (3.4)

In all the three settings we assume that the transition rates νL(t), νR(t) fulfill the(time local) detailed balance condition (2.4), i.e.,

νD(t) = exp {−σ [E1(t) − E2(t)]} νU(t) = exp [2σE2(t)] νU(t). (3.5)

In such case the rate matrix (3.1) reads

L(t) = νU(t)

(1 − exp (−σU ) exp

[−σV (t − t ′)]

−1 exp (−σU ) exp[−σV (t − t ′)

])

(3.6)

and the only function which remains to be specified is the rate νU(t). Assume thatthe two levels are separated by the free energy barrier (see Fig.3.1)

�(t) = � − V�(t − t ′), (3.7)

3.1 Two-Level System 49

which decreases/increases with the constant velocity V�. Intuitively, the transitionrate νU(t) should be a decreasing function of the barrier height �(t). In the followingwe give three examples of possible physical choices of νU(t).

3.1.2.1 Metropolis Scenario

The simplest possible choice reads νU(t) = exp [−2σE2(t)], i.e., νD(t) = 1. It isinspired by Monte Carlo simulations, namely by the so-called Metropolis algorithm[5]. In the simulations, the Metropolis algorithm is used for its fast relaxation ofthe system towards the thermal equilibrium. It does not describe well the relaxationprocesses themselves. However, it turns out that the choice νU(t) = exp [−2σE2(t)]is mathematically very similar to the more physical choice

νU(t) = exp [−σ�(t)] , V� = V . (3.8)

We call this choice of the transition scenario the Metropolis scenario.The resulting Eq. (2.13) for this model can be written in the form

∂

∂ξG(ρ, ξ ) = −

⎧(1 00 −1

)∂

∂ρ+ a

(exp(ξ ) −c

− exp(ξ ) c

)⎪G(ρ, ξ ), (3.9)

with the initial condition G(ρ, 0) = ς(ρ)11. In Eq.(3.9) we used the abbreviations

ξ = ξ (t, t ′) = σ|V |(t − t ′),ρ = ρ(w,w′) = 2σ(w − w′),

a = β

σ|V | exp(−σ�), (3.10)

c = exp

(−σU

|V |V

).

Solution of Eq.(3.9) for U = � = 0 was derived in [2] (see also [3]). However,following similar lines as in [2] one can solve Eq.(3.9) also for a general U and �.This solution, for V → 0, reads

G11(ρ, ξ ) = ς(ξ − ρ) exp[a(1 − eξ )] + a2c

2�(−ξ , ξ ; ρ) yac

(1

x− 1

)

× exp

⎧a

(1 − 1

x

)⎪

1F1 (1 − ac, 2;χ) , (3.11)

G12(ρ, ξ ) = ac

2�(−ξ , ξ ; ρ) yac exp

⎧a

(1 − 1

x

)⎪

1F1 (1 − ac, 1;χ) , (3.12)

G21(ρ, ξ ) = a

2�(−ξ , ξ ; ρ)

yac

xexp

⎧a

(1 − 1

x

)⎪

1F1 (−ac, 1;χ) , (3.13)

50 3 Discrete State Space Models

G22(ρ, ξ ) = ς(ξ + ρ)e−acξ + a2c

2�(−ξ , ξ ; ρ)

yac

x

(1

y− 1

)

× exp

⎧a

(1 − 1

x

)⎪

1F1 (1 − ac, 2;χ) , (3.14)

where the auxiliary functions x , y and χ are defined by the formulas

x = x(ρ, ξ ) = exp

(−ξ + ρ

2

), (3.15)

y = y(ρ, ξ ) = exp

(−ξ − ρ

2

), (3.16)

χ = χ(ρ, ξ ) = −a

(1 − 1

x

) (1 − 1

y

). (3.17)

The solution of Eq.(3.9) for V < 0 follows from interchanging the indices 1 and 2 inEqs.(3.11)–(3.14). In these equation the terms proportional to the Dirac ς-function,ς(·), represent singular parts of the diagonal elements of the matrix G(ρ, ξ ). Theirweights equal to the relative frequencies of the trajectories of the process D(t) thatdwell in a given microstate during the whole time interval [t ′, t]. The non-singularparts of the functions (3.11)–(3.14) possess finite supports, i.e., they are proportionalto the differences of the unit-step functions �(a, b; x) = �(x − a) − �(x − b).The probability density for the work done on a system with a discrete state space, forfinite time processes, always consists of these two parts. The singular part of the workprobability density (WPD) can be always obtained analytically (see Appendix B).The symbol 1F1(a, b; z) stands for the Kummer function of the variable z with theparameters a and b [6, 7]. Finally, the matrix G(w, t | w′, t ′) can be calculated fromthe matrix G(ρ, ξ ) as

G(w, t | w′, t ′) = 2σ G[2σ(w − w′),σ|V |(t − t ′)

]. (3.18)

3.1.2.2 Heat Bath Scenario

The choice νU(t) = 1/(1 + exp{σ�(t)]}−1, which is often referred to as theGlauber form [8], is also inspired by the Monte Carlo simulations, namely by theso-called heat-bath algorithm [9]. This algorithm is customarily used to simulatenon-equilibrium dynamics of, for example, spin systems [10]. For the spin sys-tems one often identifies the free energy barrier �(t) with the energy differenceE2(t) − E1(t) = 2E2(t). In such case the transition rate νU(t) reads

νU(t) = 1

1 + exp[2σE2(t)] . (3.19)

We call this choice of the transition scenario the Heat bath scenario.

3.1 Two-Level System 51

It is interesting to note that the finite time thermodynamics of this system at afixed temperature has been studied recently in connection with a single-level fermionmodel for a quantum dot that interacts with a thermal reservoir (metal lead) througha tunneling junction [11]. Within a description based on a quantum master equa-tion (neglect of quantum coherency and entanglement between the system and thereservoir), the level E2(t) in the rates (3.19), being valid in a wide band approxima-tion, then represents the chemical potential of the metal lead. When changing thegap between the level and the chemical potential in a given time interval, the optimalprotocol [E1(t)−E2(t)] for extracting the maximal work was shown to exhibit jumpsat the beginning and end of the time interval (see [11] for a further discussion and[12] for related findings). In this thesis, in Sect.5.1, we present a detailed analysis ofa heat engine based on the hereby described two-level system (see also [13–15]).

The resulting Eq.(2.13) for this model can be written in the form

∂

∂ξG(ρ, ξ ) = −

⎧(1 00 −1

)∂

∂ρ+ a

1 + ce−ξ

(1 −ce−ξ

−1 ce−ξ

)⎪G(ρ, ξ ), (3.20)

with the initial condition G(ρ, 0) = ς(ρ)11. In this equation we again used theabbreviations (3.10), but now with � = 0. Solution of Eq.(3.20) for U = 0 (c = 1)was derived in [3]. The solution for a general U was obtained in [13] (see also [15])and, for V → 0, reads

G11(ρ, ξ ) =⎧(1 + c) exp(−ξ )

1 + c exp(−ξ )

⎪a

ς(ξ − ρ) + �(−ξ , ξ ; ρ)ac

2xa(1 − x)y

×⎧− 1

(1 + cx)1+a(1 + cy)1−a 2F1(1 + a,−a; 1;χ) (3.21)

+ (1 + a)(1 + c)(1 + cxy)

(1 + cx)2+a(1 + cy)2−a 2F1(2 + a, 1 − a; 2;χ)

⎪,

G12(ρ, ξ ) = 1

2�(−ξ , ξ ; ρ) acxa y

2F1(a, 1 − a; 1;χ)

(1 + cx)a(1 + cy)1−a, (3.22)

G21(ρ, ξ ) = 1

2�(−ξ , ξ ; ρ) axa 2F1(1 + a,−a; 1;χ)

(1 + cx)1+a(1 + cy)−a, (3.23)

G22(ρ, ξ ) =⎧

1 + c exp(−ξ )

1 + c

⎪a

ς(ξ + ρ) + �(−ξ , ξ ; ρ)ac

2xa(1 − y)

×⎧+ 1

(1 + cx)1+a(1 + cy)1−a 2F1(a, 1 − a; 1;χ) − (3.24)

− (1 − a)(1 + c)(1 + cxy)

(1 + cx)2+a(1 + cy)2−a 2F1(1 + a, 2 − a; 2;χ)

⎪,

where the auxiliary functions x and y are again given by Eqs.(3.15) and (3.16). Thefunction χ now reads

52 3 Discrete State Space Models

χ = χ(ρ, ξ ) = −c1 − x

1 + cx

1 − y

1 + cy. (3.25)

The solution for V < 0 follows from interchanging the indices 1 and 2 inEqs.(3.21)–(3.24). In these Eqs.the terms proportional to the Dirac ς-function, ς(·),and those proportional to the differences of the unit-step functions, �(a, b; x), havethe same meaning as the corresponding terms in Eqs.(3.11)–(3.14). The symbol2F1(a, b, c; z) stands for the Gauss hypergeometric function of the variable z with theparameters a, b and c [7, 16]. Finally, the matrix G(w, t | w′, t ′) can be calculatedfrom the matrix G(ρ, ξ ) using Eq.(3.18).

3.1.2.3 Diffusive Scenario

The function νU(t) used for diffusive systems assumes the form [17]

νU(t) = exp[−σ�(t)], (3.26)

where the parameters of the barrier height � and V� are general. We call this choiceof the transition scenario the Diffusive scenario. The resulting Eq.(2.13) for thismodel can be written in the form

∂

∂ξG(ρ, ξ ) = −

⎧(1 00 −1

)∂

∂ρ+ a e

V�V ξ

(1 −ce−ξ

−1 ce−ξ

)⎪G(ρ, ξ ), (3.27)

with the initial condition G(ρ, 0) = ς(ρ)11. Here we used again the abbreviations(3.10). For V� = V one regains the exactly solvable Eq.(3.9). However, for V� ∞= Vnew computational difficulties in comparison with Eqs.(3.9) and (3.20) arises andEq.(3.27) was not solved, yet. An approximate solution using saddle point techniquewas given in [4], where the authors theoretically studied unfolding of a specificallydesigned DNA hairpin that, during experiments with optical tweezers, shows two-state cooperative unfolding [18]. Analysis of such systems shows usefulness of themethod of computer simulations described in Appendix C (see also [19]).

3.2 Ehrenfest Model

Consider the traditional discrete-time Ehrenfest urn model [20–22] with the totalnumber of balls in the two urns NB (see the illustration in Fig.3.2). In each roundof the system evolution, a ball is selected with the probability 1/NB and moved tothe other urn. The stochastic process of moving the balls between the two urns canbe described as a Markov chain. We adopt the continuous-time generalization of theEhrenfest model introduced in [23], where the time intervals between the individualmoves are taken as independent exponentially distributed random variables.

3.2 Ehrenfest Model 53

Fig. 3.2 The Ehrenfest model

Let us denote the (random) number of balls in the left urn at the time t as M(t). Wewill investigate the stochastic process D(t) = M(t) + 1, which assumes the values1, 2, . . . , NB + 1. In order to describe dynamics and energetics of this process, wecan use all the formulas introduced in Sect.2.1. Assume that the balls in the left(right) urn are moved to the other urn with the transition rate βνR(t) [βνL(t)], wherethe time dependence of the rates stems from an external manipulation with the twourns. In such case the rate matrix L(t), which determines via Eq.(2.3) the systemdynamics, for the Ehrenfest model reads [23]

⎨

⎩⎩⎩⎩⎩⎩⎩⎩⎩⎩⎩⎩⎝

νL(t) −νR(t)

NB0 . . . 0

−νL(t)

⎧νR(t)

NB+ (NB − 1)νL(t)

NB

⎪−2νR(t)

NB

. . ....

0 − (NB − 1)νL(t)

NB

. . .. . . 0

.... . .

. . .. . . −νR(t)

0 . . . 0 −νL(t)

NBνR(t)

⎞

⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎠

. (3.28)

3.2.1 Dynamics

The Ehrenfest model describes dynamics of NB uncoupled two-level systems (balls)[23]. Hence it should be possible to compose the solution of Eq.(2.3) for a generalNB from its solutions for NB = 1. Indeed, for time independent transition rates, suchsolution was constructed in [23]. For time dependent transition rates one can proceedin a similar way. Specifically, let T(t | t ′) denote the solution of Eq.(2.3) with therate matrix

54 3 Discrete State Space Models

LT(t) = 1

NB

(νL(t) −νR(t)

−νL(t) νR(t)

), (3.29)

i.e., the matrix T(t | t ′) is given by Eq.(3.2) with δ±(t) = β/NB[ νL(t) ± νR(t) ].Then the solution of Eq.(2.3) with the rate matrix (3.28) reads

Rm+1n+1(t | t ′) =m∑

k=0

(n

k

)(NB − n

m − k

)[T11(t | t ′)

]NB−n−m+k [T21(t | t ′)

]m−k

× [T12(t | t ′)

]n−k [T22(t | t ′)

]k, (3.30)

where m (n) stands for the number of balls in the left urn at the time t (t ′). In Eq.(3.30)we assume that

(ab

)equals zero whenever b > a.

3.2.2 Energetics

In order to study the energetics of the Ehrenfest model, one has to specify the levelenergies (2.2). We assume that whenever the ball is in the left urn, its energy equals−E(t)/2 and, similarly, whenever the ball is in the right urn, its energy reads E(t)/2(see Fig.3.2). Such setting yields the level energies

Ei (t) = −(i − 1)E(t)

2+ (NB − i + 1)

E(t)

2= E(t)

(NB

2− i + 1

)(3.31)

and, hence, the matrix E(t) in Eq.(2.13) assumes the form

E(t) = E(t)

⎧NB

211 − diag{0, 1, . . . , NB}

⎪. (3.32)

As in the case of the master Eq.(2.3), it should be possible to compose the solutionof Eq.(2.13), with the matrices L(t) and E(t) given by Eqs.(3.28) and (3.32), from“one ball solutions”. Indeed, let J(w, t | w′, t ′) solve Eq.(2.13) with the rate matrix(3.29) and with the matrix E(t) defined as

ET(t) = E(t)

2diag{1,−1}. (3.33)

Then the matrix Gm+1n+1(w, t | w′, t ′

)for the Ehrenfest model with NB ball reads

Gm+1n+1(w, t | w′, t ′) =m∑

k=0

(n

k

)(NB − n

m − k

)(J11 ∈ · · · ∈ J11︸ ︷︷ ︸

NB−n−m+k

∈ J21 ∈ · · · ∈ J21︸ ︷︷ ︸m−k

∈

3.2 Ehrenfest Model 55

∈ J12 ∈ · · · ∈ J12︸ ︷︷ ︸n−k

∈ J22 ∈ · · · ∈ J22︸ ︷︷ ︸k

) ([w], t

∣∣∣∣

w′

NB, t ′

),

(3.34)

where (A1 ∈ A2 ∈ · · · ∈ An)([w], t | w′, t ′) = ∫ ∞−∞ dw′′ A1(w − w′′ | w′, t ′) B(w′′,

t | w′, t ′), B(w, t | w′, t ′) = (A2 ∈ · · · ∈ An)([w], t | w′, t ′), denotes the multi-convolution in the work variable w. Validity of this result can be proven by the directsubstitution of the Laplace (Fourier) transformed formula (3.34) into the Laplace(Fourier) transformed system (2.13). Two specific examples of the transition ratesνL(t) and νR(t) and of the driving E(t) when the matrix J(w, t | w′, t ′) can beobtained analytically are discussed in Sect.3.1.

If the transition rates assume the Glauber form (3.19) (see also [8] and Sect.3.1.2)the presented Ehrenfest model describes dynamics and energetics of the system of NBnon-interacting spins in a time dependent magnetic field. In such case M(t) and D(t)reflect the total magnetization of the system at the time t and W(t, t ′) correspondsto the (random) work needed to arrange the individual spins. The work distribution(2.16) for this system for the case of large NB was investigated analytically in theworks [24, 25]. It was found that the WPD exhibits exponential tails: π(w, t) ∝exp[−NB χ(w, t)]. On the other hand, for an arbitrary NB, the work distribution wasinvestigated only using Monte Carlo simulations [26]. As far as we know this is forthe first time the exact WPD for an arbitrary NB, given by Eqs.(3.21)–(3.24) and(3.34), is presented.

3.3 Infinite-Level Model

Consider a system with infinite number of energy levels and with the infinite three-diagonal transition rate matrix

L(t) = [νR(t) + νL(t)]11 − νR(t)ER − νL(t)EL, (3.35)

where 11 stands for the infinite unity matrix and the matrices ER and EL have thematrix elements

[ER]i j = ςi j+1, [EL]i j = ςi j−1, i = . . . ,−1, 0, 1, . . . . (3.36)

Here ςi j stands for the Kronecker ς. The transition rate from the level i to the leveli + 1 thus is, for any integer i , given by βνR(t). Similarly, the transitions from thelevel i to the level i − 1 occur, for any integer i , with the rate βνL(t).

56 3 Discrete State Space Models

3.3.1 Dynamics

The dynamics of the system is described by the master Eq.(2.3) with the rate matrix(3.35). Owing to the special form of the matrices ER and EL this Eq.can be solvedanalytically regardless the specific form of the functions νR(t) and νL(t). Below wederive the solution using the algebraic approach introduced in [27].

Formally, the solution of Eq.(2.3) reads

R(t | t ′) = exp

⎡

⎣−β

t∫

t ′dt ′′ L(t ′′)

⎤

⎦

= exp [−(�R + �L)11 + �RER + �LEL] , (3.37)

where the functions �R and �L are given by

�R = �R(t, t ′) = β

t∫

t ′dt ′′ νR(t ′′),

�L = �L(t, t ′) = β

t∫

t ′dt ′′ νL(t ′′). (3.38)

All the matrices in the exponent in Eq.(3.37) commute, i.e., EREL −ELER = 0 etc,and therefore one can rewrite the formal solution (3.37) as

R(t | t ′) = exp[−(�R + �L)] exp(�RER) exp(�LEL). (3.39)

This formula already represents the exact solution of the master Eq.(2.3). The indi-vidual matrix elements of R(t | t ′) read

[R(t | t ′)]i j = exp[−(�R + �L)]⎧�R

�L

⎪ i− j2

Ii− j

(2√

�R�L

), (3.40)

where Iβ(x) stands for the modified Bessel function of the first kind [7].

3.3.2 Energetics

Let the energies (2.2) of the individual levels of the system read

Ei (t) = −η i F(t) = −η i (F0 + V t). (3.41)

3.3 Infinite-Level Model 57

In such case the matrix E(t) in Eq.(2.13), which describes the energetics of thesystem, assumes the form

E(t) = −η VZ, [Z]i j = iςi j . (3.42)

Instead of solving Eq.(2.13) directly, we focus on its Laplace transform.1 Thismeans that we face the set of ordinary differential equations

d

dtG(s, t | t ′) = − [−ηV sZ + β{[νR(t) + νL(t)]11

−νR(t)ER − νL(t)EL}] G(s, t | t ′), (3.43)

with the initial condition G(s, t ′ | t ′) = 11. The Laplace variable s corresponds tothe work variable (w − w′). The matrices on the right-hand side of Eq.(3.43) do notcommute. In order to solve this equation we again follow the procedure proposed in[27]. Let us define two auxiliary matrices, H(s, t | t ′) and L(t), by the formulas

G(s, t | t ′) = exp[x(t − t ′)Z] H(s, t | t ′),L(t) = exp[−x(t − t ′)Z]L(t) exp[x(t − t ′)Z], (3.44)

wherex = x(s) = ηV s. (3.45)

The matrix H(s, t | t ′) obeys the set of ordinary differential equations

d

dtH(s, t | t ′) = −β

{[νR(t) + νL(t)]11 − νR(t)ER − νL(t)EL

}H(s, t | t ′),

(3.46)with the initial condition H(s, t ′ | t ′) = 11. Here the auxiliary functions νL(t) andνR(t) are given by

νR(t) = νR(s, t) = νR(t) exp(xt),

νL(t) = νL(s, t) = νL(t) exp(−xt). (3.47)

Equation (3.46) can be derived using the commutation relations ERZ−ZER = −ERand ELZ−ZEL = EL. Note that for u = 0 this equation coincide with Eq.(2.3) forthe matrix (3.37). All the matrices on the right hand side of Eq.(3.46) commute andhence, as in the case of the matrix R(t | t ′), we have

H(s, t | t ′) = exp[−(�R + �L)] exp(�RER

)exp

(�LEL

), (3.48)

where the two auxiliary functions �R and �L are defined as

1 Here we assume that the transformation exists.

58 3 Discrete State Space Models

�R = �R(s, t, t ′) = β

t∫

t ′dt ′′ νR(s, t ′′),

�L = �L(s, t, t ′) = β

t∫

t ′dt ′′ νL(s, t ′′). (3.49)

Finally, the matrix elements of the matrix G(s, t | t ′) = exp[x(t − t ′)Z] H(s, t | t ′)are given by

[G(s, t | t ′)]i j = exp[−(�R + �L)] exp(i xt)

[�R

�L

] i− j2

Ii− j

(2√

�R�L

). (3.50)

Note that G(0, t | t ′) = R(t | t ′). Once the two transition rates νL(t) and νR(t) arespecified, the matrixG(w, t | w′, t ′) can be obtained from the matrix G(s, t | t ′) usingthe inverse Laplace transform.

In closing this section let us note that if one adopts the specific form of thetransition rates

νR = D

η2 q0 exp

(−1

2ησvt

),

νL = D

η2 q−10 exp

(1

2ησvt

), (3.51)

where q0 = exp (−ησ f0/2), the present model becomes in the limit η ≥ 0equivalent to the diffusion of a particle driven by the time-dependent potentialF(x, t) = E(x, t) = η i f (t) = x f (t), f (t) = f0 + vt . In such case D standsfor the diffusion constant, the particle position reads x = η i and − f (t) is a spa-tially independent force applied on the particle. It can be seen from Eqs.(2.37) and(2.43) that, for this setting, both the particle position X(t) and the thermodynamicwork done on the particle W(t, t ′) are linear functions of the Gaussian white noiseN(t). Therefore both the solution of Eq.(2.38) and that of Eq.(2.45) are given by aGaussian distribution [28]. Calculation of the WPD for a similar model is presentedin Sect.4.2.

3.4 Kittel Zipper

The following work was published with minor modifications in [29].

3.4 Kittel Zipper 59

3.4.1 Introduction

The Watson-Crick double-stranded form represents the thermodynamically stablestate of DNA in a wide range of temperature and salt conditions. However, even atstandard physiologic conditions, there always exists possibility that the double-helixis locally unzipped into two strands both at its ends and in its interior [30–34]. If theinterior unzipping is neglected, the unfolding of the two strands can be described bya simple zipper model [35–37]. This model, while including several simplifications,emphasizes the essential ingredient of the unzipping process, that means the com-petition between the entropic forces which tend to open the macromolecule, and theenergetic forces that tend to condense it into its double-stranded form.

In the last two decades, new experimental techniques have been developed fordetailed analysis of unzipping processes. A powerful technique are single mole-cule manipulations by optical tweezers [38–43], where biomolecules are unfoldedand refolded by applying mechanical forces. Because single molecules are sub-ject to strong fluctuations, a stochastic description of the unfolding/refolding kinet-ics becomes necessary. Investigation of these stochastic processes can be useful tounderstand how biomolecules unfold and fold under locally applied forces [44–47],as, for example, when mRNA passes through the ribosome during the translationprocess [48, 49], or when DNA is unzipped by helicase during the replication process[50, 51].

A particularly interesting part in the analysis of unzipping processes is the appli-cation of integral or detailed fluctuation theorems [43, 52–56] (see also Sect.2.3)to estimate free energy differences between folded and unfolded states (see, e.g.,[57]). Their advantage is that they can be applied also to protocols, which drivethe considered system far from equilibrium. It is thus not necessary to perform theunfolding/folding under quasi-static near-equilibrium conditions, where the processbecomes reversible. Most popular theorems are the Crooks fluctuation theorem [58]and Jarzynski equality [59]. These relate to the distribution of work performed onthe molecule during the process. Unfortunately it is not easy to get theoretical insightinto characteristics of the underlying work distributions in far-from-equilibriumprocesses, since the work is a functional of the whole stochastic trajectory. Inves-tigations have been conducted for simple spin systems driven by a time-dependentexternal field [2, 3, 15, 26, 60, 61] and for diffusion processes in the presence ofa time-dependent potential [28, 62]. Analytical solutions are known for two-levelsystems [2–4, 15] and for systems with one continuous state variable [28, 62].

In this section we present analytical results for the work distribution for a multi-state system, which is motivated by a model proposed by Kittel for describing themelting transition of DNA molecules [37]. In order to derive analytical results, wehad to consider a stochastic process with directed forward and forbidden backwardtransitions between states. As a consequence, detailed balance (2.4) is broken andfluctuation theorems, as mentioned above, will not hold true. On the other hand,kinetic Monte Carlo simulations of the stochastic process show that, for at least certainparameter settings in the model, the restriction of forbidden backward transitions is

60 3 Discrete State Space Models

Fig. 3.3 The zipper modelF

F

pH

not so severe and events dominating the integrand in the Jarzynski equality are notvery rare. It is important to point out that these parameter settings are not relatedto any experimental conditions. In more realistic settings rare events would playthe decisive role and in such cases it becomes difficult to determine tails of thework distribution with sufficient accuracy (see, for example, Appendix B). Also therelation of the work considered in our study to the thermodynamic work measured inan unzipping experiments needs to be treated with care. These problems are discussedin detail in Sect.3.4.2 and they imply that the theory cannot be applied to experimentsat this stage. Nevertheless, our findings should be useful because the connection toexperiments seems not completely out of reach and because they widen the range ofhopping models, where analytical results for work distributions are available.

3.4.2 Unzipping in an Extended Kittel Model

Kittel’s model [37] is a simplified version of the Poland-Scheraga model [30] for theequilibrium properties of DNA molecules, which got renewed attention in the last tenyears [63]. In contrast to the Poland-Scheraga model, it disregards the possibility ofany interior openings (bubbles) of double-stranded parts of the molecule. Despite thissimplification, it is already sufficient to understand the origin of a melting transition.

A molecule in Kittel’s model [37] has N links in its fully folded state, see Fig.3.3.Different energy levels k = 1, . . . , N of the molecule refer to configurations, where(k − 1) of the links are opened in a row. The difference Uk − U1 of the internalenergy of a microstate with (k − 1) open links and the ground level k = 1 (no openlinks) is equal to (k − 1)�, corresponding to a loss of chemical bond energy � perbroken link. With each broken link, the molecule gains G degrees of freedom, whichcharacterize the win of conformational degrees of freedom when double-strandedDNA is transferred to single stranded DNA. The entropy difference Sk −S1 betweenthe level k and the ground level then becomes kB ln Gk−1 = kB(k − 1) ln G, i.e., themultiplicity�k of the level k is Gk−1. The free energyGk of the level k at a temperatureT is thus given by Gk = G0 + (k − 1)[�− kBT ln G] and the equilibrium propertiescan be readily worked out by considering the partition sum Z = ∑N

k=1 exp(−σGk).In extending this model to treat unfolding kinetics, the molecule is supposed to

unzip successively, one link in each step, as a consequence of a strong externaldriving. Such external driving can be caused by a local force or a pH gradient, asindicated in Fig.3.3. With respect to pulling experiments on single DNA molecules

3.4 Kittel Zipper 61

out of equilibrium, there is evidence that a neglect of interior openings can becomeeven less relevant than for the equilibrium properties. For example, far away fromthe melting temperature, e.g. at standard room temperature conditions (298 K), theunfolding kinetics of DNA hairpin molecules could be successfully described byassuming no interior openings and thermally activated unzipping transitions [4, 45].Moreover, molecular fraying can be identified in individual unzipping trajectoriesby considering the size of force jumps. Thereby microstates with interior openingscan be systematically excluded from the analysis [64].

We assume that the driving lowers the energy differences by an amount propor-tional to (k − 1)t , i.e., open microstates with a larger number of broken links arefavored with increasing time. If we take the ground level energy as the referencepoint, E1 = 0, we obtain for the free energies (2.1) and energies (2.2) of the individ-ual levels

Fk(t) = (k − 1)(� − vt − kBT ln G) , k = 1, . . . , N , (3.52)

Ek(t) = (k − 1)(� − vt) , k = 1, . . . , N , (3.53)

where the parameter v has the dimension of an energy rate and characterizes differentspeeds of the unfolding in response to different strengths of the external driving.

Following established theoretical descriptions for the rupture kinetics of the bonds[4, 18, 64], the Kramers-Bell form [65] is used for the time-dependent transition rateνkk+1 from level k to level (k + 1),

νkk+1(t) = β exp[−σFkk+1(t)] . (3.54)

Here β is an attempt frequency and Fk,k+1(t) denotes the free energy barrier for thetransition, i.e., the difference of the free energy at the saddle point separating levelsk, (k + 1) and the free energy Fk in level k, see Fig.3.1. The barrier Fkk+1(t) isconsidered to be composed of a bare, k-independent free energy barrier, Fb, which ismodified by an amount proportional to the free energy difference, [Fk+1(t)−Fk(t)],

Fkk+1(t) = Fb + δ[Fk+1(t) − Fk(t)] = Fb + δ[� − vt − kBT ln G]. (3.55)

In the following we set δ = 1.2 Note that Fkk+1(t) and thus ν(t) ≤ νkk+1(t) inEq.(3.54) are independent of k. The attempt frequency β in Eq.(3.54) was reported[66] to be approximately proportional to the ratio of the diffusion constant of themolecule in water to the water viscosity. We assume here a linear dependence of thisratio on the temperature T , i.e., we take β = β(T/T0), where β and T0 are positiveconstants. Thus we obtain

ν(t) = νkk+1(t) = g exp[−σ(d − vt)], (3.56)

2 Smaller (larger) δ would correspond to saddle points lying closer to level k(k +1) in configurationspace. The derivations in Sects.3.4.3 and 3.4.4 can be performed analogously for any δ and Fb.

62 3 Discrete State Space Models

where

g = βGT

T0, d = � + Fb . (3.57)

If the (time local) detailed balance (2.4) is obeyed, i.e., νkk+1(t) exp[−σFk(t)] =νk+1k(t) exp[−σFk+1(t)], the rates νk+1k(t) ≤ νb for backward (refolding) transi-tions become both independent of k and t ,

νb = νk+1,k(t) = g

Gexp(−σFb). (3.58)

Note that G appears here in the denominator, which means that the ratio νb/ν(t) =exp(σ�−σvt)/G of backward to forward rates becomes small for large degeneracyfactors G. This reflects the fact that it is difficult for the flexible unfolded part ofthe molecule to find proper configurations, which would allow for a reformation of(hydrogen) bonds. The model with the rates given by (3.56) and (3.57) for N = 2(the molecule represented by a two-level system) is exactly solvable (for details seeSect.3.1.2).

If the model would refer to the hopping motion of a particle between time-dependent energy levels Ek(t), the work performed on the system (for one realizationof the stochastic process), (2.10), would be

W(t, 0) = W(t) =t∫

0

dt ′N∑

k=1

Ek(t′)ςkD(t ′)

= [ED(t)(t) − ED(t)(tD(t))] +D(t)−1∑

k=1

[Ek(tk+1) − Ek(tk)]

= [FD(t)(t) − FD(t)(tD(t))] +D(t)−1∑

k=1

[Fk(tk+1) − Fk(tk)], (3.59)

where D(t) denotes the (random) microstate of the molecule at time t and tk is the(random) time at which the transition from level k to (k + 1) (rupture of kth bond)takes place (t1 = t ′ = 0 being the initial time). The last line in Eq.(3.59) followsfrom the fact that the entropy in the extended Kittel model is not dependent on time,i.e., for given k, differences between internal energies and between free energies atdistinct times equal.

The work in Eq.(3.59) can be related to the thermodynamic work W f in an unzip-ping experiment under force control. As pointed out in [67], work in thermodynamicsis the internal energy transferred to a system upon changing the control parameters forgiven system configuration (see also Sects. 2.2.3 and 4.1). This means that, when theforce f = f (t) is the control variable and the molecular extension X the conjugate

3.4 Kittel Zipper 63

configurational variable, one has W f (t) = − ∫ f (t)f (0) d f X.3 Stochastic changes of the

molecular extension occur mainly due to bond rupture, while in between transitionsthe response will be rather smooth and, under neglect of small thermal fluctuations,can be represented by a function x(k, f ). This specifies the mean end-to-end distanceof the unfolded part in microstate k at force f (as often modeled by, e.g., the freelyjointed or worm-like chain models from polymer physics). With fk ≤ f (tk) we thenhave

W f (t) = −f (t)∫

f (0)

d f X = −f (t)∫

fD(t)

d f X(D(t), f ) −D(t)−1∑

k=1

fk+1∫

fk

d f x(k, f ). (3.60)

In more detailed energy landscape models, the free energy Ftot(k, f ) of the mole-cule in level k under loading f can be represented as Ftot(k, f ) = FF0(k) −f x(k, f )+Fstr(k, f ) (see, e.g., [45]), where F0(k) is the free energy in the absenceof loading and Fstr(k, f ) the elastic energy of the unfolded part upon stretching. Forsmooth response under stretching the latter is given by [45]

Fstr(k, f ) =x(k, f )∫

x(k,0)

dx f (k, x) = f x(k, f ) −f∫

0

d f ′ x(k, f ′) , (3.61)

where f (k, x) denotes the inverse function of x(k, f ) with respect to f . Accordingly,at a given k, the work for stretching upon increasing the force from fa to fb becomes

−fb∫

fa

d f x(k, f ) = Fstr(k, fb) − fbx(k, fb) − Fstr(k, fa) + fa x(k, fa)

= Ftot(k, fb) − Ftot(k, fa). (3.62)

This just means that the stretching at fixed k is assumed to take place quasi-statically,i.e., the variation of the force is supposed to occur on a time scale much slowerthan the correlation time of end-to-end distance fluctuations. Inserting this resultinto Eq.(3.60) yields

W f (t) = [Ftot(k, f (t)) − Ftot(k, D(t))]

+D(t)−1∑

k=1

[Ftot(k, f (tk+1)) − Ftot(k, f (tk))] , (3.63)

3 Force controlled experiments are achievable with magnetic tweezers or with optical tweezersoperating in the force clamp mode. In the latter case, the conjugate variable to the applied force fshould better be taken as the trap-pipette distance (rather than the molecular extension) because forthis definition the fluctuation theorems are obeyed. Both definitions differ only by a boundary terminvolving the final and initial forces ff and fi, respectively (for details, see [67]).

64 3 Discrete State Space Models

from which it becomes clear that W f (t) equals W(t), if Ftot[k, f (t)] is identifiedwith the free energy Fk(t) in the extended Kittel model.

As said above, we were able to find analytical results for the work distributionsif N = 2 (Sect.3.1.2), or if the backward rates in Eq.(3.58) were negligible. Weare interested here in the model itself and will not make attempts to assign valuesto the parameters in the transition rates ν(t) [Eq.(3.56)] and νb [Eq.(3.58)], andappearing also in the level free energies in Eq.(3.52), which are connected to realexperiments. Nevertheless, with respect to the value of the analytical results forN > 2 in connection with fluctuation theorems, the question arises whether a neglectof backward rates could be acceptable, at least for certain parameter settings. To checkthis, we have performed kinetic Monte Carlo simulations of the stochastic process.Because the forward rate from Eq.(3.56) can be written as ν(t) = νbG exp(−σ� +σvt) and the free energy differences appearing in Eq.(3.59) by [Fk(tk+1)−Fk(tk)] =−v(k−1)(tk+1−tk), the dynamics and energetics of the model is completely specifiedby the parameters νb, G, v, and T (and N if we consider a complete unfolding). Forillustration and discussion of representative results, we use �, �/kB, and β−1 asunits for energy, temperature and time, respectively.

Simulations were performed for fixed N = 10, T = 1 and νb = 1, and a setof v and G values varying in the intervals v = 0.01 − 3.3 and G = 10 − 1000,respectively. We always started the unfolding from the fully closed microstate, i.e.,pk(0) = ςk,1.4 Probabilities pN (t) of complete unfolding (occupation of level N )until time t were determined and an unfolding time tU defined by requiring that att = tU the zipper has unfolded with a probability of 99.9 %, i.e., pN (tU) = 0.999(see Sect.3.4.5). We then considered the work distributions π(w, tU) at t = tU andthe weighted distributions exp(−σw)π(w, tU), corresponding to the integrand inthe average ≡exp(−σw)≈ = ∫

dw exp(−σw)π(w, tU) as it appears in the Jarzynskiequality. It was found that backward rates turn out to have a minor importancewhen the forward rates are much larger than the backward rates during the wholeunfolding process or when v is large enough. Specifically, the error in calculating≡exp(−σw)≈ is smaller than 5 % when v � 3 for G = 10, v � 0.1 for G = 100, andv � 0.01 for G = 1000. As representative examples we show in Fig.3.4 simulatedresults (blue lines) with nonzero backward rates in comparison with analytical results(green circles) for zero backward rates (see Sect.3.4.4) for pN (t), π(w, tU), andexp(−σw)π(w, tU), and parameters v = 0.25, G = 10 [panels labeled with a)] andG = 1000 [panels labeled with b)]. As can be seen from the figure, for the case oflarge G (small backward transitions), the simulated results for pN (t), π(w, tU), andexp(−σw)π(w, tU) are almost indistinguishable from the analytical results.

4 In this case one can show (cf. Sect.2.3) that ≡exp(−σw)≈ = χ(t) exp{−σ[F(t) − F1(0)]} =χ(t) exp[−σF(t)], where F1(0) = 0 from Eq.(3.52) and F(t) = −σ−1 ln[(AN − 1)/(A − 1)],A = exp[−σ(� + vt)]/G, is the free energy for an equilibrated system with level free energiesFk(t) (protocol variables) at time t ; χ(t) is the probability for the zipper to be in the fully closedmicrostate k = 1 under the reversed protocol, if the system is initially in to the equilibrium stateexp{σ[F(t) −Fk(t)]}. The situation when the system is initially in thermal equilibrium, and hencethe standard Jarzynski equality (2.61) is valid, is discussed in Appendix B. It turns out that in thiscase, in order to get the tails of the WPD correctly, the refolding can not be neglected.

3.4 Kittel Zipper 65

0 0.1 0.2 0.3 0.40

0.5

1

t

p N(t

)

−1 −0.5 00

2

4

6

w

0 0.2 0.40

50

t

rate

s

−1 −0.5 00

5

10

15

w

−25 −20 −15 −10 −5 00

0.1

0.2

w

ρ(w

, tU

)

ρ(w

, tU

)

−25 −20 −15 −10 −5 00

1

2x 10

6

w

e−β

wρ(

w, t

U)

e−β

wρ(

w, t

U)

0 5 100

0.5

1

t

p N(t

)

0 5 100

5

10

tra

tes (a1)

(a2)

(a3)

(b1)

(b2)

(b3)

Fig. 3.4 Simulated results of the model presented in Sect.3.4.2 with backward transition rateνb = 0.133 (blue lines) in comparison with analytical results (see Sect.3.4.4) when neglectingbackward transitions (green circles). Panels labeled with (a) and (b) refer to degeneracy factorsG = 10 and G = 1000, respectively. The remaining parameters are T = 1, v = 0.25, and N = 10.Panels (a1) and (b1) show the probability pN (t) that the zipper has unfolded up to time t and theinsets depict the time-dependence of the forward transition rates (solid line); the constant backwardrate is indicated by the dashed line. Panels (a2) and (b2) display the probability densities π(w, tU)

at the unfolding time tU (tU = 10.67 for G = 10 and tU = 0.47 for G = 1000), and panels (a3)and (b3) the weighted probability densities exp(−σw)π(w, tU)

The remaining part of this section is organized as follows. In Sect.3.4.3 we specifythe equations for the time evolution of the system state [(Eq.(2.3)] and for the quanti-ties describing the energy transformations [(Eq.(2.13)]. In Sect.3.4.4 we derive exactanalytical solutions of these equations and in Sect.3.4.5 we discuss our findings.

3.4.3 Formalization of the Model

Let pk(t), k = 1, . . . , N , be the occupation probabilities of the k-th level. Thedynamics of these functions for the above described model is governed by masterEq.(2.3) with the (N × N ) matrix of the transition rates

66 3 Discrete State Space Models

L(t) = 1

β

⎨

⎩⎩⎩⎩⎩⎩⎩⎝

ν(t) 0 . . . . . . 0

−ν(t) ν(t). . .

...

0. . .

. . .. . .

......

. . .. . . ν(t) 0

0 . . . 0 −ν(t) 0

⎞

⎟⎟⎟⎟⎟⎟⎟⎠

, (3.64)

where ν(t) is given by (3.56). Without loose of generality we will consider onlythe case when the initial condition is set at the time t ′ = 0. In such case we cansimplify the notation used in Sect.2.1. We will write R(t | 0) ≤ R(t) for the solutionof Eq.(2.3), G(w, t | 0, 0) ≤ G(w, t) for the solution of Eq.(2.13) and we will omitthe variable t ′ for other quantities such as work and heat. Moreover, in the following,we always start with the completely closed zipper, that means pk(0) = ςk1. Theindividual occupation probabilities (2.6) then read

pi (t) = Ri1(t). (3.65)

The energetics of the zipper is described by Eq.(2.13), where the matrix E(t) isthe (N × N ) diagonal matrix

E(t) = diag{E1(t), . . . , EN (t)} = −v diag{0, 1 . . . , N − 1}. (3.66)

Exact solutions of Eqs.(2.3) and (2.13) with the (N ×N ) matricesL(t) and E(t) givenby Eqs.(3.64) and (3.66) will be derived the following Sect.3.4.4. The WPD π(w, t)follows from from Eq.(2.16), the mean values and variances of the internal energy,the work and the heat can be calculated from Eqs.(2.22)–(2.24) and (2.28)–(2.30).

3.4.4 Solution of the Model

Owing to the simple two-diagonal structure of the matrix L(t) in Eq.(3.64), thesystem (2.3) can be solved by simple integrations. By defining

�(t, t ′) =t∫

t ′dt ′′ν(t ′′) = α

σv

[exp(σvt) − exp(σvt ′)

], (3.67)

where α = g exp(−σd), a recursive treatment of Eq.(2.3) yields

3.4 Kittel Zipper 67

Ri j (t) = 0, i < j,R j j (t) = e−�(t,0), j = 1, . . . , N ,

Ri j (t) =t∫

0

dt ′e−�(t,t ′) ν(t ′) Ri−1 j (t′), j < i < N ,

RN j (t) =t∫

0

dt ′ν(t ′) RN−1 j (t′), j = 1, . . . , N .

(3.68)

When solving these recursive relations, we obtain the lower triangular matrix withthe nonzero matrix elements

R j+k j (t) = [�(t, 0)]k

k! e−�(t,0), j = 1, . . . , N − 1 ; k = 0, . . . , N − 1 − j,

(3.69)

RN j (t) = [�(t, 0)]N− j

(N − j)! e−�(t,0)1F1[1, N + 1 − j;�(t, 0)], j = 1, . . . , N ,

(3.70)

where 1F1(a, b; x) denotes the confluent hypergeometric function [6, 7].For solving Eq.(2.13) we first perform a Laplace transform with respect to the

work variable w. It yields the system of ordinary differential equations

d

dtG(s, t) = [−s E(t) + L(t)

]G(s, t), G(s, 0) = I. (3.71)

The matrix which multiplies G(s, t) on the right hand side is again a lower two-diagonal one. Therefore, as in the case of Eq.(3.68), we find the recursive relation

Gi j (s, t) = 0, i < j,G j j (s, t) = e−�(t,0)e−s[E j (t)−E j (0)], j = 1, . . . , N ,

Gi j (s, t) =t∫

0dt ′e−�(t,t ′)e−s[Ei (t)−Ei (t ′)] ν(t ′) Gi−1 j (s, t ′), j < i < N ,

G N j (s, t) =t∫

0dt ′e−s[EN (t)−EN (t ′)] ν(t ′) G N−1 j (s, t ′), j = 1, . . . , N .

(3.72)Notice that the matrix G(s, t) is again a lower triangular one. We now want to solvethese recursive relations. It turns out that all matrix elements of the matrix G(s, t),except the matrix elements G N j (s, t), j = 1, . . . , N − 1, can be explicitly evaluatedby simple integrations:

G j+k j (s, t) = e−�(t,0) exp [s( j − 1)vt]

k!{

α

v(σ − s)

[exp(σvt) − exp(svt)

]}k

,

(3.73)

68 3 Discrete State Space Models

for j = N , k = 0 and also for j = 1, . . . , N − 1, k = 0, . . . , N − 1 − j . Moreover,we were able to carry out the inverse Laplace transform of these functions. Theresulting diagonal elements G j j (w, t), j = 1, . . . , N are proportional to Dirac ς-functions. The remaining ones possess a finite support, i.e., they are proportional tothe differences of the unit-step functions �(a, b; x) = �(x − a) − �(x − b),

G j j (w, t) = ς[w + ( j − 1)vt] e−�(t,0), j = 1, . . . , N , (3.74)

G j+k j (w, t) = e−�(t,0)(−α

v

)k exp {σ[w + (k + j − 1)vt]}(k − 1)!

×k∑

l=0

(−1)l

l! (k − l)!�[(1 − j − l)vt, (1 − j)vt;w]

[w + ( j + l − 1)vt]1−k. (3.75)

Equation (3.75) is valid for j = 1, . . . , N − 1 and k = 1, . . . , N − 1 − j .It remains to calculate the matrix elements G N j (s, t), j = 1, . . . , N . The inverse

Laplace transform of the last formula in the recursive scheme (3.72) is

G N j (w, t) =t∫

0

dt ′ν(t ′) G N−1 j{w − [EN (t) − EN (t ′)], t ′

}, (3.76)

where j = 1, . . . , N −1. We insert herein the explicit forms of Eqs.(3.74) and (3.75).After some algebra we finally obtain

G N N−1(w, t) = α

vexp {σ[w + (N − 1)vt]}

× exp{−�

[w

v+ (N − 1)t, 0

]}× �[−(N − 1)vt,−(N − 2)vt;w],

(3.77)

G N j (w, t) =α

(−α

v

)N−1− j

(N − 2 − j)! exp{σ[w + (N − 1)vt]}N−1− j∑

l=0

(−1)l

l! (N − 1 − j − l)!

×{

Fjl

⎧w + (N − 1)vt

v(N − j),w + (N − 1)vt

v(N − j − l); w, t

⎪�[−(N − 1)vt, −( j + l − 1)vt; w]

+ Fjl

⎧w + (N − 1)vt

v(N − j), t; w, t

⎪�[−( j + l − 1)vt, −( j − 1)vt; w]

}, (3.78)

In Eq.(3.78) j ∧ {1, . . . , N − 2} and we have introduced the abbreviation

Fjl(a, b;w, t) =b∫

a

dt ′ exp[−�

(t ′, 0

)]

× [w + (N − 1)vt − (N − j − l)vt ′

]N− j−2. (3.79)

3.4 Kittel Zipper 69

The main results of this sections are Eqs.(2.69) and (2.70), which give the solutionof the master Eqs.(2.3) and (3.74)–(3.79) which present the solution of Eq.(2.13).We now turn to the discussion of these results.

3.4.5 Discussion

In Kittel’s work [37] the equilibrium properties of the zipper are studied. The meannumber of open links in equilibrium always increases with increasing temperature,but the form of this increase is different for the degeneracy factor G = 1 (cf.Fig.3.5a1) and for G > 1 (cf. Fig.3.5a2). For G = 1, the mean number of openlinks increases smoothly with a concave curvature, while for G > 1, the curveexhibits a sharp increase in a narrow temperature interval and resembles a first-orderphase transition. In the following discussion of representative results for the non-equilibrium dynamics and energetics, we use d, d/kB, and β−1 as units for energy,temperature and time, respectively.

In the unidirectional unzipping process, the time evolution towards the completelyunzipped microstate is again sensitive to the degeneracy factor G. Let us define anunfolding time tU by the condition that at t = tU the zipper has completely unfoldedwith a probability of 99.9 %, i.e.,

pN (tU) = RN1(tU) = 1 − η, (3.80)

where η = 0.001. With respect to the N dependence of tU (and other quantities to bediscussed below), we found that its behavior is similar for N = 2 and N > 2, andwe therefore restrict the following discussion to the two-level case N = 2. Equa-tion (3.80) can then be inverted after inserting RN1(tU) from Eq.(3.70) in Eq.(3.80),yielding

tU = 1

σvln

⎧1 − σv

αln η

⎪. (3.81)

For small temperatures [large argument of the exponential and/or small prefactor gin Eq.(3.56) the transitions are driven predominantly by the magnitude of the energygap between the closed and the opened microstate. For large temperatures [smallargument of the exponential and/or large prefactor g in Eq.(3.56) by contrast, thedynamics is governed by the entropy difference between the closed and the openedmicrostate, and hence by the degeneracy factor G. The dependence of the unfoldingtime tU on G and T is plotted in Fig.3.5b for a representative set of parameters.For any fixed nonzero temperature, the unfolding time decreases with increasingG, while for fixed G, the temperature-dependence of the unfolding time exhibits amaximum at a temperature T = Tmax(G), where Tmax(G) decreases with increasingG, see Fig.3.5b.

Let us now consider a certain temperature T0 and call, for this temperature, thefast-unzipping regime and slow-unzipping regime the ranges of G-values, where

70 3 Discrete State Space Models

(b)a1)

a2)

Fig. 3.5 Left two panels Mean number of open links in equilibrium for Kittel’s molecular zipperas a function of the reservoir temperature T for N = 50 links, and a1 G = 1 and a2 G = 2. Rightpanel b: Unfolding time tU , Eq.(3.81), as a function of T and G for T0 = 7.5, v = 0.25, and N = 2.Above (below) the horizontal plane in the graph, the opened (closed) microstate is energeticallyfavored. In the base plane, the temperature Tmax of maximal unfolding time in dependence of thedegeneracy factor G is shown

Tmax(G) < T0 and Tmax(G) > T0, respectively. In these two regimes the dynamicsand energetics of the molecular zipper exhibit a qualitatively different behavior. Inparticular we find (i) a different time-dependence of the N -th level’s occupationprobability, cf. Figs.3.6b and 3.7b, (ii) a different form of the curves describing thework needed to open the zipper, cf. Figs.3.6c and 3.7c, (iii) a different mean valueof heat accepted by the zipper during the unzipping, cf. Fig. 3.8a1 and b1, and (iv)different values of the variances of the internal energy and heat during the unzipping,cf. Fig. 3.8a2 and b2. These features will be now discussed in more detail.

Figure 3.6 illustrates the slow-unzipping regime. The probability pN (t) that thezipper has reached the opened microstate until time t first increases slowly. Afterthe time d/v the energy of the opened microstate becomes lower than that of theclosed one. This leads to more frequent transitions and accordingly pN (t) increasesmore rapidly. Notice that the curves exhibit a change of their second derivative.The WPD during the unzipping has a maximum located inside its finite support, cf.Fig.3.6c. The value of the work at the left (right) border of the support equals thework done on the zipper when it dwells during the time interval [0, tU ] in the opened(closed) microstate. From the position of the WPD peak we can conclude that, for atypical trajectory of the stochastic process D(t), the work consists of two comparablefractions. The first (second) part of the work is performed while the system dwellsin the closed (opened) microstate. At time tU, 0.1 % of the trajectories will give

3.4 Kittel Zipper 71

−3.75 −2.5 −1.25 00

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

wρ(w

,tU

)0 5 10 15

0

0.5

1

t

p N( t

)

0 5 10 15−3

−2

−1

0

(a)

(b)

(c)1

t

i(t) 0.25

0.5

0.75

1

1.25

tU

Fig. 3.6 Dynamics of a the energy levels [Eq.(3.53)], b the occupation probability pN (t) of theopened microstate [Eq.(3.65)], and c the probability density π(w, tU) of the work [Eq.(2.16)] forG = 1 and several values of the reservoir temperature T . The other parameters are the same as inFig.3.5. The assignment of the line styles to the temperature given in the legend of c applies alsoto parts a and b. In a we compare the dynamics of the two energy levels with the unfolding timetU [(Eq.(3.81)] at different temperatures marked by the vertical lines. The solid (dashed) line givesthe energy E1(t) [E2(t)] of the closed (unzipped) microstate. The arrows in c represent the weightsand the positions of the ς-functions, which form the singular part of the probability density

molecules still residing in the zipped microstate. These trajectories contribute to thesingular (ς-function) components of WPDs, which are depicted in Fig.3.6c by thevertical arrows [2, 3, 15, 61]. Note that the curves plotted for the temperatures T = 1and T = 1.25, which are close to the temperature Tmax(G) for G = 1, cf. Fig.3.5b,become similar to the curves for these temperatures obtained in the fast-unzippingregime for G = 50, see Fig.3.7.

Figure 3.7 illustrates the fast-unzipping regime. The probability pN (t) rapidlyincreases from the very beginning of the process, cf. Fig.3.7b, i.e., the zipper opensbefore the opened microstate becomes energetically preferred. The maximum of theWPD is located at the left border of its support, cf. Fig.3.7c. This means that, for themajority of the trajectories, the substantial part of the work is done while the systemdwells in the opened microstate. Note that the curves plotted for the temperatureT = 0.25, which is close to the boundary temperature Tmax(G) for G = 50, cf.Fig.3.5b, become similar to the curves for this temperature in the fast-unzippingregime for G = 1, see Fig.3.6.

Figure 3.8 illustrates the dynamics of the thermodynamic quantities (2.22)–(2.30)in the two unzipping regimes. For an arbitrary trajectory of D(t) which residesduring the time interval [t ′, t] in the i-th level, the work performed on the system isEi (t)−Ei (t ′), cf. Eq.(3.59). In our model, the energies of the levels decrease linearly

72 3 Discrete State Space Models

−1.5 −1 −0.5 00

2

4

6

8

10

12

14

16

wρ(w

,tU

)0 2 4 6

0

0.5

1

t

p N( t

)

0

(a)

(b)

(c)

2 4 6−0.5

0

0.5

1

t

i(t) 0.25

0.5

0.75

11.25

tU

Fig. 3.7 Same quantities as in Fig.3.6 for G = 50 and otherwise the same set of parameters

0 2 4 6 8 10 12−2.25

−1.25

−0.5

0.25

t

mea

nva

lues

0 5 100

0.19

0.38

0.56

t

vari

ance

s

0 1 2 3−1

−0.5

0

0.5

1

t

mea

nva

lues

0 1 2 30

0.06

0.13

0.19

0.25

t

vari

ance

s

U(t)

W(t )

Q(t )

[ΔU(t )]2

[ΔW(t )]2

[ΔQ(t )]2

b1)

b2)a2)

a1)

Fig. 3.8 Upper two panels Mean value of the internal energy [(Eq.(2.22)], work [(Eq.(2.23)] andheat [(Eq.(2.24)] as a function of time for a1 G = 1 and b1 G = 50. Lower two panels Variancesof internal energy [(Eq.(2.28)], work [(Eq.(2.29)] and heat [(Eq.(2.30)] as a function of time fora2 G = 1 and b2 G = 50. The temperature is T = 0.75 and the other parameters are the same asin Fig.3.5

with time and accordingly the mean work is a monotonically decreasing function oftime, see Fig. 3.8a1 and b1. Heat is exchanged with the reservoir when the moleculechanges its microstate. It is absorbed by the molecule if the transition brings the

3.4 Kittel Zipper 73

molecular zipper to a level with higher energy. Since in our setting the transitionsare unidirectional, the molecule necessarily absorbs heat up to the time tE = d/v,where the energies of the levels become the same, cf. Figs.3.6a and 3.7a. For timest > tE , the molecule delivers heat to the environment. In the slow-unzipping regimewe have tU > tE . This implies that the mean heat first increases and then decreases,cf. Fig.3.8a1. By contrast, in the fast-unzipping regime where tU < tE , the meanheat monotonically increases, cf. Fig.3.8b1. Finally, due to the transitions to the levelwith higher energy at the very beginning of the process, cf. Figs.3.6 and 3.7, the meaninternal energy (2.22) develops a single maximum.

The variances of U(t), W(t) and Q(t), cf. Eqs.(2.28)–(2.30), are plotted inFig. 3.8a2 and b2. If the variance of the internal energy approaches zero, the varianceof the work becomes equal to that of the heat. All variances approach constant valuesat large times. In fact, for t > tU , almost all trajectories have brought the moleculeinto the opened microstate. As a consequence, the increments to the work, heat andinternal energy are nearly constant during the time interval [tU, t]. Therefore the formof their probability densities does not change, the curves just move along the energyaxis.

The probability density for the internal energy has no continuous component. Ingeneral it consists of N ς-functions located at the energies of the individual levels.The corresponding weights are given by the occupation probabilities of the levels. Inthe slow-unzipping regime, [�U (t)]2 vanishes at time t = 0 and at time tE = d/v,when the level energies are equal, and it becomes very close to zero for t � tU.In between these time instants, the variance develops a maximum, cf. Fig.3.8a2. Inthe fast-unzipping regime, the function [�U (t)]2 displays just one maximum, cf.Fig.3.8b2, because tU < tE .

In both the fast and slow unzipping regime, the variance of the work monotonicallyincreases and approaches a constant for large times, cf. Fig. 3.8a2 and b2. The absolutevalue of the first derivative of the variance [�Q(t)]2 is given by the product of thetwo quantities, the energy difference between the closed and the opened microstatesat time t and the probability that the zipper opens at the time t , which were shown inFigs.3.6 and 3.7. In the slow-unzipping regime, the majority of the transitions occursduring the time interval [tE , tU]. During this time interval, the energy differencebetween the levels monotonically increases. In the fast-unzipping regime, by contrast,nearly all transitions take place before time tE/2. The sooner the transition, the largeris the amount of transferred heat. As a result, at small times, the heat probabilitydensity has one peak at a large value, coming from the trajectories with a transitionto the opened microstate, and another peak at zero heat exchange, originating fromtrajectories without a transition. With increasing time the amount of the trajectorieswith a transition to the opened microstate increases rapidly, and the peak close tozero heat moves towards the peak at a large heat value. This explains the behaviorof the heat variances in Fig. 3.8a2 and b2.

Finally, one may ask how our results are affected if the variation (“static disorder”)in base pairing energies is included in the modeling. To this end we have performedMonte Carlo simulations [19] of the stochastic process for a molecular zipper withN = 10 energy levels, where the initial energies Ek(0) = Uk , k = 1, . . . , N , in

74 3 Discrete State Space Models

Eq. (3.53) are given by Uk = (k − 1)(� + ρk), corresponding to different lossesof energies due to variations in base pair bondings. The ρk were chosen as randomnumbers from a box distribution in the interval [−�/3,�/3]. Results from thesesimulations for a number of realizations of this disorder were compared to the pre-dictions of the analytical theory for the "ordered case", where Uk = (k − 1)�. Wefound that, for typical realizations of sets of ρk , the shapes of the work distributionsare very similar, while the peak positions and peak heights are shifted slightly. Alsothe probability distributions pN (t) for the zipper to fully open until time t are, forthese sets of ρk , very similar in shape. Analogous to the peak positions of the workdistributions, the onset of opening shifts slightly from realization to realization. Theshifts of the peaks and of the onset of the opening are controlled by the largest basepair bonding (largest ρk), which governs the unfolding time.

References

1. Brey, J. J., & Prados, A. (1991). Residual properties of a two-level system. Physics Review B,43, 8350–8361, doi:10.1103/PhysRevB.43.8350, http://link.aps.org/doi/10.1103/PhysRevB.43.8350.

2. Chvosta, P., Reineker, P., & Schulz, M. (2007). Probability distribution of work done on atwo-level system during a nonequilibrium isothermal process. Physics Review E, 75, 041124,doi :10.1103/PhysRevE.75.041124, http://link.aps.org/doi/10.1103/PhysRevE.75.041124.

3. Šubrt, E., & Chvosta, P. (2007). Exact analysis of work fluctuations in two-level systems.Journal of Statistical Mechanics: Theory and Experiment, 2007(nr 09) P09019. http://stacks.iop.org/1742-5468/2007/i=09/a=P09019.

4. Manosas, M., Mossa, A., & Forns, N., et. al. (2009). Dynamic force spectroscopy of DNAhairpins: II. Irreversibility and dissipation. Journal of Statistical Mechanics: Theory and Exper-iment, 2009(nr. 02), P02061. http://stacks.iop.org/1742-5468/2009/i=02/a=P02061.

5. Metropolis, N., Rosenbluth, A. W., & Rosenbluth, M. N., et. al. (1953). Equation of statecalculations by fast computing machines. The Journal of Chemical Physics, 21(6), 1087–1092.doi:10.1063/1.1699114, http://link.aip.org/link/?JCP/21/1087/1.

6. Slater, L. (1960). Confluent hypergeometric functions. University Press. http://books.google.cz/books?id=BKUNAQAAIAAJ.

7. Abramowitz, M., & Stegun, I. (1964). Handbook of mathematical functions: With formulars,graphs, and mathematical tables. Applied mathematics series, New York: Dover Publications,Incorporated, ISBN 9780486612720. http://books.google.se/books?id=MtU8uP7XMvoC.

8. Glauber, R. J. (1963). Time-dependent statistics of the ising model. Journal of MathematicalPhysics, 4(2), 294–307. doi:10.1063/1.1703954, http://link.aip.org/link/?JMP/4/294/1.

9. Creutz, M., Jacobs, L., & Rebbi, C. (1979). Experiments with a Gauge-Invariant Ising system.Physics Review Letter, 42, 1390–1393, doi:10.1103/PhysRevLett.42.1390, http://link.aps.org/doi/10.1103/PhysRevLett.42.1390.

10. Michael, C. (1986). Fast heat-bath algorithm for the Ising model. Physics Review B, 33, 7861–7862, doi:10.1103/PhysRevB.33.7861, http://link.aps.org/doi/10.1103/PhysRevB.33.7861.

11. Esposito, M., Kawai, R., & Lindenberg, K., et. al. (2010). Quantum-dot carnot engine atmaximum power. Physics Review E, 81, 041106, doi:10.1103/PhysRevE.81.041106.,http://link.aps.org/doi/10.1103/PhysRevE.81.041106.

12. Schmiedl, T.,& Seifert, U. (2008). Efficiency at maximum power: An analytically solvablemodel for stochastic heat engines. (EPL) Europhysics Letters, 81(nr. 2), 20003. http://stacks.iop.org/0295-5075/81/i=2/a=20003.

References 75

13. Holubec, V. (2009) Nonequilibrium thermodynamics of small systems. Diploma thesis, CharlesUniversity in Prague, Faculty of Mathematics and Physics.

14. Chvosta, P., Holubec, V., & Ryabov, A. et. al. (2010). Thermodynamics of two-stroke enginebased on periodically driven two-level system. Physica E: Low-dimensional Systems andNanostructures, 42(3), 472–476. ISSN 1386–9477, doi :http://dx.doi.org/10.1016/j.physe.2009.06.031, In Proceedings of the International Conference Frontiers of Quantum andMesoscopic Thermodynamics FQMT ’08. http://www.sciencedirect.com/science/article/pii/S1386947709002380.

15. Chvosta, P., Einax, M.,& Holubec, V. et. al. (2010). Energetics and performance of a micro-scopic heat engine based on exact calculations of work and heat distributions. Journal ofStatistical Mechanics: Theory and Experiment, 2010(nr. 03), P03002. http://stacks.iop.org/1742-5468/2010/i=03/a=P03002.

16. Slater, L. (1966). Generalized hypergeometric functions. London: Cambridge University Press.http://books.google.cz/books?id=Hq0NAQAAIAAJ.

17. Hänggi, P., Talkner, P., & Borkovec, M. (1990). Reaction-rate theory: fifty years after Kramers.Review of Modern Physics, 62, 251–341. doi:10.1103/RevModPhys.62.251, http://link.aps.org/doi/10.1103/RevModPhys.62.251.

18. Mossa, A., Manosas, M., & Forns, N. et. al. (2009). Dynamic force spectroscopy of DNAhairpins: I. Force kinetics and free energy landscapes. Journal of Statistical Mechanics: Theoryand Experiment, 2009(nr. 02), P02060. http://stacks.iop.org/1742-5468/2009/i=02/a=P02060.

19. Holubec, V., Chvosta, P., & Einax, M. et. al. (2011). Attempt time monte carlo: An alternativefor simulation of stochastic jump processes with time-dependent transition rates. (EPL) Euro-physics Letters, volume 93(nr. 4), 40003. http://stacks.iop.org/0295-5075/93/i=4/a=40003.

20. Ehrenfest, P., & Ehrenfest, T. (1907). Über zwei bekannte Einwände gegen das BoltzmannscheH-Theorem. Physics Z, 8, 311.

21. Kac, M. (1959). Probability and related topics in physical sciences. Lectures in applied mathe-matics series, (Vol. 1A). London: Interscience Publishers, ISBN 9780821800478. http://books.google.cz/books?id=xm-Y9-GKr_EC.

22. Schuster, H. (2001). Complex adaptive systems: An introduction. Scator Verlag, ISBN9783980793605. http://books.google.cz/books?id=9GuePQAACAAJ.

23. Hauert, C., Nagler, J., & Schuster, H. (2004). Of dogs and fleas: The dynamics ofN uncoupled two-State systems. Journal of Statistical Physics, 116(5–6), 1453–1469.ISSN 0022–4715, doi:10.1023/B:JOSS.0000041725.70622.c4, http://dx.doi.org/10.1023/B%3AJOSS.0000041725.70622.c4.

24. Imparato, A., & Peliti, L. (2005). Work-probability distribution in systems driven out of equi-librium. Physics Review E, 72, 046114, doi:10.1103/PhysRevE.72.046114. http://link.aps.org/doi/10.1103/PhysRevE.72.046114.

25. Ritort, F. (2004). Work and heat fluctuations in two-state systems: a trajectorythermodynamicsformalism. Journal of Statistical Mechanics: Theory and Experiment, 2004(nr. 10), P10016.http://stacks.iop.org/1742-5468/2004/i=10/a=P10016.

26. Híjar, H., Quintana-H, J., & Sutmann, G. (2007). Non-equilibrium work theorems for the two-dimensional Ising model. Journal of Statistical Mechanics: Theory and Experiment, 2007(nr.04), P04010. http://stacks.iop.org/1742-5468/2007/i=04/a=P04010.

27. Yellin, J. (1995). Two exact lattice propagators. Physics Review E, 52, 2208–2215, doi:10.1103/PhysRevE.52.2208, http://link.aps.org/doi/10.1103/PhysRevE.52.2208.

28. Mazonka, O., & Jarzynski, C. (1999). Exactly solvable model illustrating far-from-equilibriumpredictions. eprint arXiv:cond-mat/9912121, arXiv:cond-mat/9912121, http://arxiv.org/abs/cond-mat/9912121.

29. Holubec, V., Chvosta, P., & Maass, P. (2012). Dynamics and energetics for a molecular zip-per model under external driving. Journal of Statistical Mechanics: Theory and Experiment,2012(nr. 11), P11009. http://stacks.iop.org/1742-5468/2012/i=11/a=P11009.

30. Poland, D., & Scheraga, H. A. (1966). Occurrence of a phase transition in nucleic acid models.The Journal of Chemical Physics, 45(5), 1464–1469. doi:10.1063/1.1727786. http://link.aip.org/link/?JCP/45/1464/1.

76 3 Discrete State Space Models

31. Hanke, A., & Metzler, R. (2003). Bubble dynamics in DNA. Journal of Physics A: Mathematicaland General, 36(nr. 36), L473. http://stacks.iop.org/0305-4470/36/i=36/a=101.

32. Bicout, D. J., & Kats, E. (2004). Bubble relaxation dynamics in double-stranded DNA.Physics Review E, 70, 010902, doi:10.1103/PhysRevE.70.010902, http://link.aps.org/doi/10.1103/PhysRevE.70.010902.

33. Fogedby, H. C., & Metzler, R. (2007). DNA bubble dynamics as a quantum coulomb problem.Physics Review Letter, 98, 070601, doi:10.1103/PhysRevLett.98.070601. http://link.aps.org/doi/10.1103/PhysRevLett.98.070601.

34. Metzler, R., Ambjörnsson, T., & Hanke, A. et. al. (2009). Single DNA denaturation and bubbledynamics. Journal of Physics: Condensed Matter, 21(nr. 3), 034111. http://stacks.iop.org/0953-8984/21/i=3/a=034111.

35. Gibbs, J. H., & DiMarzio, E. A. (1959). Statistical mechanics of helix-coil transitions in bio-logical macromolecules. The Journal of Chemical Physics, 30(1), 271–282. doi:10.1063/1.1729886, http://link.aip.org/link/?JCP/30/271/1.

36. Crothers, D., Kallenbach, N. R., & Zimm, B. (1965). The melting transition of low-molecular-weight DNA: Theory and experiment. Journal of Molecular Biology, 11(4), 802–820. ISSN 0022–2836, doi:10.1016/S0022-2836(65)80037-7, http://www.sciencedirect.com/science/article/pii/S0022283665800377.

37. Kittel, C. (1969). Phase transition of a molecular zipper. American Journal of Physics, 37(9),917–920. doi:10.1119/1.1975930, http://link.aip.org/link/?AJP/37/917/1.

38. Liphardt, J., Onoa, B., & Smith, S. B. et. al. (2001) Reversible unfolding of sin-gle RNA molecules by mechanical force. Science, 292(5517), 733–737. doi:10.1126/science.1058498, http://www.sciencemag.org/content/292/5517/733.full.pdf., http://www.sciencemag.org/content/292/5517/733.abstract.

39. Liphardt, J., Dumont, S., & Smith, S. B. et. al. (2002). Equilibrium information from nonequilib-rium measurements in an experimental test of jarzynski’s equality. Science, 296(5574), 1832–1835. doi:10.1126/science.1071152, http://www.sciencemag.org/content/296/5574/1832.full.pdf.http://www.sciencemag.org/content/296/5574/1832.abstract.

40. Lang, M. J., & Block, S. M. (2003). Resource letter: LBOT-1: Laser-based optical tweez-ers. American Journal of Physics, 71(3), 201–215. doi:10.1119/1.1532323, http://link.aip.org/link/?AJP/71/201/1.

41. Onoa, B., Dumont, S., & Liphardt, J. et. al. (2003). Identifying Kinetic barriers to mechani-cal unfolding of the T. thermophila ribozyme. Science, 299(5614), 1892–1895. doi:10.1126/science.1081338, http://www.sciencemag.org/content/299/5614/1892.full.pdf., http://www.sciencemag.org/content/299/5614/1892.abstract.

42. Ritort, F. (2006). Single-molecule experiments in biological physics: methods and applications.Journal of Physics: Condensed Matter, 18(nr. 32), R531. http://stacks.iop.org/0953-8984/18/i=32/a=R01.

43. Ritort, F. (2008). Advances in chemical physics, Chapter nonequilibrium fluctuations in smallsystems: From Physics to Biology. (vol. 137, pp. 31–123). Wiley. ISBN 9780470238080,doi:10.1002/9780470238080.ch2. http://dx.doi.org/10.1002/9780470238080.ch2.

44. Tinoco Jr, I., & Bustamante, C. (1999). How RNA folds. Journal of Molecular Biology,293(2), 271–281. ISSN 0022–2836, doi:10.1006/jmbi.1999.3001, http://www.sciencedirect.com/science/article/pii/S0022283699930012.

45. Manosas, M., & Ritort, F. (2005). Thermodynamic and Kinetic Aspects of RNA Pulling exper-iments. Biophysical Journal, 88(5), 3224–3242. ISSN 0006–3495, doi:10.1529/biophysj.104.045344, http://www.sciencedirect.com/science/article/pii/S0006349505733749.

46. Thirumalai, D., & Hyeon, C. (2005). RNA and protein folding: common themes and variations.Biochemistry, 44(13), 4957–4970. doi:10.1021/bi047314+, pMID: 15794634, http://pubs.acs.org/doi/pdf/10.1021/bi047314%2B, http://pubs.acs.org/doi/abs/10.1021/bi047314%2B.

47. Finkelstein, A. (2003). Course 12: Proteins: Structural, thermodynamic and kinetic aspects. InJ.-L. Barrat, M. Feigelman, J. Kurchan, J. Dalibard (Eds.) Slow Relaxations and nonequilibriumdynamics in condensed matter, Les Houches-École d’Été de Physique Theorique, (vol. 77, pp.649–703). Springer, Berlin. ISBN 978-3-540-40141-4, doi:10.1007/978-3-540-44835-8_12,http://adsabs.harvard.edu/abs/2004srnd.conf.649F.

References 77

48. Green R. & Noller, H. F. (1997). Ribosomes and translation. Annual Review of Biochem-istry, 66(1), 679–716. doi:10.1146/annurev.biochem.66.1.679, pMID: 9242921, http://www.annualreviews.org/doi/pdf/10.1146/annurev.biochem.66.1.679., http://www.annualreviews.org/doi/abs/10.1146/annurev.biochem.66.1.679.

49. Ramakrishnan, V. (2002). Ribosome structure and the mechanism of translation. Cell, 108,557–572. http://linkinghub.elsevier.com/retrieve/pii/S0092867402006190.

50. Kornberg, A. & Baker, T. (2005). Dna replication. University Science Books. ISBN9781891389443. http://books.google.cz/books?id=KDsubusF0YsC.

51. Tackett, A. J., Morris, P. D., & Dennis, R. et. al. (2001). Unwinding of unnatural substratesby a DNA helicase. Biochemistry, 40(2), 543–548. doi:10.1021/bi002122+, pMID: 11148049,http://pubs.acs.org/doi/pdf/10.1021/bi002122%2B.

52. Palassini, M., & Ritort, F. (2011). Improving free-energy estimates from unidirectional workmeasurements: Theory and experiment. Physics Review Letter, 107, 060601, doi:10.1103/PhysRevLett.107.060601, http://link.aps.org/doi/10.1103/PhysRevLett.107.060601.

53. Esposito, M., & Van den Broeck, C. (2010). Three detailed fluctuation theorems. PhysicsReview Letter, 104, 090601, doi:10.1103/PhysRevLett.104.090601, http://link.aps.org/doi/10.1103/PhysRevLett.104.090601.

54. Esposito, M., & Van den Broeck, C. (2010). Three faces of the second law. I. Master equationformulation. Physics Review E, 82, 011143, doi:10.1103/PhysRevE.82.011143, http://link.aps.org/doi/10.1103/PhysRevE.82.011143.

55. Van den Broeck, C. & Esposito, M. (2010). Three faces of the second law. II. Fokker-Planckformulation. Physics Review E, 82, 011144, doi:10.1103/PhysRevE.82.011144, http://link.aps.org/doi/10.1103/PhysRevE.82.011144.

56. Seifert, U. (2008). Stochastic thermodynamics: principles and perspectives. The EuropeanPhysical Journal B—Condensed Matter and Complex Systems, 64, 423–431, ISSN 1434–6028,DOI:10.1140/epjb/e2008-00001-9, http://dx.doi.org/10.1140/epjb/e2008-00001-9.

57. Braun, O., Hanke, A., & Seifert, U. (2004). Probing molecular free energy landscapes byperiodic loading. Physics Review Letter, 93, 158105, doi:10.1103/PhysRevLett.93.158105,http://link.aps.org/doi/10.1103/PhysRevLett.93.158105.

58. Crooks, G. E. (1999). Entropy production fluctuation theorem and the nonequilibrium workrelation for free energy differences. Physics Review E, 60, 2721–2726, doi:10.1103/PhysRevE.60.2721, http://link.aps.org/doi/10.1103/PhysRevE.60.2721.

59. Jarzynski, C. (1997). Nonequilibrium equality for free energy differences. Physics ReviewLetter, 78, 2690–2693, doi:10.1103/PhysRevLett.78.2690. http://link.aps.org/doi/10.1103/PhysRevLett.78.2690.

60. Chatelain, C., & Karevski, D. (2006) Probability distributions of the work in the two-dimensional Ising model. Journal of Statistical Mechanics: Theory and Experiment, 2006(nr.06), P06005. http://stacks.iop.org/1742-5468/2006/i=06/a=P06005.

61. Einax, M., & Maass, P. (2009). Work distributions for Ising chains in a time-dependent magneticfield. Physics Review E, 80, 020101, doi:10.1103/PhysRevE.80.020101. http://link.aps.org/doi/10.1103/PhysRevE.80.020101.

62. Baule, A., & Cohen, E. G. D. (2009). Steady-state work fluctuations of a dragged particle underexternal and thermal noise. Physics Review E, 80, 011110, doi:10.1103/PhysRevE.80.011110,http://link.aps.org/doi/10.1103/PhysRevE.80.011110.

63. Lubensky, D. K., & Nelson, D. R. (2000). Pulling pinned polymers and unzipping DNA. PhysicsReview Letter, 85, 1572–1575, doi:10.1103/PhysRevLett.85.1572, http://link.aps.org/doi/10.1103/PhysRevLett.85.1572.

64. Engel, S., Alemany, A., & Forns, N. et. al. (2011). Folding and unfolding of a triple-branch DNA molecule with four conformational states. Philosophical Magazine, 91(13–15),2049–2065. doi:10.1080/14786435.2011.557671, http://www.tandfonline.com/doi/pdf/10.1080/14786435.2011.557671.http://www.tandfonline.com/doi/abs/10.1080/14786435.2011.557671.

65. Bell, G. (1978). Models for the specific adhesion of cells to cells. Science, 200(4342),618–627. doi:10.1126/science.347575, http://www.sciencemag.org/content/200/4342/618.full.pdf., http://www.sciencemag.org/content/200/4342/618.abstract.

78 3 Discrete State Space Models

66. Cocco, S., Monasson, R., & Marko, J. F. (2001). Force and kinetic barriers to unzipping ofthe DNA double helix. Proceedings of the National Academy of Sciences, 98(15), 8608–8613.doi:10.1073/pnas.151257598, http://www.pnas.org/content/98/15/8608.full.pdf+html., http://www.pnas.org/content/98/15/8608.abstract.

67. Alemany, A., Ribezzi, M., & Ritort, F. (2011). Recent progress in fluctuation theorems andfree energy recovery. AIP Conference Proceedings, 1332(1), 96–110. doi:10.1063/1.3569489,http://link.aip.org/link/?APC/1332/96/1.

Chapter 4Continuous State Space Models

4.1 The Two Works: A Toy Model

In this section we illustrate the difference between the two work definitions (2.44) and(2.53) on two specific models. Specifically, we demonstrate validity of the individualformulas presented in Sect. 2.2.3. Moreover, we introduce two examples of externallycontrolled parameters Y(t) used in single molecule experiments [1–4].

Consider a microscopic spring in contact with a heat reservoir, which is, by one ofits ends, attached to a wall located at x = 0. For sake of mathematical simplicity weassume that the force exerted by the spring equals zero when its elongation vanish,i.e., when its free end is at x = 0, and that the spring can be stretched either in thepositive and in the negative direction. Our goal will be simple: to stretch the spring.

In such situation the microstate of the system is determined by the random springelongation X(t) ∈ (−→,→) and the energy (free energy) landscape of the systemalone (the spring) assumes the form

U(x) = G(x) = 1

2kmx2, (4.1)

where km denotes the spring stiffness. Having in mind that we perform our experimentat microscale, it seems difficult, although possible, to apply a controllable force onthe free end of the spring directly [4]. Instead one would rather stretch the springindirectly by inserting its free end into an externally controlled potential. This setupis sketched in Fig. 4.1, the setup when the spring is stretched by a directly controlledforce is depicted in Fig. 4.2. We first analyze the setup where the force is appliedindirectly.

4.1.1 Indirectly Controlled Force

Assume that the interaction potential applied on the spring is parabolic and thatthe single control parameter is the position of its minimum Y(t) = Y (t) = Y ,

V. Holubec, Non-equilibrium Energy Transformation Processes, Springer Theses, 79DOI: 10.1007/978-3-319-07091-9_4, © Springer International Publishing Switzerland 2014

80 4 Continuous State Space Models

Fig. 4.1 Stretching of aspring via force controlledindirectly by the position ofthe minimum of an externalpotential

Fig. 4.2 Stretching of a springvia force controlled directly

force

i.e.,

V(x, Y ) = 1

2ko(x − Y )2. (4.2)

Equations (4.1) and (4.2) yield the total energy (free energy) of the compound systemvia Eqs. (2.35) and (2.36). Specifically we have

E(x, t) = F(x, t) = 1

2kmx2 + 1

2ko[x − Y (t)]2. (4.3)

The whole setting is depicted in Fig. 4.1. Assume that the minimum of the potential isshifted during the time interval [t ∞, t] from the position Y (t ∞) = 0 to a new one, Y (t).In such case the thermodynamic work (2.44) and the mechanical work (2.53) aregiven by

W(t, t ∞) = ko

t∫

t ∞dt ∞∞ [Y (t ∞∞) − X(t ∞∞)]Y (t ∞∞), (4.4)

WE(t, t ∞) = ko

t∫

t ∞dt ∞∞ [Y (t ∞∞) − X(t ∞∞)]dX(t ∞∞)

dt ∞∞, (4.5)

respectively.

4.1.1.1 Mean Values

Assume that the potential changes quasi-statically, then the mean values of these twointegrals can be evaluated using the mean equilibrium length of the spring

4.1 The Two Works: A Toy Model 81

∈[X(t)]eq〉 = ko

ko + kmY (t). (4.6)

If we substitute this expression for X(t) in Eqs. (4.4) and (4.5), we obtain

∈[W(t, t ∞)]eq〉 = 1

2

kokm

ko + km[Y (t)]2, (4.7)

∈[WE(t, t ∞)]eq〉 = 1

2

k2okm

(ko + km)2 [Y (t)]2. (4.8)

On the other hand, the increase of the free energy of the compound system (system-interaction) and that of the system itself are given by

�F(t, t ∞) = F (∈[X(t)]eq〉, t) − F (∈[X(t ∞)]eq〉, t ∞

) = 1

2

kokm

ko + km[Y (t)]2, (4.9)

�F0(t, t ∞) = G (∈[X(t)]eq〉) − G (∈[X(t ∞)]eq〉

) = 1

2

k2okm

(ko + km)2 [Y (t)]2, (4.10)

respectively, and hence the relations (2.55) and (2.57) for the given model are verified.Similarly, one can check also the relation (2.60):

−V (∈[X(t)]eq〉, t)+V (∈[X(t ∞)]eq〉, t ∞

)+�F(t, t ∞) = 1

2

k2okm

(ko + km)2 [Y (t)]2. (4.11)

4.1.1.2 Fluctuations

The dynamics of the spring is described by the Langevin Eq. (2.37) (we take � = 1)

d

dtX(t) = −kmX(t) + ko[Y (t) − X(t)] + N(t). (4.12)

The spring elongation X(t) can be formally integrated. It turns out that X(t) is a linearfunction of the Gaussian white noise N(t) and hence its probability distribution isGaussian. Similarly, the thermodynamic work (4.4) is a linear function of the systemstate X(t). Therefore its probability density must be also Gaussian [5] specified solelyby the mean work and work variance, which can be calculated from Eqs. (4.4) and(4.12) for an arbitrary function Y (t) [5, 6]. Derivation of the WPD for a similarmodel is presented in Sect. 4.2. The mechanical work is a quadratic function of X(t)and its probability density is not Gaussian. WPD for a similar model, the so called“breathing parabola”, was investigated in [7–10] (see also Sect. 5.2). The exact WPDcan be also obtained by numerical integration of Eqs. (4.5) and (4.12). Nevertheless,no matter the densities are, they must obey the fluctuation theorems (2.61)–(2.65).

82 4 Continuous State Space Models

4.1.2 Directly Controlled Force

Assume that the single control parameter Y(t) = Y (t) = Y is the force exerted onthe spring itself (see Fig. 4.2). In such case the interaction energy (—work done onthe spring by the constant force Y to stretch it by the amount x) reads

V(x, Y ) = −xY. (4.13)

Equations (4.1) and (4.13) yield the total energy (free energy) of the compound systemvia Eqs. (2.35) and (2.36). Specifically we have

E(x, t) = F(x, t) = 1

2kmx2 − xY (t). (4.14)

Assume that the force is increased during the time interval [t ∞, t] from the valueY (t ∞) = 0 to a new one, Y (t). In such case the thermodynamic work (2.44) and themechanical work (2.53) are given by

W(t, t ∞) = −t∫

t ∞dt ∞∞ X(t ∞∞)Y (t ∞∞), (4.15)

WE(t, t ∞) =t∫

t ∞dt ∞∞ dX(t ∞∞)t ∞∞

dY(t ∞∞), (4.16)

respectively.

4.1.2.1 Mean Values

Assume that the force changes quasi-statically, then the mean values of these twointegrals can be evaluated using the mean equilibrium length of the spring

∈[X(t)]eq〉 = 1

kmY (t). (4.17)

If we substitute this expression for X(t) in Eqs. (4.15)–(4.16), we obtain

⟨[W(t, t ∞)]eq⟩ = − 1

2km[Y (t)]2, (4.18)

⟨[WE(t, t ∞)]eq⟩ = 1

2km[Y (t)]2. (4.19)

4.1 The Two Works: A Toy Model 83

On the other hand, the increase of the free energy of the compound system (systemand interaction) and that of the system itself are given by

�F(t, t ∞) = F (∈[X(t)]eq〉, t) − F (∈[X(t ∞)]eq〉, t ∞

) = − 1

2km[Y (t)]2, (4.20)

�F0(t, t ∞) = G (∈[X(t)]eq〉) − G (∈[X(t ∞)]eq〉

) = 1

2km[Y (t)]2, (4.21)

respectively, and hence the relations (2.55) and (2.57) for the given model are verified.Similarly, one can check also the relation (2.60):

− V (∈[X(t)]eq〉, t) + V (∈[X(t ∞)]eq〉, t ∞

) + �F(t, t ∞) = 1

2km[Y (t)]2. (4.22)

4.1.2.2 Fluctuations

The dynamics of the spring can be described by the Langevin Eq. (2.37) (we take� = 1)

dX(t)

dt= −kmX(t) + Y (t) + N(t). (4.23)

X(t) is a linear function of the Gaussian white noise and its probability distribu-tion is Gaussian. Further, both the thermodynamic work (4.15) and the mechanicalwork (4.16) are linear functions of the system state X(t). Therefore their probabilitydensities must also be Gaussian [5] specified solely by the individual mean worksand work variances, which can be calculated from Eqs. (4.15), (4.16) and (4.23) foran arbitrary function Y (t). Derivation of the WPD for a similar model is presentedin Sect. 4.2. However, no matter the densities are, they must obey the fluctuationtheorems (2.61)–(2.65).

4.2 Sliding Parabola

The following work was published, with small modifications, in [6]. Consider aparticle in contact with a thermal bath at the temperature T . Let the particle move inthe externally driven time-dependent parabolic potential [5]

V[x, Y (t)] = k

2[x − Y (t)]2. (4.24)

We assume that G(x) = 0 and thus E(x, t) = F(x, t) = V[x, Y (t)], cf. Eqs. (2.35)and (2.36). One can regard the particle as being attached to a spring the other endof which moves with an instantaneous velocity Y (t). Such situation is depicted in

84 4 Continuous State Space Models

Fig. 2.1 if one assumes that Y (t) denotes the speed of the car. The Langevin Eq. (2.37)for the particle position is given by

�d

dtX(t) = −k

[X(t) − Y (t)

] + N(t) (4.25)

with the initial condition X(t ∞) = x ∞. The work (2.44) is driven by the formula

d

dtW(t, t ∞) = −kY (t)[X(t) − Y (t)] (4.26)

with the initial condition W(t ∞, t ∞) = 0. The Fokker-Planck Eq. (2.45) correspondingto Eqs. (4.25) and (2.43) reads

∂

∂tG(x, w, t | x ∞, w∞, t ∞) =

{D

∂2

∂x2 + k

�

∂

∂x[x − Y (t)]

+ [x − Y (t)] Y (t)∂

∂w

}G(x, w, t | x ∞, w∞, t ∞) (4.27)

with the initial condition G(x, w, t ∞ | x ∞, w∞, t ∞) = δ(x − x ∞)δ(w−w∞). This equationcan be solved by several methods. For example, one can use the Lie algebra operatormethods [11, 12], or one can calculate the joint generating functional for the coupledprocess in question [13]. In the following we present two methods of the solution.

The first one is based on the following property of Eq. (4.27): if at an arbi-trary fixed instant the probability density G(x, w, t | x ∞, w∞, t ∞) is of the Gaussianform, then it will preserve this form for all subsequent times. This follows from thefact that all the coefficients on the right hand side of Eq. (4.27) are polynomials ofthe degree at most one in the independent variables x and w [14]. Accordingly, thefunction G(x, w, t | x ∞, w∞, t ∞) corresponds to the bivariate Gaussian distribution [5]

G(x, w, t | x ∞, w∞, t ∞) = 1

2π√[�X (t, t ∞)]2 [�W (t, t ∞)]2 − [cxw(t, t ∞)]2

× exp

{−1

2

[�W (t, t ∞)]2[w − W (t, t ∞)]2 + [�X (t, t ∞)]2[x − X (t, t ∞)]2

[�X (t, t ∞)]2 [�W (t, t ∞)]2 − [cyw(t, t ∞)]2

}

× exp

{cxw(t, t ∞)[w − W (t, t ∞)][x − X (t, t ∞)]

[�X (t, t ∞)]2 [�W (t, t ∞)]2 − [cyw(t, t ∞)]2

}, (4.28)

which is uniquely defined by the averages:

X (t, t ∞) = ∈X(t)〉,W (t, t ∞) = ∈W(t, t ∞)〉,

[�X (t, t ∞)]2 = ∈[X(t)]2〉 − [X (t, t ∞)]2, (4.29)

[�W (t, t ∞)]2 = ∈[W(t, t ∞)]2〉 − [W (t, t ∞)]2,

cxw(t, t ∞) = ∈X(t)W(t, t ∞)〉 − X (t, t ∞)W (t, t ∞).

4.2 Sliding Parabola 85

Table 4.1 Closed Lie algebra generated by the differential operators contained in Eq. (4.27)

∂xx ∂x ∂x x x∂w ∂w ∂wx ∂ww

∂xx 0 0 2 ∂xx 2 ∂wx 0 0 0∂x 0 0 ∂x ∂w 0 0 0∂x x −2 ∂xx − ∂x 0 x∂w 0 − ∂wx 0x∂w −2 ∂wx − ∂w − x∂w 0 0 − ∂ww 0∂w 0 0 0 0 0 0 0∂wx 0 0 ∂wx ∂ww 0 0 0∂ww 0 0 0 0 0 0 0

Here ∂wx = ∂2/∂w∂x etc.

The simplest way to calculate these averages is to use Eqs. (4.25) and (4.26) [14, 15].The result is given by Eqs. (4.32) and (4.38). An alternative way to solve Eq. (4.27)is to adopt the operator method introduced in [11, 12] and used in [10] during thederivation of WPD for the diffusion in a time-dependent log-harmonic potential.The differential operators ∂xx = ∂2/∂x2, ∂x = ∂/∂x , x∂/∂x , ∂w = ∂/∂w, x∂/∂w

acting in Eq. (4.27) generate a closed Lie algebra. Their commutation relations aregiven in Table 4.1. Because the operators form the closed Lie algebra, one can assumethe solution of Eq. (4.27) in the form [11, 12]

G(x, w, t ∞ | x ∞, w∞, t ∞) = eawx (t) ∂wx eaww(t) ∂ww eaxx (t) ∂xx

× eax (t) ∂x eaw(t) ∂w ex ax (t) ∂x [x−Y (t)]ex aw(t)[x−Y (t)] ∂w

× G(x, w, t ∞ | x ∞, w∞, t ∞). (4.30)

Inserting this ansatz into Eq. (4.27), performing the partial derivatives and collectingthe terms containing the individual differential operators brings us to the follow-ing system of the first order ordinary differential equations for the time dependentcoefficients x ay(t):

x ax (t) = k

�,

exp [x ax (t)] x aw(t) = k Y (t),

ax (t) + ax (t) x ax (t) = Y (t),

aw(t) + ax (t) exp [x ax (t)] x aw(t) = 0, (4.31)

axx (t) + 2axx (t) x ax (t) = D,

awx (t) + awx (t) x ax (t) + 2axx (t) exp [x ax (t)] x aw(t) = 0,

aww(t) + awx (t) exp [x ax (t)] x aw(t) = 0.

Initial condition for all the coefficients is x ay(t ∞) = 0.

86 4 Continuous State Space Models

Solution of the system (4.31) reads

x ax (t) = k

�(t − t ∞),

x aw(t) = k

t∫

t ∞dt ∞∞ Y (t ∞∞) exp

[− k

�(t ∞∞ − t ∞)

],

ax (t) = exp

[− k

�(t − t ∞)

] t∫

t ∞dt ∞∞ Y (t ∞∞) exp

[k

�(t ∞∞ − t ∞)

],

aw(t) = −k

t∫

t ∞dt ∞∞ Y (t ∞∞) ax (t

∞∞), (4.32)

axx (t) = �D

2k

(1 − exp

[−2

k

�(t − t ∞)

]),

awx (t) = −2�D exp

[− k

�(t − t ∞)

] t∫

t ∞dt ∞∞ Y (t ∞∞) sinh

[k

�(t ∞∞ − t ∞)

],

aww(t) = −k

t∫

t ∞dt ∞∞ Y (t ∞∞) awx (t

∞∞).

The explicit form of the function G(x, w, t | x ∞, w∞, t ∞) is found by applying theindividual operators in Eq. (4.30) on the initial condition G(x, w, t ∞ | x ∞, w∞, t ∞) =δ(w − w∞)δ(x − x ∞). The operators involved act in the following way

exp [a ∂x ] g(x) = g(x + a), (4.33)

exp [ax ∂x ] g(x) = g(ea x), (4.34)

exp [a∂xx ] g(x) = 1∝4πa

→∫

−→dy g(y) exp

[− (x − y)2

4a

], (4.35)

exp [−a∂x x] g(x) = e−ag(xe−a), (4.36)

and

exp [cxw ∂xw] exp

[− (w − W )2

2σ2w

]exp

[− (x − X)2

2σ2x

]=

√σ2

xσ2w

σ2xσ

2w − c2

xw

× exp

[−1

2

σ2w(w − W )2 − 2cxw(w − w)(x − X) + σ2

x (x − X)2

σ2xσ

2w − c2

xw

]. (4.37)

4.2 Sliding Parabola 87

The resulting function G(x, w, t | x ∞, w∞, t ∞) is given by Eq. (4.28), where

X (t, t ∞) = Y (t) + [x ∞ − Y (t ∞)] exp[−x ax (t)] − ax (t),

W (t, t ∞) = w∞ − aw(t) − x aw(t)[x − Y (t ∞)],[�X (t, t ∞)]2 = 2axx (t), (4.38)

[�W (t, t ∞)]2 = 2aww(t),

cxw(t, t ∞) = awx (t).

Surprisingly, the variance [�X (t, t ∞)]2 does not depend on the function Y (t). More-over, in the asymptotic regime t ≥ �/k, the variance [�X (t, t ∞)]2 attains the satu-rated value �D/k. This means that the marginal probability density for the particleposition, R(x, t | x ∞, t ∞) = ∫ →

−→ dw G(x, w, t | x ∞, 0, t ∞), eventually attains a time-independent shape.

Up to now our considerations were valid for an arbitrary form of the function Y (t).We now focus on the piecewise linear periodic driving. We take Y (t + tp) = Y (t) and

Y (t) = −2vt for t ∈ [0, t1], Y (t) = −2vt1 + vt for t ∈ [t1, tp], (4.39)

where v > 0 and 0 < t1 < tp. The parabola is first moving to the left with thevelocity 2v during the time interval [0, t1). Then, at the time t1, it abruptly changesits velocity and moves to the right with the velocity v during the rest of the periodtp, cf. Fig. 4.3d.

Due to the periodic driving the system response (4.38) eventually approaches thelimit cycle (cf. Sect. 2.4.1). Figure 4.3 illustrates the response during two such limitcycles, i.e., we take t ≥ t ∞ = 0. First, note that the mean position of the particleX (t, 0) “lags behind” the minimum of the potential well Y (t) [cf. the panel a], thefact which once again reminds the horse-carrot analogy (Fig. 1.1). The magnitudeof this phase shift is given by the second term in X (t, t ∞), Eq. (4.38), and thereforeit is proportional to the velocity v. In the quasi-static limit of the infinitely slowvelocity v ≤ 0 the probability distribution for the particle position is centered at theinstantaneous minimum of the parabola.

Consider now the mean work done on the system by the external agent dur-ing the time interval [0, t] [panel b]. W (t, 0) increases if either simultaneouslyY (t) > X (t, 0) and Y (t) > 0, or if simultaneously Y (t) < X (t, 0) and Y (t) < 0.For instance, assume the parabola moves to the right and, at the same time, theprobability packet for the particle coordinate is concentrated on the left from theinstantaneous position of the parabola minimum Y (t). Then the dragging rises thepotential energy of the particle, i.e., the work is done on it and the mean input poweris positive. Similar reasoning holds if either simultaneously Y (t) > X (t, 0) andY (t) < 0, or if simultaneously Y (t) < X (t, 0) and Y (t) > 0. Than the mean workW (t, 0) decreases and hence the mean input power is negative. The magnitude of theinstantaneous input power is proportional to the instantaneous velocity Y (t). There-fore it is bigger during the first part of the period of the limit cycle in comparisonwith the second part of the period. Finally, let us stress that the mean work per cycle,

88 4 Continuous State Space Models

20 25 30 35 40−5

−4

−3

−2

−1

0

t

X(t

,0)

20 25 30 35 4020

25

30

35

40

t

W(t

, 0)

0 10 200

0.2

0.4

0.6

0.8

1

t

[ΔX

(t,0

)]2

20 25 30 35 4015

20

25

30

35

t

[ΔW

(t,0

)]2

20 25 30 35 40−1

0

1

2

t

c xw

(t,0

)

20 25 30 35 40−6

−4

−2

0

t

Y(t

)(a) (b) (c)

(d) (e) (f)

Fig. 4.3 The central moments (4.38) in the time-asymptotic regime. The driving is representedby the position of the potential minimum Y (t) and it is depicted in the panel (d). In all panels[except the panel c] the curves are plotted for two periods tp of the driving. The panel a shows themean position of the particle, X (t, 0), which lags behind the minimum of the potential well. Thepanel b shows the mean work W (t, 0) done on the particle by the external agent. In the panel c weobserve the saturation of the variance of the particle position [�X (t, 0)]2. In the panel e we presentthe correlation function cxw(t, 0). The panel f illustrates the variance [�W (t, 0)]2 of the workdone on the particle by the external agent. The parameters used are: k = 1 kg s−2 D = 1 m2 s−1,� = 1 kg s−1, v = 0.825 m s−1, tp = 10 s, t1 = 10/3 s

Wp = W (t + tp, t) = W (t + tp, 0) − W (t, 0), is always positive, as required by thesecond law of thermodynamics. The variance of the work done on the particle by theexternal agent, [�W (t, 0)]2, shows qualitatively the same behavior as W (t, 0).

References

1. Collin, D., Ritort, F., Jarzynski, C., et al. (2005). Verification of the Crooks fluctuationtheorem and recovery of RNA folding free energies. Nature, 473, 231. http://dx.doi.org/10.1038/nature04061M3.

2. Ritort, F. (2006). Single-molecule experiments in biological physics: Methods and applications.Journal of Physics: Condensed Matter, 18(32), R531. http://stacks.iop.org/0953-8984/18/i=32/a=R01.

3. Ritort, F. (2008). Nonequilibrium fluctuations in small systems: From physics to biology. InA. Rice Stuart (Ed.), Advances in chemical physics (Vol. 137, pp. 31–123). Hoboken: JohnWiley & Sons Inc. ISBN 9780470238080, doi:10.1002/9780470238080.ch2.

4. Alemany, A., Ribezzi, M., & Ritort, F. (2011). Recent progress in fluctuation theorems andfree energy recovery. AIP Conference Proceedings, 1332(1), 96–110. doi:10.1063/1.3569489,http://link.aip.org/link/?APC/1332/96/1.

References 89

5. Mazonka, O., & Jarzynski, C. (1999). Exactly solvable model illustrating far-from-equilibriumpredictions. eprint arXiv:cond-mat/9912121, December 1999.

6. Holubec, V., Chvosta, P., & Ryabov, A. (2010). Four exactly solvable examples in non-equilibrium thermodynamics of small systems. In Thermodynamics (pp. 153–176). InTech.doi:10.5772/13374, http://www.intechopen.com/books/thermodynamics.

7. Engel, A. (2009). Asymptotics of work distributions in nonequilibrium systems. PhysicalReview E, 80, 021120. doi:10.1103/PhysRevE.80.021120.

8. Nickelsen, D., & Engel, A. (2011). Asymptotics of work distributions: The pre-exponentialfactor. The European Physical Journal B, 82, 207–218. ISSN 1434–6028, doi:10.1140/epjb/e2011-20133-y.

9. Speck, T. (2011). Work distribution for the driven harmonic oscillator with time-dependentstrength: Exact solution and slow driving. Journal of Physics A: Mathematical and Theoretical,44(30), 305001. http://stacks.iop.org/1751-8121/44/i=30/a=305001.

10. Ryabov, A., Dierl, M., Chvosta, P., et al. (2013). Work distribution in a time-dependentlogarithmic-harmonic potential: Exact results and asymptotic analysis. Journal of PhysicsA: Mathematical and Theoretical, 46(7), 075002. http://stacks.iop.org/1751-8121/46/i=7/a=075002.

11. Wilcox, R. M. (1967). Exponential operators and parameter differentiation in quantum physics.Journal of Mathematical Physics, 8(4), 962–982. doi:10.1063/1.1705306, http://link.aip.org/link/?JMP/8/962/1.

12. Wolf, F. (1988). Lie algebraic solutions of linear Fokker-Planck equations. Jour-nal of Mathematical Physics, 29(2), 305–307. doi:10.1063/1.528067, http://link.aip.org/link/?JMP/29/305/1.

13. Baule, A., & Cohen, E. G. D. (2009). Fluctuation properties of an effective nonlinear systemsubject to Poisson noise. Physical Review E, 79, 030103. doi:10.1103/PhysRevE.79.030103.

14. Van Kampen, N. (2011). Stochastic processes in physics and chemistry. North-Holland PersonalLibrary. Amsterdam: Elsevier Science. ISBN 9780080475363, http://books.google.cz/books?id=N6II-6HlPxEC.

15. Gillespie, D. T. (1992). Markov processes: An introduction for physical scientist. San Diego:Academic press, Inc.

Chapter 5Heat Engines

5.1 Two-Level Heat Engine

The following work was published with minor modifications in [1, 2].

5.1.1 Introduction

In this section we study a simple model of mesoscopic heat engine (Sect. 2.4)operating between two different heat baths under non-equilibrium conditions. Theworking medium consist of the two-level system described in Sect. 3.1.2. We con-sider that the cycle of operation includes just two isothermal branches, or strokes.Within each stroke, the system is driven by changing the energies of the two statesand we assume a constant driving rate, i.e., a linear time-dependence of the energies.The response of the working medium is governed by the master Eq. (2.3) with time-dependent transition rates given by Eqs. (3.5) and (3.19). The specific form of therates guarantees that, provided the two energies were fixed, the system would relaxtowards the Gibbs equilibrium state, cf. Fig.1.1. Of course, during the motor oper-ation, the Gibbs equilibrium is never achieved because the energies are cyclicallymodulated. At a given instant, the system dynamics just reflects the instantaneousposition of the energy levels. After a transient regime, with the periodicity of thedriving force, the system (engine) state approaches a limit cycle (see Sect. 2.4.1).We will focus on its properties. In particular, we calculate the distribution of the workduring the limit cycle.

Our two-isotherm setting imposes one important feature which is worth empha-sizing. As stated above, at the end of each branch we remove the present bath andwe allow the thermal interaction with another reservoir. This exchange of reservoirsnecessarily implies a finite difference between the new reservoir temperature and theactual system (effective) temperature. Even if the driving period tends to infinity, wewill observe a positive entropy production originating from the relaxation processes

V. Holubec, Non-equilibrium Energy Transformation Processes, Springer Theses, 91DOI: 10.1007/978-3-319-07091-9_5, © Springer International Publishing Switzerland 2014

92 5 Heat Engines

initiated by the abrupt change of the contact temperature. Differently speaking, ourengine operates in an inherently irreversible way and there exists no reversible limit.1

This section is organized as follows. In Sect. 5.1.2 we solve the dynamical equa-tion for the externally driven working medium. For the sake of clarity we first givethe solution just for an unrestricted linear driving protocol using a generic drivingrate and a generic reservoir. Thereupon we particularize the generic solution to indi-vidual branches and, using the Chapman-Kolmogorov condition (2.7), we derive thesolution for the limit cycle. In Sect. 5.1.3 we employ the recently derived (see [4] andSect. 3.1.2) analytical result for the work probability density under linear driving.Again, we first give the result for the generic linear driving and then we combinetwo such particular solutions into the final work distribution valid for the limit cycle.The results from Sects. 5.1.2 and 5.1.3 enable a detailed calculation of the energyand entropy flows during the limit cycle in Sect. 5.1.4 and allow for a discussion ofthe engine performance in Sect. 5.1.5.

5.1.2 Description of the Engine and Its Limit Cycle

Consider a two-level system with the time-dependent energies E i (t), i = 1, 2, incontact with a single thermal reservoir at the temperature T . In general, the heatreservoir temperature T may also be time-dependent. The time evolution of theoccupation probabilities pi (t), i = 1, 2, is governed by the master Eq. (2.3) withtime-dependent rate matrix (3.1) specified by the reservoir temperature and by theexternal parameters. We will adopt the Glauber form of the transition rates, which, incontrast to the more common exponential rates (∝ exp{± − β(t)[E2(t) − E1(t)/2]),saturate at large energy differences (“driving forces”; see [5] and Sect. 2.1.2 for afurther discussion),

λU(t) = 1

1 + exp {−β(t) [E1(t) − E2(t)]} , (5.1)

λD(t) = exp {−β(t) [E1(t) − E2(t)]}1 + exp {−β(t) [E1(t) − E2(t)]} . (5.2)

The general solution of the master Eq. (2.3) for these transition rates is given byEq. (3.2). It can be rewritten as R

R(t | t →) = 1 − 1

2

(1 −1

−1 1

){1 − e−ν(t−t →)

}+ 1

2

(−1 −11 1

)ξ(t, t →) , (5.3)

where

1 In Sect. 5.2 we show that if one considers also adiabatic branches, the reversible limit can beobtained [3].

5.1 Two-Level Heat Engine 93

ξ(t, t →) = ν

t∫

t →dτ exp [−ν(t − τ )] tanh

⎡β(τ )

2[E1(τ ) − E2(τ )]

⎣. (5.4)

The resulting propagator satisfies the Chapman-Kolmogorov condition (2.7) for anyintermediate time t →→. Its validity can be easily checked by direct matrix multiplication.The condition simply states that the initial state for the evolution in the time interval[t →→, t] can be taken as the final state reached in the interval [t →, t →→]. This is trueeven if the parameters of the process in the second interval differ from those in thefirst one, cf. Sect. 2.4.1. Of course, if this is the case, we should use an appropriatenotation which distinguishes the two corresponding propagators. This procedure willbe actually implemented below. Keeping in mind this possibility, we will first analyzethe propagator for a generic linear driving protocol.

5.1.2.1 Generic Case: Linear Driving Protocol

Let us consider the linear driving protocol (3.3). Concretely, we take E1(t) = h +v(t − t →), and E2(t) = −E1(t), where h = E1(0) denotes the energy of the first levelat the initial time t →, and v is the driving velocity (energy change per time). The rates(5.1) can then be written in the form

λU(t) = ν1

1 + c exp[−�(t − t →)] , λD(t) = νc exp[−�(t − t →)]

1 + c exp[−�(t − t →)] , (5.5)

where � = 2βv is the temperature-reduced driving velocity, and c = exp(−2βh)

incorporates the initial values of the energies.Under this linear driving protocol one can evaluate the definite integral in (5.4)

explicitly. Specifically, writing ξ(t, t →) = 1 − exp[−ν(t − t →)] − 2γ(t, t →), we are tocalculate the definite integral

γ(t, t →) = ν c

t∫

t →dτ exp [−ν(t − τ )]

exp[−�(τ − t →)

⎤

1 + c exp [−�(τ − t →)]

= a c exp⎦�t →⎫

exp (−νt)

�t∫

�t →dτ

exp [(a − 1)τ ]

1 + c exp (�t →) exp (−τ ).

Here we have introduced the dimensionless ratio a = ν/� of the attempt frequencycharacterizing the time scale of the system dynamics and the time scale of the externaldriving, respectively. Naturally, this ratio will describe the degree of irreversibilityof the process. Depending on the value a ∞ (0,∈), the explicit form of the functionγ(t, t →) reads

94 5 Heat Engines

⎬⎭⎭⎭⎭⎭⎭⎭⎭⎭⎭⎭

⎭⎭⎭⎭⎭⎭⎭⎭⎭⎭⎭

ac1−a exp

⎦�t →⎫

exp (−νt)⎧exp[(a − 1)�t →

⎤2F1(1, 1 − a; 2 − a; −c)

− exp [(a − 1)�t] 2F1(1, 1 − a; 2 − a; −c exp[−�(t − t →)

⎤)⎪

, a ∞ (0, 1),

c exp[−�(t − t →)

⎤⎨

�(t − t →) + ln1 + c exp

[−�(t − t →)⎤

1 + c

⎩

, a = 1 ,

2F1

(1, a; 1 + a; −1

cexp[�(t − t →)

⎤)

− exp[−a �(t − t →)

⎤2F1

(1, a; 1 + a; −1

c

), a > 1,

(5.6)

where 2F1(α,β; γ; z) denotes the Gauss hypergeometric function [6, 7].

5.1.2.2 Piecewise Linear Periodic Driving

We now specify the setup for the operational cycle of the engine under periodicdriving introduced in 2.4.1. Within a given period, two branches with linear time-dependence of the state energies are considered with different velocities. Startingfrom the value h1, the energy E1(t) linearly increases in the first branch until itattains the value h2 > h1, at time t+ and in the second branch, the energy E1(t)linearly decreases towards its original value h1 at a time t− (see Fig. 5.1). We alwaysassume E2(t) = −E1(t), i.e.,

E1(t) = −E2(t) =⎬

h1 + h2 − h1

t+ t, t ∞ [0, t+⎤,

h2 − h2 − h1t− (t − t+), t ∞ [t+, t+ + t−

⎤.

(5.7)

This pattern will be periodically repeated, the period being tp = t+ + t−.As the second ingredient, we need to specify the temperature schedule. The two-

level system will be alternately exposed to a hot and a cold reservoir, which meansthat the function β(t) in Eq. (5.1) will be a piecewise constant periodic function.During the first branch, it assumes the value β+, during the second branch it attainsthe value β−.

This completes the description of the model. Any quantity describing the engineperformance can only depend on the parameters h1, h2, β±, t±, and ν. In the follow-ing we will focus on the characterization of the limit cycle, which the engine willapproach at long times after a transient period.

We start from the general solution (5.3) of the master Eq. (2.3). Owing to theChapman-Kolmogorov condition (2.75), the propagator within the cycle is

Rp(t) ≡ Rp(t | 0) =⎡R+(t) , t ∞ [0, t+

⎤,

R−(t)R+(t+) , t ∞ [t+, tp⎤.

(5.8)

Here the matrices R±(t) evolve the state vector within the respective branches andhave the form

5.1 Two-Level Heat Engine 95

R+(t) = 1 − 1

2

(1 −1

−1 1

)[1 − e−νt ⎤+ 1

2

(−1 −11 1

)ξ+(t) , (5.9)

R−(t) = 1 − 1

2

(1 −1

−1 1

){1 − e−ν(t−t+)

}+ 1

2

(−1 −11 1

)ξ−(t) , (5.10)

where

ξ+(t) = ν

t∫

0

dτ exp [−ν(t − τ )] tanh

⎡β+⎝

h1 + h2 − h1

t+τ

⎞⎣, (5.11)

ξ−(t) = ν

t∫

t+

dτ exp [−ν(t − τ )] tanh

⎡β−⎝

h2 − h2 − h1

t−(τ − t+)

⎞⎣. (5.12)

Notice that the both propagatorsR+(t) andR−(t) are given by the generic propagator(5.3). In order to get R+(t), we replace in Eq. (5.4) the initial position of the firstenergy h by h1, the driving velocity v by v+ = (h2 − h1)/t+, and we set t → = 0.Analogously, the propagatorR−(t) follows from the generic propagator, if we replaceh by h2, v by v− = (h1 − h2)/t−, and t → by t+.

The periodic state of the system during the cycle, p(t) = Rp(t) pstat, is determinedby the solution of the eigenvalue problem (2.76). The solution reads

pstat1 = 1 − pstat

2 = 1

2

⎨

1 − ξ+(t+) exp (−νt−) + ξ−(t−)

1 − exp⎦−νtp

⎫

⎩

. (5.13)

These probabilities, and hence also the specific form of the limit cycle, depend solelyon the model parameters. The driving only contain a single component and hence thelimit cycle can be depicted using a single diagram (Sect. 2.4.2). The parametric plotof the occupation difference p(t) ≡ p1(t) − p2(t) (the response) versus the energyof the first level E1(t) = E(t) (the driving). It exhibits two possible forms whichare exemplified in Fig.5.1. First, we have a one-loop form which is oriented eitherclockwise or anticlockwise. For clockwise orientation, the work done by the engineon the environment during the limit cycle is negative, while for counter-clockwiseorientation it is positive. Secondly, we can obtain a two-loops shape exhibiting againeither positive or negative work on the environment. Slowing down the driving,the branches gradually approach the corresponding equilibrium isotherms p±(E) =− tanh(β±E/2). We postpone the further discussion of the limit cycle to Sect. 5.1.4.In next two sections we investigate fluctuations of work and heat during the cycle.

96 5 Heat Engines

0 5 10 15 20

2

4

t [s]

(t)

[J]

0 5 10 15 20−1

−0.5

0

t [s]

p(t)

0 2 4 6−1

−0.8

−0.6

−0.4

−0.2

0

(t) [J ]

p(t)

0 5 10 15 20−2

024

t [s]

(t)

[J]

0 5 10 15 20−1

0

1

t [s]

p(t)

−2 0 2 4−1

−0.5

0

0.5

(t) [J ]

p (t)

Fig. 5.1 The limit cycle for the two-stroke engine. The three graphs in the upper panel illustratethe case where h2 > h1 > 0 and the energy levels do not cross during their driving. On the leftside we show E(t) = E1(t) = −E2(t) and the response p(t) = p1(t) − p2(t). On the right handside the parametric plot of the limit cycle in the p−E plane is displayed. The cycle starts in theupper vertex and proceeds counterclockwise, cf. the arrows. The dashed and the dot-dashed curvesshow the equilibrium isotherms corresponding to the baths during the first and the second stroke,respectively. The parameters are: h1 = 1 J, h2 = 5 J, t+ = 5 s, t− = 15 s, β+ = 0.5 J−1,β− = 0.1 J−1, ν = 1 s−1. The three graphs in the lower panel depict the case where h1 < 0 < h2and the energies cross twice during the cycle. Except h1 = −2.5 J, all parameters are as above

5.1.3 Probability Density for Work and Heat

For the linear driving protocol E1(t) = h + v(t − t →) = −E2(t) the solutionof Eq. (2.13) is given by Eqs. (3.21)–(3.24) where the reduced work variable ηis given by η = η(w,w→) = 2β(w − w→), the reduced time variable τ equalsτ = τ (t, t →) = 2β|v|(t − t →) and the parameters a and c are a = ν/2β|v| andc = exp(−2βh|v|/v), respectively. The generic result (3.21)–(3.24) immediatelyyields the work propagator for the individual branches in the protocol accordingto Eq. (5.7). We simply carry out the replacements described in the text follow-ing Eq. (5.11). We denote the corresponding matrices as G±(w, t | w→). Then theChapman-Kolmogorov condition (2.77) yields the propagator

Gp(w, t) =

⎬⎭⎭

⎭⎭

G+(w, t | 0) , t ∞ [0, t+] ,h2−h1∫

−(h2−h1)

dw→ G−(w, t | w→)G+(w→, t+ | 0) , t ∞ [t+, tp] .(5.14)

5.1 Two-Level Heat Engine 97

As demonstrated in Sects. 2.1 and 2.4.1, the energetics during the limit cycle isdescribed by the function

ξ(u, w, t; u→) =2⎟

i, j=1

δ [u − Ei (t)] δ[u→ − E j (0)

⎤ [Gp(w, t)

⎤i j pstat

j . (5.15)

Specifically the probability density for the work reads

ρp(w, t) =∈∫

−∈du

∈∫

−∈du→ ξ(u, w, t; u→). (5.16)

Similarly, the probability density for the heat accepted within the limit cycle is

χp(q, t) =∈∫

−∈du

∈∫

−∈du→

∈∫

−∈dw δ

[q − ⎦u − u→ − w

⎫⎤ξ(u, w, t; u→). (5.17)

These two functions represent the main results of the present section. They areillustrated in Figs. 5.2, 5.4 and 5.4. We discuss their main features in Sect. 5.1.5.

5.1.4 Engine Performance

As shown in Sect. 5.1.2, the occupation probabilities during the limit cycle are p(t) =R+(t) pstat with Rp(t) given by Eq. (5.8). Via Eqs.(2.22)–(2.24) these probabilitiesrender the energetics in terms of mean values as we discuss now. We denote as Q(t) =Q(t + tp, tp) the mean heat received from the reservoirs during the period betweenthe beginning of the limit cycle and the time t . Analogously, W (t) = W (t + tp, tp)stands for the mean work done on the system from the beginning of the limit cycletill the time t . If W (t) < 0, the positive work −W (t) is done by the system on theenvironment. Therefore the oriented areas enclosed by the limit cycle in Figs. 5.1 and5.4 represent the work Wout = −W (tp) done by the engine on the environment percycle, cf. Sect.2.4.2. These areas approach maximal absolute values in the quasi-staticlimit. The internal energy, being a state function, fulfills U (tp) = U (0). Therefore, ifthe work Wout is positive, the same total amount of heat has been transferred from thetwo reservoirs during the cycle. The case Wout > 0 cannot occur if both reservoirswould have the same temperature. That the perpetuum mobile is actually forbiddencan be traced back to the detailed balance condition in (3.5).

We denote the system entropy at time t as Ss(t), and the reservoir entropy at timet as Sr(t) = Sr(t, 0). They are given by [cf. Eqs.(2.31) and (2.33)]

98 5 Heat Engines

−5 0 50

0,1

0,2

0,3

0,4

0,5(a) (b)

(c) (d)w [J]

ρp( w

,t+

/2)

[J−

1]

−5 0 50

0.1

0.2

0.3

0.4

0.5

w [J]

ρp(w

,t+

)[J

−1]

−5 0 50

0.1

0.2

0.3

0.4

0.5c)

w [J]

ρp( w

,t+

+t −

/2)

[J−

1]

−5 0 50

0.1

0.2

0.3

0.4

0.5d)

w [J]

ρp(w

,tp

)[J

−1]

Fig. 5.2 The probability density ρp(w, t) as a function of the work w for the same parameters as inFig.5.1 (with positive h1): a t = t+/2 (middle of the first stroke), b t = t+ (end of the first stroke), ct = t+ + t−/2 (middle of the second stroke), and d t = t+ + t− (end of the limit cycle). The triangleon the work axis marks the mean work W (t) ≡ W (t + tp, tp) at the corresponding times. Thesingular parts of ρp(w, t) are marked by arrows, where the arrow heights equal the weights of thecorresponding delta functions [for example, in panel (a), the left arrow height gives the probabilitythat the system is initially in the second state and remains in it between the beginning of the cycleand the time t = t+/2; then the work done on the system equals −(h2 − h1)/2]

Ss(t)

kB= − [p1(t) ln p1(t) + p2(t) ln p2(t)] , (5.18)

Sr(t)

kB=⎬

−β+⎠ t

0 dt →E1(t →) p(t →), t ∞ [0, t+]

−β+⎠ t+

0 dt →E1(t →) p(t →) − β−⎠ t

t+ dt →E1(t →) p(t →), t ∞ [t+, tp],(5.19)

where p(t) = p1(t) − p2(t). Upon completing the cycle, the system entropyre-assumes its value at the beginning of the cycle. On the other hand, the reser-voir entropy is controlled by the heat exchange. Owing to the inherent irreversibilityof the cycle we observe always a positive entropy production per cycle, Sr(tp) −Sr(0) = Sr(tp) > 0. The total entropy produced by the engine up to the time t ,Stot(t) = Stot(t, 0) = Ss(t) − Ss(0) + Sr(t) increases for any t ∞ [0, tp]. The rate ofthe increase is the larger the stronger the representative point in the p−E diagramdeviates from the corresponding equilibrium isotherm (a strong deviation, e.g., canbe seen in the p −E diagram in Fig. 5.4c). Due to the instantaneous exchange ofthe baths at times t+ and t+ + t− and absence of adiabatic branches [3], a strongincrease of Stot(t) always occurs after these time instants. A representative example

5.1 Two-Level Heat Engine 99

−10 −5 0 5 100

0.1

0.2

0.3

0.4

0.5

q [J]

χ p(q

,t+

/2)

[J−

1]

−10 −5 0 5 100

0.1

0.2

0.3

0.4

0.5

q [J]

χ p(q

,t+

)[J

−1]

−10 −5 0 5 100

0.1

0.2

0.3

0.4

0.5

q [J]

χ p(q

,t+

+t −

/2)

[J−

1]

−10 −5 0 5 100

0.1

0.2

0.3

0.4

0.5

q [J]

χ p(q

,tp)

[J−

1]

(a)

(c) (d)

(b)

Fig. 5.3 The probability density χp(q, t) as a function of the heat q and for the same parametersas in Fig.5.1 (with positive h1): a t = t+/2 (middle of the first stroke), b t = t+ (end of the firststroke), c t = t+ + t−/2 (middle of the second stroke), and d t = t+ + t− (end of the limit cycle).The triangles on the heat axis mark the mean heat Q(t) ≡ Q(t + tp, tp) at the corresponding times.The singular parts of χp(q, t) are marked by the arrow, where the arrow height equals the weightof the corresponding δ-function. For example, in (a), the height of the arrow gives the probabilitythat there was no transition between the states from the beginning of the cycle till the observationtime t = t+/2. The heat exchanged in this case is zero

of the overall behavior of the thermodynamic quantities (mean work and heat, andentropies) during the limit cycle is shown in Fig. 5.5. See also Figs. 5.12 and 5.10for a different model where the adiabatic branches are considered.

Important characteristics of the engine are its power output Pout and its efficiencyη. They are defined by Eqs.(2.69) and (2.70), i.e.,

Pout = Wout

tp, η = Wout

Qin, (5.20)

where Qin is the total heat absorbed by the system per cycle. The performance ofthe engine characterized by the output work, efficiency, output power, and entropiesfrom Eqs. (5.18) and (5.19) are shown in Figs.5.6 and 5.7.

In Fig. 5.6 the performance is displayed as a function of the cycle duration tp fort+ = t− = tp/2. With increasing tp, the output work and the efficiency increasewhereas the output power and the entropy production first increase up to a maximumand thereafter they decrease when approaching the quasi-static limit (tp ∝ ∈).Notice that the maximum efficiency and output power occur at different values oftp. In Fig.5.6a we show also the standard deviation of the output work, which was

100 5 Heat Engines

0 2 4 6−1

−0.5

0

ε (t) [J]−5 0 5

0

0.5

w [J]−10 −5 0 5 100

0.5

q [J]

χ p(q

,tp)

[J−

1]

(a)

0 2 4 6−1

−0.5

0

ε (t) [J]−5 0 5

0

0.5

w [J]−10 −5 0 5 100

0.5

q [J]

(b)

0 2 4 6−1

−0.5

0

ε (t) [J]−5 0 5

0

0.5

w [J]−10 −5 0 5 100

0.5

q [J]

(c)

0 2 4 6−1

−0.5

0

ε (t) [J]

p(t)

−5 0 50

0.5

w [J]

ρ p(w

,tp)

[J−

1]

−10 −5 0 5 100

0.5(d)

q [J]

Fig. 5.4 Probability densities ρp(w, tp) and χp(q, tp) for the work and heat for four representativesets of the engine parameters. For every set we show also the limit cycle in the p−E plane, where thecorresponding equilibrium isotherms are marked by dashed (first stroke) and dot-dashed (secondstroke) lines. In all cases we choose h1 = 1 J, h2 = 5 J, and ν = 1 s−1. The remaining parametersare a t+ = 50 s, t− = 10 s, β+ = 0.5 J−1, β− = 0.1 J−1 (bath of the first stroke is colder than ofthe second stroke), b t+ = 50 s, t− = 10 s, β+ = 0.1 J−1, β− = 0.5 J−1 [exchange of β+ and β−as compared to case (a), leading to a change of the traversing of the cycle from counter-clockwiseto clockwise and a sign reversal of the mean values W (tp) and Q(tp) denoted by the triangles onthe work and heat axis], c t+ = 2 s, t− = 2 s, β+ = 0.2 J−1, β− = 0.1 J−1 (a strongly irreversiblecycle traversed clockwise with positive work), d t+ = 20 s, t− = 1 s, β± = 0.1 J−1 (no change intemperatures, but large difference in duration of the two strokes; W (tp) is necessarily positive)

calculated from the work probability density ρp(w, tp). Finally, let us note that thevalues β+ = 0.5 J−1 and β− = 0.1 J−1 used in Fig.5.6 give the Carnot efficiencyηC = 0.8. This should be compared with the efficiency of the engine for a long periodtp, that is, with the value η ≥ 0.6. As discussed above, the Carnot efficiency cannotbe reached here even for tp ∝ ∈, due to the immediate temperature changes at thetimes t+ and tp = t+ + t−.

In Fig. 5.7 we have fixed tp and plotted the performance of the engine as a functionof the time asymmetry (or time splitting) parameter � = (t+ − t−)/tp. As can beseen from the upper three panels in Fig. 5.7, there exist also a maximal efficiency anda maximal output power with respect to a variation of the time asymmetry parameter(as long as the engine performs work, i.e., Wout > 0). Again, the optimal parameter�, where these maxima occur, is different for the efficiency and for the output power.In a reversed situation, considered in the lower three panels in Fig. 5.7, where thework is performed on the engine (Wout < 0), minima of the efficiency and outputpower occur.

5.1 Two-Level Heat Engine 101

0 5 10 15 20−5

−4

−3

−2

−1

0

1

2

t [s]

mea

nva

lues

[J]

W (tp)

Q(tp)

U(t)W (t)Q(t)

0 5 10 15 200

0.2

0.4

0.6

0.8

1

1.2

1.4

entr

opie

s[J

K−

1]t [s]

S s(t)S r (t)S tot (t) + S s(0)

Fig. 5.5 Thermodynamic quantities as functions of time during the limit cycle for the same set ofparameters as in the upper panel of Fig. 5.1 (positive h1). Left panel internal energy, mean workdone on the system, and mean heat received from both reservoirs; the final position of the mean workcurve marks the work done on the system per cycle W (tp). Since W (tp) < 0, the work Wout hasbeen done on the environment. The internal energy returns to its original value and, after completionof the cycle, the absorbed heat Q(tp) equals the negative work −W (tp). Right panel entropy Ss(t)of the system and Sr(t) of the bath, and their sum Stot(t) + S(0); after completing the cycle, thesystem entropy re-assumes its initial value. Stot(tp) > 0 equals the entropy production per cycle. Itis always positive and quantifies the degree of irreversibility of the cycle

5.1.5 Discussion

The overall properties of the engine critically depend on the two dimensionlessparameters a± = ν/(2β±|v±|). We call them reversibility parameters.2 For a givenbranch, say the first one, the parameter a+ represents the ratio of two characteristictime scales. The first one, 1/ν, is given by the attempt rate of the internal transitions.3

The second scale is proportional to the reciprocal driving velocity. Contrary to the firstscale, the second one is fully under the external control. Moreover, the reversibilityparameter is proportional to the absolute temperature of the heat bath.

Let us first consider the work probability density (5.16) within the first stroke inthe case h2 > h1, cf. Fig.5.2a. In essence, ρp(w, t) is given by a linear combinationof the functions (3.21)–(3.24). It vanishes outside the common support [ −v+t, v+t ]which broadens linearly in time. Besides the continuous part located within thesupport, the diagonal elements [Gp(w, t | , 0)]i i display a singular part represented

2 The reversibility here refers to the individual branches. As pointed out in Sect.5.1.1, the abruptchange in temperature, when switching between the branches, implies that there exists no reversiblelimit for the complete cycle.3 Note that due to the choice of the Glauber rates in Eq. 5.1, the relaxation rate [λU(t)+λD(t)] (forfrozen energy levels at any time instant t) is bounded by 2ν.

102 5 Heat Engines

0 10 20 30 400

1

2

3(a)

tp [s]

Wou

t[J

]

0 10 20 30 40

−0.05

0

0.05

0.1(b)

tp [s]

Pou

t[J

s−1 ]

0 10 20 30 40−0.2

0

0.2

0.4

0.6(c)

tp [s]

η

0 10 20 30 400.2

0.3

0.4

0.5

0.6

(d)

tp [s]

Sto

t(t p

)[J

K−

1 ]

Fig. 5.6 The engine performance versus the duration of the limit cycle t± for t+ = t− = 12 tp and

otherwise the same parameters as in the upper part of Fig.5.6 (positive h1). Both the output work Woutin (a) and the efficiency η in (b) increase with tp. The output power Pout in (c) assumes a maximumat a special cycle duration. The dashed line in (a) marks the standard deviation of the output work,calculated from the work density ρp(w, tp). Notice that the work fluctuation is comparatively highclose to the cycle duration, where the maximal output power is found. In the long-period limittp ∝ ∈, the cycle still represents a non-equilibrium process (due to the construction of the model,see text), and hence the entropy production Stot(tp) in (d) remains positive, approaching a specificasymptotic value

by δ-functions at the borders of the support. The δ-functions correspond to the pathswith no transitions between the states, cf. Appendix B. Specifically, the weight ofthe δ-function located at w = v+t represents the probability that the system starts inthe first state and remains there up to time t . The weight corresponding to the firstlevel decreases with increasing time and vanishes for t ∝ ∈. On the contrary, theweight of the delta function at −v+t approaches the nonzero limit (1 + c+)−a+ fort ∝ ∈, which is the probability that a trajectory starts in the second state and neverleaves it.

Within the second stroke, the density ρp(w, t) results from the integral of thepropagators for the individual strokes, cf. Eq. (5.14). Due to the integration, thesingular parts of the cycle propagatorG+(w, t | 0) are now situated inside the support,at the values w = ±[−v+t+ +v−(t − t+)]. The two δ-functions approach each otherand, upon completing the cycle, they coincide at the point w = 0. The non-singularcomponent of the density is no more continuous.4 The jumps are located at the

4 Only δ-functions are referred to as singularities here.

5.1 Two-Level Heat Engine 103

−1 0 1

0

0.1

0.2

0.3

0.4

(a1)

(a2) (b2) (c2)

Δ = (t+ − t− )/t p

η

−1 0 10

0.05

0.1

0.15

0.2(b1)

Δ = (t+ − t− )/t p

Pou

t[J

s−1 ]

−1 0 1

0.4

0.5

0.6

0.7(c1)

Δ = (t+ − t− )/t p

Sto

t(t p

)JK

−1

−1 0 1

−2

−1.5

−1

Δ = (t+ − t− )/t p

η

−1 0 1

−0.1

0

0.1

Δ = (t+ − t− )/t p

Pou

t[J

s−1 ]

−1 0 1

0.4

0.6

0.8

1

Δ = (t+ − t− )/t p

Sto

t(t p

)JK

−1

Fig. 5.7 The engine performance characterized by the efficiency η, output power Pout, and entropyproduction Stot(tp), as a function of the asymmetry parameter � = (t+ − t−)/tp for a fixed periodtp = 20 s and the same parameters h1 = 1 J, h2 = 5 J, ν = 1 s−1 as in the upper panel of Fig. 5.1.In (a1)–(c1) the bath during the first stroke is colder than during the second stroke: β+ = 0.5 J−1

and β− = 0.1 J−1. Notice that the value � of maximum efficiency does not correspond to that ofmaximum output power. In (a2)–(c2) the reciprocal bath temperatures are interchanged comparedto cases (a1)–(c1), β+ = 0.1 J−1 and β− = 0.5 J−1. The dashed curves in (b1) and (b2) show thestandard deviation of the output power calculated from ρp(w, tp)

positions of the δ-functions and their magnitudes correspond to the weights of theδ-functions (for a discussion of the origin of these jumps, see [8]).

If both reversibility parameters a± are small, the isothermal processes during bothbranches strongly differ from the equilibrium ones. The signature of this case is a flatcontinuous component of the density ρp(w, t) and a well pronounced singular part.The strongly irreversible dynamics occurs if one or more of the following conditionshold. First, if ν is small, the transitions are rare and the occupation probabilitiesof the individual energy levels are effectively frozen during long periods of time.Therefore they lag behind the Boltzmann distribution which would correspond to theinstantaneous positions of the energy levels, cf. Fig.1. More precisely, the populationof the ascending (descending) energy level is larger (smaller) than it would be duringthe corresponding reversible process. As a result, the mean work done on the systemis necessarily larger than the equilibrium work. Secondly, a similar situation occursfor large driving velocities v±. Due to the rapid motion of the energy levels, theoccupation probabilities again lag behind the equilibrium ones. Thirdly, the strong

104 5 Heat Engines

irreversibility occurs also in the low temperature limit. In the limit a± ∝ 0, thecontinuous part vanishes and ρp(w, tp) = δ(w).

In the opposite case of large reversibility parameters a±, both branches in thep −E plane are located close to the reversible isotherms. The singular part of thedensity ρp(w, t) is suppressed and the continuous part exhibits a well pronouncedpeak. From general considerations [9, 10], the density must approach, around itsmaximum, a Gaussian shape. Our results allow a detailed study of this approach.Let us denote as F(β, E) the free energy of a two level system with energies ±Eat temperature T = 1/(kBβ), i.e., F(β, E) = − ln[2 cosh(βE)]/β. Let us furtherdefine the function Wrev(t) as

⎡F[β+, E1(t)] − F[β+, E1(0)] , t ∞ [0, t+],F[β−, E1(t)] − F[β−, E1(t+)] + F[β+, E1(t+)] − F[β+, E1(0)], t ∞ [t+, tp]. (5.21)

This is simply the reversible work (2.55) done on the system if we transform itsstate from the initial equilibrium state [with the energies fixed at ±E1(0)] to anotherequilibrium state [with the energies fixed at the values ±E1(t)]. For large reversibilityparameters a±, the peak of the work density ρp(w, tp) occurs in the vicinity of thevalue Wrev(t) and with increasing a±, the peak collapses to a δ-function (for a generalprove see Appendix A)

lima±∝∈ ρp(w, t) = δ[w − Wrev(t)]. (5.22)

The main features of the heat probability density χp(q, t) from Eq. (5.17) are,as we have seen in Sect. 5.1.3, closely related to the work through simple shifts ofthe independent variable q. However, there are some interesting differences. Whilethe work is conditioned by the external driving, the heat exchange occurs as a con-sequence of the transitions between the system energy levels. The instantaneouspositions of the energies at the instant of the transition give the magnitude of theheat exchange related with the given transition. From this perspective, if there areno transitions, the exchanged heat is zero. As a consequence, the singular part ofthe probability density χp(q, t) is always situated at q = 0 and the weight of the δ-function at origin equals the sum of the weights of the δ-functions in the work densityρp(w, t). The support of the heat density is given by the largest possible value of thelevel splitting during the limit cycle. Within the first stroke the support broadens lin-early with time as [−2h1 −2v+t, 2h1 +2v+t], up to its maximum width [−2h2, 2h2]at the end of the stroke. Within the second stroke the energy difference decreasesand the support remains unchanged. The non-singular part of the heat density alwaysdisplays discontinuities inside the support, even during the first stroke.

In the strongly reversible regime each element ≤ i |Gp(w, t)| j ≡ exhibits aGaussian shape situated at Wrev(t). The transformation (5.15) maps the Gaussianfunction onto four different positions in the heat probability density χp(q, t). In thereversible limit we have

5.1 Two-Level Heat Engine 105

lima±∝∈ χp(q, t)≡ =

2⎟

i, j=1

δ⎧

q − [Ei (t) − E j (0) − Wrev(t))⎤⎪ [Rp(t)]i j pstat

j .

(5.23)If we calculate the mean accepted heat using this form, we get Q(t) = U (t)−U (0)−Wrev(t). In the opposite limit, if a± ∝ 0, we have χp(q, t) ∝ δ(q) for any t .

According to the second law of thermodynamics, the mean work W (t) must fulfill|W (t)| ≈ |Wrev(t)|. On the other hand, there always exists a fraction of trajectorieswhich, individually, display the inequality |w(tr, t)| < |Wrev(t)|, where w(tr, t)denotes the work done on the system if it evolves along the indicated trajectory.Using the exact work probability density, we can calculate the total weight of thesetrajectories. Specifically, in the case Wrev(t) > 0,

Prob{W(t, 0) < Wrev(t)} =Wrev(t)∫

−∈dw ρp(w, t) . (5.24)

If Wrev(t) < 0, we would have to integrate over the interval (Wrev(t),∈).Let us finally note that in view of the rather complex structure of the work and

heat probability densities, we performed several independent tests. First of all, thedensities ρp(w, t) and χp(q, t) must be non-negative functions fulfilling the nor-malization conditions, i.e.,

⎠∈−∈ dw ρp(w, t) = 1 for any t ∞ [0, tp]. Secondly, we

have two different procedures to calculate the first moment W (t). One can eitherstart with the density ρp(w, t) and evaluate the required w-integral, or one directlyemploys the solution of the rate equation as in Sect. 5.1.4. Another inspection isbased on the Jarzynski identity [11, 12]. In our setting, consider the case β± = β.After completing the cycle, the system returns to the original state. Therefore we haveWrev(tp) = F[β, E1(tp)]− F[β, E1(0)] = 0 and the Jarzynski identity (2.61) reducesto ≤ exp

[−β W(tp, 0)⎤ ≡ = 1. Using the explicit form of the work probability density

we have verified that the integral⎠∈−∈ dw exp(−βw)ρp(w, tp) actually equals one.

Finally, we have studied the probability densities ρp(w, t), χp(q, t) by computersimulation. In fact, we have developed two exact simulation methods which are dis-cussed in Appendix C. Each of them uses a specific algorithm to generate paths ofthe time-non-homogeneous Markov process D(t). Parts of these simulation resultshave been published in [8, 13] and confirm the analytical results.

5.1.6 Conclusions

We have investigated a simple example of a microscopic heat engine, which is exactlysolvable. Based on mean thermodynamic quantities, the engine performance is char-acterized by the occupation probabilities of the energy levels following from themaster equation. The more challenging exact calculation of the work and heat prob-ability densities allowed us to study the fluctuation properties in detail. A notable

106 5 Heat Engines

result is that the engine can be tuned to maximize its output power, but the fluctua-tions of this quantity in the corresponding optimal regime of control parameters arecomparatively high.

The present setting can be expanded in various directions. One can address variousproblems concerning the thermodynamic optimization. Another option would bethe embodiment of additional (e.g., adiabatic) branches. The role of the workingmedium can be assigned to other systems that exhibit more complicated dynamics(e.g., diffusing particles in the presence of time-dependent forces, or, variants ofthe generalized master equation). It would be also interesting to investigate settingswith a nonlinear driving of the energy levels. A nontrivial generalization wouldbe the inclusion of a third energy level. Having the three levels one can couple thesystem (different pairs of forth-back transitions between the levels) simultaneously toreservoirs at different temperatures, so that the system approaches a non-equilibriumsteady state without driving [14]. Including a driving and forming an operationalcycle, there is no serious obstacle in repeating the present analysis for this system,which has some additional intriguing properties compared to the two-level systemconsidered here (as, e.g., negative specific heats).

Another possibility is an incorporation of specific forms of transition rates [15]that describe the stretching of biomolecules in some realistic manner. In such prob-lem, the histogram of the work is experimentally accessible [15]. Particularly, inthe experiments one can also determine the probability of having certain numberof transitions between the folded and the unfolded conformation of the biomole-cule during its mechanical stretching [15]. In our formulation, this information isencoded in the counting statistics of the underlying random point process [16] andcan be extracted from the perturbation expansion of the propagators which solve ourdynamical equations.

5.2 Diffusive Heat Engine

5.2.1 Introduction

In this section we focus on stochastic heat engines based on Brownian particlesdiffusing in a periodically driven potential. In [17] strong general results concerningthe efficiency at maximum power for a wide class of such engines were obtained.Nevertheless it turned out that exactly solvable is only an engine driven by a symmetricharmonic potential [17], which was also realized experimentally [18]. We give anexactly solvable example of a stochastic heat engine based on a particle diffusingin the asymmetric log-harmonic potential (5.25). Such potential was considered intheoretical studies [19–21] and can be also realized experimentally using the opticaltweezers [22]. Within our specific setting we verify the universal results regarding theefficiency at maximum power obtained in [17, 23, 24] (see also Sect. 2.4) and discussproperties of the optimal driving. In particular we propose a physical explanation for

5.2 Diffusive Heat Engine 107

occurrence of the jumps in the optimal driving as a consequence of the equivalencebetween the power maximization and minimization of the total entropy production[25]. Moreover, we investigate the performance of the engine for two other protocolsand discuss the possibility to minimize fluctuations of the output power.

This section is organized as follows: in Sect. 5.2.2 we describe the workingmedium of the motor and present the Green’s function for its dynamics, which wefurther use for constructing the working cycle of the engine. In Sect. 5.2.3 we specifythe thermodynamic quantities used in our analysis. Sect. 5.2.4 contains the discussionconcerning the possibility to depict the operational cycle of the engine by a stochasticthermodynamics analogue of the well known p-V diagrams (see also Sect. 1.4.2). InSect. 5.2.5 we introduce three examples of the driving. First, we derive the drivingwhich maximizes the output power. Second, we define the driving, which yields thesmallest power fluctuations from the three protocols. These two protocols inevitablyincorporate two isothermal and two adiabatic branches. The third protocol is differ-ent and may consists of two isotherms only. Finally, in Sect. 5.2.6, we discuss theobtained results.

5.2.2 Description of the Engine and Its Limit Cycle

Consider an externally driven Brownian particle diffusing on the interval [0,∈]with a reflecting boundary at x = 0. Assume that the external driving is representedby a time-periodic log-harmonic potential [19–21]. Let the microstate degeneracies�(x) = 1. Moreover, assume the FEL (2.1) is given solely by the external potential,V(x, t), i.e., we have

F(x, t) = E(x, t) = V(x, t) = −g(t) log x + k(t)

2x2, (5.25)

where −g(t) < kBT and k(t) > 0. The requirement −g(t) < kBT secures theexistence of a nontrivial steady state. For −g(t) ≈ kBT are, in the long time limit,all particles trapped at x = 0. Dynamics of the particle is described by Eq. (2.38). Infurther we take � = kB = 1. Then Eq. (2.38) reads

∂

∂tR(x, t | x →, t →) =

⎡T (t)

∂2

∂2x− ∂

∂x

⎝g(t)

x− k(t) x

⎞⎣R(x, t | x →, t →) (5.26)

with the initial condition R(x, t | x →, t →) = δ(x − x →). Note that the temperature inEq. (5.26) is also time dependent. The control parameters are the functions g(t) andk(t), i.e., the vector of control parameters, Y(t), has two components (cf. Sect. 2.4.1).

108 5 Heat Engines

5.2.2.1 Generic Case: Isothermal Process

In order to solve Eq. (5.26) we first consider an isothermal process [T (t) = T ] withthe driving

V(x, t) = −g log x + k(t)

2x2. (5.27)

In this generic case [T (t) = T , g(t) = g] it is possible to solve Eq. (5.26) analytically(see, for example, [21]). The solution is

R(x, t | x →, t →) = x →e ν+22 a(t,t →)

2 b(T ; t, t →)

( x

x →)ν+1

× exp

⎨

− x2ea(t,t →) + (x →)2

4 b(T ; t, t →)

⎩

Iν

⎨xx →e 1

2 a(t,t →)

2 b(T ; t, t →)

⎩

, (5.28)

where Iν(x) stands for the modified Bessel function of the first kind [7],

ν = 1

2

( g

T− 1)

, ν > −1 (5.29)

and

a(t, t →) = 2

t∫

t →dt →→ k(t →→),

b(T ; t, t →) = T

t∫

t →dt →→ exp

⎨

2∫ t →→

t →dt →→→ k(t →→→)

⎩

. (5.30)

Note that the parameter g enters the generic solution (5.28) only through the dimen-sionless combination ν.

5.2.2.2 Operational Cycle of the Engine

Let the periodic driving of the engine (5.25) consist of two isotherms and two adia-batic branches as it is described in Sect. 2.4.1. To completely specify the protocol it isenough to describe the concrete form of the driving during the isothermal branches.To this end we assume that, during the isotherms, the potential energy V(x, t) hasthe generic form (5.27), i.e., we have

5.2 Diffusive Heat Engine 109

0

0.5

1

1.5

p(x,

0)

0 0.5 1 1.5 20

5

10

x

V(x

,0)

0

0.5

1

1.5

p(x,

t− +)

0 0.5 1 1.5 20

5

10

x

V(x

,t− +

)

00.5

11.5

p(x,

t+ +)

0 0.5 1 1.5 20

5

10

x

V(x

,t+ +

)

00.5

11.5

p(x,

t− p)

0 0.5 1 1.5 20

5

10

x

V(x

,t− p

)

IV II

T+

I

T−

III

Fig. 5.8 Scheme of the operating cycle of the engine driven by the optimal protocol (5.57). Theparticle distribution (5.34) is depicted by the filled curve. The solid line represents the potentialenergy (5.25). Parameters used: t+ = 6, t− = 4, T+ = 1, T− = 0.5, f0 = 0.1, f1 = 0.2, ν = 0.2

V(x, t) =

⎬⎭⎭⎭

⎭⎭⎭

−g+ log x + k(t)

2x2, t ∞ [0, t−+ ]

−g− log x + k(t)

2x2, t ∞ [t++ , t−p ]

. (5.31)

During the first adiabatic branch the potential changes from V(x, t−+ ) to V(x, t++ ).During the second adiabatic branch the potential regains its initial value, i.e., itchanges from V(x, t−p ) to V(x, tp) = V(x, 0). This pattern is periodically repeated,

the period being tp. The durations of the adiabatic branches t++ − t−+ and tp − t−pare considered as infinitesimally short as compared to the durations of the isothermst+ = t−+ and t− = t−p − t++ . During the first (the second) isotherm the temperature Tassumes the value T+ (T−). For the sake of mathematical simplicity we choose theparameters g± proportional to the corresponding reservoir temperatures. Specificallywe take g± = (2ν + 1)T±.5 This setting allows us to investigate the cases whenthe logarithmic part of the potential V(x, t) is during the whole cycle repulsive(attractive). The model with general g± would describe also the situation when thetwo constants have different sign and thus the logarithmic part of the potential (5.31)is during the first isotherm repulsive (attractive) and vice versa during the secondone.

The hereby considered driving is depicted in Fig. 5.8. The resulting limit cyclefor ν = −1/2 coincides with that discussed in [17]. The fact that the protocolcontains adiabatic branches II and IV, where the potential and the temperature change

5 It turns out that for general parameters g± the Green’s function (5.32) is given by a sum of Gausshypergeometric functions [26] and the integral Eq. (2.81) becomes quite complicated.

110 5 Heat Engines

infinitely fast while the particle distribution remains unchanged, may look artificial.However, it is not as was shown in experiments [18]. Onwards we will focus on thecharacterization of the limit cycle, which the engine will approach at long times aftera transient period.

We start from the generic solution (5.28) of the Fokker-Planck Eq. (5.26). Owingto the Chapman-Kolmogorov condition (2.80), the propagator within the cycle stillassumes the form (5.28). Specifically it reads

Rp(x, t | x →, t →) = x →e ν+22 a(t,t →)

2b(t, t →)

( x

x →)ν+1

× exp

⎨

− x2ea(t,t →) + (x →)2

4b(t, t →)

⎩

Iν

⎨xx →e 1

2 a(t,t →)

2b(t, t →)

⎩

, (5.32)

where the function b(t, t →) is given by

b(t, t →) =

⎬⎭⎭

⎭⎭

b(T+; t, t →), t →, t ∞ [0, t++ ],b(T+; t+, t →) + ea(t+,t →) b(T−; t, t+), t → ∞ [0, t++ ] ∧ t ∞ [t++ , tp],b(T−; t, t →), t →, t ∞ [t++ , tp],

(5.33)and the functions a(t, t →) and b(T ; t, t →) are defined in Eq. (5.30).

The periodic state of the system during the limit cycle is determined by the solutionof the integral equation (2.81), which reads p(x, t) = ⎠∈

0 dx → Rp(x, t | x →, 0)p(x, 0).For our present setting, Eq. (2.81) can be solved analytically. The result

p(x, t) = 1

�(ν + 1)

⎝1

f (t)

⎞ν+1 ( x

2

)2ν+1exp

⎝− x2

4 f (t)

⎞(5.34)

was already mentioned in [21] as a time-asymptotic distribution for a periodicallydriven system. The function f (t) =< [x(t)]2 > /(4ν + 4) determines the width ofthe distribution (5.34). It obeys the formula

f (t) =⎡ [ f0 + b(t, 0)] exp[−a(t, 0)], t ∞ [0, t+]

[ f1 + b(t, t+)] exp[−a(t, t+)], t ∞ [t+, tp] , (5.35)

where f0 = f (0) = f (tp) = ⎧b(t+, 0) + b(tp, t+) exp[a(t+, 0)]⎪ / f , f1 = f (t−+ ) =f (t++ ) = ⎧b(tp, t+) + b(t+, 0) exp[a(tp, t+)]⎪ / f , and f = exp[a(tp, 0)] − 1. Notethat the system response is continuous regardless the discontinuities in the driving.The probability distribution (5.34), and hence also the specific form of the limit cycle,is determined solely by the parameters ν, T±, t±, and by the function k(t).

5.2 Diffusive Heat Engine 111

5.2.3 Thermodynamic Quantities

5.2.3.1 Mean Values

As in the model discussed in Sect.(5.1), the probability distribution p(x, t) rendersthe energetics of the engine in terms of mean values as we discuss now. The periodicpotential energy V(x, t) can be rewritten as V(x, t) = V[x, g(t), k(t)]. Accordingto Eq. (2.83) the thermodynamic work done by the engine during the time interval[t, t →], Wout(t, t →) = −W (t, t →), reads

Wout(t, t →) =t∫

t →dt →→ ≤log x(t →→)≡ dg(t →→)

dt →→−

t∫

t →dt →→ 1

2≤[x(t)]2≡dk(t →→)

dt →→, (5.36)

where the two averages are given by

≤log x(t)≡ =∈∫

0

dx log x ρ(x, t) = 1

2

⎡log[4 f (t)] + d

dνlog �(ν + 1)

⎣, (5.37)

1

2≤[x(t)]2≡ =

∈∫

0

dx x2ρ(x, t) = 2(ν + 1) f (t). (5.38)

These two averages also determine the mean internal energy of the system at thetime t , U (t) = −g(t)≤log x(t)≡+ 1

2 k(t)≤[x(t)]2≡, cf. Eq. (2.47). Its increase from thebeginning of the cycle

�U (t) = U (t) − U (0) (5.39)

and the mean work done from the beginning of the cycle up to the time t , Wout(t) ≡Wout(t, 0), yield via the second law of thermodynamics the mean heat uptake duringthe time interval [0, t]

Q(t) ≡ Q(t, 0) = �U (t) + Wout(t). (5.40)

The total work done by the engine per cycle, Wout = Wout(tp), determines itsaverage output power: Pout = Wout/tp . The engine efficiency is given by Eq. (2.69),i.e., η = Wout/Qin, where Qin stands for the total heat transferred to the systemfrom the reservoirs, cf. Eq. (2.68). The entropy of the system at the time t , (2.52), isSs(t) = − ⎠∈

0 dx p(x, t) log [p(x, t)]. Its increase from the beginning of the cycle[Eq. (2.32)] reads

Ss(t, 0) = Ss(t) − Ss(0) = 1

2log

f (t)

f (0). (5.41)

112 5 Heat Engines

Due to the time dependence of the temperature during the cycle the entropy trans-ferred to the reservoirs during the time interval [0, t] Eq. (2.33) needs to be redefinedas

Sr (t) = Sr (t, 0) = −t∫

0

dt → 1

T (t →)d

dt →Q(t →). (5.42)

Finally, the total entropy produced by the engine during the time interval [0, t] isgiven by Eq. (2.34), i.e.,

Stot(t) = Stot(t, 0) = Ss(t, 0) + Sr (t) ≈ 0. (5.43)

5.2.3.2 Work and Power Fluctuations

In the previous model Sect. (5.1) we examined the work and power fluctuations usingthe work probability density (5.16). Here we employ Eq. (2.50), i.e., we study thework and power fluctuations using the propagator Rp(x, t | x →, t →), which yields thetime-correlation function (2.51). Specifically we calculate the relative fluctuation ofthe work done by the engine during the time interval (t →, t)

σw(t, t →) =√≤[W(t, t →)]2≡ − [Wout(t, t →)]2

|Wout(t, t →)| (5.44)

which determines the relative power fluctuation

δPout = σw(tp, 0) (5.45)

and, hence, in a sense, also the stability of engine performance. In Eq. (5.45) thesecond moment ≤[W(t, t →)]2≡ is given by Eq. (2.50), i.e., it reads

≤[W(t, t →]2≡ =t∫

t →dt →→

t∫

t →dt →→→

⟨∂V[X(t →→), t →→]∂t →→

∂V[X(t →→→), t →→→]∂t →→→

⟩

C, (5.46)

where ∂V[x(t), t]/∂t = − log[x(t)] dg(t)/dt + [x(t)]2/2 dk(t)/dt and the time-correlation function ≤h([X(t)] f [(X(t →)]≡C obeys Eq. (2.51). Note that, due to thejumps in the driving, the functions dg(t)/dt and dk(t)/dt may also contain singularterms proportional to δ-functions. For example dk(t)/dt = [k(t++ )−k(t−+ )]δ(t−t+)+[k(t+p )−k(t−p )]δ(t − tp)+dk(t)/dt

[�(t+ − t) + �(tp − t)�(t − t+)

⎤, where �(t)

equals 1 for t > 0 and 0 otherwise. For an isothermal process the function σw(t, t →)can be obtained from the work characteristic function derived in [21].

5.2 Diffusive Heat Engine 113

5.2.4 Diagrams of the Limit Cycle

The driving Y(t) has two components, g(t) and k(t). The mean work done by theengine per cycle can thus be written as Wout = Wg + Wk (cf. Sect. 2.4.2). FromEq. (5.36) the two terms are identified:

Wg =tp∫

0

dt ≤log x(t)≡⎝

dg(t)

dt

⎞, (5.47)

Wk = −tp∫

0

dt1

2≤[x(t)]2≡dk(t)

dt. (5.48)

The first contribution equals the area enclosed by the parametric plot of the average≤log x(t)≡ (system response) versus the driving component g(t), where the parame-ter t runs from 0 to tp. Similarly the second contribution corresponds to the areaenclosed by the parametric plot of the response ≤[x(t)]2≡/2 versus the driving com-ponent −k(t), cf. Figs. 5.9 and 5.11. In context of stochastic thermodynamics suchparametric plots, which represent an analogy of the well known p-V diagrams, wereintroduced in [2]. From now on we call the parametric plot corresponding to Wg(Wk) as g-cycle (k-cycle). Wg and Wk are proportional to (2ν + 1) and to (ν + 1),respectively. Consequently, it is possible to tune the sign of the total output powerPout = (Wg + Wk)/tp by changing the parameter ν > −1, cf. Fig.5.14.

An important eye-guide in Figs. 5.9 and 5.11 are the two equilibrium isotherms≤log x(t)≡eq and ≤[x(t)]2≡eq/2, where ≤•≡eq = − ⎠∈

0 dx •exp [−V(x, t)/T (t)] /Z(t),Z(t) = �(ν + 1)2ν[T (t)/k(t)]ν+1. They correspond to the values of the averages(5.37)–(5.38) if a given cycle would be carried out quasi-statically. The specific formof ≤log x(t)≡eq and ≤[x(t)]2≡eq/2 is obtained if one substitutes

feq(t) = T (t)

2k(t)(5.49)

for the function f (t) in Eqs. (5.37) and (5.38), respectively. The farther a non-equilibrium isotherm is from the corresponding equilibrium one the more irreversiblethis branch is, cf. Fig. 5.11. Close to equilibrium the work probability density is rea-sonably approximated by a Gaussian distribution [9, 27, 28]. From Jarzynski equality(2.61), valid during the isothermal branches, then follows T (t) log[Z(t →)/Z(t)]≥ −Wout(t, t →)−[Wout(t, t →) σw(t, t →)]2/[2T (t)]. We have used this formula for verifica-tion of the calculated functions (5.36) and (5.44). The further discussion of the cyclediagrams is postponed to Sect.5.2.6. In the next section we specify several forms ofthe driving component k(t) with respect to the engine performance.

114 5 Heat Engines

0 0.5 105

1015

t

k( t

)

0 0.5 10.20.50.7

t

f(t

)

−15 −10 −5 00

0.1

0.2

− k (t)

[ x(t

)]2

/2

−1.36 −1.34 −1.320

0.2

0.4

0.6

0.8

− k (t)

[x(t

)]2

/2

0 0.5 10.2

0.50.7

t

f(t

)

−4 −2 0−3.5

−3

−2.5

−2

(2 + 1) T (t)

log[

x(t

)]

−4 −2 0−4

−3.5

−3

−2.5

−2

−1.5

(2ν

ν

+ 1) T (t)

log[

x(t

)]

0 0.5 11.3

1.35

1.4

t

k(t

)

Fig. 5.9 Upper figures depict the driving k(t), the systems response f (t) Eq. (5.35), the g-cycle(5.47) formed by one clockwise loop (Wg > 0) and the k-cycle (5.48) formed by one clockwise loopand one counter-clockwise loop (Wk > 0) for the optimal protocol (5.57), respectively. Lower threefigures show the same for the adiabatic driving (5.62). The total work output for the both cycles ispositive (Wout > 0). In the left figures the red (blue) curve corresponds to the first (second) isotherm.The curves corresponding to the (infinitely fast) adiabatic branches are depicted in black. Red/bluedashed lines in the middle and in the right figures stand for the hot/cold equilibrium isotherms.In the both figures the black circles denote the initial points of the cycles, the directions of thecirculations are marked by the arrows. During the isothermal branches of the both g-cycles (andalso during the isotherms of the k-cycle for the adiabatic driving) the driving is constant and henceno work is produced. These branches thus represent an analogue of isochores from the classicalthermodynamics. Another consequence of the constant k(t) during the isotherms of the adiabaticdriving is the fact that the equilibrium isotherms corresponding to the g-cycle degenerate to points.The system response f (t) for the two different protocols [in particular compare the ranges of usedk(t) values] is quite similar. Parameters used: t+ = 0.15, t− = 0.85, T+ = 5, T− = 0.1, f0 = 0.1,f1 = 0.6864, ν = −0.8. Here and in all other figures the remaining parameters are calculated fromthe closure conditions on the driving and system response which read k(tp) = k(0), f (t−+ ) = f (t++ )

and f (tp) = f (0)

5.2.5 External Driving

The quantities Pout, δPout and η naturally determine the “quality” of a Brownianengine. The ideal engine should work, at the same time, at the largest possible outputpower (2.70) with the smallest possible fluctuation (5.45) and with the largest possibleefficiency (2.69). However, such engine can not be constructed. For example, themaximum possible efficiency (2.71) is obtained in the reversible limit, when theoutput power vanishes. Therefore one has to either maximize a single characteristicof the engine alone or settle with a compromise.

5.2 Diffusive Heat Engine 115

0 0.5 1−5

0

5

10

t

Wou

t(t)

,Δ

U(t

),Q

(t)

0 0.5 10

1

2

3

t

σσ

w(t

,0)

0 0.5 1

−5

0

5

10

t

Wou

t(t)

,Δ

U( t

),Q

(t)

0 0.5 10

2

4

6

t

w(t

,0)

0 0.5 1−1

0

1

2

3

t

Ss(

t),

Sr(

t),S

tot(

t)

0 0.5 1−1

0

1

2

3

t

Ss(

t),

Sr(

t),S

tot(

t)

Fig. 5.10 Time-resolved thermodynamics of the engines depicted in Fig. 5.9. In all the panels thered (blue) curve stands for the first (second) isotherm. The curves corresponding to the (infinitelyfast) adiabatic branches are depicted in black. The upper (lower) figures in Fig. 5.9 stand forthe upper (lower) figures herein. Left The mean work done on the environment (solid line), themean heat accepted by the system (dot-dashed line) and the internal energy of the system (brokenline). Note that these curves verify the first law of thermodynamics. Middle The entropy of thesystem (solid line), the entropy transferred to the reservoirs (dot-dashed line) and the total entropyproduction of the engine (solid line). The total entropy produced per cycle, Stot(tp), is alwayspositive and equals the amount of entropy transferred to the reservoirs, Sr(tp). Right The relativework fluctuation. The cycle driven by the optimal protocol (upper figures) produces larger workwith smaller relative fluctuation than that driven by the adiabatic driving (lower figures), whichperforms work only during the adiabatic branches. More entropy is produced during the cycle withthe adiabatic driving. Note that the relative work fluctuation changes, during the adiabatic branchesof the cycle, discontinuously. Although this function can also decrease, the total fluctuation at theend of the cycle is always positive

5.2.5.1 Maximum Power

Let the parameters f0 = f (0) = f (tp), f1 = f (t+), t±, T± and ν are given. In thissection we will find the specific form of the function k(t) which yields the maximumpower (2.70) for these parameters. Later on we will identify also ideal durations of thetwo isothermal branches, t±. The mean work (5.36) represents a complicated, non-local, functional of k(t) (see for example [29] where the procedure is performed for anisothermal process). Therefore, instead of finding the optimal protocol k(t) directly,we will adopt a little bit tricky approach introduced in [17] (see also [30]). Thelimit cycle (5.34) corresponding to the (unknown) optimal protocol k(t) is inevitablydescribed by a certain function f (t). This function can be obtained from k(t) viaEq. (5.35) and, similarly, the function k(t) can be obtained from the function f (t)

116 5 Heat Engines

0 250 5000123

t

k(t

)

0 250 5000123

t

f(t

)

−3 −2 −1 00

1

2

3

− k (t)

[x(t

)]2

/2

0 250 5000.480.49

0.5

t

k(t

)

0 250 5000123

t

f(t

)

−0.5 −0.495 −0.49 −0.485

1

2

3

− k (t)

[x(t

)]2

/2

−0.4 −0.2 0−0.8

−0.6

−0.4

−0.2

0

0.2

(2ν + 1) T (t)

log[

x(t

)]

−0.2 0−0.8

−0.6

−0.4

−0.2

0

0.2

(2ν + 1) T (t)

log[

x(t

)]

Fig. 5.11 The same quantities as in Fig. 5.9. Upper (lower) three figures correspond to the optimalprotocol (5.57) [fractional driving (5.63)]. For the both cycles we have taken very slow driving(t+ = 200, t− = 300). During the cycle driven by the optimal protocol the system is at all timesvery close to the equilibrium state. The cycle is nearly quasi-static and its efficiency η ≥ 0.830almost reaches the Carnot’s upper bound ηC ≥ 0.833. On the other hand, during the cycle drivenby the fractional driving the system is brought far from equilibrium at the instants when the heatreservoirs are interchanged. During the emerging relaxation processes a large amount of entropy isproduced (cf. Fig. 5.12) and the engine efficiency η ≥ 0.182 is far from ηC. Note also that the systemresponse f (t) after these time instants changes quite rapidly as compared to the system responsefor the optimal protocol. It may seem that the equilibrium isotherms in the g-cycle corresponding tothe fractional driving again degenerate to points. This illusion is caused by the fact that the drivingk(t) changes during these isotherms only slightly. The range of k(t) used for the optimal protocolis much larger. Other parameters used: T+ = 3, T− = 0.5, f0 = 0.5, f1 = 3.09, ν = −0.55

using the formula

k(t) = − f (t) − T (t)

2 f (t). (5.50)

Eqs.(5.35) and (5.50) represent a one-to-one correspondence between the functionsk(t) and f (t) and hence it does not matter if one first finds the optimal driving k(t) orthe optimal response f (t). During the isothermal branches the work (5.36) assumesthe generic form

Wout(tf , ti) = −tf∫

ti

dt1

f (t)

[f (t)⎤2 + T

2log

ff

fi

+⎡(

ν + 1

2

)T log

ff

fi− 2(ν + 1)[k(tf) ff − k(ti) fi]

⎣.

(5.51)

5.2 Diffusive Heat Engine 117

0 250 500−1

0

1

2

3

4

t

Wou

t(t)

,Δ

U(t

),Q

(t)

0 250 5000

0.5

1

1.5

t

Wou

t(t)

,Δ

U(t

),Q

(t)

0 250 500−1

−0.5

0

0.5

1

t

Ss(

t),

Sr(

t),S

tot(

t)

0 250 500−0.5

0

0.5

1

1.5

2

t

Ss(

t),

Sr(

t),S

tot(

t)

0 250 5000

0.4

0.8

1.2

t

σ w(t

,0)

0 250 5000

0.5

1

1.51.7

t

σ w(t

,0)

Fig. 5.12 Same quantities as in Fig. 5.10 for the engines depicted in Fig. 5.11. The engine drivenby the optimal protocol works during the whole cycle close to the quasi-static regime (cf. Fig. 5.11).On the contrary the cycle driven by the fractional driving is brought far from equilibrium at theinstants when the heat reservoirs are interchanged. This is reflected in large (small) output workand small (large) entropy production corresponding to the optimal (fractional) protocol. Note thata considerable amount of the total entropy produced during the cycle with the fraction driving iscreated just after the two heat reservoirs are interchanged. The optimal protocol is also favored bythe relative work fluctuation, which is smaller than that corresponding to the fractional driving

The work done during the first isothermal branch (branch I) is obtained after thesubstitution ti = 0, tf = t−+ , fi = f (0) = f0, ff = f (t+) = f1 and T = T+. Thesubstitution ti = t++ , tf = t−p , fi = f (t+) = f1, ff = f (tp) = f0 and T = T−yields the work done during the second isothermal branch (branch III). The integralin Eq. (5.51) equals the irreversible work

Wir(tf , ti) = −tf∫

ti

dt1

f (t)

[f (t)⎤2 = −T [Stot(tf) − Stot(ti)] ≤ 0, (5.52)

the second term is given by T Ss(tf , ti), and the third term corresponds to the increaseof the internal energy, −[U (tf)−U (ti)]. Using the definition (5.52), the total entropyproduced per cycle, the work done per cycle and the engine efficiency can be rewrittenas

118 5 Heat Engines

0.15 0.4 0.65 0.85−0.5

−0.1

0.3

0.7

0.95

Δ

η

0.15 0.4 0.65 0.85−0.3

−0.1

0.1

0.3

0.5

Δ

Pou

t

0.15 0.4 0.65 0.85

1

2

3

4

Δ

Sto

t

0.15 0.4 0.65 0.850.6

1

2

4

8

Δ

δPou

t

(a) (b)

(b) (c)

Fig. 5.13 Performance of the engine as a function of the asymmetry parameter � = t+/tp forthe optimal driving (5.57) (dashed lines), for the adiabatic driving (5.62) (solid lines) and for thefractional driving (5.63) (dashed-dotted lines). The fixed tp = 2.2344 is chosen in order to obtainthe optimal time distribution (5.59) for � = 0.5. a The efficiency (2.69) of the engine. The upper,middle and lower dotted horizontal lines correspond to the Carnot efficiency ηC, Curzon-Ahlbornefficiency ηCA and efficiency at maximum power ηP given by (5.60), respectively. The upper greenand lower red solid horizontal lines correspond to the upper and lower bounds for the efficiency atmaximum power (2.74). For � = 0.5 the efficiency for the optimal driving fulfills this limitation.Note that the upper bound for the efficiency at maximum power is larger than the Curzon-Ahlbornefficiency. b The output power (2.70). The maximal power for the optimal driving is achieved for� = 0.5. c The total entropy produced per cycle (5.43). d The relative power fluctuation (5.45)tends to +∈ if the corresponding output power vanishes. We restricted the vertical limits in orderto show the most interesting region of the figure in sufficient detail. The smallest power fluctuationis achieved by the adiabatic driving. For all the protocols we took T+ = 4, T− = 0.5, f0 = 0.5,ν = −0.499. Moreover, for the optimal and for the adiabatic driving we used f1 = 1.5, and for thefractional driving we took k1 = 1.8, which results in a different f1. According to the discussionin Sect.5.2.5.1, the optimal protocol yields, for a given f0, f1, t±, T± and ν, maximum efficiency,maximum output power and minimum amount of entropy. This is verified on the adiabatic drivingwhich corresponds to the same parameters as the used optimal protocol. On the other hand, for thefractional driving we have used different f1, which means that, for some values of the parameter �,the resulting efficiency and even the output power may exceed the values obtained for the optimalprotocol

Stot = Stot(tp) = −⎝

Wir(t+, 0)

T++ Wir(tp, t+)

T−

⎞, (5.53)

Wout = (T+ − T−)Ss(t+, 0) + [Wir(t+, 0) + Wir(tp, t+)], (5.54)

η =(

1 − T−T+

)Wout

Wout + T−Stot, (5.55)

5.2 Diffusive Heat Engine 119

−0.65 −0.55 −0.45 −0.35−0.8

−0.4

0

0.4

0.8(a) (b)

(c) (d)ν

η

−0,65 −0,55 −0,45 −0,35−0.3

−0.1

0.1

0.3

0.5

ν

Pou

t

−0.65 −0.55 −0.45 −0.35

1

1.5

2

2.5

3

ν

Sto

t

−0.65 −0.55 −0.45 −0.350.8

2

4

7

ν

δPou

t

Fig. 5.14 Performance of the engine as a function of the parameter ν (5.29). For ν > −0.5(ν < −0.5) the logarithmic part of the potential (5.25) is repulsive (attractive). The meaning ofthe individual curves is the same as in Fig. 5.13. The duration of the isothermal branches t+ =t− = 1.1172 is chosen in order to obtain the optimal time distribution (5.59) for ν = −0.499.The efficiency, the output power and the entropy produced per cycle are monotonic functions ofν. Note that the efficiency at maximum power (ν = −0.499) fulfills the restriction (2.74). For theparameters taken, the optimal protocol yields the largest efficiency, the largest output power andthe smallest entropy production from the three drivings. The power fluctuation both for the optimalprotocol and for the adiabatic driving exhibits well pronounced minimum, which is deeper for theadiabatic driving. Other parameters used for all protocols: T+ = 4, T− = 0.5, f0 = 0.5. For theoptimal and adiabatic drivings we, moreover, used f1 = 1.5. For the fractional driving we tookk1 = 1.8

respectively. The increase of the system entropy during the first isothermal branch,Ss(t+, 0), equals 1/2 log( f1/ f0). Assume that the parameters T±, t±, f0 and f1 aregiven. Moreover, let the function f (t) maximize the irreversible work (5.52) duringthe both isothermal branches. Then, for these given parameters, this function alsominimizes the total entropy production (5.53) and maximizes the output work (5.54)and the efficiency (5.55), cf. [25] and Figs.5.13 and 5.14.

Heaving in mind the above discussion, we find the function k(t), which maximizesthe work per cycle for a given f0, f1, t±, T± and ν, in the following two steps. First,we find the function f (t) which maximizes the work for a given f0 and f1, etc andsecond, we use Eq. (5.50) and calculate the corresponding function k(t).

The function f (t) maximizing the integral (5.52) solves the Euler-Lagrange equa-tion [ f (t)]2−2 f (t) [ f (t)] = 0 with the boundary conditions f (ti) = fi, f (tf) = ff .Its solution for the isothermal branches I and III reads [17, 29, 31]

120 5 Heat Engines

f (t) =

⎬⎭⎭⎭⎭

⎭⎭⎭⎭

f0 (1 + A1t)2 , A1 = 1

t+

(√f1

f0− 1

)

, t ∞ [0, t+]

f1[1 + A2(t − t+)

⎤2, A2 = 1

t−

(√f0

f1− 1

)

, t ∞ [t+, tp].

(5.56)For this optimal response, Eq. (5.50) yields the following optimal protocol:

k(t) =

⎬⎭⎭⎭

⎭⎭⎭

T+2 f0

1

(1 + A1t)2 − A1

1 + A1t, t ∞ [0, t−+ ]

T−2 f1

1

[1 + A2(t − t+)]2 − A2

1 + A2(t − t+), t ∞ [t++ , t−p ]

. (5.57)

This protocol already determines the jumps in k(t) and thus the adiabatic branches IIand IV. The assumptions we have imposed on the function k(t) during the derivationof the optimal protocol (5.57) are the following: (1) k(t) is continuous within theisothermal branches (otherwise the time derivative f (t) in Eq. (5.52) would notexist); (2) the optimal system response f (t) corresponding to k(t) is periodic andassumes the values f0 and f1 at the times 0 and t+, respectively.

Equation (5.57) represents the solution of the problem proposed in the beginningof this section, i.e., it describes the driving k(t) which, for the given parameters f0,f1, t±, T± and ν, yields the maximal output work and thus also the maximum power.

The power corresponding to the optimal protocol (5.57) reads

Pout = T+ − T−t+ + t−

�S − Air1

t+t−, (5.58)

where �S = S(t+, 0) is given by (5.41) and Air = 4(ν+1)⎦√

f1 − √f0⎫2 stands for

the irreversible action [17], which determines the heat uptake during the isothermalbranches, Q± = ± T±�S − Air/t±, and thus also the total entropy produced percycle, Stot = −Q+/T+ − Q−/T− = [1/(t+T+) + 1

/(t−T−)]Air . The authors

of [17] already noted that, in the long time limit t± ∝ ∈, the produced entropyconverges to zero and the cycle becomes reversible regardless the instantaneousinterchanges of the reservoirs. This feature of the driving, which in fact means thatthe system is in the long time limit during the whole cycle in thermal equilibrium,cf. Figs.5.11 and 5.12, stems from the fact that, during the derivation, one actuallyminimizes the irreversible entropy production Stot, cf. [25]. Indeed, from Eqs.(5.56)and (5.57) follow that limt±∝∈ f (t) = feq(t). As was already noted by Sekimoto[3], this would not be possible if the driving would not include the jumps, whichensure that the system response f (t) converges to its equilibrium value (5.49) alsoat the instants when the temperature suddenly changes. For the protocols wherelimt±∝∈ f (t) �= feq(t) there is, due to the sudden changes of the temperature andsubsequent inevitable relaxation processes, always produced a positive amount ofentropy Stot, cf. [2], Sect.5.1 and Figs.5.11 and 5.12.

5.2 Diffusive Heat Engine 121

Maximization of (5.58) with respect to the times t± leads to the optimal durationof the branches

t+ = t− = 4Air

(T+ − T−)�S. (5.59)

Without loose of generality we assume that the first reservoir is hot (T+ > T−). Insuch case the efficiency (2.69) corresponding to the optimal protocol (5.57) with theoptimal period durations (5.59) can be written as

ηP = 2ηC

4 − ηC, (5.60)

where ηC = 1 − T−/T+ denotes the Carnot efficiency (2.71). Note that ηp depends,contrary to the corresponding optimal power Pout = (T+−T−)2(�S)2/[16Air], onlyon the reservoir temperatures T+ and T−, not on the parameters f0, f1 or k0, k1 etc.Under the assumption of small temperature difference (ηC small) one can performTaylor expansion of the efficiency at maximum power (5.60), the result is

ηP ≥ ηCA ≥ ηC

2+ η2

C

8+ O

(η3

C

), (5.61)

where ηCA = 1 − √T−/T+ stands for the Curzon-Ahlborn efficiency (2.71). The

formulas (5.60) and (5.61) represent another example which verifies validity of gen-eral considerations of the works [17, 23, 24, 32] where the authors prove that theefficiency at maximum power (5.60) is bounded as ηC/2 < ηP < ηC/(2 − ηC) andthat the expansion (5.61) is universal for strong coupling models that possess a left-right symmetry, cf. Sect. 2.4. In Fig. 5.8 we show one example of the potential (5.25)and of the corresponding particle distribution (5.34) during the limit cycle driven bythe optimal protocol (5.57).

5.2.5.2 Minimum Relative Power Fluctuation

Minimization of the relative power fluctuation (5.45) is a task which is beyond thescope of the present Sect. (the functional, which yields the protocol minimizing thepower fluctuation, is to complicated). Using the physical intuition, one can guess thatthe driving, which minimizes the work fluctuation |Wtot| σw(tp, 0), should perform allthe work at the instants when the width of the particle distribution (5.38) is minimal.However, such driving would also perform zero amount of work Wout. Nevertheless,we decided to investigate a class of protocols where the work is performed onlyduring the adiabatic branches. To this end we introduce the adiabatic driving:

k(t) =⎡

r1 , t ∞ [0, t−+ ]r2 , t ∞ [t++ , t−p ] . (5.62)

122 5 Heat Engines

The two isotherms where no work is produced can also be referred to as isochoricbranches. It is worth noting that, due to the jumps in k(t), the resulting cycle can beperformed quasi-statically. However, the only equilibrium cycle consistent with thedriving (5.62) dwells during the whole period in the same state.

5.2.5.3 Fractional Driving

Except the protocols (5.57) and (5.62) we have examined also the fractional driving:

k(t) =

⎬⎭⎭

⎭⎭

k1

1 + γ1t, t ∞ [0, t−+ ]

k2

1 + γ2(t − t++ ), t ∞ [t++ , t−p ]

. (5.63)

Our results for this protocol coincide with those obtained in [21]. Contrary to theprotocols (5.57) and (5.62), this driving may consists of two isotherms only (cf. withthe driving used in Sect. 5.1). In the figures we always take continuous k(t), thenthe adiabatic branches vanish for ν = −1/2. If the jumps in the driving are allowed,the engine corresponding to the fractional driving can be tuned so it differs from theone driven by the optimal protocol (5.57) nearly negligibly. In the next section wediscuss the results obtained for these three protocols.

5.2.6 Discussion

Assume that we want to realize the described engine experimentally using the opticaltweezers. Then the control parameters g(t) and k(t) can be driven by the laser inten-sity. The response function f (t) is proportional to the particle variance. Thereforeits maximum value in a sense determines the engine size. In this section we com-pare performance of the three protocols described above. In the most illustrations weassume that the protocols are specified by the parameters f0, f1, t±, T± and ν. In allthe figures the constant f0 ( f1) represents the smallest (largest) value of the functionf (t) during the cycle. This means that we compare the engines characterized by thesame minimum and maximum particle variance (system size).

For a given f0, f1, t±, T± and ν the parameters of the individual drivings arecalculated using the closure conditions on the driving and system response whichread k(tp) = k(0), f (t−+ ) = f (t++ ), f (tp) = f (0) and, for the fractional driving,k(t−+ ) = k(t++ ), k(t−p ) = k(tp). For arbitrary reasonable parameters (positive f0, f1,t±, T±) these formulas yield solution only for the optimal protocol. Therefore, inFigs. 5.13 and 5.14, we were pushed to take different f1 for the fractional drivingthan that used for the other two protocols.

5.2 Diffusive Heat Engine 123

Two representative working cycles for the optimal protocol (5.57) are depicted inFigs. 5.9 and 5.11. The time resolved thermodynamic quantities (5.36), (5.39), (5.40),(5.41–5.43) and (5.44), for these two engines, are depicted in Figs. 5.10 and 5.12. InFigs. 5.9, 5.10, 5.11 and 5.12 we show an example of the working cycle together withthe corresponding time resolved thermodynamic quantities for the fractional driving(5.63) and for the adiabatic driving (5.62), respectively. Contrary to the equilibriumsituation, where two isotherms corresponding to different temperatures can nevercross, the simplest non-equilibrium cycle can be formed just by two isothermalbranches. For the drivings used the g-cycle forms always a rectangle, while thek-cycle can be formed by two loops of a general shape. An example of such two-loop cycle is depicted in Fig.5.9.

Figs. 5.11 and 5.12 demonstrate the case of very slow driving. The cycle drivenby the optimal protocol is close to the quasi-static realization and its efficiencynearly attains the Carnot’s upper bound (2.71). In the limit of infinitely slow driving(t± ∝ ∈), the adiabatic changes of the optimal driving at the instants when thereservoirs are interchanged guarantees that the equilibrium states before and afterthe adiabatic branches coincide, cf. [3]. On the other hand, during the cycle driven bythe fractional driving, where the adiabatic branches are not considered, the system isbrought far from equilibrium whenever the temperature changes and the quasi-staticlimit does not exist. In Sect. 5.2.5.1 we have shown that, for a given parameters f0,f1, t±, T± and ν, the optimal protocol (5.57) yields the maximal efficiency and outputpower and that it minimizes the entropy produced per cycle. This result is verifiedin Figs.5.10 and 5.12. More detailed discussion of Figs.5.9, 5.10, 5.11 and 5.12 iscontained in the captions below the individual figures.

The efficiency (2.69), the output power (2.70), the power fluctuation (5.45) andthe total entropy production (5.43) of engines driven by the protocols (5.57), (5.62)and (5.63) are depicted in Figs.5.13 and 5.14. In Fig.5.13 we study the dependenceof these variables on the allocation of a given period tp between the two isotherms.In Fig. 5.14 the engine performance as a function of the parameter ν is studied. Itturns out that all the depicted quantities except the relative power fluctuation, whichexhibits a well pronounced minimum, are monotonic functions of ν. The curvescorresponding to the optimal protocol and to the adiabatic driving in Figs.5.13 and5.14 verify that, for a given parameters f0, f1, t±, T± and ν, the optimal protocol(5.57) yields the maximal efficiency, output power and that it minimizes the entropyproduced per cycle. On the other hand, for the fractional driving we use different f1and hence the corresponding efficiency and even the power output exceeds, for someparameters, the results obtained for the optimal protocol. For the parameters takenin Figs. 5.13 and 5.14 the maximum power (5.58) is obtained for � = 0.5 and forν = −0.499, respectively. In the both Figs.the efficiency at maximum power (5.60)lies between the general bounds (2.74), which are depicted by the solid red and greenhorizontal lines. For more detailed discussion see the captions below the individualfigures.

The smallest relative power fluctuation observed for the three protocols is achievedby the adiabatic driving. From macroscopic world we know that the power fluctu-ations towards large values can be handled by enlarging the capacity of the system

124 5 Heat Engines

where the power is delivered. On the other hand the fluctuations towards small powervalues can be balanced only by adapting a standby power supply. An example is thediscussion about wind power plants, which cause current fluctuations in transmission-grids [33]. The natural Brownian motors [34–37] are exposed to large fluctuationsof the environment and hence they are quite adapted, indeed. Nevertheless it is aninteresting question whether the principle of minimal power fluctuations, the prin-ciple of maximum output power, or some other principle eventually applies in thisfield. In any case, these principles should be considered during the design of artificialBrownian motors [36].

To conclude we have studied a simple, exactly solvable, model of a stochasticheat engine with the continuous state space. The working medium of the engineis a particle diffusing in a time-periodic log-harmonic potential described by twocontrol parameters. We have examined the efficiency at maximum power for thisengine. Specifically, we have verified the general results obtained by Schmiedl andSeifert [17] and by Esposito et al. [23, 24]. Further we have focused on output powerfluctuations and discussed the possibility to minimize them. Concretely, we havepresented a protocol which yields smaller power fluctuations than the one whichgives the maximum power. Finally, we have introduced the driving which, contraryto the two mentioned protocols, does not incorporate the adiabatic branches and thusthe corresponding limit cycle can never approach the quasi-static limit. For the threedrivings we have discussed the possibility to depict the corresponding limit cyclesin a PV-like diagram.

References

1. Chvosta, P., Holubec, V., Ryabov, A., et. al. (2010). Thermodynamics of two-strokeengine based on periodically driven two-level system. Physica E: Low-dimensional Sys-tems and Nanostructures, 42(3), 472–476. ISSN 1386–9477, http://dx.doi.org/10.1016/j.physe.2009.06.031, proceedings of the international conference Frontiers of Quantum andMesoscopic Thermodynamics FQMT ’08. http://www.sciencedirect.com/science/article/pii/S1386947709002380.

2. Chvosta, P., Einax, M., Holubec, V., et. al. (2010). Energetics and performance of a microscopicheat engine based on exact calculations of work and heat distributions. Journal of StatisticalMechanics: Theory and Experiment, 2010(nr. 03), P03002. http://stacks.iop.org/1742-5468/2010/i=03/a=P03002.

3. Sekimoto, K., Takagi, F., & Hondou, T. (2000). Carnot’s cycle for small systems: Irreversibilityand cost of operations. Physical Review E, 62, 7759–7768. doi:10.1103/PhysRevE.62.7759,http://link.aps.org/doi/10.1103/PhysRevE.62.7759.

4. Šubrt, E., & Chvosta, P. (2007). Exact analysis of work fluctuations in two-level systems.Journal of Statistical Mechanics: Theory and Experiment, 2007(nr. 09), P09019. http://stacks.iop.org/1742-5468/2007/i=09/a=P09019.

5. Einax, M., Korner, M., Maass, P., et. al. (2010). Nonlinear hopping transport in ring systemsand open channels. Physical Chemistry Chemical Physics: PCCP, 12, 645–654, doi:10.1039/B916827C, http://dx.doi.org/10.1039/B916827C.

6. Slater, L. (1960). Confluent hypergeometric functions. New York: Cambridge University Press.http://books.google.cz/books?id=BKUNAQAAIAAJ.

References 125

7. Abramowitz, M., & Stegun, I. (1964). Handbook of mathematical functions: With formulars,graphs, and mathematical tables. Applied mathematics series. New York: Dover Publications,Incorporated, ISBN 9780486612720, http://books.google.se/books?id=MtU8uP7XMvoC.

8. Einax, M., & Maass, P. (2009). Work distributions for Ising chains in a time-dependent magneticfield. Physical Review E, 80, 020101, doi:10.1103/PhysRevE.80.020101, http://link.aps.org/doi/10.1103/PhysRevE.80.020101.

9. Speck, T., & Seifert, U. (2004). Distribution of work in isothermal nonequilibrium processes.Physical Review E, 70, 066112, doi:10.1103/PhysRevE.70.066112, http://link.aps.org/doi/10.1103/PhysRevE.70.066112.

10. Engel, A. (2009). Asymptotics of work distributions in nonequilibrium systems. PhysicalReview E, 80, 021120, doi:10.1103/PhysRevE.80.021120, http://link.aps.org/doi/10.1103/PhysRevE.80.021120.

11. Jarzynski, C. (1997). Nonequilibrium Equality for Free Energy Differences. Physical ReviewLetter, 78, 2690–2693, doi:10.1103/PhysRevLett.78.2690, http://link.aps.org/doi/10.1103/PhysRevLett.78.2690.

12. Crooks, G. E. (1999). Entropy production fluctuation theorem and the nonequilibrium workrelation for free energy differences. Physical Review E, 60, 2721–2726, doi:10.1103/PhysRevE.60.2721, http://link.aps.org/doi/10.1103/PhysRevE.60.2721.

13. Holubec, V., Chvosta, P., Einax, M., et. al. (2011). Attempt time Monte Carlo: An alternative forsimulation of stochastic jump processes with time-dependent transition rates. EPL (EurophysicsLetters), 93(nr. 4), 40003. http://stacks.iop.org/0295-5075/93/i=4/a=40003.

14. Zia, R. K. P., Praestgaard, E. L., & Mouritsen, O. G. (2002) Getting more from pushing less:Negative specific heat and conductivity in nonequilibrium steady states. American Journal ofPhysics, 70(4), 384–392. doi:10.1119/1.1427088, http://link.aip.org/link/?AJP/70/384/1.

15. Manosas, M., Mossa, A., Forns, N., et. al. (2009) Dynamic force spectroscopy of DNA hairpins:II. Irreversibility and dissipation. Journal of Statistical Mechanics: Theory and Experiment,2009(nr. 02), P02061. http://stacks.iop.org/1742-5468/2009/i=02/a=P02061.

16. Snyder, D. (1975). Random point processes. New York: Wiley. ISBN 9780471810216. http://books.google.cz/books?id=veRQAAAAMAAJ.

17. Schmiedl, T., & Seifert, U. (2008). Efficiency at maximum power: An analytically solvablemodel for stochastic heat engines. EPL (Europhysics Letters), 81(nr. 2) 20003. http://stacks.iop.org/0295-5075/81/i=2/a=20003.

18. Blickle, V., & Bechinger, C. (2011). Realization of a micrometre-sized stochastic heat engine.Nature Physics, 8(2), 143–146. doi:10.1038/nphys2163, http://dx.doi.org/10.1038/nphys2163.

19. Giampaoli, J. A., Strier, D. E., Batista, C., et. al. (1999). Exact expression for the diffusionpropagator in a family of time-dependent anharmonic potentials. Physical Review E, 60, 2540–2546, doi:10.1103/PhysRevE.60.2540, http://link.aps.org/doi/10.1103/PhysRevE.60.2540.

20. Strier, D. E., Drazer, G., & Wio, H. S. (2000). An analytical study of stochastic resonance in amonostable non-harmonic system. Physica A: Statistical Mechanics and its Applications, 283,255–260, ISSN 0378–4371, doi:10.1016/S0378-4371(00)00163-1, http://www.sciencedirect.com/science/article/pii/S0378437100001631.

21. Ryabov, A., Dierl, M., Chvosta, P., et. al. (2013). Work distribution in a time-dependentlogarithmic-harmonic potential: exact results and asymptotic analysis. Journal of Physics A:Mathematical and Theoretical, 46(nr. 7), 075002. http://stacks.iop.org/1751-8121/46/i=7/a=075002.

22. Cohen, A. E. (2005). Control of Nanoparticles with Arbitrary Two-Dimensional Force Fields.Physical Review Letter, 94, 118102, doi:10.1103/PhysRevLett.94.118102, http://link.aps.org/doi/10.1103/PhysRevLett.94.118102.

23. Esposito, M., Lindenberg, K., & Van den Broeck, C. (2009). Universality of efficiency atmaximum power. Physical Review Letter, 102, 130602, doi:10.1103/PhysRevLett.102.130602,http://link.aps.org/doi/10.1103/PhysRevLett.102.130602.

24. Esposito, M., Kawai, R., Lindenberg, K., et. al. (2010). Efficiency at maximum power of low-dissipation carnot engines. Physical Review Letter, 105, 150603, doi:10.1103/PhysRevLett.105.150603, http://link.aps.org/doi/10.1103/PhysRevLett.105.150603.

126 5 Heat Engines

25. Wang, Y., & Tu, Z. C. (2012) Efficiency at maximum power output of linear irreversible Carnot-like heat engines. Physical Review E, 85, 011127, doi:10.1103/PhysRevE.85.011127, http://link.aps.org/doi/10.1103/PhysRevE.85.011127.

26. Prudnikov, A., Brychkov, I., & Maricev, O. (1986). Integrals and series: Special functions.Volume 2. Integrals and series. Boca Raton: C R C Press LLC, ISBN 9782881240904. http://books.google.cz/books?id=2t2cNs00aTgC.

27. Speck, T. (2011). Work distribution for the driven harmonic oscillator with time-dependentstrength: exact solution and slow driving. Journal of Physics A: Mathematical and Theoretical,44(nr. 30) 305001. http://stacks.iop.org/1751-8121/44/i=30/a=305001.

28. Nickelsen, D., & Engel, A. (2011). Asymptotics of work distributions: the pre-exponentialfactor. The European Physical Journal B, 82, 207–218, ISSN 1434–6028, doi:10.1140/epjb/e2011-20133-y, http://dx.doi.org/10.1140/epjb/e2011-20133-y.

29. Then, H., & Engel, A. (2008). Computing the optimal protocol for finite-time processes in sto-chastic thermodynamics. Physical Review E, 77, 041105, doi:10.1103/PhysRevE.77.041105,http://link.aps.org/doi/10.1103/PhysRevE.77.041105.

30. Zhan-Chun, T. (2012). Recent advance on the efficiency at maximum power of heat engines.Chinese Physics B, 21(nr. 2), 020513. http://stacks.iop.org/1674-1056/21/i=2/a=020513.

31. Schmiedl, T., & Seifert, U. (2007). Optimal finite-time processes in stochastic thermodynamics.Physical Review Letter, 98, 108301, doi:10.1103/PhysRevLett.98.108301, http://link.aps.org/doi/10.1103/PhysRevLett.98.108301.

32. Izumida, Y., & Okuda, K. (2012). Efficiency at maximum power of minimally nonlinear irre-versible heat engines. EPL (Europhysics Letters), 97(nr. 1), 10004. http://stacks.iop.org/0295-5075/97/i=1/a=10004.

33. Eriksen, P., Ackermann, T., Abildgaard, H., et. al. (2005). System operation with high windpenetration. IEEE Power and Energy Magazine, 3(nr. 6), 65–74, ISSN 1540–7977, doi:10.1109/MPAE.2005.1524622.

34. Jung, P., & Hänggi, P. (1990). Resonantly driven Brownian motion: Basic concepts and exactresults. Physical Review A, 41, 2977–2988, doi:10.1103/PhysRevA.41.2977, http://link.aps.org/doi/10.1103/PhysRevA.41.2977.

35. Hänggi, P., Marchesoni, F., & Nori, F. (2005). Brownian motors. Annalen der Physik, 14(1–3), 51–70. ISSN 1521–3889, doi:10.1002/andp.200410121, http://dx.doi.org/10.1002/andp.200410121.

36. Hänggi, P., & Marchesoni, F. (2009). Artificial Brownian motors: Controlling transport on thenanoscale. Reviews of Modern Physics, 81, 387–442, doi:10.1103/RevModPhys.81.387, http://link.aps.org/doi/10.1103/RevModPhys.81.387.

37. Astumian, R. D., & Hanggi, P. (2002). Brownian Motors. Physics Today, 55(11), 33–39. doi:10.1063/1.1535005, http://link.aip.org/link/?PTO/55/33/1.

Chapter 6Conclusion

In this thesis we have presented several exactly solvable models from the field of sto-chastic thermodynamics, which we have reviewed in Chaps. 1 and 2. First, in Chaps. 3and 4, we have focused on driven mesoscopic systems with both discrete (Chap. 3)and continuous (Chap. 4) state space which, moreover, interact with a thermal reser-voir at a constant temperature. We have analytically investigated the dynamics andenergetics during the emerging isothermal processes. In particular, we have calcu-lated the probability density for work, heat and internal energy. Although, from thetheoretical point of view, these results are themselves interesting, they can also helpto better utilize the experimental data. For instance in many experiments the Jarzyn-ski equality (2.61) is used in order to extract the equilibrium free energy differencesfrom non-equilibrium stretching experiments [1]. The extracted free energy differ-ence strongly depends on the tail of the work distribution for large negative workvalues, which correspond to highly improbable realizations of the experiment andhence can not be measured accurately enough. Therefore it is important to use thecorrect fit (the Jarzynski estimator) of the measured work distributions in order toobtain the tail accurately enough [2]. Although complicated from the mathematicalpoint of view, the models where the work fluctuations can be treated analyticallyare often too simple from the physical point of view. More realistic models can bestudied using computer simulations, which can also help to find suitable Jarzynskiestimators. In Appendix C we present a new algorithm which can be used for thesesimulations.

In the second part of the thesis (Chap. 5) we have focused on periodically drivenmesoscopic systems which communicate with two heat reservoirs at different tem-peratures. During the cyclic process the system can perform a positive mean workon the environment. Such stochastic heat engines have been studied during the lastdecade. We have presented two exactly solvable models. In the first one we havederived the probability density for the work performed per operational cycle, and wehave discussed its properties. In the second example we have found the exact form ofthe periodic driving which yields the maximum output power of the engine and wehave verified the recent general results concerning the corresponding efficiency [3].

V. Holubec, Non-equilibrium Energy Transformation Processes, Springer Theses 127DOI: 10.1007/978-3-319-07091-9_6, © Springer International Publishing Switzerland 2014

128 6 Conclusion

In the both models we have also discussed the possibility to minimize the powerfluctuations. Majority of the available studies concerning stochastic heat enginesfocus on calculating the mean values of the thermodynamic quantities. Our analysisof the corresponding fluctuations provides new insights into the performance of theengines.

The results presented in the thesis can be extended in several directions. For in-stance one can focus on stochastic thermodynamics of interacting systems, which didnot received much attention in the literature, yet. Further, it would be interesting toinvestigate the possibility to optimize the performance of the stochastic heat enginesunder different requirements than that of maximum power or minimal power fluc-tuations. One can draw inspiration from many studies concerning macroscopic heatengines, where several ecological, economical and size criteria are considered [4].“Natural” engines working in living cells usually do not use temperature gradients.Therefore, it would be interesting to extend the presented analysis on the processesdriven by chemical gradients, which, moreover, involve chemical reactions. Severalstudies in this direction were performed in the field of Brownian motors [5–8]. How-ever, in these papers, the work fluctuations are studied only seldom, which openspossibilities for further research.

References

1. Ritort, F. (2008). Nonequilibrium fluctuations in small systems: From physics to biology. InA. Rice Stuart (Ed.), Advances in Chemical Physics (Vol. 137, pp. 31–123). Hoboken: JohnWiley & Sons Inc. ISBN 9780470238080, doi:10.1002/9780470238080.ch2, http://dx.doi.org/10.1002/9780470238080.ch2

2. Palassini, M., & Ritort, F. (2011). Improving free-energy estimates from unidirectional workmeasurements: theory and experiment. Physical Review Letters, 107, 060601,doi:10.1103/PhysRevLett.107.060601, http://link.aps.org/doi/10.1103/PhysRevLett.107.060601

3. Zhan-Chun, T. (2012). Recent advance on the efficiency at maximum power of heat engines.Chinese Physics B, 21(2), 020513. http://stacks.iop.org/1674-1056/21/i=2/a=020513

4. Durmayaz, A., Sogut, O. S., & Sahin, B. et. al. (2004). Optimization of thermal systems based onfinite-time thermodynamics and thermoeconomics. Progress in Energy and Combustion Science,30(2), 175–217. ISSN 0360–1285, doi:10.1016/j.pecs.2003.10.003, http://www.sciencedirect.com/science/article/pii/S0360128503000777

5. Jung, P., & Hänggi, P. (1990). Resonantly driven Brownian motion: Basic concepts and exactresults. Physical Review A, 41, 2977–2988, doi:10.1103/PhysRevA.41.2977, http://link.aps.org/doi/10.1103/PhysRevA.41.2977.

6. Hänggi, P., Marchesoni, F., & Nori, F. (2005). Brownian motors. Annalen der Physik, 14(1–3), 51–70. ISSN 1521–3889, doi:10.1002/andp.200410121, http://dx.doi.org/10.1002/andp.200410121

7. Hänggi, P. & Marchesoni, F. (2009). Artificial Brownian motors: Controlling transport on thenanoscale. Review of Modern Physics, 81, 387–442, doi:10.1103/RevModPhys.81.387, http://link.aps.org/doi/10.1103/RevModPhys.81.387

8. Astumian, R. D. & Hanggi, P. (2002). Brownian Motors. Physics Today, 55(11), doi:10.1063/1.1535005, http://link.aip.org/link/?PTO/55/33/1

Appendix AWork Distributions for Slow and for FastProcesses

In this appendix we investigate the probability density for the work done during aninfinitely fast (slow) isothermal process. Without loose of generality we consideronly the situation when the initial condition is set at the time t′ = 0. In such case wecan simplify the notation. We write R(t | 0) → R(t) and G(w, t | 0, 0) → G(w, t) forthe solution of the master Eq. (2.3) and that of Eq. (2.13), respectively. Further weassume that the detail balance condition (2.4) is fulfilled.

Laplace transform of the probability distribution for the work given initial andfinal state of the system [elements of the matrix G(w, t)], if exists, obeys the set ofordinary differential equations

d

dtG(s, t) = − [

s E(t) + νL(t)]G(s, t), G(s, 0) = 1, (A.1)

where s is the Laplace variable corresponding to the work variable w. Note thatG(0, t) = R(t). Its solution for an infinitely slow (fast) process can be obtained bytaking the limit ν ∞ ∈ (ν ∞ 0) in Eq. (A.1), i.e., by assuming infinitely fast (slow)intrinsic relaxation processes.

A.1 Infinitely Fast Process

If ν = 0, the solution of the master Eq. (2.3) is the constant matrix R(t) = 1/N 1

and Eq. (A.1) reduces to

d

dtG(s, t) = −s E(t) G(s, t), G(s, 0) = 1. (A.2)

Its solution reads

G(s, t) = 1

Nexp {−s [E(t) − E(0)]} 1, (A.3)

V. Holubec, Non-equilibrium Energy Transformation Processes, Springer Theses, 129DOI: 10.1007/978-3-319-07091-9, © Springer International Publishing Switzerland 2014

130 Appendix A: Work Distributions for Slow and for Fast Processes

which gives after the inverse Laplace transform [the matrix E(t) is diagonal]

[G(w, t)]ij = 1

Nδ {w − [E i(t) − Ei(0)]} δij. (A.4)

As expected, during an infinitely fast process, the system dwells in its initial stateand the work equals the change of its internal energy (there is no time to accept anyheat from the reservoir). In this thesis we call these processes as adiabatic.

A.2 Infinitely Slow Process

If ν = ∈, the master Eq. (2.3) translates into the eigenvalue problem

L(t)R(t) = 0. (A.5)

Assuming the detailed balance condition (2.4), the solution of Eq. (A.5) is thematrix containing the equilibrium occupation probability vector corresponding tothe instantaneous value of the potential in all its columns: [R(t)]ij = πj(t) =exp

[−βFj(t)]/Z(t), where Z(t) = ∑N

i=1 exp[−βFi(t)] denotes the partitionfunction. Differently speaking, any initial state p(0) immediately relaxes into theequilibrium state π(0) = R(0) p(0). Note that the individual matrix elements arediscontinuous. Initially R(t) equals the unity matrix and at any larger time (t > 0) itconsists of the equilibrium vectors π(t).

Let us integrate the work variable out of the work propagatorG(w, t), what remainsis the transition matrix R(t):

∈∫

−∈dwG(w, t) = R(t). (A.6)

Now we multiply this Equation from the left by the rate matrix L(t) and we employEq. (A.5). The result is

∈∫

−∈dwL(t)G(w, t) = 0. (A.7)

This formula is valid for any external driving Ei(t), even for those for which thematrix L(t)G(w, t) does not vanish after the integration. This means that the matrixG(w, t) factorizes as G(w, t) = R(t)F(w, t), where the unknown matrix F(w, t)obeys the formula

∫∈−∈ dwF(w, t) = 1. Using the same argumentation as above, in

order to fulfill the last formula, the matrix F(w, t) must be diagonal. Moreover, ifwe consider that any initial state p(0) immediately relaxes into the equilibrium stateπ(0), it is reasonable to assume that for any initial state of the system one obtains

Appendix A: Work Distributions for Slow and for Fast Processes 131

the same work distribution. Altogether we conclude that the work propagator shouldhave the form G(w, t) = f (w, t)R(t), where f (w, t) is an unknown function. Afterinserting this ansatz into Eq. (A.1) we obtain

f (s, t)d

dtR(t) +

[d

dtf (s, t)

]R(t) = −sf (s, t)

[d

dtE(t)

]R(t), (A.8)

where f (s, t) stands for the Laplace transformed function f (w, t). The work prob-ability density is obtained from the work propagator as ρ(w, t) = 1T G(w, t) p(0),where 1T stands for the line vector of ones, cf. Eqs. (2.15) and (2.16). Therefore, inorder to obtain the function f (s, t) we multiply Eq. (A.8) by 1T from the left and byp(0) from the right. We get the formula

d

dtf (s, t) = −s

N∑

i=1

Ei(t)πi(t)f (s, t), (A.9)

where we have used the following relations

1T R(t) p(0) = 1, (A.10)

1T R(t) p(0) = d

dt

[1T R(t) p(0)

]= 0, (A.11)

1T E(t)R(t) p(0) = 1T E(t)π =N∑

i=1

Ei(t)πi(t). (A.12)

Solution of Eq. (A.9) is similar to that of Eq. (A.1). We will continue with its inverseLaplace transform

f (w, t) = ρ(w, t) = δ

⎧⎨

⎩w −

t∫

0

dt′N∑

i=1

Ei(t′)πi(t

′)

⎫⎬

⎭. (A.13)

The integral inside the δ-function yields the mean work done on the system (1.23).Since the process is quasi-static, the mean work equals the increase of the Helmholtzfree energy F(t) = − log[Z(t)]/β of the system. We get the expected result

ρ(w, t) = δ {w − [F(t) − F(0)]} . (A.14)

This results was already obtained by Speck and Seifert [1] using Langevin andFokker-Planck formalism. In their setting these authors have also shown that forslow (but finite) driving WPD has always Gaussian form. We were not able to repeatsuch calculation within the master equation dynamics. For example if one attemptsto expand the matrices R(t) and G(w, t) in the small parameter 1/ν, he is pushedto calculate the matrix [L(t)]−1. However, the rate matrix L(t) is singular and thus

132 Appendix A: Work Distributions for Slow and for Fast Processes

it can not be simply inverted.1 Nevertheless, such result could be expected, becausethe WPD in discrete models poses (for finite times) a finite support and hence it canbe approximated by a Gaussian near its maximum at best [3].

References

1. Speck, T., & Seifert, U. (2004). Distribution of work in isothermal nonequilibriumprocesses. Physical Review E, 70, 066112. doi:10.1103/PhysRevE.70.066112.http://link.aps.org/doi/10.1103/PhysRevE.70.066112.

2. Pešek, J. (2013). Heat processes in non-equilibrium stochastic systems. Ph.D.thesis, Czech Republic: Charles University in Prague, Faculty of Mathematicsand Physics.

3. Nickelsen, D., & Engel, A. (2011). Asymptotics of work distributions: the pre-exponential factor. The European Physical Journal B, 82, 207–218. ISSN 1434–6028, doi:10.1140/epjb/e2011-20133-y.

1 One possibility how a generalized matrix inversion of L(t) can be calculated is discussed inPešek’s PhD thesis [2].

Appendix BValidity of Jarzynski Equalityfor a Unidirectional Process

In this appendix we investigate whether the analytical results obtained for thework probability density in the Kittel zipper model (Sect. 3.4) can be used as anapproximation of work distributions measured during single molecule experiments[1–4]. Specifically, we examine the validity of the experimentally important Jarzynskiequality (2.61) for this approximate model. To this end we consider a more realisticmodel of a DNA hairpin unfolded by a linearly increasing force f (t) = f0 + rt,where r is the loading rate (cf. Fig. 3.3). With the help of Mfold folding predictions[5, 6], it is possible to design a periodic DNA sequence [7] which unfolds like aN-level system with the (forward) free energy barriers between the individual levels(cf. Fig. 3.1) and the level free energies given by

�(t) = E − Ff (t) = E − Ff0 − Frt, (B.1)

Fi(t) = (i − 1)[A − Bf (t)] = (i − 1)(A − Bf0 − Brt), (B.2)

respectively. The unfolding of such periodic DNA molecule can be described byEq. (2.3) with the rate matrix

L(t) = 1

ν

⎛

⎜⎜⎜⎜⎜⎜⎜⎝

λU(t) −λR(t)

−λU(t) λU(t) + λR(t). . .

. . .. . .

. . .

. . . λU(t) + λR(t) −λR(t)−λU(t) λR(t)

⎞

⎟⎟⎟⎟⎟⎟⎟⎠

, (B.3)

where the empty elements are 0. The unfolding and refolding rates in (B.3) are

λU(t) = ν exp[−β(E − Ff0)] exp(βFrt), (B.4)

λR(t) = exp[β(A − Bf0)] exp(−βFrt)λU(t). (B.5)

V. Holubec, Non-equilibrium Energy Transformation Processes, Springer Theses, 133DOI: 10.1007/978-3-319-07091-9, © Springer International Publishing Switzerland 2014

134 Appendix B: Validity of Jarzynski Equality for a Unidirectional Process

The work fluctuations during the unfolding can be studied using the solution ofEq. (2.13) with the matrix L(t) given by Eq. (B.3) and with

E(t) = diag{F1(t), . . . , FN (t)} = −Br diag{0, . . . , N − 1}. (B.6)

From Sect. 3.1.2 we know that, even for N = 2, this model is exactly solvable onlyfor the case F = B. The approximate solution for the case λU(t) � λR(t) ∝ 0 isdiscussed in Sect. 3.4.

WPD for any N-level system assumes the generic form

ρ(w, t) = ρC(w, t) +N∑

i=1

Ri(t) pi(0) δ{w − [Fi(t) − Fi(0)]}, (B.7)

where ρC(w, t) stands for the non-singular part of the probability density and thesingular part, proportional to the δ-functions, is determined by the probabilities thatthe system dwells in a given microstate during the whole time interval (0, t), Ri(t),and by the initial state of the system, pi(0). Contrary to the function ρC(w, t), thesingular part of ρ(w, t) can be always obtained analytically. Specifically, let us definethe two auxiliary functions

�U(t) =t∫

0

dt′ λU(t′) = αU

βFr

[exp(βFrt) − 1

], (B.8)

�R(t) =t∫

0

dt′ λR(t′) = αR

β(F − B)r{exp[β(F − B)rt] − 1} , (B.9)

where αU = ν exp[−β(E − Ff0)] and αR = ν exp[−β(E − Ff0)] exp[β(A − Bf0)].Then the weights Ri(t) for the present model read

R1(t) = exp[−�U(t)], (B.10)

Ri(t) = exp[−�U(t) − �R(t))], i = 2, . . . , N − 1, (B.11)

RN (t) = exp[−�R(t)]. (B.12)

Realistic values of the model parameters obtained in [7] are ν = 5 × 106 Hz, A =66.75 kBT , B = 4.64 kBT/pN, E = 34.61 kBT and F = 1.55 kBT/pN. Moreover,assume that the constant pulling rate is r = 5 pN/s, the force applied on the moleculeis initially f (0) = f0 = 10 pN and that we stop the pulling at the time tU = 1.0876 swhen the molecule is already unfolded with the probability larger than 99 %. In orderto fulfill the Jarzynski equality

Appendix B: Validity of Jarzynski Equality for a Unidirectional Process 135

∈∫

−∈dw exp (−βw) ρ(w, tU) = exp[−β�F(tU, 0)], (B.13)

the molecule must be initially in thermal equilibrium. For the parameters above, wehave for the partition function Z(0) = ∑N

i=1 exp[−βFi(0)] ∝ exp[−βF1(0)] andthus pi(0) ∝ exp[−βFi(0)]/ exp[−βF1(0)]. Similarly, Z(tU) ∝ exp[−βFN (tU)]and thus we can write for the equilibrium free energy difference �F(tU, 0) =log[Z(tF)/Z(0)] ∝ FN (tF) − F1(0). For the present model Eq. (B.13) is preciselyfulfilled. Let us now examine Eq. (B.13) for the unidirectional model, i.e., we takeλR(t) = 0. In that case �R(t) = 0 and the weights Ri(t) equal to the diagonal ele-ments of the matrix R(t), i.e., we have Ri(t) = Rii(t) = exp[−�U(t)], ≥ i ≤= Nand RN (t) = RNN (t) = 1. For the parameters above we have �U(tU) ∝ 15,�R(tU) ∝ 106 and hence these weights of the δ-functions differ significantly from thevalues valid for the exact model. Using Eq. (B.7) the integral in Eq. (B.13) assumesthe form

Jint =∈∫

−∈dw exp (−βw) ρ(w, tU) = IC + IS, (B.14)

where the two non-negative terms are IC = ∫∈−∈ dw exp (−βw) ρC(w, tU) and

IS = ∑Ni=1 Ri(tU)pi(0) exp{−β[Fi(tU) − Fi(0)]}. Using the parameters above and

considering the unidirectional model, the second term can be rewritten as

IS ∝ RN (tU)pN (0) exp{−β[FN (tU) − FN (0)]} (B.15)

= exp {−β [FN (tF) − F1(0)]} ∝ exp [−β�F(tU, 0)] .

The right-hand side of the Jarzynski identity (B.13) is thus, for the unidirectionalmodel, given solely by the singular part of the work distribution. This means that,although for high enough pulling rates r the work distribution for the unidirectionalprocess and that for the exact model are very similar on a first glance, their tailscorresponding to the large negative work values differ significantly. These resultsare depicted in Fig. B.1.

For small pulling rates (r = 5 pN/s) both the probability that the molecule is at thetime t fully unfolded, pN (t), and the WPD for the two models significantly differ, cf.Fig. B.1a1 and a2. On the other hand for large pulling rates (r = 500 pN/s) these twofunctions are, on a first glance, identical, cf. Fig. B.1b1 and b2. Note that neither inthe case r = 5 pN/s nor in the case r = 500 pN/s the refolding rates can be neglectedduring the whole pulling, see Fig. B.1a1 and b1. Indeed, if the backward rates could beneglected also in the very beginning of the pulling experiment, the molecule wouldbe with large probability already unfolded, cf. Fig. 3.4 and the discussion of thebackward rates in Sect. 3.4.2. Figure B.1a demonstrates that the correct value of theJarzynski integral (B.13) is for the unidirectional model given solely by the singular

136 Appendix B: Validity of Jarzynski Equality for a Unidirectional Process

−150 −100 −50 00

0.02

0.04

−100 −50 00

0.05

0.1

0 0.005 0.01 0.0150

0.5

1

0 0.5 10

0.5

1

0 0.5 110

15

1020

0 0.01

1020

0 100 200 300 400 50010

−10

100

1010

1020

1030

1040

1050

0 500

−100

−50

0

(c)

(b2)

(a2)(a1)

(b1)

Fig. B.1 Comparison of the unidirectional model (analytical results) and exact model (Monte Carloresults) for the molecule described in appendix B. Panels a1 and a2 [b1 and b2] correspond to thepulling rate r = 5 pN/s [r = 500 pN/s]. The panels a1 and b1 show the probability pN (t) as afunction of time. In the panels a2 and b2 the probability density ρ(w, tU) for the work done onthe molecule during the pulling experiment is presented. In all four panels the results for the exact(unidirectional) model are depicted by the green circles (solid blue lines). The insets in the panels a1and b1 show the transition rates λU(t) (solid blue line) and λR(t) (dashed red line) as the functionsof time. The unfolding time tU in all the panels is given by the implicit equation pN (tU) = 0.99.Panel c depicts the Jarzynski integral (B.14) calculated for the unidirectional process as the functionof the pulling rate r. Specifically, the red triangles stand for the whole integral, Jint , the dashed blackline stands for the continuous part of the integral, IC, and the solid blue line depicts the singular partof the integral, IS. The green circles stand for the value of the Jarzynski integral for the exact model.The inset shows the free energies calculated from the individual parts of the Jarzynski integral usingEq. (B.13). For each point in the panel c a different unfolding time tU is used. Note that the Jarzynskiintegral is, in the unidirectional model, always overestimated, i.e., the corresponding free energy isunderestimated [cf. Eq. (B.14)]

part of the WPD, cf. Eqs. (B.14) and (B.15). Note that the Jarzynski integral is inthe unidirectional model always overestimated, i.e., the corresponding free energyis underestimated [cf. Eq. (B.14)].

References

1. Collin, D., Ritort, F., & Jarzynski, C., et. al. (2005). Verification of the Crooksfluctuation theorem and recovery of RNA folding free energies. Nature, 473, 231.http://dx.doi.org/10.1038/nature04061 M3.

2. Ritort, F. (2006). Single-molecule experiments in biological physics: methodsand applications. Journal of Physics: Condensed Matter, 18(32), R531. http://stacks.iop.org/0953-8984/18/i=32/a=R01.

Appendix B: Validity of Jarzynski Equality for a Unidirectional Process 137

3. Ritort, F. (2008). Nonequilibrium fluctuations in small systems: From physicsto biology. In A. Rice Stuart (Ed.), Advances in chemical physics (Vol. 137, pp.31–123). Hoboken: John Wiley & Sons Inc. ISBN 9780470238080, http://dx.doi.org/10.1002/9780470238080.ch2.

4. Alemany, A., Ribezzi, M., & Ritort, F. (2011). Recent progress in fluctuationtheorems and free energy recovery. AIP Conference Proceedings, 1332(1), 96–110. doi:10.1063/1.3569489.

5. SantaLucia, J. (1998). A unified view of polymer, dumbbell, and oligonucleotideDNA nearest-neighbor thermodynamics. Proceedings of the National Acad-emy of Sciences, 95(4), 1460–1465. http://www.pnas.org/content/95/4/1460.full.pdf+html. http://www.pnas.org/content/95/4/1460.abstract.

6. Zuker, M. (2003). Mfold web server for nucleic acid folding and hybridizationprediction. Nucleic Acids Research, 31(13), 3406–3415. doi:10.1093/nar/gkg595.

7. Nostheide, S., Holubec, V., Chvosta, P. & Maass, P. (2014). Unfolding kineticsof periodic DNA hairpins. Journal of Physics: Condensed Matter, 26, 205102.http://stacks.iop.org/0953-8984/26/i=20/a=205102.

Appendix CSimulations

The following work was, in a slightly modified version, published in [1].

C.1 Introduction

Stochastic jump processes with time-dependent transition rates are of generalimportance for many applications in physics and chemistry, in particular for describ-ing the kinetics of chemical reactions [2–4] and the non-equilibrium dynamics ofdriven systems in statistical mechanics [5–7]. With respect to applications in inter-disciplinary fields they play an important role in connection with queuing theories.

In general a system with N-levels is considered that, at random time instants,performs transitions from one microstate to another. In case of a Markovian jumpdynamics the probability for the system to change its microstate in the time interval[t, t + �t) is independent of the history and given by λij(t)�t + o(�t), where jand i ≤= j are the initial and target levels, respectively, and λij(t) the correspondingtransition rate at the time t (λjj(t) = 0). That is, if the system is in the level j at thetime t′, it will stay in this level until a time t > t′ with the probability φj(t, t′) =exp[− ∫ t

t′ dτ λtotj (τ )], where λtot

j (τ ) = ∑Ni=1 λij(τ ) is the total escape rate from the

level j at the time τ . The probability to perform a transition to the target level i in thetime interval [t, t + dt) then is λij(t)φj(t, t′) dt, i.e.,

ψij(t, t′) = λij(t) exp

⎡

⎣−t∫

t′dτ λtot

j (τ )

⎤

⎦ (C.1)

stands for the probability density for the first transition to the level i to occur at thetime t after the system was in the level j at the time t′. Any algorithm that evolvesthe system according to Eq. (C.1) generates stochastic trajectories with the correctpath probabilities.

V. Holubec, Non-equilibrium Energy Transformation Processes, Springer Theses, 139DOI: 10.1007/978-3-319-07091-9, © Springer International Publishing Switzerland 2014

140 Appendix C: Simulations

The first algorithm of this kind was developed by Gillespie [8] in generalization ofthe continuous-time Monte-Carlo algorithm introduced by Bortz et al. [9] for time-independent rates. We call it the reaction time algorithm (RTA) in the following.The RTA consists of drawing a random time t from the first transition time prob-ability density ψtot

j (t, t′) = ∑Ni=1 ψij(t, t′) = λtot

j (t)φj(t, t′) = −∂tφj(t, t′) to anyother microstate i ≤= j, and a subsequent random selection of the target level i withprobability λij(t)/λtot

j (t). In practice these two steps can be performed by generatingtwo uncorrelated and uniformly distributed random numbers r1, r2 in the unit inter-val [0, 1) with some random number generator, where the first one is used to specifythe transition time t via

�j(t, t′) =t∫

t′dτλtot

j (τ ) = − log(1 − r1) (C.2)

and the second one is used to select the target level i by requiring

i−1∑

k=1

λkj(t)

λtotj (t)

≡ r2 <

i∑

k=1

λkj(t)

λtotj (t)

. (C.3)

Both steps, however, lead to some unpleasant problems in the practical realization.The first step according to Eq. (C.2) requires the calculation of �j(t, t′) and the

determination of its inverse �j(t, t′) with respect to t in order to obtain the transitiontime t = �j[− log(1 − r1), t′]. While this is always possible, since λtot

j > 0 andaccordingly �j(t, t′) is a monotonously increasing function of t, it can be CPU timeconsuming in the case when �j(t, t′) cannot be explicitly given in an analytical formand one needs to implement a root finding procedure.

The second step according to Eq. (C.3) can be cumbersome if there are manylevels (N large) and a systematic grouping of the λij(t) to only a few classes isnot possible. This situation in particular applies to many-particle systems, where Ntypically grows exponentially with the number of particles, and the interactions (ora coupling to spatially inhomogeneous time-dependent external fields) can lead to alarge number of different transitions rates. Moreover, even for systems with simpleinteractions (as, for example, Ising spin systems), where a grouping is in principlepossible, the subdivision of the unit interval underlying Eq. (C.3) cannot be stronglysimplified for time-dependent rates.

A way to circumvent Eq. (C.3) is the use of the First Reaction Time Algorithm(FRTA) for time dependent rates [10], or modifications of it [3]. In the FRTA onefirst draws random transition times tk from the probability densities ψkj(tk, t′) =λkj(tk) exp[− ∫ tk

t′ dτ λkj(τ )] for the individual transitions to each of the targetmicrostates k and then performs the transition i with the smallest ti = mink{tk}at the time ti. This is statistically equivalent to the RTA, since for the given initialmicrostate j, the possible transitions to all target microstates are independent of each

Appendix C: Simulations 141

other. In short-range interacting systems, in particular, many of the random times tkcan be kept for determining the next transition following i. In fact, all transitions fromthe new level i to target levels k can be kept for which λki(τ ) = λkj(τ ) for τ > t (seeRef. [11] for details). However, the random times tk need to be drawn from ψkj(tk, t′)and this unfortunately involves the same problems as discussed above in connectionwith Eq. (C.2).

C.2 Algorithms

We now present a new “attempt time algorithm” (ATA) that allows one to avoid theproblems associated with the generation of the transition time in Eq. (C.2). Startingwith the system in the level j at the time t0, one first considers a large time intervalT and determines a number μtot

j satisfying

μtotj ≈ max

t0≡τ≡t0+T{λtot

j (τ )}. (C.4)

In general this can by done easily, since λtotj (τ ) is a known function. In particular for

bounded transition rates it poses no difficulty, as, for example, in the case of Glauberrates or a periodic external driving, where T could be chosen as the time period. Ifan unlimited growth of λtot

j with time were present (an unphysical situation for longtimes), T can be chosen self-consistently by requiring that the time t for the nexttransition to another level i ≤= j (see below) must be smaller than t0 + T .

Next an attempt time interval �t1 is drawn from the exponential density Fj(�t1) =μtot

j exp(−μtotj �t1) and the resulting attempt transition time t1 = t0 +�t1 is rejected

with probability prejj (t1) = 1 − λtot

j (t1)/μtotj . If it is rejected, a further attempt time

interval �t2 is drawn from Fj(�t2), corresponding to an attempt transition timet2 = t1 +�t2, and so on until an attempt time t < t0 +T is eventually accepted. Thena transition to a target level i is performed at the time t with probability λij(t)/λtot

j (t),using the target level selection of Eq. (C.3).

In order to show that this method yields the correct first transition probabilitydensity ψij(t, t0) from Eq. (C.1), let us first consider a sequence, where exactly n ≈ 0attempts at some times t1 < . . . < tn are rejected and then the (n+1)th attempt leadsto a transition to the target level i in the time interval [t, t + dt). The correspondingprobability density ψ

(n)ij (t, t0) is given by

ψ(n)ij (t, t0) =

t∫

t0

dtn

tn−1∫

t0

dtn−1 . . .

t2∫

t0

dt1λij(t)

λtotj (t)

×[1 − prej

j (t)]

Fj(t − tn)n∏

m=1

prejj (tm)Fj(tm − tm−1)

142 Appendix C: Simulations

= λij(t)e−μtot

j (t−t0)

n!

⎡

⎣t∫

t0

dτ μtotj prej

j (τ )

⎤

⎦

n

= λij(t)e−μtot

j (t−t0)

n!

⎡

⎣μtotj (t − t0) −

t∫

t0

dτ λtotj (τ )

⎤

⎦

n

. (C.5)

Summing over all possible n hence yields

ψj(t, t0) =∈∑

n=0

ψ(n)ij (t, t0) = λij(t) exp

⎡

⎣−t∫

t0

dτ λtotj (τ )

⎤

⎦ (C.6)

from Eq. (C.1).It is clear that for avoiding the root finding of Eq. (C.2) by use of the ATA, one

has to pay the price for introducing rejections. If the typical number of rejectionscan be kept small and an explicit analytical expression for t cannot be derived fromEq. (C.2), the ATA should become favorable in comparison to the RTA. Moreover,the ATA can be implemented in a software routine independent of the special formof the λij(τ ) for applicants who are not interested to invest special thoughts on howto solve Eq. (C.2).

One may object that the ATA still entails the problem connected with the cum-bersome target microstate selection by Eq. (C.3). However, as the RTA has the firstreaction variant FRTA, the ATA has a first attempt variant. In this first attempt timealgorithm (FATA) one first determines, instead of μtot

j from Eq. (C.4), upper boundsfor the individual transitions to all target microstates k ≤= j (μjj = 0),

μkj ≈ maxt0≡τ≡t0+T

{λkj(τ )}. (C.7)

Thereupon random time intervals �tk are drawn from the probability densityFkj(�tk) = μkj exp(−μkj�tk), yielding corresponding attempt times t(1)

k = t0 +�tk .

The transition to the target level k′ with the minimal t(1)

k′ = mink{t(1)k } = t1

is attempted and rejected with probability prejk′k(t

(1)

k′ ) = 1 − λk′k(t(1)

k′ )/μk′k . If

it is rejected, a further time interval �t(2)

k′ is drawn from Fk′j(�t(2)

k′ ), yielding

t(2)

k′ = t(1)

k′ + �t(2)

k′ , while the other attempt transition times are kept, t(2)k = t(1)

kfor k ≤= k′ (it is not necessary to draw new time intervals for these target levelsdue to the absence of memory in the Poisson process). The target level k′′ with thenew minimal t(2)

k′′ = mink{t(2)k } = t2 is then attempted and so on until eventually a

transition to a target level i is accepted at a time t < t0 + T . The determination of theminimal times can be done effectively by keeping an ordered stack of the attempttimes. Furthermore, as in the FRTA, one can, after a successful transition to a targetlevel i at the time t, keep the (last updated) attempt times tk for all target levels that are

Appendix C: Simulations 143

not affected by this transition (i.e., for which λki(τ ) = λkj(τ ) for τ ≈ t). Overall onecan view the procedure implied by the FATA as that each level k has a next attempttime tk (with tj = ∈ if the system is in the level j) and that the next attempt is madeto the target level with the minimal tk . After each attempt, updates of some of the tkare made as described above in dependence of whether the attempt was rejected oraccepted.

In order to prove that the FATA gives the ψj(t, t0) from Eq. (C.1), we show

that the probability densities χij(t, tn) = [λij(t)/λtotj (t)](1 − prej

j (t))Fj(t − tn) =λij(t) exp[−μtot

j (t − tn)] and ηj(tm, tm−1) = prejj (tm)Fj(tm − tm−1) = [μtot

j − λtotj ]

exp[−μtotj (tm − tm−1)] appearing in Eq. (C.5) are generated, if we set μtot

j = ∑Nk μkj

[note that Eq. (C.4) is automatically satisfied by this choice]. These probability den-sities have the following meaning: χij(t, tn)dt is the probability that, if the systemis in the level j at the time tn, the next attempt to a target microstate occurs in thetime interval [t, t + dt), the attempt is accepted, and it changes the level from j to i;ηj(tm, tm−1) dtm is the probability that, after the attempt time tm, the next attemptoccurs in [tm, tm + dtm) with tm > tm−1 and is rejected.

In the FATA the probability κlj(tm, tm−1) dtm that, when starting at the time tm−1,the next attempt is occurring in [tm, tm + dtm) to a target level l is given by

κlj(tm, tm−1) = μlj exp[−μlj(tm − tm−1)]

×∏

k ≤= l

∈∫

tm−tm−1

dτ μkj exp(−μkjτ )

= μlj exp[−μtotj (tm − tm−1)]. (C.8)

The product ensures that tm is the minimal time (the lower bound in the integral canbe set equal (tm − tm−1) for all k ≤= l due to the absence of memory in the Poissonprocess). The probability that this attempted transition is rejected is prej

lj (tm) = 1 −λlj(tm)/μlj and accordingly, by summing over all target levels l, we obtain

ηj(tm, tm−1) =N∑

l=1

prejlj (tm)μlj exp[−μtot

j )(tm − tm−1)]

= [μtotj − λtot

j (tm)] exp[−μtotj (tm − tm−1)] (C.9)

in agreement with the expression appearing in Eq. (C.5). Furthermore, when startingfrom time tn, the probability density χij(t, tn) referring to the joint probability thatthe next attempted transition occurs in [t, t +dt) to level i and is accepted is given by

χij(t, tn) = λij(t)

μijκij(t, tn) = λij(t) exp[−μtot

j (t − tn)]. (C.10)

144 Appendix C: Simulations

Hence one recovers the decomposition in Eq. (C.5) with μtotj = ∑N

k=1 μkj.Before discussing an example, it is instructive to see how the ATA (and RTA) can

be associated with a solution of the underlying master Eq. (1.3)

d

dtR(t | t′) = −M(t)R(t | t′), R(t′ | t′) = 1 (C.11)

where R(t | t′) is the matrix of transition probabilities Rij(t | t′) for the system to bein the level i at the time t if it was in the level j at the time t′ ≡ t, and M(t) = νL(t)is the transition rate matrix with elements Mij(t) = −λij(t) for i ≤= j and Mjj(t) =−∑

≥ i ≤=j Mij(t) = λtotj (t). Let us decompose M(t) as M(t) = D+A(t), where D =

diag{μtot

1 , . . . ,μtotN

}. IfA(t) were missing, the solution of the master equation (C.11)

would be R0(t | t′) = diag{exp(−μtot

1 (t − t′), . . . , exp(−μtotN (t − t′)

}. Hence, when

introducing A(t | t′) = R−10 (t | t′)A(t)R0(t | t′) = R0(t′ | t)A(t)R0(t | t′) in the

“interaction picture”, the solution of the master equation can be written as

R(t | t′) = R0(t | t′)

⎡

⎣1 +t∫

t′dt1A(t1 | t′)

+t∫

t′dt2

t2∫

t′dt1A(t2 | t′)A(t1 | t′) + · · ·

⎤

⎦ . (C.12)

Inserting 1 = D−1D after each matrix A, one arrives at

R(t | t′) = R0(t | t′) +t∫

t′dt1R0(t | t1)B(t1)F0(t1 | t′)

+t∫

t′dt2

t2∫

t′dt1R0(t | t2)B(t2)F0(t2 | t1)B(t1)F0(t1 | t′)

+ · · · (C.13)

where F0(t | t′) = DR0(t | t′) = diag{μtot1 exp[−μtot

1 (t − t′)], . . . , μtotN exp[−μtot

N(t − t′)]}, and B(t) = A(t)D−1 has the matrix elements Bij(t) = −λij(t)/μtot

j for

i ≤= j and Bjj(t) = 1 − λtotj (t)/μtot

j .Equation (C.13) resembles the ATA: The transition probabilities Rij(t | t′) are

decomposed into paths with an arbitrary number n = 0, 1, 2, . . . of “Poisson points”,where transitions are attempted. The times between successive attempted transitionsare exponentially distributed according to the matrix elements ofF0 and the attemptedtransitions are accepted or rejected according to the probabilities encoded in thediagonal and non-diagonal elements of the B matrix, respectively. The R0 entering

Appendix C: Simulations 145

Eq. (C.13) takes care that after the last attempt in a path with exactly n attemptedtransitions no further attempt occurs and the system remains in the target level i.The RTA can be associated with an analogous formal solution of the master equation

if one replaces R0(t | t′) by RRTA0 (t | t′) = diag

{λtot

1 (t) exp[− ∫ tt′ dτ λtot

1 (τ )], . . . ,λtot

N (t) exp[− ∫ tt′ dτ λtot

N (τ )]}

andB(t) byBRTA(t) with elements BRTAij (t) = (1−δij)

λij(t)/λtotj (t) (the diagonal elements are zero since the RTA is rejection-free).

C.3 Example

An implementation of FATA was used for simulated results depicted in Figs. 3.4and B.1. Another example is presented in [1]. Here we will introduce an examplewhere Eq. (C.2) must be inevitably solved numerically and, hence, ATA is advan-taged before RTA. Specifically, we consider a two-level system with stochastic time-dependent level energies (free energies) E1(t) and E2(t). We choose a simple linearprotocol

E1(t) = 0, E2(t) = at + �(t)�(τ − t) I(t). (C.14)

The constant a governs the deterministic component of E2(t) and I(t) denotes its sto-chastic (zero-mean) component. We considered the evolution within the time-intervalt ∧ [0, τ ]. The unit-step functions �(z) secure that the stochastic component of thedriving is switched on (of) after (before) the beginning (end) of the process. The time-evolution of the system for a given noise realization is driven by Eq. (2.3). The workdone on the system fulfills Eq. (2.10). We consider the detailed balanced Glaubertransition rates (3.19). Due to the stochastic level energies, the rates themselves arestochastic:

ν L21(t) = ν

1 + exp{β[(E2(t) − E1(t))]} , (C.15)

ν L12(t) = ν

1 + exp{β[(E1(t) − E2(t))]} .

Our goal in this example is to verify the Crooks relation (2.62) as it can be, using theresults derived in [12], generalized for models with stochastic driving. Let us definethe entropy for a given realization of the external noise, I → I(t), t ∧ (0, τ ), by theformula

σ(I) = ln

[Prob {I(t) = I(t), ≥ t ∧ (0, τ )}Prob {I(t) = I(t), ≥ t ∧ (0, τ )}

], (C.16)

where I → I(t) = I(τ − t), t ∧ (0, τ ). The external noise is time-reversible if itsarbitrary realization I is as probable to occur as the corresponding time-reversedrealization I , i.e., if the entropy (C.16) vanishes for any realization of the noise (see[13, 14]). The generalized Crooks relation reads [15]

146 Appendix C: Simulations

−10 −5 0 5 10 150

0.05

0.1

0.15

0.2

0.25

−10 −5 0 5 10 150

0.05

0.1

0.15

0.2

0.25(a) (b)

Fig. C.1 The illustration of the fluctuation theorem (C.17). The blue solid line stands for thefunction ρR(−w). The red dot-dashed line corresponds to ρR(−w) exp{−β[w − �F(τ , 0)]}. Theblue circle (red square) on the horizontal axis depicts the mean work W(τ , 0) [free energy difference�F(τ , 0)]. a The noise I(t) is the stationary reversible Ornstein-Uhlenbeck process (〈σ(I)〉I = 0).We take V0 = 5. b The noise I(t) is the non-stationary irreversible Ornstein-Uhlenbeck processwith 〈σ(I)〉I = 20.76 and V0 = 0.1. Other parameters used: D = 0.5, k = 0.05, a = 1, β = 0.3,τ = 10, ν = 1

ρF(w)

ρR(−w)〈exp[−σ(I)]〉σ(I)|w = exp{−β[�F(τ , 0) − w]}. (C.17)

Here, ρF(w) stands for the probability density for the work (2.10) done during theforward stochastic process D(t) when the system starts form thermal equilibriumdistribution π1 = π2 = 1/2 corresponding to the instantaneous value of the energies(C.14) at the time t = 0 (cf. Sect. 2.3). SimilarlyρR(w)denotes the probability densityfor the work done during the time-reversed process when the system starts form thethermal equilibrium distribution π1 = 1/(1 + e−βaτ ) π2 = e−βaτ/(1 + e−βaτ )

corresponding to the instantaneous value of the energies (C.14) at the time t = τ . Itis assumed that during the reversed process only the deterministic part of the energies(C.14) is reversed. The average 〈exp[−σ(I)]〉σ(I)|w is taken over all trajectories ofthe forward process D(t) (for each of these trajectories a new realization I of thenoise I(t) is taken) that yield the work w.

The generalized Crooks theorem (C.17) implies that the Crooks theorem ρF(w)/

ρR(−w) = exp{−β[�F(τ , 0)− w]} is exact if and only if the external noise is time-reversible (the entropy production for each individual realization I vanishes). This isverified in Fig. C.1 where the external noise I(t) is the Ornstein-Uhlenbeck process[16, 17]. The process driven by the Langevin Eq. (2.37) with the parabolic potentialF(x, t) = kx2/2. Using ATA we calculate the probability densities of work for boththe forward process, ρF(w), and the corresponding time-reversed process, ρR(−w).

Appendix C: Simulations 147

During the time-reversed process we reverse only the deterministic component ofE2(t). The stochastic component for the both processes starts at the initial time t = 0from the value drawn from a Gaussian distribution with zero mean and a givenvariance V0. Whenever the initial variance differs from D/(2k) the correspondingOrnstein-Uhlenbeck process is non-stationary, the relaxation occurs, and the entropyσ(I) is positive for most realizations of the process.

In the illustrations we take a=1, so in the forward process the deterministic partof E2(t) increases. Notice that under the deterministic driving alone, the support ofthe work distributions ρF(w) and ρR(−w) would be w ∧ [0, τ ]. The tail w < 0 arisespurely due to the action of the noise. In Fig. C.1 there are two δ-functions situatedat the origin. Their weights correspond to the relative frequencies of the trajectoriesthat reside in the level 1 during [0, τ ]. If the external noise is time-irreversible asillustrated in Fig. C.1b then, for a given w, one of the three cases

〈e−σ(I)〉σ(I)|w < 1 . . . ρF(w) e−β(w−�F(τ ,0)) > ρR(−w),

〈e−σ(I)〉σ(I)|w = 1 . . . ρF(w) e−β(w−�F(τ ,0)) = ρR(−w),

〈e−σ(I)〉σ(I)|w > 1 . . . ρF(w) e−β(w−�F(τ ,0)) < ρR(−w),

may be observed.The case 〈e−σ(I)〉σ(I)|w < 1 is normal, that is, a typical situation which is

highly probable to be observed in experiments or simulations. In this case, major-ity of the noise realizations possess a positive entropy. The values of w, for which〈e−σ(I)〉σ(I)|w < 1 holds, are highly probable values situated mostly in a vicinityof W(τ , 0) and/or �F(τ , 0) [W(τ , 0) denotes the mean work performed during theforward process]. The case 〈e−σ(I)〉σ(I)|w = 1 is marginal. For such w the Crooksrelation is valid pointwise. This case serves as a borderline between the first and thethird cases. The case 〈e−σ(I)〉σ(I)|w > 1 is anomalous. For these values of w theentropy corresponding to the majority of the noise realizations is negative, i.e., theserealizations occur with a lower probability then their time-reversals. The values of wfor which this case arises are rather unexpected. They are situated in the tails of thework distributions. For more irreversible noise the effect of such noise realizationson the tail w < 0 (works that are inaccessible without the noise) becomes morepronounced.

C.4 Summary

In summary, we have presented new simulation algorithms for Markovian jumpprocesses with time-dependent transition rates, which avoid the often cumbersomeor unhandy calculation of inverse functions. The ATA and FATA rely on the con-struction of a series of Poisson points, where transitions are attempted and rejectedwith certain probabilities. As a consequence, both algorithms are easy to implement,and their efficiency will be good as long as the number of rejections can be kept

148 Appendix C: Simulations

small. For complex interacting systems, the FATA has the same merits as the FRTAwith respect to the FRA. Both the ATA and FATA generate exact realizations of thestochastic process. Their connection to perturbative solutions of the underlying mas-ter equation may allow one to include in future work also non-Markovian featuresof a stochastic dynamics by letting the rejection probabilities to depend on the his-tory [18]. Compared to the RTA and FRTA, the new algorithms should in particularbe favorable, when considering periodically driven systems with interactions. Suchsystems are of much current interest in the study of non-equilibrium stationary statesand we thus hope that our findings will help to investigate them more convenientlyand efficiently.

References

1. Holubec, V., Chvosta, P., & Einax, M., et. al. (2011) Attempt time Monte Carlo:An alternative for simulation of stochastic jump processes with time-dependenttransition rates. Europhysics Letters, 93(4), 40003. http://stacks.iop.org/0295-5075/93/i=4/a=40003.

2. Gibson, M. A., & Bruck, J. (2000). Efficient exact stochastic simulation of chem-ical systems with many species and many channels. The Journal of PhysicalChemistry A, 104(9), 1876–1889. doi: 10.1021/jp993732q, http://pubs.acs.org/doi/pdf/10.1021/jp993732q.

3. Anderson, D. F. (2007). A modified next reaction method for simulating chemicalsystems with time dependent propensities and delays. The Journal of ChemicalPhysics, 127(21), 214107.doi:10.1063/1.2799998.

4. Astumian, R. D. (2007). Adiabatic operation of a molecular machine. Proceed-ings of National Academy of Sciences U.S.A., 104, 19715. http://www.pnas.org/content/104/50/19715.full.pdf+html.

5. Crooks, G. E. (1999). Entropy production fluctuation theorem and the non-equilibrium work relation for free energy differences. Physical Review E, 60,2721–2726, doi:10.1103/PhysRevE.60.2721. http://link.aps.org/doi/10.1103/PhysRevE.60.2721.

6. Seifert, U. (2005). Entropy production along a stochastic trajectory and anintegral fluctuation theorem. Physical Review Letters, 95, 040602, doi:10.1103/PhysRevLett.95.040602. http://link.aps.org/doi/10.1103/PhysRevLett.95.040602.

7. Esposito, M., & Van den Broeck, C. (2010). Three detailed fluctuation theorems.Physical Review Letters, 104, 090601, doi: 10.1103/PhysRevLett.104.090601.http://link.aps.org/doi/10.1103/PhysRevLett.104.090601.

8. Gillespie, D. T. (1978). Monte Carlo simulation of random walks with residencetime dependent transition probability rates. Journal of Computational Physics,28(3), 395–407. ISSN 0021–9991, doi:10.1016/0021-9991(78), 90060–8. http://www.sciencedirect.com/science/article/pii/0021999178900608.

9. Bortz, A., Kalos, M., & Lebowitz, J. (1975). A new algorithm for Monte Carlosimulation of Ising spin systems. Journal of Computational Physics, 17(1),

Appendix C: Simulations 149

10–18. ISSN 0021–9991, doi:10.1016/0021-9991(75), 90060–1. http://www.sciencedirect.com/science/article/pii/0021999175900601.

10. Jansen, A. (1995). Monte Carlo simulations of chemical reactions on a surfacewith time-dependent reaction-rate constants. Computer Physics Communica-tions, 86, 1–12, ISSN 0010–4655, doi:10.1016/0010-4655(94)00155-U. http://www.sciencedirect.com/science/article/pii/001046559400155U.

11. Einax, M., & Maass, P. (2009). Work distributions for Ising chains in atime-dependent magnetic field. Physical Review E, 80, 020101, doi:10.1103/PhysRevE.80.020101. http://link.aps.org/doi/10.1103/PhysRevE.80.020101.

12. Harris, R. J., & Schntz, G. M. (2007). Fluctuation theorems for stochastic dynam-ics. Journal of Statistical Mechanics: Theory and Experiment, 2007(07), P07020.http://stacks.iop.org/1742-5468/2007/i=07/a=P07020.

13. Kelly, F. P. (1979). Reversibility and stochastic networks. New York: Wiley.doi:10.1002/net.3230130110.

14. Dubkov, A. A. (2010). Statistical time-reversal symmetry and its physical appli-cations. Chemical Physics, 375(2–3), 364–369. ISSN 0301–0104, http://dx.doi.org/10.1016/j.chemphys.2010.05.033, stochastic processes in Physics andChemistry (in honor of Peter Hänggi).http://www.sciencedirect.com/science/article/pii/S0301010410002661.

15. Holubec, V., & Ryabov, A. (2012). Work fluctuations in systems under externalstochastic driving, unpublished.

16. Gillespie, D. T. (1992). Markov processes: An introduction for physical scientist.San Diego: Academic press, Inc.

17. Van Kampen, N. (2011). Stochastic processes in physics and chemistry. North-Holland Personal Library, San Diego: Elsevier Science. ISBN 9780080475363.http://books.google.cz/books?id=N6II-6HlPxEC.

18. Chvosta, P., & Reineker, P. (1999). Dynamics under the influence of semi-Markov noise. Physica A: Statistical Mechanics and its Applications, 268, 103–120, ISSN 0378–4371, doi:10.1016/S0378-4371(99), 00021–7. http://www.sciencedirect.com/science/article/pii/S0378437199000217.

Curriculum Vitae

Family name: HolubecGiven name: ViktorDate of birth: 22 January 1985Nationality: Czech RepublicAffiliation: Faculty of Mathematics and Physics, Charles University in PragueAddress: V Holešovickách 2, CZ-180 00 Prague, Czech RepublicEmail address: Viktor.Holubec@gmail.comTelephone: +420 774 237 329

Education

• Elementary school: 1992–2000• Secondary school: 2000–2004• Bachelor degree (General physics), Charles University in Prague, 2004–2007• Master degree (Theoretical physics), Charles University in Prague, 2007–2009• RNDr. (Theoretical physics), Charles University in Prague, 2012• Ph.D. (Theoretical physics, Astrophysics and Astronomy), Charles University in

Prague, 2009–2013

Publications

• Bachelor thesis: Parrondo’s paradox and simple discrete model of a molecularmotor, Prague 2007.

• Diploma thesis: Nonequilibrium Thermodynamics of Small Systems, Prague 2009.• Papers

– V. Holubec, P. Chvosta, P. Maass, J. Stat. Mech. (2012) P11009.– V. Holubec, P. Chvosta, P. Maass, M. Einax, EPL 93 (2011) 40003.– P. Chvosta, M. Einax, V. Holubec, A. Ryabov and P. Maass, J. Stat. Mech. (2010)

P03002.– P. Chvosta, V. Holubec, A. Ryabov, M. Einax and P. Maass, Physica E 42 (2010)

472–476.

V. Holubec, Non-equilibrium Energy Transformation Processes, Springer Theses, 151DOI: 10.1007/978-3-319-07091-9, © Springer International Publishing Switzerland 2014

152 Curriculum Vitae

• Book chapters

– V. Holubec, A. Ryabov and P. Chvosta (2011). Four Exactly Solvable Exam-ples in Non-Equilibrium Thermodynamics of Small Systems, Thermodynamics,Mizutani Tadashi (Ed.), ISBN: 978-953-307-544-0, InTech.