a conceptual perspectiveassets.cambridge.org/.../9780521884730_frontmatter.pdf · computational...

Computational Complexity

A Conceptual Perspective

Complexity Theory is a central field of the theoretical foundations of computer science.It is concerned with the general study of the intrinsic complexity of computational tasks;that is, it addresses the question of what can be achieved within limited time (and/or withother limited natural computational resources).

This book offers a conceptual perspective on Complexity Theory. It is intended to serveas an introduction for advanced undergraduate and graduate students, either as a textbookor for self-study. The book will also be useful to experts, since it provides expositions ofthe various sub-areas of Complexity Theory such as hardness amplification, pseudoran-domness, and probabilistic proof systems.

In each case, the author starts by posing the intuitive questions that are addressed by thesub-area and then discusses the choices made in the actual formulation of these questions,the approaches that lead to the answers, and the ideas that are embedded in these answers.

Oded Goldreich is a Professor of Computer Science at the Weizmann Institute of Scienceand an Incumbent of the Meyer W. Weisgal Professorial Chair. He is an editor for theSIAM Journal on Computing, the Journal of Cryptology, and Computational Complex-ity and previously authored the books Modern Cryptography, Probabilistic Proofs andPseudorandomness, and the two-volume work Foundations of Cryptography.

www.cambridge.org© Cambridge University Press

Cambridge University Press978-0-521-88473-0 - Computational Complexity: A Conceptual PerspectiveOded GoldreichFrontmatterMore information

http://www.cambridge.org/052188473X

http://www.cambridge.org


Computational Complexity

A Conceptual Perspective

Oded GoldreichWeizmann Institute of Science






CAMBRIDGE UNIVERSITY PRESS

Cambridge, New York, Melbourne, Madrid, Cape Town, Singapore, Sao Paulo, Delhi

Cambridge University Press32 Avenue of the Americas, New York, NY 10013-2473, USA

www.cambridge.orgInformation on this title: www.cambridge.org/9780521884730

C© Oded Goldreich 2008

This publication is in copyright. Subject to statutory exceptionand to the provisions of relevant collective licensing agreements,

no reproduction of any part may take place withoutthe written permission of Cambridge University Press.

First published 2008

Printed in the United States of America

A catalog record for this publication is available from the British Library.

Library of Congress Cataloging in Publication Data

Goldreich, Oded.Computational complexity : a conceptual perspective / Oded Goldreich.

p. cm.Includes bibliographical references and index.

ISBN 978-0-521-88473-0 (hardback)1. Computational complexity. 2. Turing machines. I. Title.

QA267.7.G65 2008511.3′52–dc22 2008006750

ISBN 978-0-521-88473-0 hardback

Cambridge University Press has no responsibility forthe persistence or accuracy of URLs for external or

third-party Internet Web sites referred to in this publicationand does not guarantee that any content on such

Web sites is, or will remain, accurate or appropriate.






to Dana






Contents

List of Figures page xiiiPreface xvOrganization and Chapter Summaries xviiAcknowledgments xxiii

1 Introduction and Preliminaries 1

1.1 Introduction 11.1.1 A Brief Overview of Complexity Theory 21.1.2 Characteristics of Complexity Theory 61.1.3 Contents of This Book 81.1.4 Approach and Style of This Book 121.1.5 Standard Notations and Other Conventions 16

1.2 Computational Tasks and Models 171.2.1 Representation 181.2.2 Computational Tasks 181.2.3 Uniform Models (Algorithms) 201.2.4 Non-uniform Models (Circuits and Advice) 361.2.5 Complexity Classes 42

Chapter Notes 43

2 P, NP, and NP-Completeness 44

2.1 The P Versus NP Question 462.1.1 The Search Version: Finding Versus Checking 472.1.2 The Decision Version: Proving Versus Verifying 502.1.3 Equivalence of the Two Formulations 542.1.4 Two Technical Comments Regarding NP 552.1.5 The Traditional Definition of NP 552.1.6 In Support of P Different from NP 572.1.7 Philosophical Meditations 58

2.2 Polynomial-Time Reductions 582.2.1 The General Notion of a Reduction 592.2.2 Reducing Optimization Problems to Search Problems 612.2.3 Self-Reducibility of Search Problems 632.2.4 Digest and General Perspective 67

vii






CONTENTS

2.3 NP-Completeness 672.3.1 Definitions 682.3.2 The Existence of NP-Complete Problems 692.3.3 Some Natural NP-Complete Problems 712.3.4 NP Sets That Are Neither in P nor NP-Complete 812.3.5 Reflections on Complete Problems 85

2.4 Three Relatively Advanced Topics 872.4.1 Promise Problems 872.4.2 Optimal Search Algorithms for NP 922.4.3 The Class coNP and Its Intersection with NP 94

Chapter Notes 97Exercises 99

3 Variations on P and NP 108

3.1 Non-uniform Polynomial Time (P/poly) 1083.1.1 Boolean Circuits 1093.1.2 Machines That Take Advice 111

3.2 The Polynomial-Time Hierarchy (PH) 1133.2.1 Alternation of Quantifiers 1143.2.2 Non-deterministic Oracle Machines 1173.2.3 The P/poly Versus NP Question and PH 119


4 More Resources, More Power? 127

4.1 Non-uniform Complexity Hierarchies 1284.2 Time Hierarchies and Gaps 129

4.2.1 Time Hierarchies 1294.2.2 Time Gaps and Speedup 136

4.3 Space Hierarchies and Gaps 139Chapter Notes 139Exercises 140

5 Space Complexity 143

5.1 General Preliminaries and Issues 1445.1.1 Important Conventions 1445.1.2 On the Minimal Amount of Useful Computation Space 1455.1.3 Time Versus Space 1465.1.4 Circuit Evaluation 153

5.2 Logarithmic Space 1535.2.1 The Class L 1545.2.2 Log-Space Reductions 1545.2.3 Log-Space Uniformity and Stronger Notions 1555.2.4 Undirected Connectivity 155

5.3 Non-deterministic Space Complexity 1625.3.1 Two Models 1625.3.2 NL and Directed Connectivity 1645.3.3 A Retrospective Discussion 171

viii






CONTENTS

5.4 PSPACE and Games 172Chapter Notes 175Exercises 175

6 Randomness and Counting 184

6.1 Probabilistic Polynomial Time 1856.1.1 Basic Modeling Issues 1866.1.2 Two-Sided Error: The Complexity Class BPP 1896.1.3 One-Sided Error: The Complexity Classes RP and coRP 1936.1.4 Zero-Sided Error: The Complexity Class ZPP 1996.1.5 Randomized Log-Space 199

6.2 Counting 2026.2.1 Exact Counting 2026.2.2 Approximate Counting 2116.2.3 Searching for Unique Solutions 2176.2.4 Uniform Generation of Solutions 220


7 The Bright Side of Hardness 241

7.1 One-Way Functions 2427.1.1 Generating Hard Instances and One-Way Functions 2437.1.2 Amplification of Weak One-Way Functions 2457.1.3 Hard-Core Predicates 2507.1.4 Reflections on Hardness Amplification 255

7.2 Hard Problems in E 2557.2.1 Amplification with Respect to Polynomial-Size Circuits 2577.2.2 Amplification with Respect to Exponential-Size Circuits 270


8 Pseudorandom Generators 284

Introduction 2858.1 The General Paradigm 2888.2 General-Purpose Pseudorandom Generators 290

8.2.1 The Basic Definition 2918.2.2 The Archetypical Application 2928.2.3 Computational Indistinguishability 2958.2.4 Amplifying the Stretch Function 2998.2.5 Constructions 3018.2.6 Non-uniformly Strong Pseudorandom Generators 3048.2.7 Stronger Notions and Conceptual Reflections 305

8.3 Derandomization of Time-Complexity Classes 3078.3.1 Defining Canonical Derandomizers 3088.3.2 Constructing Canonical Derandomizers 3108.3.3 Technical Variations and Conceptual Reflections 313

8.4 Space-Bounded Distinguishers 3158.4.1 Definitional Issues 316

ix






CONTENTS

8.4.2 Two Constructions 3188.5 Special-Purpose Generators 325

8.5.1 Pairwise Independence Generators 3268.5.2 Small-Bias Generators 3298.5.3 Random Walks on Expanders 332


9 Probabilistic Proof Systems 349

Introduction and Preliminaries 3509.1 Interactive Proof Systems 352

9.1.1 Motivation and Perspective 3529.1.2 Definition 3549.1.3 The Power of Interactive Proofs 3579.1.4 Variants and Finer Structure: An Overview 3639.1.5 On Computationally Bounded Provers: An Overview 366

9.2 Zero-Knowledge Proof Systems 3689.2.1 Definitional Issues 3699.2.2 The Power of Zero-Knowledge 3729.2.3 Proofs of Knowledge – A Parenthetical Subsection 378

9.3 Probabilistically Checkable Proof Systems 3809.3.1 Definition 3819.3.2 The Power of Probabilistically Checkable Proofs 3839.3.3 PCP and Approximation 3989.3.4 More on PCP Itself: An Overview 401


10 Relaxing the Requirements 416

10.1 Approximation 41710.1.1 Search or Optimization 41810.1.2 Decision or Property Testing 423

10.2 Average-Case Complexity 42810.2.1 The Basic Theory 43010.2.2 Ramifications 442


Epilogue 461

Appendix A: Glossary of Complexity Classes 463

A.1 Preliminaries 463A.2 Algorithm-Based Classes 464

A.2.1 Time Complexity Classes 464A.2.2 Space Complexity Classes 467

A.3 Circuit-Based Classes 467

Appendix B: On the Quest for Lower Bounds 469

B.1 Preliminaries 469

x






CONTENTS

B.2 Boolean Circuit Complexity 471B.2.1 Basic Results and Questions 472B.2.2 Monotone Circuits 473B.2.3 Bounded-Depth Circuits 473B.2.4 Formula Size 474

B.3 Arithmetic Circuits 475B.3.1 Univariate Polynomials 476B.3.2 Multivariate Polynomials 476

B.4 Proof Complexity 478B.4.1 Logical Proof Systems 480B.4.2 Algebraic Proof Systems 480B.4.3 Geometric Proof Systems 481

Appendix C: On the Foundations of Modern Cryptography 482

C.1 Introduction and Preliminaries 482C.1.1 The Underlying Principles 483C.1.2 The Computational Model 485C.1.3 Organization and Beyond 486

C.2 Computational Difficulty 487C.2.1 One-Way Functions 487C.2.2 Hard-Core Predicates 489

C.3 Pseudorandomness 490C.3.1 Computational Indistinguishability 490C.3.2 Pseudorandom Generators 491C.3.3 Pseudorandom Functions 492

C.4 Zero-Knowledge 494C.4.1 The Simulation Paradigm 494C.4.2 The Actual Definition 494C.4.3 A General Result and a Generic Application 495C.4.4 Definitional Variations and Related Notions 497

C.5 Encryption Schemes 500C.5.1 Definitions 502C.5.2 Constructions 504C.5.3 Beyond Eavesdropping Security 505

C.6 Signatures and Message Authentication 507C.6.1 Definitions 508C.6.2 Constructions 509

C.7 General Cryptographic Protocols 511C.7.1 The Definitional Approach and Some Models 512C.7.2 Some Known Results 517C.7.3 Construction Paradigms and Two Simple Protocols 517C.7.4 Concluding Remarks 522

Appendix D: Probabilistic Preliminaries and Advanced Topics inRandomization 523

D.1 Probabilistic Preliminaries 523D.1.1 Notational Conventions 523D.1.2 Three Inequalities 524

xi






CONTENTS

D.2.1 Definitions 528D.2.2 Constructions 529D.2.3 The Leftover Hash Lemma 529

D.3 Sampling 533D.3.1 Formal Setting 533D.3.2 Known Results 534D.3.3 Hitters 535

D.4 Randomness Extractors 536D.4.1 Definitions and Various Perspectives 537D.4.2 Constructions 541

Appendix E: Explicit Constructions 545

E.1 Error-Correcting Codes 546E.1.1 Basic Notions 546E.1.2 A Few Popular Codes 547E.1.3 Two Additional Computational Problems 551E.1.4 A List-Decoding Bound 553

E.2 Expander Graphs 554E.2.1 Definitions and Properties 555E.2.2 Constructions 561

Appendix F: Some Omitted Proofs 566

F.1 Proving That PH Reduces to #P 566F.2 Proving That IP( f ) ⊆ AM(O( f )) ⊆ AM( f ) 572

F.2.1 Emulating General Interactive Proofs by AM-Games 572F.2.2 Linear Speedup for AM 578

Appendix G: Some Computational Problems 583

G.1 Graphs 583G.2 Boolean Formulae 585G.3 Finite Fields, Polynomials, and Vector Spaces 586G.4 The Determinant and the Permanent 587G.5 Primes and Composite Numbers 587

Bibliography 589Index 601

xii

D.2 Hashing 528






List of Figures

1.1 Dependencies among the advanced chapters. page 91.2 A single step by a Turing machine. 231.3 A circuit computing f (x1, x2, x3, x4) = (x1 ⊕ x2, x1 ∧ ¬x2 ∧ x4). 381.4 Recursive construction of parity circuits and formulae. 422.1 Consecutive computation steps of a Turing machine. 742.2 The idea underlying the reduction of CSAT to SAT. 762.3 The reduction to G3C – the clause gadget and its sub-gadget. 812.4 The reduction to G3C – connecting the gadgets. 822.5 The world view under P �= coNP ∩ NP �= NP . 973.1 Two levels of the Polynomial-time Hierarchy. 1194.1 The Gap Theorem – determining the value of t(n). 1375.1 Algorithmic composition for space-bounded computation. 1475.2 The recursive procedure in NL ⊆ DSPACE(O(log2)). 1675.3 The main step in proving NL = coNL. 1706.1 The reduction to the Permanent – tracks connecting gadgets. 2076.2 The reduction to the Permanent – the gadget’s effect. 2076.3 The reduction to the Permanent – A Deus ex Machina gadget. 2086.4 The reduction to the Permanent – a structured gadget. 2096.5 The reduction to the Permanent – the box’s effect. 2097.1 The hard-core of a one-way function – an illustration. 2507.2 Proofs of hardness amplification: Organization. 2588.1 Pseudorandom generators – an illustration. 2878.2 Analysis of stretch amplification – the i th hybrid. 3008.3 Derandomization of BPL – the generator. 3218.4 An affine transformation affected by a Toeplitz matrix. 3278.5 The LFSR small-bias generator. 3308.6 Pseudorandom generators at a glance. 3359.1 Arithmetization of CNF formulae. 3609.2 Zero-knowledge proofs – an illustration. 3699.3 The PCP model – an illustration. 3809.4 Testing consistency of linear and quadratic forms. 3879.5 Composition of PCP system – an illustration. 3899.6 The amplifying reduction in the second proof of the PCP Theorem. 397

xiii






LIST OF FIGURES

10.1 Two types of average-case completeness. 44610.2 Worst-case versus average-case assumptions. 451E.1 Detail of the Zig-Zag product of G ′ and G. 562F.1 The transformation of an MA-game into an AM-game. 579F.2 The transformation of MAMA into AMA. 581

xiv






Preface

The quest for efficiency is ancient and universal, as time and other resources are alwaysin shortage. Thus, the question of which tasks can be performed efficiently is central tothe human experience.

A key step toward the systematic study of the aforementioned question is a rigorousdefinition of the notion of a task and of procedures for solving tasks. These definitionswere provided by computability theory, which emerged in the 1930s. This theory focuseson computational tasks, and considers automated procedures (i.e., computing devices andalgorithms) that may solve such tasks.

In focusing attention on computational tasks and algorithms, computability theory hasset the stage for the study of the computational resources (like time) that are requiredby such algorithms. When this study focuses on the resources that are necessary for anyalgorithm that solves a particular task (or a task of a particular type), the study becomespart of the theory of Computational Complexity (also known as Complexity Theory).1

Complexity Theory is a central field of the theoretical foundations of computer science.It is concerned with the study of the intrinsic complexity of computational tasks. Thatis, a typical complexity theoretic study refers to the computational resources required tosolve a computational task (or a class of such tasks), rather than referring to a specificalgorithm or an algorithmic schema. Actually, research in Complexity Theory tends tostart with and focus on the computational resources themselves, and addresses the effectof limiting these resources on the class of tasks that can be solved. Thus, ComputationalComplexity is the general study of what can be achieved within limited time (and/or otherlimited natural computational resources).

The (half-century) history of Complexity Theory has witnessed two main researchefforts (or directions). The first direction is aimed toward actually establishing concretelower bounds on the complexity of computational problems, via an analysis of the evolutionof the process of computation. Thus, in a sense, the heart of this direction is a “low-level”analysis of computation. Most research in circuit complexity and in proof complexityfalls within this category. In contrast, a second research effort is aimed at exploring theconnections among computational problems and notions, without being able to provideabsolute statements regarding the individual problems or notions. This effort may be

1In contrast, when the focus is on the design and analysis of specific algorithms (rather than on the intrinsic com-plexity of the task), the study becomes part of a related subfield that may be called Algorithmic Design and Analysis.Furthermore, Algorithmic Design and Analysis tends to be sub-divided according to the domain of mathematics,science, and engineering in which the computational tasks arise. In contrast, Complexity Theory typically maintainsa unity of the study of tasks solvable within certain resources (regardless of the origins of these tasks).

xv






PREFACE

viewed as a “high-level” study of computation. The theory of NP-completeness as wellas the studies of approximation, probabilistic proof systems, pseudorandomness, andcryptography all fall within this category.

The current book focuses on the latter effort (or direction). The main reason for ourdecision to focus on the “high-level” direction is the clear conceptual significance of theknown results. That is, many known results in this direction have an extremely appealingconceptual message, which can be appreciated also by non-experts. Furthermore, theseconceptual aspects can be explained without entering excessive technical detail. Conse-quently, the “high-level” direction is more suitable for an exposition in a book of thecurrent nature.2

The last paragraph brings us to a discussion of the nature of the current book, whichis captured by the subtitle (i.e., “a conceptual perspective”). Our main thesis is thatComplexity Theory is extremely rich in conceptual content, and that this content should beexplicitly communicated in expositions and courses on the subject. The desire to providea corresponding textbook is indeed the motivation for writing the current book and itsmain governing principle.

This book offers a conceptual perspective on Complexity Theory, and the presentationis designed to highlight this perspective. It is intended to serve as an introduction to thefield, and can be used either as a textbook or for self-study. Indeed, the book’s primarytarget audience consists of students who wish to learn Complexity Theory and educatorswho intend to teach a course on Complexity Theory. Still, we hope that the book will beuseful also to experts, especially to experts in one sub-area of Complexity Theory whoseek an introduction to and/or an overview of some other sub-area.

It is also hoped that the book may help promote general interest in Complexity Theoryand make this field acccessible to general readers with adequate background (whichconsists mainly of being comfortable with abstract discussions, definitions, and proofs).However, we do expect most readers to have a basic knowledge of algorithms, or at leastbe fairly comfortable with the notion of an algorithm.

The book focuses on several sub-areas of Complexity Theory (see the following or-ganization and chapter summaries). In each case, the exposition starts from the intuitivequestions addressed by the sub-area, as embodied in the concepts that it studies. Theexposition discusses the fundamental importance of these questions, the choices madein the actual formulation of these questions and notions, the approaches that underliethe answers, and the ideas that are embedded in these answers. Our view is that these(“non-technical”) aspects are the core of the field, and the presentation attempts to reflectthis view.

We note that being guided by the conceptual contents of the material leads, in somecases, to technical simplifications. Indeed, for many of the results presented in this book,the presentation of the proof is different (and arguably easier to understand) than thestandard presentations.

Web site for notices regarding this book. We intend to maintain a Web site listingcorrections of various types. The location of the site is

http://www.wisdom.weizmann.ac.il/∼oded/cc-book.html

2In addition, we mention a subjective reason for our decision: The “high-level” direction is within our ownexpertise, while this cannot be said about the “low-level” direction.

xvi






Organization and Chapter Summaries

This book consists of ten chapters and seven appendices. The chapters constitute the coreof this book and are written in a style adequate for a textbook, whereas the appendicesprovide either relevant background or additional perspective and are written in the styleof a survey article. The relative length and ordering of the chapters (and appendices) donot reflect their relative importance, but rather an attempt at the best logical order (i.e.,minimizing the number of forward pointers).

Following are brief summaries of the book’s chapters and appendices. These summariesare more novice-friendly than those provided in Section 1.1.3 but less detailed than thesummaries provided at the beginning of each chapter.

Chapter 1: Introduction and Preliminaries. The introduction provides a high-leveloverview of some of the content of Complexity Theory as well as a discussion of someof the characteristic features of this field. In addition, the introduction contains severalimportant comments regarding the approach and conventions of the current book. Thepreliminaries provide the relevant background on computability theory, which is the settingin which complexity theoretic questions are being studied. Most importantly, centralnotions such as search and decision problems, algorithms that solve such problems, andtheir complexity are defined. In addition, this part presents the basic notions underlyingnon-uniform models of computation (like Boolean circuits).

Chapter 2: P, NP, and NP-Completeness. The P versus NP Question can be phrased asasking whether or not finding solutions is harder than checking the correctness of solutions.An alternative formulation asks whether or not discovering proofs is harder than verifyingtheir correctness, that is, is proving harder than verifying. It is widely believed that theanswer to the two equivalent formulations is that finding (resp., proving) is harder thanchecking (resp., verifying); that is, it is believed that P is different from NP. At present,when faced with a hard problem in NP, we can only hope to prove that it is not in Passuming that NP is different from P. This is where the theory of NP-completeness, whichis based on the notion of a reduction, comes into the picture. In general, one computationalproblem is reducible to another problem if it is possible to efficiently solve the formerwhen provided with an (efficient) algorithm for solving the latter. A problem (in NP) isNP-complete if any problem in NP is reducible to it. Amazingly enough, NP-completeproblems exist, and furthermore, hundreds of natural computational problems arising inmany different areas of mathematics and science are NP-complete.

xvii






ORGANIZATION AND CHAPTER SUMMARIES

Chapter 3: Variations on P and NP. Non-uniform polynomial time (P/poly) capturesefficient computations that are carried out by devices that handle specific input lengths. Thebasic formalism ignores the complexity of constructing such devices (i.e., a uniformitycondition), but a finer formalism (based on “machines that take advice”) allows us toquantify the amount of non-uniformity. This provides a generalization of P. In contrast,the Polynomial-time Hierarchy (PH) generalizes NP by considering statements expressedby a quantified Boolean formula with a fixed number of alternations of existential anduniversal quantifiers. It is widely believed that each quantifier alternation adds expressivepower to the class of such formulae. The two different classes are related by showing thatif NP is contained in P/poly then the Polynomial-time Hierarchy collapses to its secondlevel (i.e., �2).

Chapter 4: More Resources, More Power? When using “nice” functions to determinean algorithm’s resources, it is indeed the case that more resources allow for more tasks to beperformed. However, when “ugly” functions are used for the same purpose, increasing theresources may have no effect. By nice functions we mean functions that can be computedwithout exceeding the amount of resources that they specify. Thus, we get results asserting,for example, that there are problems that are solvable in cubic time but not in quadratictime. In the case of non-uniform models of computation, the issue of “nicety” does notarise, and it is easy to establish separation results.

Chapter 5: Space Complexity. This chapter is devoted to the study of the space com-plexity of computations, while focusing on two rather extreme cases. The first case isthat of algorithms having logarithmic space complexity, which seem a proper and naturalsubset of the set of polynomial-time algorithms. The second case is that of algorithmshaving polynomial space complexity, which in turn can solve almost all computationalproblems considered in this book. Among the many results presented in this chapter area log-space algorithm for exploring (undirected) graphs, and a log-space reduction of theset of directed graphs that are not strongly connected to the set of directed graphs that arestrongly connected. These results capture fundamental properties of space complexity,which seems to differentiate it from time complexity.

Chapter 6: Randomness and Counting. Probabilistic polynomial-time algorithms withvarious types of failure give rise to complexity classes such as BPP , RP , and ZPP .The results presented include the emulation of probabilistic choices by non-uniformadvice (i.e., BPP ⊂ P/poly) and the emulation of two-sided probabilistic error by an ∃∀-sequence of quantifiers (i.e., BPP ⊆ �2). Turning to counting problems (i.e., countingthe number of solutions for NP-type problems), we distinguish between exact countingand approximate counting (in the sense of relative approximation). While any problemin PH is reducible to the exact counting class #P , approximate counting (for #P) is(probabilistically) reducible to NP . Additional related topics include #P-completeness,the complexity of searching for unique solutions, and the relation between approximatecounting and generating almost uniformly distributed solutions.

Chapter 7: The Bright Side of Hardness. It turns out that hard problems can be “putto work” to our benefit, most notably in cryptography. One key issue that arises in thiscontext is bridging the gap between “occasional” hardness (e.g., worst-case hardness ormild average-case hardness) and “typical” hardness (i.e., strong average-case hardness).

xviii







We consider two conjectures that are related to P �= NP . The first conjecture is that thereare problems that are solvable in exponential time but are not solvable by (non-uniform)families of small (say, polynomial-size) circuits. We show that these types of worst-caseconjectures can be transformed into average-case hardness results that yield non-trivialderandomizations of BPP (and even BPP = P). The second conjecture is that there areproblems in NP for which it is easy to generate (solved) instances that are hard to solvefor other people. This conjecture is captured in the notion of one-way functions, whichare functions that are easy to evaluate but hard to invert (in an average-case sense). Weshow that functions that are hard to invert in a relatively mild average-case sense yieldfunctions that are hard to invert almost everywhere, and that the latter yield predicatesthat are very hard to approximate (called hard-core predicates). The latter are useful forthe construction of general-purpose pseudorandom generators, as well as for a host ofcryptographic applications.

Chapter 8: Pseudorandom Generators. A fresh view of the question of randomness wastaken in the theory of computing: It has been postulated that a distribution is pseudorandomif it cannot be told apart from the uniform distribution by any efficient procedure. Theparadigm, originally associating efficient procedures with polynomial-time algorithms,has been applied also with respect to a variety of limited classes of such distinguishingprocedures. The archetypical case of pseudorandom generators refers to efficient genera-tors that fool any feasible procedure; that is, the potential distinguisher is any probabilisticpolynomial-time algorithm, which may be more complex than the generator itself. Thesegenerators are called general-purpose, because their output can be safely used in anyefficient application. In contrast, for purposes of derandomization, one may use pseu-dorandom generators that are somewhat more complex than the potential distinguisher(which represents the algorithm to be derandomized). Following this approach and usingvarious hardness assumptions, one may obtain corresponding derandomizations of BPP(including a full derandomization; i.e., BPP = P). Other forms of pseudorandom gener-ators include ones that fool space-bounded distinguishers, and even weaker ones that onlyexhibit some limited random behavior (e.g., outputting a pairwise independent sequence).

Chapter 9: Probabilistic Proof Systems. Randomized and interactive verification pro-cedures, giving rise to interactive proof systems, seem much more powerful than theirdeterministic counterparts. In particular, interactive proof systems exist for any set inPSPACE ⊇ coNP (e.g., for the set of unsatisfied propositional formulae), whereas it iswidely believed that some sets in coNP do not have NP-proof systems. Interactive proofsallow the meaningful conceptualization of zero-knowledge proofs, which are interactiveproofs that yield nothing (to the verifier) beyond the fact that the assertion is indeed valid.Under reasonable complexity assumptions, every set in NP has a zero-knowledge proofsystem. (This result has many applications in cryptography.) A third type of probabilisticproof system underlies the model of PCPs, which stands for probabilistically checkableproofs. These are (redundant) NP-proofs that offer a trade-off between the number of lo-cations (randomly) examined in the proof and the confidence in its validity. In particular,a small constant error probability can be obtained by reading a constant number of bitsin the redundant NP-proof. The PCP Theorem asserts that NP-proofs can be efficientlytransformed into PCPs. The study of PCPs is closely related to the study of the complexityof approximation problems.

xix







Chapter 10: Relaxing the Requirements. In light of the apparent infeasibility of solv-ing numerous useful computational problems, it is natural to seek relaxations of thoseproblems that remain useful for the original applications and yet allow for feasible solvingprocedures. Two such types of relaxation are provided by adequate notions of approxima-tion and a theory of average-case complexity. The notions of approximation refer to thecomputational problems themselves; that is, for each problem instance we extend the setof admissible solutions. In the context of search problems this means settling for solutionsthat have a value that is “sufficiently close” to the value of the optimal solution, whereasin the context of decision problems this means settling for procedures that distinguishyes-instances from instances that are “far” from any yes-instance. Turning to average-case complexity, we note that a systematic study of this notion requires the developmentof a non-trivial conceptual framework. One major aspect of this framework is limiting theclass of distributions in a way that, on the one hand, allows for various types of naturaldistributions and, on the other hand, prevents the collapse of average-case hardness toworst-case hardness.

Appendix A: Glossary of Complexity Classes. The glossary provides self-containeddefinitions of most complexity classes mentioned in the book. The glossary is partitionedinto two parts, dealing separately with complexity classes that are defined in terms ofalgorithms and their resources (i.e., time and space complexity of Turing machines) andcomplexity classes defined in terms of non-uniform circuits (and referring to their sizeand depth). In particular, the following classes are defined: P , NP , coNP , BPP , RP ,coRP , ZPP , #P , PH, E , EXP , NEXP , L, NL, RL, PSPACE , P/poly, NCk , andACk .

Appendix B: On the Quest for Lower Bounds. This brief survey describes the mostfamous attempts at proving lower bounds on the complexity of natural computationalproblems. The first part, devoted to Circuit Complexity, reviews lower bounds for thesize of (restricted) circuits that solve natural computational problems. This represents aprogram whose long-term goal is proving that P �= NP . The second part, devoted toProof Complexity, reviews lower bounds on the length of (restricted) propositional proofsof natural tautologies. This represents a program whose long-term goal is proving thatNP �= coNP .

Appendix C: On the Foundations of Modern Cryptography. This survey of the founda-tions of cryptography focuses on the paradigms, approaches, and techniques that are usedto conceptualize, define, and provide solutions to natural security concerns. It presentssome of these conceptual tools as well as some of the fundamental results obtained usingthem. The appendix augments the partial treatment of one-way functions, pseudorandomgenerators, and zero-knowledge proofs (included in Chapters 7–9). Using these basic tools,the appendix provides a treatment of basic cryptographic applications such as encryption,signatures, and general cryptographic protocols.

Appendix D: Probabilistic Preliminaries and Advanced Topics in Randomization.The probabilistic preliminaries include conventions regarding random variables as well asthree useful inequalities (i.e., Markov’s Inequality, Chebyshev’s Inequality, and ChernoffBound). The advanced topics include constructions and lemmas regarding families ofhashing functions, a study of the sample and randomness complexities of estimating the

xx







average value of an arbitrary function, and the problem of randomness extraction (i.e.,procedures for extracting almost perfect randomness from sources of weak or defectedrandomness).

Appendix E: Explicit Constructions. Complexity Theory provides a clear perspectiveon the intuitive notion of an explicit construction. This perspective is demonstrated withrespect to error-correcting codes and expander graphs. Starting with codes, the appendixfocuses on various computational aspects, and offers a review of several popular con-structions as well as a construction of a binary code of constant rate and constant relativedistance. Also included are a brief review of the notions of locally testable and locallydecodable codes, and a useful upper bound on the number of codewords that are close toany single sequence. Turning to expander graphs, the appendix contains a review of twostandard definitions of expanders, two levels of explicitness, two properties of expandersthat are related to (single-step and multi-step) random walks on them, and two explicitconstructions of expander graphs.

Appendix F: Some Omitted Proofs. This appendix contains some proofs that were notincluded in the main text (for a variety of reasons) and still are beneficial as alternativesto the original and/or standard presentations. Included are a proof that PH is reducible to#P via randomized Karp-reductions, and the presentation of two useful transformationsregarding interactive proof systems.

Appendix G: Some Computational Problems. This appendix includes definitions ofmost of the specific computational problems that are referred to in the main text. Inparticular, it contains a brief introduction to graph algorithms, Boolean formulae, andfinite fields.

xxi






Acknowledgments

My perspective on Complexity Theory was most influenced by Shimon Even and LeonidLevin. In fact, it was hard not to be influenced by these two remarkable and highlyopinionated researchers (especially for somebody like me who was fortunate to spend alot of time with them).1

Shimon Even viewed Complexity Theory as the study of the limitations of algorithms,a study concerned with natural computational resources and natural computational tasks.Complexity Theory was there to guide the engineer and to address the deepest questionsthat bother an intellectually curious computer scientist. I believe that this book sharesShimon’s perspective of Complexity Theory as evolving around such questions.

Leonid Levin emphasized the general principles that underlie Complexity Theory,rejecting any “model-dependent effects” as well as the common coupling of ComplexityTheory with the theory of automata and formal languages. In my opinion, this book isgreatly influenced by these perspectives of Leonid.

I wish to acknowledge the influence of numerous other colleagues on my professionalperspectives and attitudes. These include Shafi Goldwasser, Dick Karp, Silvio Micali, andAvi Wigderson. Needless to say, this is but a partial list that reflects influences of which Iam most aware.

The year I spent at Radcliffe Institute for Advanced Study (of Harvard University) wasinstrumental in my decision to undertake the writing of this book. I am grateful to Radcliffefor creating such an empowering atmosphere. I also wish to thank many colleagues fortheir comments and advice (or help) regarding earlier versions of this text. A partial listincludes Noga Alon, Noam Livne, Dieter van Melkebeek, Omer Reingold, Dana Ron,Ronen Shaltiel, Amir Shpilka, Madhu Sudan, Salil Vadhan, and Avi Wigderson.

Lastly, I am grateful to Mohammad Mahmoody Ghidary and Or Meir for their carefulreading of drafts of this manuscript and for the numerous corrections and suggestions thatthey have provided.

Relation to previous texts. Some of the text of this book has been adapted from previoustexts of mine. In particular, Chapters 8 and 9 were written based on my surveys [90,Chap. 3] and [90, Chap. 2], respectively; but the exposition has been extensively revised

1Shimon Even was my graduate studies adviser (at the Technion, 1980–83), whereas I had a lot of meetings withLeonid Levin during my postdoctoral period (at MIT, 1983–86).

xxiii






ACKNOWLEDGMENTS

to fit the significantly different aims of the current book. Similarly, Section 7.1 andAppendix C were written based on my survey [90, Chap. 1] and books [91, 92]; but,again, the previous texts are very different in many ways. In contrast, Appendix B wasadapted with relatively little modifications from an early draft of a section in an article byAvi Wigderson and myself [107].

xxiv






a conceptual perspectiveassets.cambridge.org/.../9780521884730_frontmatter.pdf · computational...

Documents