foundations of computer science

TABLE OF CONTENTS

v

3 3 3 3

Table of ContentsPreface Chapter 1.1. 1.2. 1.3. 1.4. 1.5. 1.6. 1.7. 1.8. ix 1. Computer Science: The Mechanization of Abstraction What This Book Is About 3 What This Chapter Is About 6 Data Models 6 The C Data Model 13 Algorithms and the Design of Programs 20 Some C Conventions Used Throughout the Book 22 Summary of Chapter 1 23 Bibliographic Notes for Chapter 1 24 1

Chapter 2. Iteration, Induction, and Recursion 25 2.1. What This Chapter Is About 27 2.2. Iteration 27 2.3. Inductive Proofs 34 2.4. Complete Induction 44 2.5. Proving Properties of Programs 52 2.6. Recursive Denitions 59 2.7. Recursive Functions 69 2.8. Merge Sort: A Recursive Sorting Algorithm 75 2.9. Proving Properties of Recursive Programs 84 2.10. Summary of Chapter 2 87 2.11. Bibliographic Notes for Chapter 2 88 Chapter 3. The Running Time of Programs 89 3.1. What This Chapter Is About 89 3.2. Choosing an Algorithm 90 3.3. Measuring Running Time 91 3.4. Big-Oh and Approximate Running Time 96 3.5. Simplifying Big-Oh Expressions 101 3.6. Analyzing the Running Time of a Program 109 3.7. A Recursive Rule for Bounding Running Time 116 3.8. Analyzing Programs with Function Calls 127 3.9. Analyzing Recursive Functions 132 3.10. Analysis of Merge Sort 136 3.11. Solving Recurrence Relations 144 3.12. Summary of Chapter 3 154 3.13. Bibliographic Notes for Chapter 3 155 Chapter 4.1. 4.2. 4.3. 4.4. 4. Combinatorics and Probability 156 What This Chapter Is About 156 Counting Assignments 157 Counting Permutations 160 Ordered Selections 167

vi

TABLE OF CONTENTS

4.5. Unordered Selections 170 4.6. Orderings With Identical Items 178 4.7. Distribution of Objects to Bins 181 4.8. Combining Counting Rules 184 4.9. Introduction to Probability Theory 187 4.10. Conditional Probability 193 4.11. Probabilistic Reasoning 203 4.12. Expected Value Calculations 212 4.13. Some Programming Applications of Probability 4.14. Summary of Chapter 4 220 4.15. Bibliographic Notes for Chapter 4 221 Chapter 5. The Tree Data Model 223 5.1. What This Chapter Is About 223 5.2. Basic Terminology 224 5.3. Data Structures for Trees 231 5.4. Recursions on Trees 239 5.5. Structural Induction 248 5.6. Binary Trees 253 5.7. Binary Search Trees 258 5.8. Eciency of Binary Search Tree Operations 268 5.9. Priority Queues and Partially Ordered Trees 271 5.10. Heapsort: Sorting with Balanced POTs 280 5.11. Summary of Chapter 5 284 5.12. Bibliographic Notes for Chapter 5 285 Chapter 6. The List Data Model 286 6.1. What This Chapter Is About 286 6.2. Basic Terminology 287 6.3. Operations on Lists 291 6.4. The Linked-List Data Structure 293 6.5. Array-Based Implementation of Lists 301 6.6. Stacks 306 6.7. Implementing Function Calls Using a Stack 6.8. Queues 318 6.9. Longest Common Subsequences 321 6.10. Representing Character Strings 327 6.11. Summary of Chapter 6 334 6.12. Bibliographic Notes for Chapter 6 335 Chapter 7.1. 7.2. 7.3. 7.4. 7.5. 7.6. 7.7. 7.8. 7.9. 7. The Set Data Model 337 What This Chapter Is About 337 Basic Denitions 338 Operations on Sets 342 List Implementation of Sets 351 Characteristic-Vector Implementation of Sets Hashing 360 Relations and Functions 366 Implementing Functions as Data 373 Implementing Binary Relations 380

215

312

357

TABLE OF CONTENTS

vii

7.10. 7.11. 7.12. 7.13.

Some Special Properties of Binary Relations Innite Sets 396 Summary of Chapter 7 401 Bibliographic Notes for Chapter 7 402

386

Chapter 8. The Relational Data Model 403 8.1. What This Chapter Is About 403 8.2. Relations 404 8.3. Keys 411 8.4. Primary Storage Structures for Relations 414 8.5. Secondary Index Structures 419 8.6. Navigation among Relations 423 8.7. An Algebra of Relations 428 8.8. Implementing Relational Algebra Operations 436 8.9. Algebraic Laws for Relations 440 8.10. Summary of Chapter 8 449 8.11. Bibliographic Notes for Chapter 8 450 Chapter 9. The Graph Data Model 451 9.1. What This Chapter Is About 451 9.2. Basic Concepts 452 9.3. Implementation of Graphs 459 9.4. Connected Components of an Undirected Graph 9.5. Minimal Spanning Trees 478 9.6. Depth-First Search 484 9.7. Some Uses of Depth-First Search 495 9.8. Dijkstras Algorithm for Finding Shortest Paths 9.9. Floyds Algorithm for Shortest Paths 513 9.10. An Introduction to Graph Theory 521 9.11. Summary of Chapter 9 526 9.12. Bibliographic Notes for Chapter 9 527

466

502

Chapter 10. Patterns, Automata, and Regular Expressions 529 10.1. What This Chapter Is About 530 10.2. State Machines and Automata 530 10.3. Deterministic and Nondeterministic Automata 536 10.4. From Nondeterminism to Determinism 547 10.5. Regular Expressions 556 10.6. The UNIX Extensions to Regular Expressions 564 10.7. Algebraic Laws for Regular Expressions 568 10.8. From Regular Expressions to Automata 571 10.9. From Automata to Regular Expressions 582 10.10. Summary of Chapter 10 588 10.11. Bibliographic Notes for Chapter 10 589 Chapter 11. Recursive Description of Patterns 11.1. What This Chapter Is About 591 11.2. Context-Free Grammars 592 11.3. Languages from Grammars 599 11.4. Parse Trees 602 591

viii

TABLE OF CONTENTS

11.5. Ambiguity and the Design of Grammars 610 11.6. Constructing Parse Trees 617 11.7. A Table-Driven Parsing Algorithm 625 11.8. Grammars Versus Regular Expressions 634 11.9. Summary of Chapter 11 640 11.10. Bibliographic Notes for Chapter 11 641 Chapter 12. Propositional Logic 642 12.1. What This Chapter Is About 642 12.2. What Is Propositional Logic? 643 12.3. Logical Expressions 645 12.4. Truth Tables 649 12.5. From Boolean Functions to Logical Expressions 655 12.6. Designing Logical Expressions by Karnaugh Maps 660 12.7. Tautologies 669 12.8. Some Algebraic Laws for Logical Expressions 674 12.9. Tautologies and Methods of Proof 682 12.10. Deduction 686 12.11. Proofs by Resolution 692 12.12. Summary of Chapter 12 697 12.13. Bibliographic Notes for Chapter 12 698 Chapter 13. Using Logic to Design Computer Components 13.1. What This Chapter is About 699 13.2. Gates 700 13.3. Circuits 701 13.4. Logical Expressions and Circuits 705 13.5. Some Physical Constraints on Circuits 711 13.6. A Divide-and-Conquer Addition Circuit 716 13.7. Design of a Multiplexer 723 13.8. Memory Elements 730 13.9. Summary of Chapter 13 731 13.10. Bibliographic Notes for Chapter 13 732 Chapter 14. Predicate Logic 733 14.1. What This Chapter Is About 733 14.2. Predicates 734 14.3. Logical Expressions 736 14.4. Quantiers 739 14.5. Interpretations 745 14.6. Tautologies 751 14.7. Tautologies Involving Quantiers 753 14.8. Proofs in Predicate Logic 759 14.9. Proofs from Rules and Facts 762 14.10. Truth and Provability 768 14.11. Summary of Chapter 14 774 14.12. Bibliographic Notes for Chapter 14 775 Index 776 699

PrefaceThis book was motivated by the desire we and others have had to further the evolution of the core course in computer science. Many departments across the country have revised their curriculum in response to the introductory course in the science of computing discussed in the Denning Report, (Denning, P. J., D. E. Comer, D. Gries, M. C. Mulder, A. Tucker, J. Turner, and P. R. Young, Computing as a Discipline, Comm. ACM 32:1, pp. 923, January 1989.). That report draws attention to three working methodologies or processes theory, abstraction, and design as fundamental to all undergraduate programs in the discipline. More recently, the Computing Curricula 1991 report of the joint ACM/IEEE-CS Curriculum Task Force echoes the Denning Report in identifying key recurring concepts which are fundamental to computing, especially: conceptual and formal models, eciency, and levels of abstraction. The themes of these two reports summarize what we have tried to oer the student in this book. This book developed from notes for a two-quarter course at Stanford called CS109: Introduction to Computer Science that serves a number of goals. The rst goal is to give beginning computer science majors a solid foundation for further study. However, computing is becoming increasingly important in a much wider range of scientic and engineering disciplines. Therefore, a second goal is to give those students who will not take advanced courses in computer science the conceptual tools that the eld provides. Finally, a more pervasive goal is to expose all students not only to programming concepts but also to the intellectually rich foundations of the eld. Our rst version of this book was based on programming in Pascal and appeared in 1992. Our choice of Pascal as the language for example programs was motivated by that languages use in the Computer Science Advanced Placement Test as well as in a plurality of college introductions to programming. We were pleased to see that since 1992 there has been a signicant trend toward C as the introductory programming language, and we accordingly developed a new version of the book using C for programming examples. Our emphasis on abstraction and encapsulation should provide a good foundation for subsequent courses covering object-oriented technology using C++. At the same time, we decided to make two signicant improvements in the content of the book. First, although it is useful to have a grounding in machine architecture to motivate how running time is measured, we found that almost all curricula separate architecture into a separate course, so the chapter on that subject was not useful. Second, many introductory courses in the theory of computing emphasize combinatorics and probability, so we decided to increase the coverage and cluster the material into a chapter of its own. Foundations of Computer Science covers subjects that are often found split between a discrete mathematics course and a sophomore-level sequence in computer science in data structures. It has been our intention to select the mathematical foundations with an eye toward what the computer user really needs, rather than what a mathematician might choose. We have tried to integrate eectively the mathematical foundations with the computing. We thus hope to provide a better feel for the soul of computer science than might be found in a programming course, ix

x

PREFACE

a discrete mathematics course, or a course in a computer science subspecialty. We believe that, as time goes on, all scientists and engineers will take a foundational course similar to the one oered at Stanford upon which this book is based. Such a course in computer science should become as standard as similar courses in calculus and physics.

PrerequisitesStudents taking courses based on this book have ranged from rst-year undergraduates to graduate students. We assume only that students have had a solid course in programming. They should be familiar with the programming language ANSI C to use this edition. In particular, we expect students to be comfortable with C constructs such as recursive functions, structures, pointers, and operators involving pointers and structures such as dot, ->, and &.

Suggested Outlines for Foundational Courses in CSIn terms of a traditional computer science curriculum, the book combines a rst course in data structures that is, a CS2 course with a course in discrete mathematics. We believe that the integration of these subjects is extremely desirable for two reasons: 1. 2. It helps motivate the mathematics by relating it more closely to the computing. Computing and mathematics can be mutually reinforcing. Some examples are the way recursive programming and mathematical induction are related in Chapter 2 and the way the free/bound variable distinction for logic is related to the scope of variables in programming languages in Chapter 14. Suggestions for instructive programming assignments are presented throughout the book.

There are a number of ways in which this book can be used.

A Two-Quarter or Two-Semester CourseThe CS109A-B sequence at Stanford is typical of what might be done in two quarters, although these courses are rather intensive, being 4-unit, 10-week courses each. These two courses cover the entire book, the rst seven chapters in CS109A and Chapters 8 through 14 in CS109B.

A One-Semester CS2 Type CourseIt is possible to use the book for a one-semester course covering a set of topics similar to what would appear in a CS2 course. Naturally, there is too much material in the book to cover in one semester, and so we recommend the following: 1. 2. 3. Recursive algorithms and programs in Sections 2.7 and 2.8. Big-oh analysis and running time of programs: all of Chapter 3 except for Section 3.11 on solving recurrence relations. Trees in Sections 5.2 through 5.10.

PREFACE

xi

4.

Lists: all of Chapter 6. Some may wish to cover lists before trees, which is a more traditional treatment. We regard trees as the more fundamental notion, but there is little harm in switching the order. The only signicant dependency is that Chapter 6 talks about the dictionary abstract data type (set with operations insert, delete, and lookup), which is introduced in Section 5.7 as a concept in connection with binary search trees. Sets and relations. Data structures for sets and relations are emphasized in Sections 7.2 through 7.9 and 8.2 through 8.6. Graph algorithms are covered in Sections 9.2 through 9.9.

5. 6.

A One-Semester Discrete Mathematics CourseFor a one-semester course emphasizing mathematical foundations, the instructor could choose to cover: Mathematical induction and recursive programs in Chapter 2. Big-oh analysis, running time, and recurrence relations in Sections 3.4 through 3.11. 3. Combinatorics in Sections 4.2 through 4.8. 4. Discrete probability in Sections 4.9 through 4.13. 5. Mathematical aspects of trees in Sections 5.2 through 5.6. 6. Mathematical aspects of sets in Sections 7.2, 7.3, 7.7, 7.10, and 7.11. 7. The algebra of relations in Sections 8.2, 8.7, and 8.9. 8. Graph algorithms and graph theory in Chapter 9. 9. Automata and regular expressions in Chapter 10. 10. Context-free grammars in Sections 11.2 through 11.4. 11. Propositional and predicate logic in Chapters 12 and 14, respectively. 1. 2.

Features of This BookTo help the student assimilate the material, we have added the following study aids: 1. 2. 3. Each chapter has an outline section at the beginning and a summary section at the end highlighting the main points. Marginal notes mark important concepts and denitions. However, items mentioned in section or subsection headlines are not repeated in the margin. Sidebars are separated from the text by double lines. These short notes serve several purposes: 3 3 Some are elaborations on the text or make some ne points about program or algorithm design. Others are for summary or emphasis of points made in the text nearby. These include outlines of important kinds of proofs, such as the various forms of proof by induction. A few are used to give examples of fallacious arguments, and we hope that the separation from the text in this way will eliminate possible misconstruction of the point.

3

xii

PREFACE

3 4.

A few give very brief introductions to major topics like undecidability or the history of computers to which we wish we could devote a full section.

Most of the sections end with exercises. There are more than 1000 exercises or parts spread among the sections. Of these roughly 30% are marked with a single star, which indicates that they require more thought than the unstarred exercises. Approximately another 10% of the exercises are doubly starred, and these are the most challenging. Chapters end with bibliographic notes. We have not attempted to be exhaustive, but oer suggestions for more advanced texts on the subject of the chapter and mention the relevant papers with the most historical signicance.

5.

About the CoverIt is a tradition for computer science texts to have a cover with a cartoon or drawing symbolizing the content of the book. Here, we have drawn on the myth of the world as the back of a turtle, but our world is populated with representatives of some of the other, more advanced texts in computer science that this book is intended to support. They are: The teddy bear: R. Sethi, Programming Languages: Concepts and Constructs, Addison-Wesley, Reading, Mass., 1989. The baseball player: J. D. Ullman, Principles of Database and Knowledge-Base Systems, Computer Science Press, New York, 1988. The column: J. L. Hennessy and D. A. Patterson, Computer Architecture: a Quantitative Approach, Morgan-Kaufmann, San Mateo, Calif., 1990. The dragon: A. V. Aho, R. Sethi, and J. D. Ullman, Compiler Design: Principles, Techniques, and Tools, Addison-Wesley, Reading, Mass., 1986. The triceratops: J. L. Peterson and A. Silberschatz, Operating Systems Concepts, second edition, Addison-Wesley, Reading, Mass., 1985.

AcknowledgmentsWe are deeply indebted to a number of colleagues and students who have read this material and given us many valuable suggestions for improving the presentation. We owe a special debt of gratitude to Brian Kernighan, Don Knuth, Apostolos Lerios, and Bob Martin who read the original Pascal manuscript in detail and gave us many perceptive comments. We have received, and gratefully acknowledge, reports of course testing of the notes for the Pascal edition of this book by Michael Anderson, Margaret Johnson, Udi Manber, Joseph Naor, Prabhakar Ragde, Rocky Ross, and Shuky Sagiv. There are a number of other people who found errors in earlier editions, both the original notes and the various printings of the Pascal edition. In this regard, we would like to thank: Susan Aho, Michael Anderson, Aaron Edsinger, Lonnie Eldridge, Todd Feldman, Steve Friedland, Christopher Fuselier, Mike Genstil, Paul Grubb III, Barry Hayes, John Hwang, Hakan Jakobsson, Arthur Keller, Dean Kelley, James Kuner Jr., Steve Lindell, Richard Long, Mark MacDonald, Simone Martini, Hugh McGuire, Alan Morgan, Monnia Oropeza, Rodrigo Philander, Andrew Quan, Stuart Reges, John Stone, Keith Swanson, Steve Swenson, Sanjai Tiwari, Eric Traut, and Lynzi Ziegenhagen.

PREFACE

xiii

We acknowledge helpful advice from Geo Clem, Jon Kettenring, and Brian Kernighan during the preparation of the C edition of Foundations of Computer Science. Peter Ullman produced a number of the gures used in this book. We are grateful to Dan Clayton, Anthony Dayao, Mat Howard, and Ron Underwood for help with TEX fonts, and to Hester Glynn and Anne Smith for help with the manuscript preparation.

On-Line Access to Code, Errata, and NotesYou can obtain copies of the major programs in this book by anonymous ftp to host ftp-cs.stanford.edu. Login with user name anonymous and give your name and host as a password. You may then execute cd fcsc where you will nd programs from this book. We also plan to keep in this directory information about errata and what course notes we can provide. A. V. A. Chatham, NJ J. D. U. Stanford, CA July, 1994

CHAPTER

1 Computer Science: The Mechanization of AbstractionThough it is a new eld, computer science already touches virtually every aspect of human endeavor. Its impact on society is seen in the proliferation of computers, information systems, text editors, spreadsheets, and all of the wonderful application programs that have been developed to make computers easier to use and people more productive. An important part of the eld deals with how to make programming easier and software more reliable. But fundamentally, computer science is a science of abstraction creating the right model for thinking about a problem and devising the appropriate mechanizable techniques to solve it. Every other science deals with the universe as it is. The physicists job, for example, is to understand how the world works, not to invent a world in which physical laws would be simpler or more pleasant to follow. Computer scientists, on the other hand, must create abstractions of real-world problems that can be understood by computer users and, at the same time, that can be represented and manipulated inside a computer. Sometimes the process of abstraction is simple. For example, we can model the behavior of the electronic circuits used to build computers quite well by an abstraction called propositional logic. The modeling of circuits by logical expressions is not exact; it simplies, or abstracts away, many details such as the time it takes for electrons to ow through circuits and gates. Nevertheless, the propositional logic model is good enough to help us design computer circuits well. We shall have much more to say about propositional logic in Chapter 12. As another example, suppose we are faced with the problem of scheduling nal examinations for courses. That is, we must assign course exams to time slots so that two courses may have their exams scheduled in the same time slot only if there is no student taking both. At rst, it may not be apparent how we should model this problem. One approach is to draw a circle called a node for each course and draw a line called an edge connecting two nodes if the corresponding courses have a student in common. Figure 1.1 suggests a possible picture for ve courses; the picture is called a course-conict graph. Given the course-conict graph, we can solve the exam-scheduling problem by repeatedly nding and removing maximal independent sets from the graph. An 1

3 3 3 3

Abstraction

Exam scheduling

2

COMPUTER SCIENCE: THE MECHANIZATION OF ABSTRACTION

Eng

Math

CS

Econ

Phy

Fig. 1.1. Course-conict graph for ve courses. An edge between two courses indicates that at least one student is taking both courses.

Maximal independent set

independent set is a collection of nodes that have no connecting edges within the collection. An independent set is maximal if no other node from the graph can be added without including an edge between two nodes of the set. In terms of courses, a maximal independent set is any maximal set of courses with no common students. In Fig. 1.1, {Econ, Eng, P hy} is one maximal independent set. The set of courses corresponding to the selected maximal independent set is assigned to the rst time slot. We remove from the graph the nodes in the rst maximal independent set, along with all incident edges, and then nd a maximal independent set among the remaining courses. One choice for the next maximal independent set is the singleton set {CS}. The course in this maximal independent set is assigned to the second time slot. We repeat this process of nding and deleting maximal independent sets until no more nodes remain in the course-conict graph. At this point, all courses will have been assigned to time slots. In our example, after two iterations, the only remaining node in the course-conict graph is Math, and this forms the nal maximal independent set, which is assigned to the third time slot. The resulting exam schedule is thus TIME SLOT 1 2 3 COURSE EXAMS Econ, Eng, Phy CS Math

This algorithm does not necessarily partition the courses among the smallest possible number of time slots, but it is simple and does tend to produce a schedule with close to the smallest number of time slots. It is also one that can be readily programmed using the techniques presented in Chapter 9. Notice that this approach abstracts away some details of the problem that may be important. For example, it could cause one student to have ve exams in ve consecutive time slots. We could create a model that included limits on how many exams in a row one student could take, but then both the model and the solution

SEC. 1.1

WHAT THIS BOOK IS ABOUT

3

Abstraction: Not to Be FearedThe reader may cringe at the word abstraction, because we all have the intuition that abstract things are hard to understand; for example, abstract algebra (the study of groups, rings, and the like) is generally considered harder than the algebra we learned in high school. However, abstraction in the sense we use it implies simplication, the replacement of a complex and detailed real-world situation by an understandable model within which we can solve a problem. That is, we abstract away the details whose eect on the solution to a problem is minimal or nonexistent, thereby creating a model that lets us deal with the essence of the problem.

Knowledge representation

to the exam-scheduling problem would be more complicated. Often, nding a good abstraction can be quite dicult because we are forced to confront the fundamental limitations on the tasks computers can perform and the speed with which computers can perform those tasks. In the early days of computer science, some optimists believed that robots would soon have the prodigious capability and versatility of the Star Wars robot C3PO. Since then we have learned that in order to have intelligent behavior on the part of a computer (or robot), we need to provide that computer with a model of the world that is essentially as detailed as that possessed by humans, including not only facts (Sallys phone number is 555-1234), but principles and relationships (If you drop something, it usually falls downward). We have made much progress on this problem of knowledge representation. We have devised abstractions that can be used to help build programs that do certain kinds of reasoning. One example of such an abstraction is the directed graph, in which nodes represent entities (the species cat or Fluy) and arrows (called arcs) from one node to another represent relationships (Fluy is a cat, cats are animals, Fluy owns Fluys milk saucer); Figure 1.2 suggests such a graph. Another useful abstraction is formal logic, which allows us to manipulate facts by applying rules of inference, such as If X is a cat and Y is the mother of X, then Y is a cat. Nevertheless, progress on modeling, or abstracting, the real world or signicant pieces thereof remains a fundamental challenge of computer science, one that is not likely to be solved completely in the near future.

3 3 3 3

1.1

What This Book Is AboutThis book will introduce the reader, who is assumed to have a working knowledge of the programming language ANSI C, to the principal ideas and concerns of computer science. The book emphasizes three important problem-solving tools: 1. Data models, the abstractions used to describe problems. We have already mentioned two models: logic and graphs. We shall meet many others throughout this book.

4


Animal

is

Cat

is owns Fluys milk saucer

Fluy

Fig. 1.2. A graph representing knowledge about Fluy.

2.

Data structures, the programming-language constructs used to represent data models. For example, C provides built-in abstractions, such as structures and pointers, that allow us to construct data structures to represent complex abstractions such as graphs. Algorithms, the techniques used to obtain solutions by manipulating data as represented by the abstractions of a data model, by data structures, or by other means.

3.

Data ModelsWe meet data models in two contexts. Data models such as the graphs discussed in the introduction to this chapter are abstractions frequently used to help formulate solutions to problems. We shall learn about several such data models in this book: trees in Chapter 5, lists in Chapter 6, sets in Chapter 7, relations in Chapter 8, graphs in Chapter 9, nite automata in Chapter 10, grammars in Chapter 11, and logic in Chapters 12 and 14. Data models are also associated with programming languages and computers. For example, C has a data model that includes abstractions such as characters, integers of several sizes, and oating-point numbers. Integers and oating-point numbers in C are only approximations of integers and reals in mathematics because of the limited precision of arithmetic available in computers. The C data model also includes types such as structures, pointers, and functions, which we shall discuss in more detail in Section 1.4.

Data StructuresWhen the data model of the language in which we are writing a program lacks a built-in representation for the data model of the problem at hand, we must represent the needed data model using the abstractions supported by the language. For this purpose, we study data structures, which are methods for representing in the data model of a programming language abstractions that are not an explicit part of

SEC. 1.1

WHAT THIS BOOK IS ABOUT

5

that language. Dierent programming languages may have strikingly dierent data models. For example, unlike C, the language Lisp supports trees directly, and the language Prolog has logic built into its data model.

AlgorithmsAn algorithm is a precise and unambiguous specication of a sequence of steps that can be carried out mechanically. The notation in which an algorithm is expressed can be any commonly understood language, but in computer science algorithms are most often expressed formally as programs in a programming language, or in an informal style as a sequence of programming language constructs intermingled with English language statements. Most likely, you have already encountered several important algorithms while studying programming. For example, there are a number of algorithms for sorting the elements of an array, that is, putting the elements in smallest-rst order. There are clever searching algorithms such as binary search, which quickly nds a given element in a sorted array by repeatedly dividing in half the portion of the array in which the element could appear. These, and many other tricks for solving common problems, are among the tools the computer scientist uses when designing programs. We shall study many such techniques in this book, including the important methods for sorting and searching. In addition, we shall learn what makes one algorithm better than another. Frequently, the running time, or time taken by an algorithm measured as a function of the size of its input, is one important aspect of the quality of the algorithm; we discuss running time in Chapter 3. Other aspects of algorithms are also important, particularly their simplicity. Ideally, an algorithm should be easy to understand and easy to turn into a working program. Also, the resulting program should be understandable by a person reading the code that implements the algorithm. Unfortunately, our desires for a fast algorithm and a simple algorithm are often in conict, and we must choose our algorithm wisely.

Underlying ThreadsAs we progress through this book, we shall encounter a number of important unifying principles. We alert the reader to two of these here: 1. Design algebras. In certain elds in which the underlying models have become well understood, we can develop notations in which design trade-os can be expressed and evaluated. Through this understanding, we can develop a theory of design with which well-engineered systems can be constructed. Propositional logic, with the associated notation called Boolean algebra that we encounter in Chapter 12, is a good example of this kind of design algebra. With it, we can design ecient circuits for subsystems of the kind found in digital computers. Other examples of algebras found in this book are the algebra of sets in Chapter 7, the algebra of relations in Chapter 8, and the algebra of regular expressions in Chapter 10.

6


2.

Recursion is such a useful technique for dening concepts and solving problems that it deserves special mention. We discuss recursion in detail in Chapter 2 and use it throughout the rest of the book. Whenever we need to dene an object precisely or whenever we need to solve a problem, we should always ask, What does the recursive solution look like? Frequently that solution has a simplicity and eciency that makes it the method of choice.

3 3 3 3

1.2

What This Chapter Is AboutThe remainder of this chapter sets the stage for the study of computer science. The primary concepts that will be covered are 3 3 3 Data models (Section 1.3) The data model of the programming language C (Section 1.4) The principal steps in the software-creation process (Section 1.5)

We shall give examples of several dierent ways in which abstractions and models appear in computer systems. In particular, we mention the models found in programming languages, in certain kinds of systems programs, such as operating systems, and in the circuits from which computers are built. Since software is a vital component of todays computer systems, we need to understand the softwarecreation process, the role played by models and algorithms, and the aspects of software creation that computer science can address only in limited ways. In Section 1.6 there are some conventional denitions that are used in C programs throughout this book. 3 3 3 3

1.3

Data ModelsAny mathematical concept can be termed a data model. In computer science, a data model normally has two aspects: 1. The values that objects can assume. For example, many data models contain objects that have integer values. This aspect of the data model is static; it tells us what values objects may take. The static part of a programming languages data model is often called the type system. The operations on the data. For example, we normally apply operations such as addition to integers. This aspect of the model is dynamic; it tells us the ways in which we can change values and create new values.

Type system

2.

Programming Language Data ModelsEach programming language has its own data model, and these dier from one another, often in quite substantial ways. The basic principle under which most programming languages deal with data is that each program has access to boxes, which we can think of as regions of storage. Each box has a type, such as int or char. We may store in a box any value of the correct type for that box. We often refer to the values that can be stored in boxes as data objects.

Data object

SEC. 1.3

DATA MODELS

7

Name

Dereferencing

The list data model

We may also name boxes. In general, a name for a box is any expression that denotes that box. Often, we think of the names of boxes as the variables of the program, but that is not quite right. For example, if x is a variable local to a recursive function F , then there may be many boxes named x, each associated with a dierent call to F . Then the true name of such a box is a combination of x and the particular call to F . Most of the data types of C are familiar: integers, oating-point numbers, characters, arrays, structures, and pointers. These are all static notions. The operations permitted on data include the usual arithmetic operations on integers and oating-point numbers, accessing operations for elements of arrays or structures, and pointer dereferencing, that is, nding the element pointed to by a pointer. These operations are part of the dynamics of the C data model. In a programming course, we would see important data models that are not part of C, such as lists, trees, and graphs. In mathematical terms, a list is a sequence of n elements, which we shall write as (a1 , a2 , . . . , an ), where a1 is the rst element, a2 the second, and so on. Operations on lists include inserting new elements, deleting elements, and concatenating lists (that is, appending one list to the end of another).

3

Example 1.1. In C, a list of integers can be represented by a data structure called a linked list in which list elements are stored in cells. Lists and their cells can be dened by a type declaration such astypedef struct CELL *LIST; struct CELL { int element; struct LIST next; }; This declaration denes a self-referential structure CELL with two elds. The rst is element, which holds the value of an element of the list and is of type int. The second eld of each CELL is next, which holds a pointer to a cell. Note that the type LIST is really a pointer to a CELL. Thus, structures of type CELL can be linked together by their next elds to form what we usually think of as a linked list, as suggested in Fig. 1.3. The next eld can be thought of as either a pointer to the next cell or as representing the entire list that follows the cell in which it appears. Similarly, the entire list can be represented by a pointer, of type LIST, to the rst cell on the list. a1 a2 an

Fig. 1.3. A linked list representing the list (a1 , a2 , . . . , an ).

Cells are represented by rectangles, the left part of which is the element, and the right part of which holds a pointer, shown as an arrow to the next cell pointed to. A dot in the box holding a pointer means that the pointer is NULL.1 Lists will be covered in more detail in Chapter 6. 31

NULL is a symbolic constant dened in the standard header le stdio.h to be equal to a value that cannot be a pointer to anything. We shall use it to have this meaning throughout the book.

8


Data Models Versus Data StructuresDespite their similar names, a list and a linked list are very dierent concepts. A list is a mathematical abstraction, or data model. A linked list is a data structure. In particular, it is the data structure we normally use in C and many similar languages to represent abstract lists in programs. There are other languages in which it is not necessary to use a data structure to represent abstract lists. For example, the list (a1 , a2 , . . . , an ) could be represented directly in the language Lisp and in the language Prolog similarly, as [a1 , a2 , . . . , an ].

Data Models of System SoftwareOperating systems

Data models are found not only in programming languages but also in operating systems and applications programs. You are probably familiar with an operating system such as UNIX or MS-DOS (perhaps with Microsoft Windows).2 The function of an operating system is to manage and schedule the resources of a computer. The data model for an operating system like UNIX has concepts such as les, directories, and processes. /

mnt

usr

bin

ann

bob

a1

a2

a3

b1

b2

Fig. 1.4. A typical UNIX directory/le structure.

1.Files Directories

The data itself is stored in les, which in the UNIX system are strings of characters. Files are organized into directories, which are collections of les and/or other directories. The directories and les form a tree with the les at the leaves.3 Figure 1.4 suggests the tree that might represent the directory structure of a typical UNIX operating system. Directories are indicated by circles. The root directory / contains directories called mnt, usr, bin, and so on. The directory /usr contains directories ann and bob; directory ann contains three les: a1, a2, and a3.2

2.

3

If you are unfamiliar with operating systems, you can skip the next paragraphs. However, most readers have probably encountered an operating system, perhaps under another name. For example, the Macintosh system is an operating system, although dierent terminology is used. For example, a directory becomes a folder in Macintosh-ese. However, links in directories may make it appear that a le or directory is part of several dierent directories.

SEC. 1.3

DATA MODELS

9

Processes Pipes

3.

Processes are individual executions of programs. Processes take zero or more streams as input and produce zero or more streams as output. In the UNIX system, processes can be combined by pipes, where the output from one process is fed as input into the next process. The resulting composition of processes can be viewed as a single process with its own input and output.

3

Example 1.2. Consider the UNIX command linebc | word | speak The symbol | indicates a pipe, an operation that makes the output of the process on the left of this symbol be the input to the process on its right. The program bc is a desk calculator that takes arithmetic expressions, such as 2 + 3, as input and produces the answer 5 as output. The program word translates numbers into words; speak translates words into phoneme sequences, which are then uttered over a loudspeaker by a voice synthesizer. Connecting these three programs together by pipes turns this UNIX command line into a single process that behaves like a talking desk calculator. It takes as input arithmetic expressions and produces as output the spoken answers. This example also suggests that a complex task may be implemented more easily as the composition of several simpler functions. 3 There are many other aspects to an operating system, such as how it manages security of data and interaction with the user. However, even these few observations should make it apparent that the data model of an operating system is rather dierent from the data model of a programming language. Another type of data model is found in text editors. Every text editors data model incorporates a notion of text strings and editing operations on text. The data model usually includes the notion of lines, which, like most les, are character strings. However, unlike les, lines may have associated line numbers. Lines may also be organized into larger units such as paragraphs, and operations on lines are normally applicable anywhere within the line not just at the front, like the most common le operations. The typical editor supports a notion of a current line (where the cursor is) and probably a current position within that line. Operations performed by the editor include various modications to lines, such as deletion or insertion of characters within the line, deletion of lines, and creation of new lines. It is also possible in typical editors to search for features, such as specic character strings, among the lines of the le being edited. In fact, if you examine any other familiar piece of software, such as a spreadsheet or a video game, a pattern emerges. Each program that is designed to be used by others has its own data model, within which the user must work. The data models we meet are often radically dierent from one another, both in the primitives they use to represent data and in the operations on that data that are oered to the user. Yet each data model is implemented, via data structures and the programs that use them, in some programming language.

Text editors

The Data Model of CircuitsWe shall also meet in this book a data model for computer circuits. This model, called propositional logic, is most useful in the design of computers. Computers are composed of elementary components called gates. Each gate has one or more

10


inputs and one output; the value of an input or output can be only 0 or 1. A gate performs a simple function such as AND, where the output is 1 if all the inputs are 1 and the output is 0 if one or more of the inputs are 0. At one level of abstraction, computer design is the process of deciding how to connect gates to perform the basic operations of a computer. There are many other levels of abstraction associated with computer design as well. Figure 1.5 shows the usual symbol for an AND-gate, together with its truth table, which indicates the output value of the gate for each pair of input values.4 We discuss truth tables in Chapter 12 and gates and their interconnections in Chapter 13. x x AND y z 0 0 1 1 0 1 0 1 0 0 0 1 y z

Fig. 1.5. An AND-gate and its truth table.

3

Bit

Example 1.3. To execute the C assignment statement a = b+c, a computer performs the addition with an adder circuit. In the computer, all numbers are represented in binary notation using the two digits 0 and 1 (called binary digits, or bits for short). The familiar algorithm for adding decimal numbers, where we add the digits at the right end, generate a carry to the next place to the left, add that carry and the digits at that place, generate a carry to the next place to the left, and so on, works in binary as well.x y

d

c

zFig. 1.6. A one-bit adder: dz is the sum x + y + c.

One-bit adder

Out of a few gates, we can build a one-bit adder circuit, as suggested in Fig. 1.6. Two input bits, x and y, and a carry-in bit c, are summed, resulting in a sum bit z and a carry-out bit d. To be precise, d is 1 if two or more of c, x, and y are 1, while z is 1 if an odd number (one or three) of c, x, and y are 1, as suggested by4

Note that if we think of 1 as true and 0 as false, then the AND-gate performs the same logical operation as the && operator of C.

SEC. 1.3

DATA MODELS

11

x 0 0 0 0 1 1 1 1

y 0 0 1 1 0 0 1 1

c 0 1 0 1 0 1 0 1

d 0 0 0 1 0 1 1 1

z 0 1 1 0 1 0 0 1

Fig. 1.7. Truth table for the one-bit adder.

The Ripple-Carry Addition AlgorithmWe all have used the ripple-carry algorithm to add numbers in decimal. To add 456 + 829, for example, one performs the steps suggested below: 1 4 5 6 8 2 9 5 0 4 5 6 8 2 9 8 5 4 5 6 8 2 9 1 2 8 5

That is, at the rst step, we add the rightmost digits, 6+9 = 15. We write down the 5 and carry the 1 to the second column. At the second step, we add the carry-in, 1, and the two digits in the second place from the right, to get 1 + 5 + 2 = 8. We write down the 8, and the carry is 0. In the third step, we add the carry-in, 0, and the digits in the third place from the right, to get 0 + 4 + 8 = 12. We write down the 2, but since we are at the leftmost place, we do not carry the 1, but rather write it down as the leftmost digit of the answer. Binary ripple-carry addition works the same way. However, at each place, the carry and the digits being added are all either 0 or 1. The one-bit adder thus describes completely the addition table for a single place. That is, if all three bits are 0, then the sum is 0, and so we write down 0 and carry 0. If one of the three is 1, the sum is 1; we write down 1 and carry 0. If two of the three are 1, the sum is 2, or 10 in binary; we write down 0 and carry 1. If all three are 1, then the sum is 3, or 11 in binary, and so we write down 1 and carry 1. For example, to add 101 to 111 using binary ripple-carry addition, the steps are 1 1 0 1 1 1 1 0 1 1 0 1 1 1 1 0 0 1 0 1 1 1 1 1 1 0 0

the table of Fig. 1.7. The carry-out bit followed by the sum bit that is, dz forms a two-bit binary number, which is the total number of x, y, and c that are 1. In this sense, the one-bit adder adds its inputs. Many computers represent integers as 32-bit numbers. An adder circuit can

12


Ripple-carry adder

then be composed of 32 one-bit adders, as suggested in Fig. 1.8. This circuit is often called a ripple-carry adder, because the carry ripples from right to left, one bit at a time. Note that the carry into the rightmost (low-order bit) one-bit adder is always 0. The sequence of bits x31 x30 x0 represents the bits of the rst number being added, and y31 y30 y0 is the second addend. The sum is dz31 z30 z0 ; that is, the leading bit is the carry-out of the leftmost one-bit adder, and the following bits of the sum are the sum bits of the adders, from the left. x31 y31 x30 y30 x0 y0

d

0

z31

z30

z0

Fig. 1.8. A ripple-carry adder: dz31 z30 z0 = x31 x30 x0 + y31 y30 y0 .

The circuit of Fig. 1.8 is really an algorithm in the data model of bits and the primitive operations of gates. However, it is not a particularly good algorithm. The reason is that until we compute the carry-out of the rightmost place, we cannot compute z1 or the carry-out of the second place. Until we compute the carry-out of the second place, we cannot compute z2 or the carry-out of the third place, and so on. Thus, the time taken by the circuit is the length of the numbers being added 32 in our case multiplied by the time needed by a one-bit adder. One might suspect that the need to ripple the carry through each of the onebit adders, in turn, is inherent in the denition of addition. Thus, it may come as a surprise to the reader that computers have a much faster way of adding numbers. We shall cover such an improved algorithm for addition when we discuss the design of circuits in Chapter 13. 3

EXERCISES1.3.1: Explain the dierence between the static and dynamic aspects of a data model. 1.3.2: Describe the data model of your favorite video game. Distinguish between static and dynamic aspects of the model. Hint : The static parts are not just the parts of the game board that do not move. For example, in Pac Man, the static part includes not only the map, but the power pills, monsters, and so on. 1.3.3: Describe the data model of your favorite text editor. 1.3.4: Describe the data model of a spreadsheet program.

SEC. 1.4

THE C DATA MODEL

13

3 3 3 3

1.4

The C Data ModelIn this section we shall highlight important parts of the data model used by the C programming language. As an example of a C program, consider the program in Fig. 1.10 that uses the variable num to count the number of characters in its input. #include main() { int num; num = 0; while (getchar() != EOF) ++num; /* add 1 to num */ printf("%d\n", num); }Fig. 1.10. C program to count number of input characters.

EOF

The rst line tells the C preprocessor to include as part of the source the standard input/output le stdio.h, which contains the denitions of the functions getchar and printf, and the symbolic constant EOF, a value that represents the end of a le. A C program itself consists of a sequence of denitions, which can be either function denitions or data denitions. One must be a denition of a function called main. The rst statement in the function body of the program in Fig. 1.10 declares the variable num to be of type int. (All variables in a C program must be declared before their use.) The next statement initializes num to zero. The following while statement reads input characters one at a time using the library function getchar, incrementing num after each character read, until there are no more input characters. The end of le is signaled by the special value EOF on the input. The printf statement prints the value of num as a decimal integer, followed by a newline character.

The C Type SystemWe begin with the static part of the C data model, the type system, which describes the values that data may have. We then discuss the dynamics of the C data model, that is, the operations that may be performed on data. In C, there is an innite set of types, any of which could be the type associated with a particular variable. These types, and the rules by which they are constructed, form the type system of C. The type system contains basic types such as integers, and a collection of type-formation rules with which we can construct progressively more complex types from types we already know. The basic types of C are 1. 2. 3. 4. Characters (char, signed char, unsigned char) Integers (int, short int, long int, unsigned) Floating-point numbers (float, double, long double) Enumerations (enum)

14


Integers and oating-point numbers are considered to be arithmetic types. The type-formation rules assume that we already have some types, which could be basic types or other types that we have already constructed using these rules. Here are some examples of the type formation rules in C: 1. Array types. We can form an array whose elements are type T with the declaration T A[n] This statement declares an array A of n elements, each of type T . In C, array subscripts begin at 0, so the rst element is A[0] and the last element is A[n1]. Arrays can be constructed from characters, arithmetic types, pointers, structures, unions, or other arrays.Members of a structure

2.

Structure types. In C, a structure is a grouping of variables called members or elds. Within a structure dierent members can have dierent types, but each member must have elements of a single type. If T1 , T2 , . . . , Tn are types and M1 , M2 , . . . , Mn are member names, then the declaration struct S { T1 M1 ; T2 M2 ; Tn Mn ; } denes a structure whose tag (i.e., the name of its type) is S and that has n members. The ith member has the name Mi and a value of type Ti , for i = 1, 2, . . . , n. Example 1.1 is an illustration of a structure. This structure has tag CELL and two members. The rst member has name element and has integer type. The second has name next and its type is a pointer to a structure of the same type. The structure tag S is optional, but it provides a convenient shorthand for referring to the type in later declarations. For example, the declaration struct S myRecord; denes the variable myRecord to be a structure of type S.

3.

Union types. A union type allows a variable to have dierent types at dierent times during the execution of a program. The declaration union { T1 M1 ; T2 M2 ; Tn Mn ; } x; denes a variable x that can hold a value of any of the types T1 , T2 , . . . , Tn . The member names M1 , M2 , . . . , Mn help indicate which type the value of x should be regarded as being. That is, x.Mi refers to the value of x treated as a value of type Ti .

SEC. 1.4

THE C DATA MODEL

15

4.

Pointer types. C is distinctive for its reliance on pointers. A variable of type pointer contains the address of a region of storage. We can access the value of another variable indirectly through a pointer. The declaration T *p; denes the variable p to be a pointer to a variable of type T . Thus p names a box of type pointer to T and the value in box p is a pointer. We often draw the value of p as an arrow, rather than as an object of type T itself, as shown in Fig. 1.11. What really appears in the box named p is the address, or location, at which an object of type T is stored in the computer. Consider the declaration int x, *p; In C, the unary operator & is used to obtain the address of an object, so the statement p = &x; assigns the address of x to p; that is, it makes p point to x. The unary operator * applied to p fetches the value of the box pointed to by p, so the statement y = *p; assigns to y the contents of whatever box p points to. If y is a variable of type int, then p = &x; y = *p; is equivalent to the assignment y = x;

p

Object of type T

Fig. 1.11. Variable p is of type pointer to T .

3

Example 1.4. C has the typedef construct to create synonyms for type names.The declaration typedef int Distance;

16


typedef int type1[10]; typedef type1 *type2; typedef struct { int field1; type2 field2; } type3; typedef type3 type4[5];Fig. 1.12. Some C typedef declarations.

0

field1 field2

3

9 (a)

(b) field1 field2 1 3 (c)

0

2

3

field1 field2 7 (d)

4

Fig. 1.13. Visualization of type declarations in Fig. 1.12.

SEC. 1.4

THE C DATA MODEL

17

Types, Names, Variables, and IdentiersA number of terms associated with data objects have dierent meanings but are easy to confuse. First, a type describes a shape for data objects. In C, a new name T may be dened for an existing type using the typedef construct typedef TType descriptor

Here the type descriptor is an expression that tells us the shape of objects of the type T . A typedef declaration for T does not actually create any objects of that type. An object of type T is created by a declaration of the form T x; Here, x is an identier, or variable name. Possibly, x is static (not local to any function), in which case the box for x is created when the program starts. If x is not static, then it is local to some function F . When F is called, a box whose name is the x associated with this call to F is created. More precisely, the name of the box is x, but only uses of the identier x during the execution of this call to F refer to this box. As mentioned in the text, there can be many boxes each of whose name involves the identier x, since F may be recursive. There may even be other functions that also have used identier x to name one of their variables. Moreover, names are more general than identiers, since there are many kinds of expressions that could be used to name boxes. For instance, we mentioned that *p could be the name of an object pointed to by pointer p, and other names could be complex expressions such as (*p).f[2] or p->f[2]. The last two expressions are equivalent and refer to the array element number 2 of the eld f of the structure pointed to by pointer p.

allows the name Distance to be used in place of the type int. Consider the four typedef declarations in Fig. 1.12. In the conventional view of data in C, an object of type type1 is an array with 10 slots, each holding an integer, as suggested in Fig. 1.13(a). Likewise, objects of type type2 are pointers to such arrays, as in Fig. 1.13(b). Structures, like those of type3, are visualized as in Fig. 1.13(c), with a slot for each eld; note that the name of the eld (e.g., field1) does not actually appear with the value of the eld. Finally, objects of the array type type4 would have ve slots, each of which holds an object of type type3, a structure we suggest in Fig. 1.13(d). 3

FunctionsFunctions also have associated types, even though we do not associate boxes or values with functions, as we do with program variables. For any list of types T1 , T2 , . . . , Tn , we can dene a function with n parameters consisting of those types, in order. This list of types followed by the type of the value returned by the function (the return-value) is the type of the function. If the function has no return value, its type is void. In general, we can build types by applying the type-construction rules arbitrarily, but there are a number of constraints. For example, we cannot construct an

Return-value

18


array of functions, although we can construct an array of pointers to functions. The complete set of rules for constructing types in C can be found in the ANSI standard.

Operations in the C Data ModelThe operations on data in the C data model can be divided into three categories: 1. 2. 3. Operations that create or destroy a data object. Operations that access and modify parts of a data object. Operations that combine the values of data objects to form a new value for a data object.

Data Object Creation and DisposalFor data creation, C provides several rudimentary mechanisms. When a function is called, boxes for each of its local arguments (parameters) are created; these serve to hold the values of the arguments. Another mechanism for data creation is through the use of the library routine malloc(n), which returns a pointer to n consecutive character positions of unused storage that can be used to store data by the caller of malloc. Data objects can then be created in this storage region. C has the analogous methods for destroying data objects. Local parameters of a function call cease to exist when the function returns. The routine free releases the storage created by malloc. In particular, the eect of calling free(p) is to release the storage area pointed to by p. It is disastrous to use free to get rid of an object that was not created by calling malloc.

Data Access and ModicationC has mechanisms for accessing the components of objects. It uses a[i] to access the ith element of array a, x.m to access member m of a structure named x, and *p to access the object pointed to by pointer p. Modifying, or writing, values in C is done principally by the assignment operators, which allow us to change the value of an object.

3

Example 1.5. If a is a variable of type type4 dened in Example 1.4, thena[0].(*field2[3]) = 99; assigns the value 99 to the fourth element of the array pointed to by field2 in the structure that is the rst element of the array a. 3

Data CombinationC has a rich set of operators for manipulating and combining values. The principal operators are 1. Arithmetic operators. C provides:

SEC. 1.4

THE C DATA MODEL

19

a) b) c) d)

The customary binary arithmetic operators +, , , / on integers and oating-point numbers. Integer division truncates (4/3 yields 1). There are the unary + and operators. The modulus operator i % j produces the remainder when i is divided by j. The increment and decrement operators, ++ and --, applied to a single integer variable add or subtract 1 from that variable, respectively. These operators can appear before or after their operand, depending on whether we wish the value of the expression to be computed before or after the change in the variables value.

2.

Logical operators. C does not have a Boolean type; it uses zero to represent the logical value false, and nonzero to represent true.5 C uses: a) && to represent AND. For example, the expression x && y returns 1 if both operands are nonzero, 0 otherwise. However, y is not evaluated if x has the value 0. || represents OR. The expression x || y returns 1 if either x or y is nonzero, and returns 0 otherwise. However, y is not evaluated if x is nonzero. The unary negation operator !x returns 0 if x is nonzero and returns 1 if x = 0. The conditional operator is a ternary (3-argument) operator represented by a question mark and a colon. The expression x?y:z returns the value of y if x is true (i.e., it is nonzero) and returns the value of z if x is false (i.e., 0).

b)

c) d)

3.

Comparison operators. The result of applying one of the six relational comparison operators (==, !=, , =) to integers or oating point numbers is 0 if the relation is false and 1 otherwise. Bitwise manipulation operators. C provides several useful bitwise logical operators, which treat integers as if they were bits strings equal to their binary representations. These include & for bitwise AND, | for bitwise inclusive-or, ^ for bitwise exclusive-or, > for right shift, and a tilde for left shift. Assignment operators. C uses = as the assignment operator. In addition, C allows expressions such as x = x + y; to be written in a shortened form x += y; Similar forms apply to the other binary arithmetic operators.

4.

5.

5

We shall use TRUE and FALSE as dened constants 1 and 0, respectively, to represent Boolean values; see Section 1.6.

20


6.

Coercion operators. Coercion is the process of converting a value of one type into an equivalent value of another type. For example, if x is a oating-point number and i is an integer, then x = i causes the integer value of i to be converted to a oating-point number with the same value. Here, the coercion operator is not shown explicitly, but the C compiler can deduce that conversion from integer to oat is necessary and inserts the required step.

EXERCISES1.4.1: Explain the dierence between an identier of a C program and a name (for a box or data object). 1.4.2: Give an example of a C data object that has more than one name. 1.4.3: If you are familiar with another programming language besides C, describe its type system and operations. 3 3 3 3

1.5

Algorithms and the Design of ProgramsThe study of data models, their properties, and their appropriate use is one pillar of computer science. A second, equally important pillar is the study of algorithms and their associated data structures. We need to know the best ways to perform common tasks, and we need to learn the principal techniques for designing good algorithms. Further, we need to understand how the use of data structures and algorithms ts into the process of creating useful programs. The themes of data models, algorithms, data structures, and their implementation in programs are interdependent, and each appears many times throughout the book. In this section, we shall mention some generalities regarding the design and implementation of programs.

The Creation of SoftwareIn a programming class, when you were given a programming problem, you probably designed an algorithm to solve the problem, implemented the algorithm in some language, compiled and ran the program on some sample data, and then submitted the program to be graded. In a commercial setting, programs are written under rather dierent circumstances. Algorithms, at least those simple enough and common enough to have names, are usually small parts of a complete program. Programs, in turn, are usually components of a larger system, involving hardware as well as software. Both the programs and the complete systems in which they are embedded are developed by teams of programmers and engineers; there could be hundreds of people on such a team. The development of a software system typically spans several phases. Although these phases may supercially bear some resemblance to the steps involved in solving the classroom programming assignment, most of the eort in building a software system to solve a given problem is not concerned with programming. Here is an idealized scenario.

Software development process

SEC. 1.5

ALGORITHMS AND THE DESIGN OF PROGRAMS

21

Prototyping

Problem denition and specication. The hardest, but most important, part of the task of creating a software system is dening what the problem really is and then specifying what is needed to solve it. Usually, problem denition begins by analyzing the users requirements, but these requirements are often imprecise and hard to write down. The system architect may have to consult with the future users of the system and iterate the specication, until both the specier and the users are satised that the specication denes and solves the problem at hand. In the specication stage, it may be helpful to build a simple prototype or model of the nal system, to gain insight into its behavior and intended use. Data modeling is also an important tool in the problem-denition phase. Design. Once the specication is complete, a high-level design of the system is created, with the major components identied. A document outlining the high-level design is prepared, and performance requirements for the system may be included. More detailed specications of some of the major components may also be included during this phase. A cost-eective design often calls for the reuse or modication of previously constructed components. Various software methodologies such as objectoriented technology facilitate the reuse of components. Implementation. Once the design is xed, implementation of the components can proceed. Many of the algorithms discussed in this book are useful in the implementation of new components. Once a component has been implemented, it is subject to a series of tests to make sure that it behaves as specied. Integration and system testing. When the components have been implemented and individually tested, the entire system is assembled and tested. Installation and eld testing. Once the developer is satised that the system will work to the customers satisfaction, the system is installed on the customers premises and the nal eld testing takes place. Maintenance. At this point, we might think that the bulk of the work has been done. Maintenance remains, however, and in many situations maintenance can account for more than half the cost of system development. Maintenance may involve modifying components to eliminate unforeseen side-eects, to correct or improve system performance, or to add features. Because maintenance is such an important part of software systems design, it is important to write programs that are correct, rugged, ecient, modiable, and whenever possible portable from one computer to another. It is very important to catch errors as early as possible, preferably during the problem-denition phase. At each successive phase, the cost of xing a design error or programming bug rises greatly. Independent reviews of requirements and designs are benecial in reducing downstream errors.

Software reuse

Programming StyleAn individual programmer can ease the maintenance burden greatly by writing programs that others can read and modify readily. Good programming style comes only with practice, and we recommend that you begin at once to try writing programs that are easy for others to understand. There is no magic formula that will guarantee readable programs, but there are several useful rules of thumb:

22


1. 2. 3.

Modularize a program into coherent pieces. Lay out a program so that its structure is clear. Write intelligent comments to explain a program. Describe, clearly and precisely, the underlying data models, the data structures selected to represent them, and the operation performed by each procedure. When describing a procedure, state the assumptions made about its inputs, and tell how the output relates to the input. Use meaningful names for procedures and variables. Avoid explicit constants whenever possible. For example, do not use 7 for the number of dwarfs. Rather, use a dened constant such as NumberOfDwarfs, so that you can easily change all uses of this constant to 8, if you decide to add another dwarf. Avoid the use of global variables that is, variables dened for the program as a whole unless the data represented by that variable really is used by most of the procedures of the program.

4.Dened constants

5.

Global variables

6.

Test suite

Another good programming practice is to maintain a test suite of inputs that will try to exercise every line of code while you are developing a program. Whenever new features are added to the program, the test suite can be run to make sure that the new program has the same behavior as the old on previously working inputs.

3 3 3 3

1.6

Some C Conventions Used Throughout the BookThere are several denitions and conventions that we shall nd useful as we illustrate concepts with C programs. Some of these are common conventions found in the standard header le stdio.h, while others are dened specially for the purposes of this book and must be included with any C program that uses them.

NULL

1.

The identier NULL is a value that may appear anywhere a pointer can appear, but it is not a value that can ever point to anything. Thus, NULL in a eld such as next in the cells of Example 1.1 can be used to indicate the end of a list. We shall see that NULL has a number of similar uses in other data structures. NULL is properly dened in stdio.h. The identiers TRUE and FALSE are dened by #define TRUE 1 #define FALSE 0

2.

TRUE and FALSE BOOLEAN 3.

Thus, TRUE can be used anywhere a condition with logical value true is wanted, and FALSE can be used for a condition whose value is false. The type BOOLEAN is dened as typedef int BOOLEAN; We use BOOLEAN whenever we want to stress the fact that we are interested in the logical rather than the numeric value of an expression.

SEC. 1.7

SUMMARY OF CHAPTER 1

23

EOF

4.

The identier EOF is a value that is returned by le-reading functions such as getchar() when there are no more bytes left to be read from the le. An appropriate value for EOF is provided in stdio.h. We shall dene a macro that generates declarations of cells of the kind used in Example 1.1. An appropriate denition appears in Fig. 1.14. It declares cells with two elds: an element eld whose type is given by the parameter Type and a next eld to point to a cell with this structure. The macro provides two external denitions: CellName is the name of structures of this type, and ListName is a name for the type of pointers to these cells. #define DefCell(EltType, CellType, ListType) typedef struct CellType *ListType; struct CellType { EltType element; ListType next; }Fig. 1.14. A macro for dening list cells.

5.Cell denition

\ \ \ \ \

3

Example 1.6. We can dene cells of the type used in Example 1.1 by the macrouse DefCell(int, CELL, LIST); The macro then expands into typedef struct CELL, LIST; struct CELL { int element; LIST next; } As a consequence, we can use CELL as the type of integer cells, and we can use LIST as the type of pointers to these cells. For example, CELL c; LIST L; denes c to be a cell and L to be a pointer to a cell. Note that the representation of a list of cells is normally a pointer to the rst cell on the list, or NULL if the list is empty. 3

3 3 3 3

1.7

Summary of Chapter 1At this point you should be aware of the following concepts: 3 How data models, data structures, and algorithms are used to solve problems

24


3 3 3 3 3 3 3 3

The distinction between a list as a data model and a linked list as a data structure The presence of some kind of data model in every software system, be it a programming language, an operating system, or an application program The key elements of the data model supported by the programming language C The major steps in the development of a large software system

1.8

Bibliographic Notes for Chapter 1Kernighan and Ritchie [1988] is the classic reference for the C programming language. Roberts [1994] is a good introduction to programming using C. Stroustrup [1991] has created an object-oriented extension of C called C++ that is now widely used for implementing systems. Sethi [1989] provides an introduction to the data models of several major programming languages. Brooks [1974] eloquently describes the technical and managerial diculties in developing large software systems. Kernighan and Plauger [1978] provide sound advice for improving your programming style. American National Standards Institute (ANSI) [1990]. Programming Language C, American National Standards Institute, New York. Brooks, F. P. [1974]. The Mythical Man Month, Addison-Wesley, Reading, Mass. Kernighan, B. W., and P. J. Plauger [1978]. The Elements of Programming Style, second edition, McGraw-Hill, New York. Kernighan, B. W., and D. M. Ritchie [1988]. The C Programming Language, second edition, Prentice-Hall, Englewood Clis, New Jersey. Roberts, E. S. [1994]. A C-Based Introduction to Computer Science, AddisonWesley, Reading, Mass. Sethi, R. [1989]. Programming Languages: Concepts and Constructs, AddisonWesley, Reading, Mass. Stroustrup, B. [1991]. The C++ Programming Language, second edition, AddisonWesley, Reading, Mass.

CHAPTER

2 Iteration, Induction, and RecursionThe power of computers comes from their ability to execute the same task, or dierent versions of the same task, repeatedly. In computing, the theme of iteration is met in a number of guises. Many concepts in data models, such as lists, are forms of repetition, as A list either is empty or is one element followed by another, then another, and so on. Programs and algorithms use iteration to perform repetitive jobs without requiring a large number of similar steps to be specied individually, as Do the next step 1000 times. Programming languages use looping constructs, like the while- and for-statements of C, to implement iterative algorithms. Closely related to repetition is recursion, a technique in which a concept is dened, directly or indirectly, in terms of itself. For example, we could have dened a list by saying A list either is empty or is an element followed by a list. Recursion is supported by many programming languages. In C, a function F can call itself, either directly from within the body of F itself, or indirectly by calling some other function, which calls another, and another, and so on, until nally some function in the sequence calls F . Another important idea, induction, is closely related to recursion and is used in many mathematical proofs. Iteration, induction, and recursion are fundamental concepts that appear in many forms in data models, data structures, and algorithms. The following list gives some examples of uses of these concepts; each will be covered in some detail in this book. 1. 2. Iterative techniques. The simplest way to perform a sequence of operations repeatedly is to use an iterative construct such as the for-statement of C. Recursive programming. C and many other languages permit recursive functions, which call themselves either directly or indirectly. Often, beginning programmers are more secure writing iterative programs than recursive ones, but an important goal of this book is to accustom the reader to thinking and programming recursively, when appropriate. Recursive programs can be simpler to write, analyze, and understand. 25

3 3 3 3

26

ITERATION, INDUCTION, AND RECURSION

Notation: The Summation and Product SymbolsAn oversized Greek capital letter sigma is often used to denote a summation, as in n i. This particular expression represents the sum of the integers from 1 to i=1 n; that is, it stands for the sum 1 + 2 + 3 + + n. More generally, we can sum any function f (i) of the summation index i. (Of course, the index could be some b symbol other than i.) The expression i=a f (i) stands for f (a) + f (a + 1) + f (a + 2) + + f (b) For example, j=2 j 2 stands for the sum 4 + 9 + 16 + + m2 . Here, the function f is squaring, and we used index j instead of i. b As a special case, if b < a, then there are no terms in the sum i=a f (i), and the value of the expression, by convention, is taken to be 0. If b = a, then there is a exactly one term, that for i = a. Thus, the value of the sum i=a f (i) is just f (a). The analogous notation for products uses an oversized capital pi. The expresb sion i=a f (i) stands for the product f (a) f (a + 1) f (a + 2) f (b); if b < a, the product is taken to be 1.m

3.

Basis Inductive step

Proofs by induction. An important technique for showing that a statement is true is proof by induction. We shall cover inductive proofs extensively, starting in Section 2.3. The following is the simplest form of an inductive proof. We begin with a statement S(n) involving a variable n; we wish to prove that S(n) is true. We prove S(n) by rst proving a basis, that is, the statement S(n) for a particular value of n. For example, we could let n = 0 and prove the statement S(0). Second, we must prove an inductive step, in which we prove that the statement S, for one value of its argument, follows from the same statement S for the previous values of its argument; that is, S(n) implies S(n + 1) for all n 0. For example, S(n) might be the familiar summation formulan

i = n(n + 1)/2i=1

(2.1)

which says that the sum of the integers from 1 to n equals n(n+1)/2. The basis could be S(1) that is, Equation (2.1) with 1 in place of n which is just the equality 1 = 1 2/2. The inductive step is to show that n i = n(n + 1)/2 i=1 n+1 implies that i=1 i = (n + 1)(n + 2)/2; the former is S(n), which is Equation (2.1) itself, while the latter is S(n + 1), which is Equation (2.1) with n + 1 replacing n everywhere n appears. Section 2.3 will show us how to construct proofs such as this. 4. Proofs of program correctness. In computer science, we often wish to prove, formally or informally, that a statement S(n) about a program is true. The statement S(n) might, for example, describe what is true on the nth iteration of some loop or what is true for the nth recursive call to some function. Proofs of this sort are generally inductive proofs. Inductive denitions. Many important concepts of computer science, especially those involving data models, are best dened by an induction in which we give

5.

SEC. 2.2

ITERATION

27

a basis rule dening the simplest example or examples of the concept, and an inductive rule or rules, where we build larger instances of the concept from smaller ones. For instance, we noted that a list can be dened by a basis rule (an empty list is a list) together with an inductive rule (an element followed by a list is also a list). 6. Analysis of running time. An important criterion for the goodness of an algorithm is how long it takes to run on inputs of various sizes (its running time). When the algorithm involves recursion, we use a formula called a recurrence equation, which is an inductive denition that predicts how long the algorithm takes to run on inputs of dierent sizes.

Each of these subjects, except the last, is introduced in this chapter; the running time of a program is the topic of Chapter 3. 3 3 3 3

2.1

What This Chapter Is AboutIn this chapter we meet the following major concepts. 3 3 3 3 3 Iterative programming (Section 2.2) Inductive proofs (Sections 2.3 and 2.4) Inductive denitions (Section 2.6) Recursive programming (Sections 2.7 and 2.8) Proving the correctness of a program (Sections 2.5 and 2.9)

In addition, we spotlight, through examples of these concepts, several interesting and important ideas from computer science. Among these are 3 3 3 3 Sorting algorithms, including selection sort (Section 2.2) and merge sort (Section 2.8) Parity checking and detection of errors in data (Section 2.3) Arithmetic expressions and their transformation using algebraic laws (Sections 2.4 and 2.6) Balanced parentheses (Section 2.6)

3 3 3 3

2.2

IterationEach beginning programmer learns to use iteration, employing some kind of looping construct such as the for- or while-statement of C. In this section, we present an example of an iterative algorithm, called selection sort. In Section 2.5 we shall prove by induction that this algorithm does indeed sort, and we shall analyze its running time in Section 3.6. In Section 2.8, we shall show how recursion can help us devise a more ecient sorting algorithm using a technique called divide and conquer.

28


Common Themes: Self-Denition and Basis-InductionAs you study this chapter, you should be alert to two themes that run through the various concepts. The rst is self-denition, in which a concept is dened, or built, in terms of itself. For example, we mentioned that a list can be dened as being empty or as being an element followed by a list. The second theme is basis-induction. Recursive functions usually have some sort of test for a basis case where no recursive calls are made and an inductive case where one or more recursive calls are made. Inductive proofs are well known to consist of a basis and an inductive step, as do inductive denitions. This basisinduction pairing is so important that these words are highlighted in the text to introduce each occurrence of a basis case or an inductive step. There is no paradox or circularity involved in properly used self-denition, because the self-dened subparts are always smaller than the object being dened. Further, after a nite number of steps to smaller parts, we arrive at the basis case, at which the self-denition ends. For example, a list L is built from an element and a list that is one element shorter than L. When we reach a list with zero elements, we have the basis case of the denition of a list: The empty list is a list. As another example, if a recursive function works, the arguments of the call must, in some sense, be smaller than the arguments of the calling copy of the function. Moreover, after some number of recursive calls, we must get to arguments that are so small that the function does not make any more recursive calls.

SortingTo sort a list of n elements we need to permute the elements of the list so that they appear in nondecreasing order.

3

Example 2.1. Suppose we are given the list of integers (3, 1, 4, 1, 5, 9, 2, 6, 5).We sort this list by permuting it into the sequence (1, 1, 2, 3, 4, 5, 5, 6, 9). Note that sorting not only orders the values so that each is either less than or equal to the one that follows, but it also preserves the number of occurrences of each value. Thus, the sorted list has two 1s, two 5s, and one each of the numbers that appear once in the original list. 3 We can sort a list of elements of any type as long as the elements have a lessthan order dened on them, which we usually represent by the symbol 0, the rst i 1 characters of the two strings agree, but the ith character of the rst string is less than the ith character of the second string. That is, cj = dj for j = 1, 2, . . . , i 1, and ci < di . According to this rule, ball < base, because the two words rst dier at position 3, and at that position ball has an l, which precedes the character s found in the third position of base.

2.

1.Permutation

List (b1 , b2 , . . . , bn ) is sorted. List (b1 , b2 , . . . , bn ) is a permutation of the original list. That is, each value appears in list (a1 , a2 , . . . , an ) exactly as many times as that value appears in list (b1 , b2 , . . . , bn ).

2.

A sorting algorithm takes as input an arbitrary list and produces as output a sorted list that is a permutation of the input.

3

Example 2.2. Consider the list of wordsbase, ball, mound, bat, glove, batter Given this input, and using lexicographic order, a sorting algorithm would produce this output: ball, base, bat, batter, glove, mound. 3

Selection Sort: An Iterative Sorting AlgorithmSuppose we have an array A of n integers that we wish to sort into nondecreasing

30


Convention Regarding Names and ValuesWe can think of a variable as a box with a name and a value. When we refer to a variable, such as abc, we use the constant-width, or computer font for its name, as we did in this sentence. When we refer to the value of the variable abc, we shall use italics, as abc. To summarize, abc refers to the name of the box, and abc to its contents.

order. We may do so by iterating a step in which a smallest element1 not yet part of the sorted portion of the array is found and exchanged with the element in the rst position of the unsorted part of the array. In the rst iteration, we nd (select) a smallest element among the values found in the full array A[0..n-1] and exchange it with A[0].2 In the second iteration, we nd a smallest element in A[1..n-1] and exchange it with A[1]. We continue these iterations. At the start of the i + 1st iteration, A[0..i-1] contains the i smallest elements in A sorted in nondecreasing order, and the remaining elements of the array are in no particular order. A picture of A just before the i + 1st iteration is shown in Fig. 2.1. sorted 0 i unsorted n1

Fig. 2.1. Picture of array just before the i + 1st iteration of selection sort.

In the i + 1st iteration, we nd a smallest element in A[i..n-1] and exchange it with A[i]. Thus, after the i + 1st iteration, A[0..i] contains the i + 1 smallest elements sorted in nondecreasing order. After the (n 1)st iteration, the entire array is sorted. A C function for selection sort is shown in Fig. 2.2. This function, whose name is SelectionSort, takes an array A as the rst argument. The second argument, n, is the length of array A. Lines (2) through (5) select a smallest element in the unsorted part of the array, A[i..n-1]. We begin by setting the value of index small to i in line (2). The forloop of lines (3) through (5) consider all higher indexes j in turn, and small is set to j if A[j] has a smaller value than any of the array elements in the range A[i..j-1]. As a result, we set the variable small to the index of the rst occurrence of the smallest element in A[i..n-1]. After choosing a value for the index small, we exchange the element in that position with the element in A[i], in lines (6) to (8). If small = i, the exchange is performed, but has no eect on the array. Notice that in order to swap two elements, we need a temporary place to store one of them. Thus, we move the value1

We say a smallest element rather than the smallest element because there may be several occurrences of the smallest value. If so, we shall be happy with any of those occurrences. To describe a range of elements within an array, we adopt a convention from the language Pascal. If A is an array, then A[i..j] denotes those elements of A with indexes from i to j, inclusive.

2

SEC. 2.2

ITERATION

31

(1)

(2) (3) (4) (5)

(6) (7) (8)

void SelectionSort(int A[], int n) { int i, j, small, temp; for (i = 0; i < n-1; i++) { /* set small to the index of the first occur- */ /* rence of the smallest element remaining */ small = i; for (j = i+1; j < n; j++) if (A[j] < A[small]) small = j; /* when we reach here, small is the index of */ /* the first smallest element in A[i..n-1]; */ /* we now exchange A[small] with A[i] */ temp = A[small]; A[small] = A[i]; A[i] = temp; } }Fig. 2.2. Iterative selection sort.

in A[small] to temp at line (6), move the value in A[i] to A[small] at line (7), and nally move the value originally in A[small] from temp to A[i] at line (8).

3

Example 2.3. Let us

foundations of computer science

Documents