thispageintentionallyleftblank · 2.3.1 simple substitution cipher 22 2.3.2 cryptanalysis of a...

30

Upload: others

Post on 17-Mar-2020

8 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Thispageintentionallyleftblank · 2.3.1 Simple Substitution Cipher 22 2.3.2 Cryptanalysis of a Simple Substitution 24 2.3.3 Definition of Secure 25 2.3.4 Double Transposition Cipher
Page 2: Thispageintentionallyleftblank · 2.3.1 Simple Substitution Cipher 22 2.3.2 Cryptanalysis of a Simple Substitution 24 2.3.3 Definition of Secure 25 2.3.4 Double Transposition Cipher

This page intentionally left blank

Page 3: Thispageintentionallyleftblank · 2.3.1 Simple Substitution Cipher 22 2.3.2 Cryptanalysis of a Simple Substitution 24 2.3.3 Definition of Secure 25 2.3.4 Double Transposition Cipher

INFORMATION SECURITY

Page 4: Thispageintentionallyleftblank · 2.3.1 Simple Substitution Cipher 22 2.3.2 Cryptanalysis of a Simple Substitution 24 2.3.3 Definition of Secure 25 2.3.4 Double Transposition Cipher

This page intentionally left blank

Page 5: Thispageintentionallyleftblank · 2.3.1 Simple Substitution Cipher 22 2.3.2 Cryptanalysis of a Simple Substitution 24 2.3.3 Definition of Secure 25 2.3.4 Double Transposition Cipher

INFORMATION SECURITY Principles and Practice

Second Edition

Mark Stamp San Jose State University

San Jose, CA

WILEY A JOHN WILEY & SONS, INC., PUBLICATION

Page 6: Thispageintentionallyleftblank · 2.3.1 Simple Substitution Cipher 22 2.3.2 Cryptanalysis of a Simple Substitution 24 2.3.3 Definition of Secure 25 2.3.4 Double Transposition Cipher

Copyright © 2011 by John Wiley & Sons, Inc. All rights reserved.

Published by John Wiley & Sons, Inc., Hoboken, New Jersey.

Published simultaneously in Canada.

No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as per-mitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 750-4470, or on the web at www.copyright.com. Requests to the Publisher for permission should be ad-dressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008, or online at http://www.wiley.com/go/permission.

Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or com-pleteness of the contents of this book and specifically disclaim any implied warranties of merchantabili-ty or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be li-able for any loss of profit or any other commercial damages, including but not limited to special, inci-dental, consequential, or other damages.

For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at (800) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002.

Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic formats. For more information about Wiley products, visit our web site at www.wiley.com.

Library of Congress Cataloging-in-Publication Data:

Stamp, Mark. Information security: principles and practice / Mark Stamp. — 2nd ed.

p. cm. Includes bibliographical references and index. ISBN 978-0-470-62639-9 (hardback)

1. Computer security. I. Title. QA76.9.A25S69 2011 005.8—dc22 2010045221

Printed in the United States of America.

10 9 8 7 6 5 4 3 2 1

Page 7: Thispageintentionallyleftblank · 2.3.1 Simple Substitution Cipher 22 2.3.2 Cryptanalysis of a Simple Substitution 24 2.3.3 Definition of Secure 25 2.3.4 Double Transposition Cipher

To Miles, Austin, and Melody, with love.

Page 8: Thispageintentionallyleftblank · 2.3.1 Simple Substitution Cipher 22 2.3.2 Cryptanalysis of a Simple Substitution 24 2.3.3 Definition of Secure 25 2.3.4 Double Transposition Cipher

This page intentionally left blank

Page 9: Thispageintentionallyleftblank · 2.3.1 Simple Substitution Cipher 22 2.3.2 Cryptanalysis of a Simple Substitution 24 2.3.3 Definition of Secure 25 2.3.4 Double Transposition Cipher

Contents

Preface xv

About The Author xix

Acknowledgments xxi

1 Introduction 1 1.1 The Cast of Characters 1 1.2 Alice's Online Bank 2

1.2.1 Confidentiality, Integrity, and Availability 2 1.2.2 Beyond CIA 3

1.3 About This Book 4 1.3.1 Cryptography 5 1.3.2 Access Control 6 1.3.3 Protocols 7 1.3.4 Software 8

1.4 The People Problem 8 1.5 Principles and Practice 9 1.6 Problems 10

1 Crypto 17

2 Crypto Basics 19 2.1 Introduction 19 2.2 How to Speak Crypto 20 2.3 Classic Crypto 22

2.3.1 Simple Substitution Cipher 22 2.3.2 Cryptanalysis of a Simple Substitution 24 2.3.3 Definition of Secure 25 2.3.4 Double Transposition Cipher 26 2.3.5 One-Time Pad 27 2.3.6 Project VENONA 31

vii

Page 10: Thispageintentionallyleftblank · 2.3.1 Simple Substitution Cipher 22 2.3.2 Cryptanalysis of a Simple Substitution 24 2.3.3 Definition of Secure 25 2.3.4 Double Transposition Cipher

CONTENTS

2.3.7 Codebook Cipher 32 2.3.8 Ciphers of the Election of 1876 35

2.4 Modern Crypto History 37 2.5 A Taxonomy of Cryptography 40 2.6 A Taxonomy of Cryptanalysis 41 2.7 Summary 42 2.8 Problems 43

3 Symmetric Key Crypto 51 3.1 Introduction 51 3.2 Stream Ciphers 52

3.2.1 A5/1 53 3.2.2 RC4 55

3.3 Block Ciphers 57 3.3.1 Feistel Cipher 57 3.3.2 DES 58 3.3.3 Triple DES 65 3.3.4 AES 67 3.3.5 Three More Block Ciphers 69 3.3.6 TEA 70 3.3.7 Block Cipher Modes 72

3.4 Integrity 76 3.5 Summary 78 3.6 Problems 79

4 Public Key Crypto 89 4.1 Introduction 89 4.2 Knapsack 91 4.3 RSA 95

4.3.1 Textbook RSA Example 97 4.3.2 Repeated Squaring 98 4.3.3 Speeding Up RSA 99

4.4 Diffie-Hellman 100 4.5 Elliptic Curve Cryptography 102

4.5.1 Elliptic Curve Math 103 4.5.2 ECC Diffie-Hellman 105 4.5.3 Realistic Elliptic Curve Example 106

4.6 Public Key Notation 107 4.7 Uses for Public Key Crypto 107

4.7.1 Confidentiality in the Real World 108 4.7.2 Signatures and Non-repudiation 108 4.7.3 Confidentiality and Non-repudiation 109

4.8 Public Key Infrastructure 112

Page 11: Thispageintentionallyleftblank · 2.3.1 Simple Substitution Cipher 22 2.3.2 Cryptanalysis of a Simple Substitution 24 2.3.3 Definition of Secure 25 2.3.4 Double Transposition Cipher

CONTENTS ix

4.9 Summary 114 4.10 Problems 115

5 Hash Functions++ 125 5.1 Introduction 125 5.2 What is a Cryptographic Hash Function? 126 5.3 The Birthday Problem 128 5.4 A Birthday Attack 129 5.5 Non-Cryptographic Hashes 130 5.6 Tiger Hash 132 5.7 HMAC 136 5.8 Uses for Hash Functions 139

5.8.1 Online Bids 139 5.8.2 Spam Reduction 140

5.9 Miscellaneous Crypto-Related Topics 141 5.9.1 Secret Sharing 142 5.9.2 Random Numbers 145 5.9.3 Information Hiding 148

5.10 Summary 153 5.11 Problems 153

6 Advanced Cryptanalysis 167 6.1 Introduction 167 6.2 Enigma 169

6.2.1 Enigma Cipher Machine 169 6.2.2 Enigma Keyspace 172 6.2.3 Rotors 174 6.2.4 Enigma Attack 176

6.3 RC4 as Used in WEP 179 6.3.1 RC4 Algorithm 180 6.3.2 RC4 Cryptanalytic Attack 181 6.3.3 Preventing Attacks on RC4 185

6.4 Linear and Differential Cryptanalysis 186 6.4.1 Quick Review of DES 186 6.4.2 Overview of Differential Cryptanalysis 187 6.4.3 Overview of Linear Cryptanalysis 190 6.4.4 Tiny DES 192 6.4.5 Differential Cryptanalysis of TDES 194 6.4.6 Linear Cryptanalysis of TDES 199 6.4.7 Implications Block Cipher Design 202

6.5 Lattice Reduction and the Knapsack 203 6.6 RSA Timing Attacks 210

6.6.1 A Simple Timing Attack 211

Page 12: Thispageintentionallyleftblank · 2.3.1 Simple Substitution Cipher 22 2.3.2 Cryptanalysis of a Simple Substitution 24 2.3.3 Definition of Secure 25 2.3.4 Double Transposition Cipher

x CONTENTS

6.6.2 Kocher's Timing Attack 214 6.7 Summary 218 6.8 Problems 218

II Access Control 227

7 Authentication 229 7.1 Introduction 229 7.2 Authentication Methods 230 7.3 Passwords 231

7.3.1 Keys Versus Passwords 232 7.3.2 Choosing Passwords 232 7.3.3 Attacking Systems via Passwords 234 7.3.4 Password Verification 235 7.3.5 Math of Password Cracking 237 7.3.6 Other Password Issues 240

7.4 Biometrics 242 7.4.1 Types of Errors 244 7.4.2 Biometrie Examples 244 7.4.3 Biometrie Error Rates 250 7.4.4 Biometrie Conclusions 250

7.5 Something You Have 251 7.6 Two-Factor Authentication 252 7.7 Single Sign-On and Web Cookies 252 7.8 Summary 254 7.9 Problems 254

8 Authorization 265 8.1 Introduction 265 8.2 A Brief History of Authorization 266

8.2.1 The Orange Book 266 8.2.2 The Common Criteria 269

8.3 Access Control Matrix 271 8.3.1 ACLs and Capabilities 272 8.3.2 Confused Deputy 273

8.4 Multilevel Security Models 274 8.4.1 Bell-LaPadula 276 8.4.2 Biba's Model 278

8.5 Compartments 279 8.6 Covert Channel 281 8.7 Inference Control 283 8.8 CAPTCHA 285

Page 13: Thispageintentionallyleftblank · 2.3.1 Simple Substitution Cipher 22 2.3.2 Cryptanalysis of a Simple Substitution 24 2.3.3 Definition of Secure 25 2.3.4 Double Transposition Cipher

CONTENTS xi

8.9 Firewalls 287 8.9.1 Packet Filter 288 8.9.2 Stateful Packet Filter 290 8.9.3 Application Proxy 291 8.9.4 Personal Firewall 293 8.9.5 Defense in Depth 293

8.10 Intrusion Detection Systems 294 8.10.1 Signature-Based IDS 296 8.10.2 Anomaly-Based IDS 297

8.11 Summary 301 8.12 Problems 302

III Protocols 311

9 Simple Authentication Protocols 313 9.1 Introduction 313 9.2 Simple Security Protocols 315 9.3 Authentication Protocols 317

9.3.1 Authentication Using Symmetric Keys 320 9.3.2 Authentication Using Public Keys 323 9.3.3 Session Keys 325 9.3.4 Perfect Forward Secrecy 327 9.3.5 Mutual Authentication, Session Key, and PFS . . . . 329 9.3.6 Timestamps 330

9.4 Authentication and TCP 332 9.5 Zero Knowledge Proofs 335 9.6 The Best Authentication Protocol? 339 9.7 Summary 339 9.8 Problems 340

10 Real-World Security Protocols 351 10.1 Introduction 351 10.2 SSH 352 10.3 SSL 353

10.3.1 SSL and the Man-in-the-Middle 356 10.3.2 SSL Connections 357 10.3.3 SSL Versus IPSec 358

10.4 IPSec 359 10.4.1 IKE Phase 1: Digital Signature 361 10.4.2 IKE Phase 1: Symmetric Key 363 10.4.3 IKE Phase 1: Public Key Encryption 364 10.4.4 IPSec Cookies 366

Page 14: Thispageintentionallyleftblank · 2.3.1 Simple Substitution Cipher 22 2.3.2 Cryptanalysis of a Simple Substitution 24 2.3.3 Definition of Secure 25 2.3.4 Double Transposition Cipher

xii CONTENTS

10.4.5 IKE Phase 1 Summary 366 10.4.6 IKE Phase 2 367 10.4.7 IPSec and IP Datagrams 368 10.4.8 Transport and Tunnel Modes 369 10.4.9 ESP and AH 370

10.5 Kerberos 372 10.5.1 Kerberized Login 374 10.5.2 Kerberos Ticket 375 10.5.3 Kerberos Security 376

10.6 WEP 377 10.6.1 WEP Authentication 377 10.6.2 WEP Encryption 378 10.6.3 WEP Non-Integrity 379 10.6.4 Other WEP Issues 379 10.6.5 WEP: The Bottom Line 380

10.7 GSM 381 10.7.1 GSM Architecture 381 10.7.2 GSM Security Architecture 383 10.7.3 GSM Authentication Protocol 385 10.7.4 GSM Security Flaws 386 10.7.5 GSM Conclusions 388 10.7.6 3GPP 389

10.8 Summary 389 10.9 Problems 390

IV Software 401

11 Software Flaws and Malware 403 11.1 Introduction 403 11.2 Software Flaws 404

11.2.1 Buffer Overflow 407 11.2.2 Incomplete Mediation 418 11.2.3 Race Conditions 419

11.3 Malware 421 11.3.1 Brain 422 11.3.2 Morris Worm 422 11.3.3 Code Red 424 11.3.4 SQL Slammer 425 11.3.5 Trojan Example 426 11.3.6 Malware Detection 427 11.3.7 The Future of Malware 429 11.3.8 Cyber Diseases Versus Biological Diseases 432

Page 15: Thispageintentionallyleftblank · 2.3.1 Simple Substitution Cipher 22 2.3.2 Cryptanalysis of a Simple Substitution 24 2.3.3 Definition of Secure 25 2.3.4 Double Transposition Cipher

CONTENTS xiii

11.4 Botnets 433 11.5 Miscellaneous Software-Based Attacks 433

11.5.1 Salami Attacks 434 11.5.2 Linearization Attacks 434 11.5.3 Time Bombs 436 11.5.4 Trusting Software 436

11.6 Summary 437 11.7 Problems 438

12 Insecurity in Software 447 12.1 Introduction 447 12.2 Software Reverse Engineering 448

12.2.1 Reversing Java Bytecode 450 12.2.2 SRE Example 451 12.2.3 Anti-Disassembly Techniques 455 12.2.4 Anti-Debugging Techniques 456 12.2.5 Software Tamper Resistance 457 12.2.6 Metamorphism 2.0 459

12.3 Digital Rights Management 460 12.3.1 What is DRM? 460 12.3.2 A Real-World DRM System 464 12.3.3 DRM for Streaming Media 467 12.3.4 DRM for a P2P Application 469 12.3.5 Enterprise DRM 470 12.3.6 DRM Failures 471 12.3.7 DRM Conclusions 472

12.4 Software Development 472 12.4.1 Open Versus Closed Source Software 473 12.4.2 Finding Flaws 476 12.4.3 Other Software Development Issues 477

12.5 Summary 481 12.6 Problems 481

13 Operating Systems and Security 491 13.1 Introduction 491 13.2 OS Security Functions 492

13.2.1 Separation 492 13.2.2 Memory Protection 492 13.2.3 Access Control 494

13.3 Trusted Operating System 495 13.3.1 MAC, DAC, and More 495 13.3.2 Trusted Path 497 13.3.3 Trusted Computing Base 498

Page 16: Thispageintentionallyleftblank · 2.3.1 Simple Substitution Cipher 22 2.3.2 Cryptanalysis of a Simple Substitution 24 2.3.3 Definition of Secure 25 2.3.4 Double Transposition Cipher

xiv CONTENTS

13.4 Next Generation Secure Computing Base 500 13.4.1 NGSCB Feature Groups 502 13.4.2 NGSCB Compelling Applications 503 13.4.3 Criticisms of NGSCB 504

13.5 Summary 506 13.6 Problems 506

Appendix 511 A-l Network Security Basics 511

A-l.l Introduction 511 A-1.2 The Protocol Stack 512 A-l.3 Application Layer 514 A-l.4 Transport Layer 516 A-l.5 Network Layer 519 A-1.6 Link Layer 521 A-l.7 Conclusions 523

A-2 Math Essentials 523 A-2.1 Introduction 523 A-2.2 Modular Arithmetic 524 A-2.3 Permutations 526 A-2.4 Probability 526 A-2.5 Linear Algebra 527 A-2.6 Conclusions 529

Annotated Bibliography 531

Index 572

Page 17: Thispageintentionallyleftblank · 2.3.1 Simple Substitution Cipher 22 2.3.2 Cryptanalysis of a Simple Substitution 24 2.3.3 Definition of Secure 25 2.3.4 Double Transposition Cipher

Preface

Please sir or madam won't you read my book? It took me years to write, won't you take a look?

— Lennon and McCartney

I hate black boxes. One of my goals in writing this book was to illuminate some of those black boxes that are so popular in information security books today. On the other hand, I don't want to bore you to death with trivial details (if that's what you want, go read some RFCs). As a result, I often ignore details that I deem irrelevant to the topic at hand. You can judge whether I've struck the proper balance between these two competing goals.

I've strived to keep the presentation moving along so as to cover a broad selection of topics. My goal is to cover each item in just enough detail so that you can appreciate the basic security issue at hand, while not getting bogged down in details. I've also attempted to regularly emphasize and reiterate the main points so that crucial information doesn't slip by below the radar screen.

Another goal of mine was to present the topic in a reasonably lively and interesting way. If any computing subject should be exciting and fun, it's information security. Security is happening now and it's in the news—it's clearly alive and kicking.

I've also tried to inject a little humor into the material. They say that humor is derived from pain, so judging by the quality of my jokes, I'd say that I've led a charmed life. In any case, most of the really bad jokes are in footnotes so they shouldn't be too distracting.

Some security textbooks offer a large dollop of dry useless theory. Reading one of those books is about as exciting as reading a calculus textbook. Other books offer a seemingly random collection of apparently unrelated facts, giv-ing the impression that security is not really a coherent subject at all. Then there are books that present the topic as a collection of high-level managerial platitudes. Finally, some texts focus on the human factors in security. While all of these approaches have their place, I believe that, first and foremost, a

xv

Page 18: Thispageintentionallyleftblank · 2.3.1 Simple Substitution Cipher 22 2.3.2 Cryptanalysis of a Simple Substitution 24 2.3.3 Definition of Secure 25 2.3.4 Double Transposition Cipher

XVI PREFACE

security engineer must have a solid understanding of the inherent strengths and weaknesses of the underlying technology.

Information security is a huge topic, and unlike more established fields, it's not clear what material should be included in a book like this, or how best to organize it. I've chosen to organize this book around the following four major themes:

• Cryptography • Access Control • Protocols • Software

In my usage, these themes are fairly elastic. For example, under the heading of access control I've included the traditional topics of authentica-tion and authorization, along with such nontraditional topics as firewalls and CAPTCHAs. The software theme is particularly flexible, and includes such diverse topics as secure software development, malware, software reverse en-gineering, and operating systems.

Although this book is focused on practical issues, I've tried to cover enough of the fundamental principles so that you will be prepared for further study in the field. In addition, I've strived to minimize the background re-quirements as much as possible. In particular, the mathematical formalism has been kept to a bare minimum (the Appendix contains a review of all necessary math topics). Despite this self-imposed limitation, I believe this book contains more substantive cryptography than most security books out there. The required computer science background is also minimal—an in-troductory computer organization course (or comparable experience) is more than sufficient. Some programming experience is assumed and a rudimentary knowledge of assembly language would be helpful in a couple of sections, but it's not mandatory. Networking basics arise in a few sections. The Appendix contains a brief overview of networking that provides more than sufficient background material.

If you are an information technology professional who's trying to learn more about security, I would suggest that you read the entire book. However, if you want to avoid the material that's most likely to slow you down and is not critical to the overall flow of the book, you can safely skip Section 4.5, all of Chapter 6 (although Section 6.6 is highly recommended), and Section 8.4.

If you are teaching a security class, you need to realize that this book has more material than can be covered in a one-semester course. The schedule that I generally follow in my undergraduate security class appears in Table 1. This schedule allows ample time to cover a few of the optional topics.

If the syllabus in Table 1 is too busy, you could cut Section 8.9 of Chap-ter 8 and some of the topics in Chapters 12 and 13. Of course, many other variations on the syllabus are possible.

Page 19: Thispageintentionallyleftblank · 2.3.1 Simple Substitution Cipher 22 2.3.2 Cryptanalysis of a Simple Substitution 24 2.3.3 Definition of Secure 25 2.3.4 Double Transposition Cipher

PREFACE XVll

Chapter 1. Introduction 2. Classic Cryptography 3. Symmetric Key Crypto 4. Public Key Crypto 5. Hash Functions++

6. Advanced Cryptanalysis 7. Authentication 8. Authorization

9. Authentication Protocols 10. Real-World Protocols 11. Software Flaws and Malware 12. Insecurity in Software 13. OS and Security Total

Hours 1 3 4 4 3

0 4 2

4 4 4 4 3 40

Comments All All Omit Section 3.3.5 Omit Section 4.5 Omit 5.6 Omit attack details in 5.7 Omit Section 5.9.1 Omit entire chapter All Omit 8.4.1 and 8.4.2 Omit 8.10 Omit 9.4 Omit either WEP or GSM All Omit 12.3 All, time permitting

Table 1: Suggested Syllabus

Security is not a spectator sport—doing a large number of homework problems is essential to learning the material in this book. Many topics are fleshed out in the problems and additional topics are often introduced. The bottom line is that the more problems you solve, the more you'll learn.

A security course based on this book is an ideal venue for individual or group projects. Chapter 6 is a good source for crypto projects, while the annotated bibliography provides a starting point to search for additional project topics. In addition, many homework problems lend themselves well to class discussions or in-class assignments (see, for example, Problem 19 in Chapter 10 or Problem 33 in Chapter 11).

The textbook website is at

http ://www.es.sj su.edu/~stamp/infosec/

where you'll find PowerPoint slides, all of the files mentioned in the home-work problems, errata, and so on. If I were teaching this class for the first time, I would particularly appreciate the PowerPoint slides, which have been thoroughly "battle tested" and improved over several iterations. In addi-tion, a solutions manual is available to instructors (sorry, students) from the publisher.

It is also worth noting how the Appendices fit in. Appendix A-l, Network Security Basics, is relevant to Sections 8.9 and 8.10 of Chapter 8 and also for

Page 20: Thispageintentionallyleftblank · 2.3.1 Simple Substitution Cipher 22 2.3.2 Cryptanalysis of a Simple Substitution 24 2.3.3 Definition of Secure 25 2.3.4 Double Transposition Cipher

XVlll PREFACE

all of Part III. Even if students have a solid foundation in networking, it's probably worthwhile to review this material, since networking terminology is not always consistent and the focus here is on security.

The Math Essentials of Appendix A-2 are assumed in various places throughout the text. Elementary modular arithmetic (Appendix A-2.2) arises in a few sections of Chapter 3 and Chapter 5, while some of the relatively advanced concepts are required in Chapter 4 and Section 9.5 of Chapter 9. I've found that the vast majority of my students need to brush up on modular arithmetic basics. It only takes about 20 to 30 minutes of class time to cover the material on modular arithmetic and that will be time well spent prior to diving into public key cryptography. Trust me.

Permutations, which are briefly discussed in Appendix A-2.3, are most prominent in Chapter 3, while elementary discrete probability (Appendix A-2.4) appears in several places. The elementary linear algebra in Appendix A-2.5 is only required in Section 6.5.

Just as any large and complex piece of software must have bugs, this book inevitably has errors. I would like to hear about any errors—large or small— that you find. I will maintain a reasonably up-to-date errata on the textbook website. Also, don't hesitate to provide any suggestions you might have for future editions of this book.

What's New for the Second Edition?

Cats right themseJves; books don't. — John Ay cock

One major change for this second edition is that the number and quality of the homework problems have both greatly increased. In addition to the new-and-improved homework problems, new topics have been added, some new background material has been included, virtually all of the existing material has been updated and clarified, and all known errors have been corrected. Examples of new topics include a practical RS A timing attack, a discussion of botnets, and coverage of security certification. Examples of added background material include a section on the Enigma cipher and coverage of the classic "orange book" view of security.

Information security is a rapidly evolving field and there have been some significant changes since the first edition of this book was published in 2005. Nevertheless, the basic structure of the book remains intact. I believe the organization and list of topics has held up well over the past five years. Consequently, the changes to the content for this second edition are more evolutionary than revolutionary.

Mark Stamp San Jose State University

Page 21: Thispageintentionallyleftblank · 2.3.1 Simple Substitution Cipher 22 2.3.2 Cryptanalysis of a Simple Substitution 24 2.3.3 Definition of Secure 25 2.3.4 Double Transposition Cipher

About The Author

I've got nearly 20 years of experience in information security, including ex-tensive work in industry and government. My work experience includes more than seven years at the National Security Agency followed by two years at a Silicon Valley startup company. While I can't say too much about my work at NSA, I can tell you that my job title was Cryptologie Mathematician. In industry I helped design and develop a digital rights management security product. This real-world work was sandwiched between academic jobs. While in academia, my research interests have included a wide variety of security topics.

When I returned to academia in 2002, it seemed to me that none of the available security textbooks had much connection with the real world. I felt that I could write an information security book that would fill this gap, while also containing information that would be useful to working IT professionals. Based on the feedback I've received, the first edition was apparently a success.

I believe that this second edition will prove even more valuable in its dual role as a textbook and as a resource for working professionals, but then I'm biased. I can say that many of my former students who are now at leading Silicon Valley technology companies tell me that the information they learned in my courses has been useful to them. And I certainly wish that a book like this had been available when I worked in industry, since my colleagues and I would have benefitted from it.

I do have a life outside of information security.1 My family includes my wife, Melody, and two wonderful sons, Austin (whose initials are AES), and Miles (whose initials are not DES, thanks to Melody). We enjoy the outdoors, with regular local trips involving such activities as bicycling, hiking, camping, and fishing. I also spend way too much time working on my fixer-upper house in the Santa Cruz mountains.

1Well, sort of...

xix

Page 22: Thispageintentionallyleftblank · 2.3.1 Simple Substitution Cipher 22 2.3.2 Cryptanalysis of a Simple Substitution 24 2.3.3 Definition of Secure 25 2.3.4 Double Transposition Cipher

This page intentionally left blank

Page 23: Thispageintentionallyleftblank · 2.3.1 Simple Substitution Cipher 22 2.3.2 Cryptanalysis of a Simple Substitution 24 2.3.3 Definition of Secure 25 2.3.4 Double Transposition Cipher

Acknowledgments

My work in information security began when I was in graduate school. I want to thank my thesis advisor, Clyde F. Martin, for introducing me to this fascinating subject.

In my seven years at NSA, I learned more about security than I could have learned in a lifetime anywhere else. From my time in industry, I want to thank Joe Pasqua and Paul Clarke for giving me the chance to work on a fascinating and challenging project.

The following San Jose State University students helped greatly with the first edition: Fiona Wong, Martina Simova, Deepali Holankar, Xufen Gao, Subha Rajagopalan, Neerja Bhatnager, Amit Mathur, Ali Hushyar, Smita Thaker, Puneet Mishra, Jianning Yang, Konstantin Skachkov, Jian Dai, Thomas Niki, Ikai Lan, Thu Nguyen, Samuel Reed, Yue Wang, David Stillion, Edward Yin, and Randy Fort.

Richard Low, a colleague here at SJSU, provided helpful feedback on an early version of the manuscript. David Blockus (God rest his soul) deserves special mention for providing detailed comments on each chapter at a partic-ularly critical juncture in the writing of the first edition.

For this second edition, many of my SJSU masters students "volunteered" to serve as proofreaders. The following students all contributed their time and energy to correct errors in the manuscript: Naidele Manjunath, Mausami Mungale, Deepti Kundu, Jianrui (Louis) Zhang, Abhishek Shah, Sushant Priyadarshi, Mahim Patel, Lin Huang, Eilbroun Benjamin, Neha Samant, Rashmi Muralidhar, Kenny Zhang, Jyotsna Krishnaswamy, Ronak Shah, Gauri Gokhale, Arnold Suvatne, Ashish Sharma, Ankit Patel, Annie Hii, Namrata Buddhadev, Sujandharan Venkatachalam, and Sathya Anandan. In addition, Piyush Upadhyay found several errors in the first edition.

Many other people made helpful comments and suggestions. Here, I would like to specifically thank Bob Harris (Penn State University) for the visual crypto example and exercise, and a very special thanks goes to John Trono (Saint Michael's College) for his many detailed comments and questions.

Undoubtedly, errors remain. Of course, all remaining flaws are my re-sponsibility alone.

xxi

Page 24: Thispageintentionallyleftblank · 2.3.1 Simple Substitution Cipher 22 2.3.2 Cryptanalysis of a Simple Substitution 24 2.3.3 Definition of Secure 25 2.3.4 Double Transposition Cipher

This page intentionally left blank

Page 25: Thispageintentionallyleftblank · 2.3.1 Simple Substitution Cipher 22 2.3.2 Cryptanalysis of a Simple Substitution 24 2.3.3 Definition of Secure 25 2.3.4 Double Transposition Cipher

Chapter 1

Introduction

"Begin a t the beginning, " the King said, very gravely, "and go on till you come to the end: then stop."

— Lewis Carroll, Alice in Wonderland

1.1 The Cast of Characters Following tradition, Alice and Bob, who are pictured in Figure 1.1, are the good guys. Occasionally we'll require additional good guys, such as Charlie and Dave.

Alice Bob

Figure 1.1: Alice and Bob.

Trudy, pictured in Figure 1.2, is a generic bad "guy" who is trying to attack the system in some way. Some authors employ a team of bad guys where the name implies the particular nefarious activity. In this usage, Trudy is an "intruder" and Eve is an "eavesdropper" and so on. To simplify things, we'll use Trudy as our all-purpose bad guy.1

xYou might be wondering why a picture of Tweedledee and Tweedledum is used to represent Trudy. After all, Trudy is typically a female name, so why two bad guys instead of one bad girl? One possible reason is that, occasionally, we need two bad guys, so it's convenient to have both Tweedledee and Tweedledum available. Another plausible

1

Page 26: Thispageintentionallyleftblank · 2.3.1 Simple Substitution Cipher 22 2.3.2 Cryptanalysis of a Simple Substitution 24 2.3.3 Definition of Secure 25 2.3.4 Double Transposition Cipher

2 INTRODUCTION

Figure 1.2: Trudy.

Alice, Bob, Trudy, and the rest of the gang need not be humans. For example, one of many possible scenarios would have Alice as a laptop, Bob a server, and Trudy a human.

1.2 Alice's Online Bank

Suppose that Alice starts an online banking business, appropriately named Alice's Online Bank,2 or AOB. What are Alice's information security con-cerns? If Bob is Alice's customer, what are his information security con-cerns? Are Bob's concerns the same as Alice's? If we look at AOB from Trudy's perspective, what security vulnerabilities might we see?

First, let's consider the traditional triumvirate of confidentiality, integrity, and availability, or CIA,3 in the context of Alice's Bank. Then we'll point out some of the many other possible security concerns.

1.2.1 Confidentiality, Integrity, and Availability

Confidentiality deals with preventing unauthorized reading of information. AOB probably wouldn't care much about the confidentiality of the informa-tion it deals with, except for the fact that its customers certainly do. For example, Bob doesn't want Trudy to know how much he has in his savings account. Alice's Bank would also face legal problems if it failed to protect the confidentiality of such information.

Integrity deals with preventing, or at least detecting, unauthorized "writ-ing" (i.e., changes to data). Alice's Bank must protect the integrity of account information to prevent Trudy from, say, increasing the balance in her account or changing the balance in Bob's account. Note that confidentiality and in-tegrity are not the same thing. For example, even if Trudy cannot read the data, she might be able to modify this unreadable data, which, if undetected,

explanation is that you never know who might be acting as "Trudy." While these would be good reasons for choosing the Tweedle brothers, the reality is that your easily amused author finds the picture, well, amusing.

2Not to be confused with "Alice's Restaurant" [135]. 3No, not that CIA.. .

Page 27: Thispageintentionallyleftblank · 2.3.1 Simple Substitution Cipher 22 2.3.2 Cryptanalysis of a Simple Substitution 24 2.3.3 Definition of Secure 25 2.3.4 Double Transposition Cipher

1.2 ALICE'S ONLINE BANK 3

would destroy its integrity. In this case, Trudy might not know what changes she had made to the data (since she can't read it), but she might not care— sometimes just causing trouble is good enough.

Denial of service, or DoS, attacks are a relatively recent concern. Such attacks try to reduce access to information. As a result of the rise in DoS attacks, data availability has become a fundamental issue in information secu-rity. Availability is an issue for both Alice's Bank and Bob—if AOB's website is unavailable, then Alice can't make money from customer transactions and Bob can't get his business done. Bob might then take his business elsewhere. If Trudy has a grudge against Alice, or if she just wants to be malicious, she might attempt a denial of service attack on Alice's Online Bank.

1.2.2 Beyond CIA

Confidentiality, integrity, and availability are only the beginning of the in-formation security story. Beginning at the beginning, consider the situation when customer Bob logs on to his computer. How does Bob's computer de-termine that "Bob" is really Bob and not Trudy? And when Bob logs into his account at Alice's Online Bank, how does AOB know that "Bob" is really Bob, and not Trudy? Although these two authentication problems appear to be similar on the surface, under the covers they are actually completely different.

Authentication on a standalone computer typically requires that Bob's password be verified. To do so securely, some clever techniques from the field of cryptography are required. On the other hand, authentication over a network is open to many kinds of attacks that are not usually relevant on a standalone computer. Potentially, the messages sent over a network can be viewed by Trudy. To make matters worse, Trudy might be able to intercept messages, alter messages, and insert messages of her own making. If so, Trudy can simply replay Bob's old messages in an effort to, say, convince AOB that she is really Bob. Since information security people are professional paranoids,4 we always assume the worst. In any case, authentication over a network requires careful attention to protocol, that is, the composition and ordering of the exchanged messages. Cryptography also has an important role to play in security protocols.

Once Bob has been authenticated by Alice's Bank, then Alice must en-force restrictions on Bob's actions. For example, Bob can't look at Charlie's account balance or install new accounting software on the AOB system. How-ever, Sam, the AOB system administrator, can install new accounting soft-ware. Enforcing such restrictions goes by the name of authorization. Note that authorization places restrictions on the actions of authenticated users.

4Rumor has it that the security people at Yahoo proudly carry the title of "Paranoids."

Page 28: Thispageintentionallyleftblank · 2.3.1 Simple Substitution Cipher 22 2.3.2 Cryptanalysis of a Simple Substitution 24 2.3.3 Definition of Secure 25 2.3.4 Double Transposition Cipher

4 INTRODUCTION

Since authentication and authorization both deal with issues of access to resources, we'll lump them together under the clever title of access control.

All of the information security mechanisms discussed so far are imple-mented in software. And, if you think about it, other than the hardware, what isn't software in a modern computing system? Today, software systems tend to be large, complex, and rife with bugs. A software bug is not just an annoyance, it is a potential security issue, since it may cause the system to misbehave. Of course, Trudy loves misbehavior.

What software flaws are security issues, and how are they exploited? How can AOB be sure that its software is behaving correctly? How can AOB's software developers reduce (or, ideally, eliminate) security flaws in their soft-ware? We'll examine these software development related questions (and much more) in Chapter 11.

Although bugs can (and do) give rise to security flaws, these problems are created unintentionally by well-meaning developers. On the other hand, some software is written with the intent of doing evil. Examples of such malicious software, or malware, includes the all-too-familiar computer viruses and worms that plague the Internet today. How do these nasty beasts do what they do, and what can Alice's Online Bank do to limit their damage? What can Trudy do to increase the nastiness of such pests? We'll also consider these and related questions in Chapter 11.

Of course, Bob has many software concerns, too. For example, when Bob enters his password on his computer, how does he know that his password has not been captured and sent to Trudy? If Bob conducts a transaction at www.alicesonlinebank.com, how does he know that the transaction he sees on his screen is the same transaction that actually goes to the bank? That is, how can Bob be confident that his software is behaving as it should, instead of as Trudy would like it to behave? We'll consider these questions as well.

When discussing software and security, we'll need to consider operating system, or OS, topics. Operating systems are themselves large and complex pieces of software and OSs are responsible for enforcing much of the security in any system. So, some basic knowledge of OSs is necessary to fully appre-ciate the challenges of information security. We'll also briefly consider the concept of a trusted operating system, that is, an operating system that we can actually have reasonable confidence is doing the right thing.

1.3 About This Book

Lampson [180] believes that real-world security boils down to the following.

• Specification/policy — What is the system supposed to do?

• Implementation/mechanism — How does it do it?

Page 29: Thispageintentionallyleftblank · 2.3.1 Simple Substitution Cipher 22 2.3.2 Cryptanalysis of a Simple Substitution 24 2.3.3 Definition of Secure 25 2.3.4 Double Transposition Cipher

1.3 ABOUT THIS BOOK 5

• Correctness/assurance — Does it really work?

Your humble author would humbly5 add a fourth category:

• Human nature — Can the system survive "clever" users?

The focus of this book is primarily on the implementation/mechanism front. Your fearless author believes this is appropriate, nay essential, for an intro-ductory course, since the strengths, weaknesses, and inherent limitations of the mechanisms directly affect all other aspects of security. In other words, without a reasonable understanding of the mechanisms, it is not possible to have an informed discussion of other security issues.

The material in this book is divided into four major parts. The first part deals with cryptography, while the next part covers access control. Part III is on protocols, while the final part deals with the vast and relatively ill-defined topic of software. Hopefully, the previous discussion of Alice's Online Bank6 has convinced you that these major topics are all relevant to real-world information security.

In the remainder of this chapter, we'll give a quick preview of each of these four major topics. Then the chapter concludes with a summary followed by some lovely homework problems.

1.3.1 Cryptography

Cryptography or "secret codes" are a fundamental information security tool. Cryptography has many uses, including providing confidentiality and in-tegrity, among other vital information security functions. We'll discuss cryp-tography in detail, since this is essential background for any sensible discus-sion of information security.

We'll begin our coverage of cryptography with a look at a handful of classic cipher systems. In addition to their obvious historic and entertainment value, these classic ciphers illustrate the fundamental principles that are employed in modern digital cipher systems, but in a more user-friendly format.

With this background, we'll be prepared to study modern cryptography. Symmetric key cryptography and public key cryptography both play major roles in information security, and we'll spend an entire chapter on each. We'll then turn our attention to hash functions, which are another fundamental se-curity tool. Hash functions are used in many different contexts in information security, some of which are surprising and not always intuitive.

Then we'll briefly consider a few special topics that are related to cryp-tography. For example, we'll discuss information hiding, where the goal is for Alice and Bob to communicate without Trudy even knowing that any

5This sentence is brought to you by the Department of Redundancy Department. 6You did read that, right?

Page 30: Thispageintentionallyleftblank · 2.3.1 Simple Substitution Cipher 22 2.3.2 Cryptanalysis of a Simple Substitution 24 2.3.3 Definition of Secure 25 2.3.4 Double Transposition Cipher

6 INTRODUCTION

information has been passed. This is closely related to the concept of digital watermarking, which we also cover briefly.

The final chapter on cryptography deals with cryptanalysis, that is, the methods used to break cipher systems. Although this is relatively technical and specialized information, understanding these attack methods makes clear many of the design principles behind modern cryptographic systems.

1.3.2 Access Control

As mentioned above, access control deals with authentication and authoriza-tion. In the area of authentication, we'll consider the many issues related to passwords. Passwords are the most often used form of authentication today, but this is primarily because passwords are cheap, and definitely not because they are the most secure option.7

We'll consider how to securely store passwords. Then we'll delve into the issues surrounding secure password selection. Although it is possible to select reasonably strong passwords that are relatively easy to remember, it's surprisingly difficult to enforce such policies on clever users. In any case, weak passwords present a major security vulnerability in most systems.

The alternatives to passwords include biometrics and smartcards. We'll consider some of the security benefits of these alternate forms of authentica-tion. In particular, we'll discuss the details of several biometrie authentication methods.

Authorization deals with restrictions placed on authenticated users. Once Alice's Bank is convinced that Bob is really Bob, it must enforce restrictions on Bob's actions. The two classic methods for enforcing such restrictions are so-called access control lists8 and capabilities. We'll look at the plusses and minuses of each of these methods.

Authorization leads naturally to a few relatively specialized topics. We'll discuss multilevel security (and the related topic of compartments). For ex-ample, the United States government and military has TOP SECRET and SECRET information—some users can see both types of information, while other users can only see the SECRET information, and some can't view ei-ther. If both types of information are stored on a single system, how can we enforce such restrictions? This is a thorny authorization issue that has potential implications beyond classified military systems.

Multilevel security leads naturally into the rarified air of security mod-eling. The idea behind such modeling is to lay out the essential security requirements of a system. Ideally, by verifying a few simple properties we

7 If someone asks you why some weak security measure is used when better options are available, the correct answer is invariably "money."

8Access control list, or ACL, is one of many overloaded terms that arise in the field of information security.