Request full copy

Design and Implementation of the FreeBSD Operating System, The, 2nd edition

Published by Addison-Wesley Professional (September 5, 2014) © 2015

  • Marshall Kirk McKusick Consultant
  • George V. Neville-Neil Consultant
  • Robert N.M. Watson

eTextbook

$76.99

  • Available for purchase from all major ebook resellers, including InformIT.com.
  • To request a review copy, click on the "Request a Review Copy" button.
$63.99

  • A print text (hardcover or paperback) 
  • Free shipping
  • Also available for purchase as an ebook from all major ebook resellers, including InformIT.com

This book contains comprehensive, up-to-date, and authoritative technical information on the internal structure of the FreeBSD open-source operating system. Coverage includes the capabilities of the system; how to effectively and efficiently interface to the system; how to maintain, tune, and configure the operating system; and how to extend and enhance the system.

The authors provide a concise overview of FreeBSD's design and implementation. Then, while explaining key design decisions, they detail the concepts, data structures, and algorithms used in implementing the systems facilities. As a result, this book can be used as an operating systems textbook, a practical reference, or an in-depth study of a contemporary, portable, open-source operating system.

  • Adds a new chapter describing the Zettabyte filesystem
  • Adds a new chapter on security including Capsicum security sandboxes
  • Details the addition of super-page support in the virtual memory system
  • Describes techniques for doing fine-grained symmetric-multiprocessing
  • Updates information on networking, including virtual networks and¿new protocols such as SCTP
  • This edition will feature a full line of instructor resources

This edition has been extensively revamped and expanded to cover all significant improvements to FreeBSD between Version 5 and Versions 9/10. Approximately one-third of its content is entirely new, and another one-third has been extensively rewritten. New coverage includes (but is not limited to):

  • Light-weight virtualization (supporting hundreds of instances on a single system)
  • Fine-grained authority delegation (replacing the "all-or-nothing" superuser)
  • New support for NFS version 4.
  • Journaled soft updates (eliminating the need for background fsck)
  • Fully integrated support for OpenSolaris ZFS
  • New volume-management features provided through the GEOM subsystem
  • Superpage support for large processes (vastly increasing coverage of TLB)
  • Finely-tuned locking to support many-CPU-core systems
  • Fine-grained process debugging/profiling with DTrace from OpenSolaris
  • Pluggable TCP Congestion Control.
  • The SCTP transport protocol
  • New Layer 2 Networking (New ARP, crucial to router vendors)
  • Greatly expanded wireless and USB support
  • FreeBSD's completely overhauled pseudo-terminal subsystem

Preface xxi

About the Authors xxix

 

Part I: Over view 1

 

Chapter 1: History and Goals 3

1.1 History of the UNIX System 3

1.2 BSD and Other Systems 7

1.3 The Transition of BSD to Open Source 9

1.4 The FreeBSD Development Model 14

References 17

 

Chapter 2: Design Overview of FreeBSD 21

2.1 FreeBSD Facilities and the Kernel 21

2.2 Kernel Organization 23

2.3 Kernel Services 26

2.4 Process Management 26

2.5 Security 29

2.6 Memory Management 36

2.7 I/O System Overview 39

2.8 Devices 44

2.9 The Fast Filesystem 45

2.10 The Zettabyte Filesystem 49

2.11 The Network Filesystem 50

2.12 Interprocess Communication 50

2.13 Network-Layer Protocols 51

2.14 Transport-Layer Protocols 52

2.15 System Startup and Shutdown 52

Exercises 54

References 54

 

Chapter 3: Kernel Services 57

3.1 Kernel Organization 57

3.2 System Calls 62

3.3 Traps and Interrupts 64

3.4 Clock Interrupts 65

3.5 Memory-Management Services 69

3.6 Timing Services 73

3.7 Resource Services 75

3.8 Kernel Tracing Facilities 77

Exercises 84

References 85

 

Part II: Processes 87

 

Chapter 4: Process Management 89

4.1 Introduction to Process Management 89

4.2 Process State 92

4.3 Context Switching 99

4.4 Thread Scheduling 114

4.5 Process Creation 126

4.6 Process Termination 128

4.7 Signals 129

4.8 Process Groups and Sessions 136

4.9 Process Debugging 142

Exercises 144

References 146

 

Chapter 5: Security 147

5.1 Operating-System Security 148

5.2 Security Model 149

5.3 Process Credentials 151

5.4 Users and Groups 154

5.5 Privilege Model 157

5.6 Interprocess Access Control 159

5.7 Discretionary Access Control 161

5.8 Capsicum Capability Model 174

5.9 Jails 180

5.10 Mandatory Access-Control Framework 184

5.11 Security Event Auditing 200

5.12 Cryptographic Services 206

5.13 GELI Full-Disk Encryption 212

Exercises 217

References 217

 

Chapter 6: Memory Management 221

6.1 Terminology 221

6.2 Overview of the FreeBSD Virtual-Memory System 227

6.3 Kernel Memory Management 230

6.4 Per-Process Resources 244

6.5 Shared Memory 250

6.6 Creation of a New Process 258

6.7 Execution of a File 262

6.8 Process Manipulation of Its Address Space 263

6.9 Termination of a Process 266

6.10 The Pager Interface 267

6.11 Paging 276

6.12 Page Replacement 289

6.13 Portability 298

Exercises 308

References 310

 

Part III: I/OSystem 313

 

Chapter 7: I/O System Overview 315

7.1 Descriptor Management and Services 316

7.2 Local Interprocess Communication 333

7.3 The Virtual-Filesystem Interface 339

7.4 Filesystem-Independent Services 344

7.5 Stackable Filesystems 352

Exercises 358

References 359

 

Chapter 8: Devices 361

8.1 Device Overview 361

8.2 I/O Mapping from User to Device 367

8.3 Character Devices 370

8.4 Disk Devices 374

8.5 Network Devices 378

8.6 Terminal Handling 382

8.7 The GEOM Layer 391

8.8 The CAM Layer 399

8.9 Device Configuration 402

8.10 Device Virtualization 414

Exercises 428

References 429

 

Chapter 9: The Fast Filesystem 431

9.1 Hierarchical Filesystem Management 431

9.2 Structure of an Inode 433

9.3 Naming 443

9.4 Quotas 451

9.5 File Locking 454

9.6 Soft Updates 459

9.7 Filesystem Snapshots 480

9.8 Journaled Soft Updates 487

9.9 The Local Filestore 496

9.10 The Berkeley Fast Filesystem 501

Exercises 517

References 519

 

Chapter 10: The Zettabyte Filesystem 523

10.1 Introduction 523

10.2 ZFS Organization 527

10.3 ZFS Structure 532

10.4 ZFS Operation 535

10.5 ZFS Design Tradeoffs 547

Exercises 549

References 549

 

Chapter 11: The Network Filesystem 551

11.1 Overview 551

11.2 Structure and Operation 553

11.3 NFS Evolution 567

Exercises 586

References 587

 

Part IV: Interprocess Communication 591

 

Chapter 12: Interprocess Communication 593

12.1 Interprocess-Communication Model 593

12.2 Implementation Structure and Overview 599

12.3 Memory Management 601

12.4 IPC Data Structures 606

12.5 Connection Setup 612

12.6 Data Transfer 615

12.7 Socket Shutdown 620

12.8 Network-Communication Protocol Internal Structure 621

12.9 Socket-to-Protocol Interface 626

12.10 Protocol-to-Protocol Interface 631

12.11 Protocol-to-Network Interface 634

12.12 Buffering and Flow Control 643

12.13 Network Virtualization 644

Exercises 646

References 648

 

Chapter 13: Network-Layer Protocols 649

13.1 Internet Protocol Version 4 650

13.2 Internet Control Message Protocols (ICMP) 657

13.3 Internet Protocol Version 6 659

13.4 Internet Protocols Code Structure 670

13.5 Routing 675

13.6 Raw Sockets 686

13.7 Security 688

13.8 Packet-Processing Frameworks 700

Exercises 715

References 717

 

Chapter 14: Transport-Layer Protocols 721

14.1 Internet Ports and Associations 721

14.2 User Datagram Protocol (UDP) 723

14.3 Transmission Control Protocol (TCP) 725

14.4 TCP Algorithms 732

14.5 TCP Input Processing 741

14.6 TCP Output Processing 745

14.7 Stream Control Transmission Protocol (SCTP) 761

Exercises 768

References 770

 

Part V: System Operation 773

 

Chapter 15: System Startup and Shutdown 775

15.1 Firmware and BIOSes 776

15.2 Boot Loaders 777

15.3 Kernel Boot 782

15.4 User-Level Initialization 798

15.5 System Operation 800

Exercises 805

References 806

 

Glossary 807

Index 847

Marshall Kirk McKusick writes books and articles, consults, and teaches classes on UNIX- and BSD-related subjects. While at the University of California at Berkeley, he implemented the 4.2BSD fast filesystem and was the Research Computer Scientist at the Berkeley Computer Systems Research Group (CSRG), overseeing the development and release of 4.3BSD and 4.4BSD. His particular areas of interest are the virtual-memory system and the filesystem. He earned his undergraduate degree in electrical engineering from Cornell University and did his graduate work at the University of California at Berkeley, where he received master’s degrees in computer science and business administration, and a doctoral degree in computer science. He has twice been president of the board of the Usenix Association, is currently a member of the FreeBSD Foundation Board of Directors, a member of the editorial board of ACM’s Queue magazine, a senior member of the IEEE, and a member of the Usenix Association, ACM, and AAAS. In his spare time, he enjoys swimming, scuba diving, and wine collecting. The wine is stored in a specially constructed wine cellar (accessible from the Web at http://www.McKusick.com/cgi-bin/readhouse) in the basement of the house that he shares with Eric Allman, his partner of 35-and-some-odd years and husband since 2013.

George V. Neville-Neil hacks, writes, teaches, and consults in the areas of Security, Networking, and Operating Systems. Other areas of interest include embedded and real-time systems, network time protocols, and code spelunking. In 2007, he helped start the AsiaBSDCon series of conferences in Tokyo, Japan, and has served on the program committee every year since then. He is a member of the FreeBSD Foundation Board of Directors, and was a member of the FreeBSD Core Team for 4 years. Contributing broadly to open source, he is the lead developer on the Precision Time Protocol project (http://ptpd.sf.net) and the developer of the Packet Construction Set (http://pcs.sf.net). Since 2004, he has written a monthly column, ‘‘Kode Vicious,’’ that appears both in ACM’s Queue and Communications of the ACM. He serves on the editorial board of ACM’s Queue magazine, is vice-chair of ACM’s Practitioner Board, and is a member of the Usenix Association, ACM, IEEE, and AAAS. He earned his bachelor’s degree in computer science at Northeastern University in Boston, Massachusetts. He is an avid bicyclist, hiker, and traveler who has lived in Amsterdam, The Netherlands, and Tokyo, Japan. He is currently based in Brooklyn, New York, where he lives with his husband, Kaz Senju.

Robert N.M. Watson is a University Lecturer in Systems, Security, and Architecture in the Security Research Group at the University of Cambridge Computer Laboratory. He supervises doctoral students and postdoctoral researchers in cross-layer research projects spanning computer architecture, compilers, program analysis, program transformation, operating systems, networking, and security. Dr. Watson is a member of the FreeBSD Foundation Board of Directors, was a member of the FreeBSD Core Team for 10 years, and has been a FreeBSD committer for 15 years. His open-source contributions include work on FreeBSD networking, security, and multiprocessing. Having grown up in Washington, D. C., he earned his undergraduate degree in Logic and Computation, with a double major in Computer Science, at Carnegie Mellon University in Pittsburgh, Pennsylvania, and then worked at a series of industrial research labs investigating computer security. He earned his doctoral degree at the University of Cambridge, where his graduate research was in extensible operating system access control. Dr. Watson and his wife Dr. Leigh Denault have lived in Cambridge, England, for 10 years.

Need help? Get in touch

Video
Play
Privacy and cookies
By watching, you agree Pearson can share your viewership data for marketing and analytics for one year, revocable by deleting your cookies.

Video transcript (PDF | 16KB) 

Pearson eTextbook: What’s on the inside just might surprise you

They say you can’t judge a book by its cover. It’s the same with your students. Meet each one right where they are with an engaging, interactive, personalized learning experience that goes beyond the textbook to fit any schedule, any budget, and any lifestyle. 

Digital Learning NOW

Extend your professional development and meet your students where they are with free weekly Digital Learning NOW webinars. Attend live, watch on-demand, or listen at your leisure to expand your teaching strategies. Earn digital professional development badges for attending a live session.

Explore webinars