Architecture and Design of the Linux Storage Stack

Master the design and structure of Linux storage stack and explore its sophisticated architecture

Key Features

• Explore the virtual file system (VFS) and how it serves as an abstraction layer for the actual file system
• Understand how the block layer acts as an intermediary between file systems and physical storage
• Discover the physical layout and protocols linked with storage media

Book Description

The Linux storage stack serves as a prime example of meticulously coordinated layers. Embark on a journey through the kernel code with Architecture and Design of the Linux Storage Stack, crafted for anyone seeking in-depth knowledge about the layered design of Linux storage and its landscape.

You’ll explore the Linux storage stack and its various concepts. You’ll unlock the secrets of the virtual filesystem and the actual filesystem and the differences in their implementation, the role of the block layer, the Multi-Queue and Device Mapper frameworks, I/O schedulers, physical storage layout, and how to analyze all the layers in the storage stack.

By the end of this book, you’ll be acquainted with how a simple I/O request from a process travels down through all the layers and ends up in physical storage.

What you will learn

• Understand the role of the virtual filesystem
• Explore the different flavors of Linux filesystems and their key concepts
• Manage I/O operations to and from block devices using the block layer
• Deep dive into the Small Computer System Interface (SCSI) subsystem and the layout of physical devices
• Gauge I/O performance at each layer of the storage stack
• Discover the best storage practices

Who this book is for

This book is for system and storage administrators, engineers, linux professionals, linux community in general, and anyone looking to expand their understanding of Linux and its storage landscape. Prior knowledge of Linux operating system is a must.

Table of Contents

Part 1: Diving into the Virtual Filesystem

Chapter 1: Where It All Starts From – The Virtual Filesystem

Chapter 2: Explaining the Data Structures in a VFS

Chapter 3: Exploring the Actual Filesystems Under the VFS

Part 2: Navigating Through the Block layer

Chapter 4: Understanding the Block layer, Block Devices, and Data Structures

Chapter 5: Understanding the Block layer, Multi-Queue, and Device Mapper

Chapter 6: Understanding 1/0 Handling and Scheduling in the Block layer

Part 3: Descending into the Physical layer

Chapter 7: The SCSI Subsystem

Chapter 8: Illustrating the layout of Physical Media

Part 4: Analyzing and Troubleshooting Storage Performance

Chapter 9: Analyzing Physical Storage Performance

Chapter 1 0: Analyzing Filesystems and the Block layer

Chapter 11: Tuning the I/O Stack

Review

“This book is an excellent modern take on the Linux storage stack, though better suited for those who are quite new to it. Its coverage of the modern block multi-queue (blk-mq) IO framework is great, given its relevance from the 5.0 kernel onward. Also, the device mapper concepts are useful. I learned stuff regarding SCSI and NVMe, and provided the comparisons are very helpful too. Part 4, which covers storage performance analysis and troubleshooting is quite valuable!

On the plus side, this book does a great job of explaining basic concepts well, along with very good diagrams! I would wish for a bit more kernel code coverage to back up concepts, as well as more samples, like a basic block driver utilizing blk-mq, and case studies on storage performance analysis.

Overall, a wonderful book for folks relatively new to the Linux storage stack.”

—

Kaiwan Billimoria, Owner, kaiwanTECH (a division of Designer Graphix), Author, Linux Consultant and Trainer

“Umer’s work delves deep into the core components that make up the Linux storage stack, making it an indispensable resource for anyone seeking an in-depth understanding of this critical aspect of modern computing.

With a meticulous approach, the author navigates readers through the intricacies of Linux Virtual File System (VFS), the block layer, and the I/O scheduler, unravelling the complex interactions between these layers.

The final chapter of the book goes beyond the theoretical underpinnings and takes readers into the realm of real-world application by providing a detailed exploration of various Linux tools used to monitor and debug I/O delays and congestion.

In conclusion, Muhammad Umer’s book stands as an invaluable addition to the libraries of those seeking to unravel the mysteries of the Linux storage stack.

Its combination of depth, accessibility, and practicality sets it apart as a must-read for anyone interested in the intricate design and functionality that underpins modern storage systems.”

—

Ramon Fried, Head of Embedded Linux and BSP at NeuReality

About the Author

Muhammad Umer (RHCA®) is a systems engineer and trainer with more than six years of experience in working with Linux based systems, HA design & architecture, tuning operating system and underlying hardware for optimal performance and root cause analysis. Ever since a virus infected his laptop ten years ago, he switched to Linux. Turns out, it wasn’t the worst thing that ever happened. He has a particular preference for all things storage and in this book, he has tried to decode the mysteries of the Linux storage stack one disk sector at a time. When not immersed in the binary wonders of storage, you can find him savoring pizza and dreaming of a world where storage is as reliable as a perfectly timed cron job.