Understanding Software Dynamics

An Expert Guide to Software Performance Optimization

From mobile and cloud apps to video games to driverless vehicle control, more and more software is time-constrained: It must deliver reliable results seamlessly, consistently, and virtually instantaneously. If it doesn’t, customers are unhappy–and sometimes lives are put at risk. When complex software underperforms or fails, software engineers need to identify and address the root causes. This is difficult and, historically, few tools have been available to help.

In Understanding Software Dynamics, performance expert Richard L. Sites tackles the problem head on, offering expert methods and advanced tools for understanding complex, time-constrained software dynamics, improving reliability and troubleshooting challenging performance problems.

Sites draws on several decades of experience pioneering software performance optimization, as well as extensive experience teaching graduate-level developers. He introduces principles and techniques for use in any environment, from embedded devices to datacenters, illuminating them with examples based on x86 or ARM processors running Linux and linked by Ethernet. He also guides readers through building and applying a powerful, new, extremely low-overhead open-source software tool, KUtrace, to precisely trace executions on every CPU core. Using insights gleaned from this tool, readers can apply nuanced solutions–not merely brute-force techniques such as turning off caches or cores.

• Measure and address issues associated with CPUs, memory, disk/SSD, networks, and their interactions
• Fix programs that are always too slow, and those that sometimes lag for no apparent reason
• Design useful observability, logging, and time-stamping capabilities into your code
• Reason more effectively about performance data to see why reality differs from expectations
• Identify problems such as excess execution, slow instruction execution, waiting for resources, and software locks

Understanding Software Dynamics will be valuable to experienced software professionals, including application and OS developers, hardware and system architects, real-time system designers, and game developers, as well as advanced students.

Table of Contents

Part I: Measurement

1 My Program Is Too Slow

2 Measuring CPUs

3 Measuring Memory

4 CPU and Memory Interaction

5 Measuring Disk/SSD

6 Measuring Networks

7 Disk and Network Database Interaction

Part II: Observation

8 Logging

9 Aggregate Measures

10 Dashboards

11 Other Existing Tools

12 Traces

13 Observation Tool Design Principles

Part III: Kernel-User Trace

14 KUtrace: Goals, Design, Implementation

15 KUtrace: Linux Kernel Patches

16 KUtrace: Linux Loadable Module

17 KUtrace: User-Mode Runtime Control

18 KUtrace: Postprocessing

19 KUtrace: Display of Software Dynamics

Part IV: Reasoning

20 What to Look For

21 Executing Too Much

22 Executing Slowly

23 Waiting for CPU

24 Waiting for Memory

25 Waiting for Disk

26 Waiting for Network

27 Waiting for Locks

28 Waiting for Time

29 Waiting for Queues

30 Recap

Appendix A: Sample Servers

Appendix B: Trace Entries

Review

“Complex software often uses threads, events, and I/O to produce results. Even intermittent performance bugs can lead to functional failures when results are time sensitive. Here Sites shares his methods from decades of experience as a real-world performance detective to enable you to approach the deduction skills of an IT Sherlock Holmes.”

―Mark D. Hill, Partner Hardware Architect at Microsoft and Professor Emeritus at the University of Wisconsin-Madison

“I think that any senior CS student or professional can benefit by reading this book. While all the material in the first half of the book leads up to the use of KUTrace, the first two parts are worth reading on their own by anyone who wants to better understand the systems they are building and using.”―Rik Farrow, ;login

About the Author

Richard L. Sites wrote his first computer program in 1959 and has spent most of his career at the boundary between hardware and software, with a particular interest in CPU/software performance interactions. His past work includes VAX microcode, DEC Alpha co-architect, and inventing the performance counters found in nearly all processors today. He has done low-overhead microcode and software tracing at DEC, Adobe, Google, and Tesla. Dr. Sites earned his PhD at Stanford in 1974; he holds 66 patents and is a member of the US National Academy of Engineering.