Hadoop Security

As more corporations turn to Hadoop to store and process their most valuable data, the risk of a potential breach of those systems increases exponentially. This practical book not only shows Hadoop administrators and security architects how to protect Hadoop data from unauthorized access, it also shows how to limit the ability of an attacker to corrupt or modify data in the event of a security breach.

Authors Ben Spivey and Joey Echeverria provide in-depth information about the security features available in Hadoop, and organize them according to common computer security concepts. You’ll also get real-world examples that demonstrate how you can apply these concepts to your use cases.

• Understand the challenges of securing distributed systems, particularly Hadoop
• Use best practices for preparing Hadoop cluster hardware as securely as possible
• Get an overview of the Kerberos network authentication protocol
• Delve into authorization and accounting principles as they apply to Hadoop
• Learn how to use mechanisms to protect data in a Hadoop cluster, both in transit and at rest
• Integrate Hadoop data ingest into enterprise-wide security architecture
• Ensure that security architecture reaches all the way to end-user access

Table of Contents

Chapter 1. Introduction

Part I. Security Architecture

Chapter 2. Securing Distributed Systems

Chapter 3. System Architecture

Chapter 4. Kerberos

Part 11. Authentication, Authorization, and Accounting

Chapter 5. Identity and Authentication

Chapter 6. Authorization

Chapter 7. Apache Sentry {Incubating)

Chapter 8. Accounting

Part 111. Data Security

Chapter 9. Data Protection

Chapter 10. Securing Data Ingest

Chapter 11. Data Extraction and Client Access Security

Chapter 12. Cloudera Hue

Part IV. Putting It All Together

Chapter 13. Case Studies

About the Authors

Ben Spivey is currently a Solutions Architect at Cloudera. During his time with Cloudera, he has worked in a consulting capacity to assist customers with their Hadoop deployments. Ben has worked with many Fortune 500 companies across multiple industries, including financial services, retail, and health care. His primary expertise is the planning, installation, configuration, and securing of customers’ Hadoop clusters.

In addition to consulting responsibilities, Ben contributes a vast amount of technical writing on customer document deliverables, to include Hadoop best practices, security integration, and cluster administration.

Prior to Cloudera, Ben worked for the National Security Agency and with a defense contractor as a software engineer. During this time, Ben built applications that, among other things, integrated with enterprise security infrastructure to protect sensitive information.

Ben holds a Bachelor’s degree in Computer Science and a Master’s degree in Information Technology, with a focus on Information Assurance. Ben’s final Master’s project was focused on designing an enterprise IT infrastructure with a defense-in-depth approach.

Joey Echeverria is a Software Engineer at ScalingData where he builds the next generation of IT Operations Analytics on the Apache Hadoop platform. Joey is also a committer on the Kite SDK, an Apache-licensed data API for the Hadoop ecosystem. Joey was previously a Software Engineer at Cloudera where he contributed to a number of ASF projects including Apache Flume, Apache Sqoop, Apache Hadoop, and Apache HBase.

While at Cloudera, Joey also served as the Director of Federal Field Technical Services, overseeing the public sector Professional Services and Systems Engineering teams. Joey started at Cloudera as a Solutions Architect, where he helped customers to design, develop, and deploy production Hadoop applications and clusters. When needed, he has also filled in for Cloudera’s support and training teams and has taught Cloudera’s administrator and Apache HBase courses.

Joey’s background is in building and deploying secure data processing applications, with the last 7 years focused on Hadoop-based applications. In the past, he has worked on resource constrained data processing, a clustered implementation of the Snort intrusion detection system, and he built a distributed index system on Hadoop when he worked for NSA.