Duration
4 days
Overview
This open source course provides participants with a comprehensive understanding of the steps necessary to install, configure, operate and maintain Hadoop. The course begins with an overview of the Big Data landscape, and then dives into a system administration working view of running Hadoop.
Audience
This course is intended for System administrators, DevOps engineers, and software developers responsible for managing and maintaining Hadoop clusters.
Prerequisite Knowledge/Skills
None
Course Objectives
Upon successful completion of this course, participants should be able to:
- Describe the fundamental concepts of using Big Data
- Identify where Hadoop fits into a Big Data strategy
- Learn to plan your Hadoop cluster.
- Learn HDFS features.
- Learn how to get data into HDFS.
- Learn to work with MapReduce.
- Learn installation and configuration of Hadoop.
- Learn cluster maintenance.
Course Outline
The content of this course is designed to support the course objectives.
1.0. Hadoop Introduction
o1.1. A Brief History of Hadoop
o1.2. Core Hadoop Components
o1.3. Fundamental Concepts
- 2. Planning Your Hadoop Cluster
o2.1. General Planning Considerations
o2.2. Choosing Hardware
o2.3. Network Considerations
o2.4. Configuring Nodes
o2.5. Planning for Cluster Management
- 3. HDFS
o3.1. HDFS Features
o3.2. Writing and Reading Files
o3.3. NameNode Considerations
o3.4. HDFS Security
o3.5. Namenode Web UI
o3.6. Hadoop File Shell
- 4. Getting Data into HDFS
o4.1. Pulling data from External Sources with Flume
o4.2. Importing Data from Relational Databases with Sqoop
o4.3. REST Interfaces
o4.4. Best Practices
- 5. MapReduce
o5.1. MapReduce overview
o5.2. Features of MapReduce
o5.3. Architectural Overview
o5.4. YARN MapReduce Version 2
o5.5. Failure Recovery
o5.6. The JobTracker Web UI
- 6. Hadoop Installation & Initial Configuration
o6.1. Configuration & Deployment Types
o6.2. Installing Hadoop
o6.3. Specifying the Hadoop Configuration
o6.4. Initial HDFS & MapReduce Configuration
o6.5. Log Files
- 7. Installing/Configuring Hive, Impala, and Pig
o7.1. Hive
o7.2. Impala
o7.3. Pig
8.0. Hadoop Clients
o8.1. What is a Hadoop Client?
o8.2. Installing and Configuring Hadoop Clients
o8.3. Installing and Configuring Hue
o8.4. Hue Authentication and Configuration
- 9. Advanced Cluster Configuration
o9.1. Advanced Configuration Parameters
o9.2. Configuring Hadoop Ports
o9.3. Explicitly Including and Excluding Hosts
o9.4. Configuring HDFS for Rack Awareness & HDFS High Availability
- 10. Hadoop Security
o10.1. Why Hadoop Security Is Important
o10.2. Hadoop’s Security System Concepts
o10.3. What Kerberos Is and How it Works
o10.4. Securing a Hadoop Cluster with Kerberos
- 11. Managing and Scheduling Jobs
o11.1. Managing Running Jobs
o11.2. Scheduling Hadoop Jobs
o11.3. Configuring the FairScheduler
- 12. Cluster Maintenance
o12.1. Checking HDFS Status
o12.2. Copying Data Between Clusters
o12.3. Adding/Removing Cluster Nodes
o12.4. Rebalancing the Cluster
o12.5. NameNode Metadata Backup
o12.6. Cluster Upgrades
- 13. Cluster Monitoring and Troubleshooting
o13.1. General System Monitoring
o13.2. Managing Hadoop’s Log Files
o13.3. Monitoring the Clusters
o13.4. Common Troubleshooting Issues
Course Delivery Options
This course is currently available in the following formats:
MR-1CN-OSHDADMIN: Instructor Led – Includes hands-on lab exercises that reinforce the concepts covered in lectures.
MR-1LN-OSHDADMIN: Online ILT – Live course delivered via the Internet where participants attend virtual classroom interacting with instructors and other participants.
A headset with microphone is REQUIRED to speak with the instructor and the rest of the class. Text communication is also available through the virtual classroom.
MR-1VN-OSHDADMIN -1007: Video ILT – USB video provides a recording of an instructor-delivered course along with recorded labs, online quizzes, and additional content.
MR-1TN-OSHDADMIN -1007: Stream Video – Streaming video provides a recording of an instructor-delivered course along with online quizzes and additional content.
Programme scope, delivery arrangements and suitability are confirmed during enquiry.