Duration
3 days
Course Outline
Time
Day 1
08.30 am to 10.00 am
- Introduction to Hadoop and Spark Ecosystem
- Big Data Overview
- Key Roles in Big Data Project
- Key Business Use cases
- Hadoop and Spark Logical Architecture
- Typical Big Data Project Pipeline
- Cloudera Ecosystem
10.00 am to 11.00 pm
- Basic Concepts of HDFS
- HDFS Overview
- Physical Architectures of HDFS
- Exploring HUE
- Hands-on Exercises
11:15 pm to 12:30 pm
&
01:30 pm to 03:00 pm
- MapReduce v1/YARN Frameworks, Architectures and MapReduce API
- Java Basics for understanding, developing, building and deploying MapReduce Programs
- Logical Architecture of MapReduce
- Why YARN?
- Physical Architecture of MRv1 and YARN
- Compare MRv1 vs. MRv2 on YARN
- MapReduce API
- Hands-on Exercise
03:15 pm to 04:30 pm
- Sqoop
- Sqoop Basics
- Sqoop Internals
- Key Sqoop Commands
- Hands-on Exercises
04:30 pm onwards
- Ingesting Data with Flume
- Flume Overview
- Physical Architectures of Flume
- Source, Sink and Channel
- Building Data Pipeline using Flume
- Hands-on Exercises
Day 2
08.30 am to 11.00 am
- Working with Hive
- Hive Overview and Use cases
- How Hive Differs from Relational Databases
- Basic Syntax in Hive
- External and Managed Tables
- Key Built-In functions in Hive
- Hive vs. HiveServer2
- Partitioning – Static and Dynamic
- Working with HUE
- Hands-on Exercises
11:15 pm to 12:30 pm
- Introduction to Impala
- What is Impala?
- MPP Architecture
- How Impala Differs from Hive
- How Impala Differs from Relational Databases
- Limitations and Future Directions
- Using the Impala Shell
- Hands-on Exercises
01:30 pm to 4:00 pm
- Working with HBase
- HBase Overview
- Physical Architectures of HBase
- HBase Table Fundamentals
- Thinking About Table Design
- HBase Shell
- HBase Physical Architecture
- HBase Schema Design
- HBase API
- Hive on HBase
- Hands-on Exercises
4:00 pm to 5:00 pm
- Spark Overview
- What is Spark?
- Why Spark?
- Data Abstraction - RDD
- Logical Architecture of Spark
- Programming Languages in Spark
- Functional Programming with Spark
- Hands-on Exercises
Day 3
8:30 am to 11:00 am
- Interactive Data Exploration with Spark
- Key RDD API Operations
- Pair RDDs
- MapReduce Operations
- Join of Sqoop and Flume data
- Spark on YARN
- YARN client vs YARN cluster modes
- Hands-on Exercise
11:15 am to 12:30 pm
- Spark SQL
Introduction
- Dataframe API
- Performing ad-hoc query analysis using Spark SQL
- Hands-on on Dataframe API
- Hands-on on Using Avro and Parquet with Dataframe API
- Hands-on: Integrating Hive with Spark SQL
2:30 pm to 4:00 pm
- Data Ingestion using Kafka
- Kafka Overview
- Kafka Architecture
- Kafka Producer Consumer API
- Flume vs Kafka
- Hands-on Exercises
4:00 pm onwards
- Other key developments in Cloudera
- What’s new in CDH 6 (Hadoop 3)?
- Kudu
- DataFlow
- Working with GPUs
Programme scope, delivery arrangements and suitability are confirmed during enquiry.