Training & Certification

No

Title

Topics Covered

Introduction to Big Data and Hadoop

01

Introduction to

Big Data

• What is Big Data?

• Data Evolution Through Past 4 Decades

• Nature of Big Data – Volume, Variety & Velocity

• Factors Driving Use of Big Data

• Challenges faced by Conventional Data Analytics system

• Need for Big Data Technologies

• Overview of Big Data Technologies – Hadoop

02

Applications of

Big Data

• Real life business problems, which can’t be answered without data

• Case Study – Increasing Customer Life-time Value in Web Retail Business

• Case Study – Increasing Machine utilization and asset life in aManufacturing Organization

• Case Study – Real-time Credit Card Fraud Detection

03

Hadoop

Architecture

• Principles of Distributed Processing

• Overview of Hadoop Architecture

• Hadoop Core Components – HDFS & Map/Reduce

04

HDFS

• HDFS Overview

• Distributed File Storage Concepts

• Importance of Block Size

• Data Fault-tolerance using Replication

• Understanding Name node Metadata

• Cluster Fault-tolerance using Secondary Name node

05

Setting Up a

Hadoop Cluster

• Downloading, installing & Configuring Hadoop

• Setting up a Pseudo-Distributed Cluster

• Setting Up a Fully-distributed Cluster

• Working with Hadoop – DFS shell commands

• Hadoop User Interface

• Understanding Hadoop Log Files

• Changing HDFS block sizes and Replication Factors

• Adding and Removing Nodes from Hadoop Cluster

06

Introduction to

Map/Reduce

• What is Map/Reduce

• Map/Reduce examples from real world

• Basic principles of Map/Reduce

• Understanding various Map/Reduce Phases

• Running our first Map/Reduce program using Wordcount example

07

Working with

Map/Reduce

• Map/Reduce Framework

• Roles of Developer and Framework

• Hadoop Data Types – Writable & Comparable Interfaces

• Hadoop Input & Output Formats

• Writing a Map Class

• Writing a Reduce Class

• Creating a Job Configuration Object

• Building a Jar and running a Map/Reduce program

08

Map/Reduce in

Detail

• How Map/Reduce works

• Roles of Developer and Framework

• Hadoop Data Types – Writable & Comparable Interfaces

• Hadoop Input & Output Formats

• Writing a Map Class

• Writing a Reduce Class

• Creating a Job Configuration Object using inner classes

• Building a Jar and running a Map/Reduce program

09

Introduction to Pig

• What is Pig? Why Pig

• Pig Architecture & Modes of Operations

• Pig Latin Script

• Pig Data Types

• Pig Operators

• Word Count Example & Other Hands-on Exercises

• User Defined Functions and Data Types

• PIG Data Joins

10

Introduction to

Hive

• Hive Introduction

• Differences between Hive and traditional RDBMS

• Hive Architecture – components of Hive

• Hive Data Model – database, partitions, tables, fields, indexes

• Hive Schema and Meta store

• Input / Output Formats – SerDe’s

• HQL - Hive Query Language

• HQL – Hands On Exercises

• Project - Building a Tweeter Sentiment Analysis Application

11

Introduction to

No-SQL Databases

• Why HBase?

• HBase Architecture

• Role of Zookeeper

• Overview of Cassandra

• Cassandra Exercise

12

Data Import.

Export with

Flume and Sqoop

• Bringing unstructured data into Hadoop using Flume

• Flume Exercise

• Bringing Relational Data from SQL into Hadoop using Sqoop

• Sqoop Exercises

13

Real-Time

Analytics with

Spark

• Why Spark

• Spark Architecture

• Spark Application Development Environment