Thank you for sending your enquiry! One of our team member will contact you shortly.
Thank you for sending your booking! One of our team member will contact you shortly.
Course Outline
-
Scala primer
- A quick introduction to Scala
- Labs : Getting know Scala
-
Spark Basics
- Background and history
- Spark and Hadoop
- Spark concepts and architecture
- Spark eco system (core, spark sql, mlib, streaming)
- Labs : Installing and running Spark
-
First Look at Spark
- Running Spark in local mode
- Spark web UI
- Spark shell
- Analyzing dataset – part 1
- Inspecting RDDs
- Labs: Spark shell exploration
-
RDDs
- RDDs concepts
- Partitions
- RDD Operations / transformations
- RDD types
- Key-Value pair RDDs
- MapReduce on RDD
- Caching and persistence
- Labs : creating & inspecting RDDs; Caching RDDs
-
Spark API programming
- Introduction to Spark API / RDD API
- Submitting the first program to Spark
- Debugging / logging
- Configuration properties
- Labs : Programming in Spark API, Submitting jobs
-
Spark SQL
- SQL support in Spark
- Dataframes
- Defining tables and importing datasets
- Querying data frames using SQL
- Storage formats : JSON / Parquet
- Labs : Creating and querying data frames; evaluating data formats
-
MLlib
- MLlib intro
- MLlib algorithms
- Labs : Writing MLib applications
-
GraphX
- GraphX library overview
- GraphX APIs
- Labs : Processing graph data using Spark
-
Spark Streaming
- Streaming overview
- Evaluating Streaming platforms
- Streaming operations
- Sliding window operations
- Labs : Writing spark streaming applications
-
Spark and Hadoop
- Hadoop Intro (HDFS / YARN)
- Hadoop + Spark architecture
- Running Spark on Hadoop YARN
- Processing HDFS files using Spark
-
Spark Performance and Tuning
- Broadcast variables
- Accumulators
- Memory management & caching
-
Spark Operations
- Deploying Spark in production
- Sample deployment templates
- Configurations
- Monitoring
- Troubleshooting
Requirements
PRE-REQUISITES
familiarity with either Java / Scala / Python language (our labs in Scala and Python)
basic understanding of Linux development environment (command line navigation / editing files using VI or nano)
Testimonials
Gunnar was very friendly and took time to answer all questions. It was clear he knew a lot about Spark and was able to explain the content and answer questions well.
Kirsty Nangle - Capita Business Services Ltd.
The topics being taught, the examples and exercises were helpful and instructor had great knowledge of subject
Capita Business Services Ltd.
The detailed explanations. Gunnar knew the material very well and took the time to explain anything we found confusing. Most trainers object when you start asking searching questions that take half an hour to answer; Gunnar knew the answers and was willing to focus on what we wanted to learn, not what he had prepared.
Maxwell Green - Capita Business Services Ltd.
Ajay is very personable and a pleasant speaker. He is nice and seems super knowledgeable in many of these areas. He made himself available and his github is a great resource!
credit karma
Doing similar exercises different ways really help understanding what each component (Hadoop/Spark, standalone/cluster) can do on its own and together. It gave me ideas on how I should test my application on my local machine when I develop vs when it is deployed on a cluster.
Thomas Carcaud - IT Frankfurt GmbH
Related Courses
Introduction to Graph Computing
28 hours
Apache Spark MLlib
35 hours
Apache Spark SQL
7 hours
Big Data Analytics in Health
21 hours
Apache Spark in the Cloud
21 hours
Apache Spark Streaming with Scala
21 hours
SMACK Stack for Data Science
14 hours
Apache Spark Fundamentals
21 hours
Apache Spark for .NET Developers
21 hours