Course Code
introtostreamprocessing
Duration
21 hours (usually 3 days including breaks)
Requirements
- Programming experience in any language
- An understanding of Big Data concepts (Hadoop, etc.)
Overview
Stream Processing refers to the real-time processing of "data in motion", that is, performing computations on data as it is being received. Such data is read as continuous streams from data sources such as sensor events, website user activity, financial trades, credit card swipes, click streams, etc. Stream Processing frameworks are able to read large volumes of incoming data and provide valuable insights almost instantaneously.
In this instructor-led, live training (onsite or remote), participants will learn how to set up and integrate different Stream Processing frameworks with existing big data storage systems and related software applications and microservices.
By the end of this training, participants will be able to:
- Install and configure different Stream Processing frameworks, such as Spark Streaming and Kafka Streaming.
- Understand and select the most appropriate framework for the job.
- Process of data continuously, concurrently, and in a record-by-record fashion.
- Integrate Stream Processing solutions with existing databases, data warehouses, data lakes, etc.
- Integrate the most appropriate stream processing library with enterprise applications and microservices.
Audience
- Developers
- Software architects
Format of the Course
- Part lecture, part discussion, exercises and heavy hands-on practice
Notes
- To request a customized training for this course, please contact us to arrange.
Course Outline
Introduction
- Stream processing vs batch processing
- Analytics-focused stream processing
Overview Frameworks and Programming Languages
- Spark Streaming (Scala)
- Kafka Streaming (Java)
- Flink
- Storm
Comparison of Features and Strengths of Each Framework
Overview of Data Sources
- Live data as a series of events over time
- Historical data sources
Deployment Options
- In the cloud (AWS, etc.)
- On premise (private cloud, etc.)
Setting up the Development Environment
Installing and Configuring
Assessing Your Data Analysis Needs
Integrating the Streaming Framework with Big Data Tools
Transforming the Input Data
Inspecting the Output Data
Integrating the Stream Processing Framework with Existing Applications and Microservices
Troubleshooting
Summary and Conclusion
.jpg "abu dhabi company for onshore oil operations (adco)")
.png "oman airports management company (oamc)")
.png "european central bank (ecb)")
.jpg "european organization for nuclear research (cern)")
