Course Outline


Overview of Spark Streaming Features and Architecture

  • Supported data sources
  • Core APIs

Preparing the Environment

  • Dependencies
  • Spark and streaming context
  • Connecting to Kafka

Processing Messages

  • Parsing inbound messages as JSON
  • ETL processes
  • Starting the streaming context

Performing a Windowed Stream Processing

  • Slide interval
  • Checkpoint delivery configuration
  • Launching the environment

Prototyping the Processing Code

  • Connecting to a Kafka topic
  • Retrieving JSON from data source using Paw
  • Variations and additional processing

Streaming the Code

  • Job control variables
  • Defining values to match
  • Functions and conditions

Acquiring Stream Output

  • Counters
  • Kafka output (matched and non-matched)


Summary and Conclusion


  • Experience with Python and Apache Kafka
  • Familiarity with stream-processing platforms


  • Data engineers
  • Data scientists
  • Programmers
  7 Hours


Related Courses

Data Analysis with Python, Pandas, and Numpy

  14 hours

Machine Learning with Python and Pandas

  14 hours

Accelerating Python Pandas Workflows with Modin

  14 hours

Scaling Data Analysis with Python and Dask

  14 hours

Developing APIs with Python and FastAPI

  14 hours

FARM (FastAPI, React, and MongoDB) Full Stack Development

  14 hours

Scientific Computing with Python SciPy

  7 hours

Game Development with PyGame

  7 hours

Web application development with Flask

  14 hours

Build REST APIs with Python and Flask

  14 hours

Advanced Flask

  14 hours

GUI Programming with Python and Tkinter

  14 hours

Kivy: Building Android Apps with Python

  7 hours

GUI Programming with Python and PyQt

  21 hours

Web Development with Web2Py

  28 hours