Spark Streaming with Python and Kafka Training Course

This instructor-led, live training in the UAE (online or onsite) is aimed at developers who wish to integrate Apache Kafka with existing databases and applications for processing, analysis, etc.

By the end of this training, participants will be able to:

• Use Kafka Connect to ingest large amounts of data from a database into Kafka topics.
• Ingest log data generated by an application servers into Kafka topics.
• Make any collected data available for stream processing.
• Export data from Kafka topics into secondary systems for storage and analysis.

Learn to implement end-to-end big data streaming use cases. Real-time data preparation and maintenance with Informatica, Edge, Kafka and Spark. This training covers software versions 10.2.1 and up.

This instructor-led, live training (delivered online or at your site) is designed for engineers who want to leverage Confluent (a Kafka distribution) to develop and manage a real-time data processing platform for their applications.

By the conclusion of this course, participants will be able to:

• Set up and configure the Confluent Platform.
• Utilize Confluent's management tools and services to simplify Kafka operations.
• Store and process incoming data streams.
• Optimize and manage Kafka clusters effectively.
• Secure their data streams.

Course Format

• Engaging lectures and discussions.
• Ample exercises and practice sessions.
• Practical implementation in a live-lab setting.

Customization Options for the Course

• The course is based on the open-source version of Confluent: Confluent Open Source.
• To arrange for customized training, please contact us to discuss your requirements.

Apache Kafka is a distributed streaming platform that has become the standard for developing data pipelines. It addresses various use cases related to data processing, serving as a message queue, distributed log, stream processor, and more.

We will begin by exploring the theoretical foundations of data pipelines in general, followed by an in-depth look at the core concepts of Kafka. Additionally, we will delve into key components such as Kafka Streams and Kafka Connect.

In this instructor-led, live training in the UAE (onsite or remote), participants will learn how to set up and integrate different Stream Processing frameworks with existing big data storage systems and related software applications and microservices.

By the end of this training, participants will be able to:

• Install and configure different Stream Processing frameworks, such as Spark Streaming and Kafka Streaming.
• Understand and select the most appropriate framework for the job.
• Process of data continuously, concurrently, and in a record-by-record fashion.
• Integrate Stream Processing solutions with existing databases, data warehouses, data lakes, etc.
• Integrate the most appropriate stream processing library with enterprise applications and microservices.

This course is designed for enterprise architects, developers, system administrators, and anyone interested in understanding and utilizing a high-performance distributed messaging system. Should you require a focus on particular aspects (such as system administration exclusively), the course content can be customized to align more closely with your specific requirements.

This instructor-led, live training in the UAE (online or onsite) is aimed at sysadmins who wish to set up, deploy, manage and optimize an enterprise-grade Kafka cluster.

By the end of this training, participants will be able to:

• Set up and administer a Kafka Cluster.
• Evaluate the benefits and disadvantages of deploying Kafka on-premise vs in the cloud.
• Deploy and monitor Kafka in using various on-premise and cloud environment tools.

This instructor-led, live training in the UAE (online or onsite) is aimed at intermediate-level developers who wish to develop big data applications with Apache Kafka.

By the end of this training, participants will be able to:

• Develop Kafka producers and consumers to send and read data from Kafka.
• Integrate Kafka with external systems using Kafka Connect.
• Write streaming applications with Kafka Streams & ksqlDB.
• Integrate a Kafka client application with Confluent Cloud for cloud-based Kafka deployments.
• Gain practical experience through hands-on exercises and real-world use cases.

This instructor-led, live training in the UAE (online or onsite) is aimed at data engineers, data scientists, and programmers who wish to use Apache Kafka features in data streaming with Python.

By the end of this training, participants will be able to use Apache Kafka to monitor and manage conditions in continuous data streams using Python programming.

This instructor-led, live training in the UAE (online or onsite) is aimed at software testers who wish to implement network security measures into an Apache Kafka application.

By the end of this training, participants will be able to:

• Deploy Apache Kafka onto a cloud based server.
• Implement SSL encryption to prevent attacks.
• Add ACL authentication to track and control user access.
• Ensure credible clients have access to Kafka clusters with SSL and SASL authentication.

This instructor-led, live training in the UAE (online or onsite) is aimed at intermediate-level developers who wish to learn the fundamentals of Kafka and integrate it with Spring Boot.

By the end of this training, participants will be able to:

• Understand Kafka and its architecture.
• Learn how to install, configure, and set up a basic Kafka environment.
• Integrate Kafka with Spring Boot.

Kafka Streams is a client-side library designed for developing applications and microservices that exchange data with a Kafka messaging system. Traditionally, Apache Kafka has depended on Apache Spark or Apache Storm for processing data between message producers and consumers. However, by utilizing the Kafka Streams API within an application, data can be processed directly inside Kafka, eliminating the need to send it to another cluster for processing.

In this instructor-led live training session, participants will learn how to incorporate Kafka Streams into a series of sample Java applications that exchange data with Apache Kafka for stream processing.

By the end of this training, participants will be able to:

• Comprehend the features and benefits of Kafka Streams compared to other stream processing frameworks
• Process streaming data directly within a Kafka cluster
• Create Java or Scala applications or microservices that integrate with Kafka and Kafka Streams
• Write succinct code that converts input Kafka topics into output Kafka topics
• Construct, package, and deploy the application

Audience

• Developers

Course Format

• The course includes lectures, discussions, exercises, and extensive hands-on practice.

Notes

• To request a customized training for this course, please contact us to arrange the details.

This instructor-led, live training in the UAE (online or onsite) is aimed at developers who wish to implement Apache Kafka stream processing without writing code.

By the end of this training, participants will be able to:

• Install and configure Confluent KSQL.
• Set up a stream processing pipeline using only SQL commands (no Java or Python coding).
• Carry out data filtering, transformations, aggregations, joins, windowing, and sessionization entirely in SQL.
• Design and deploy interactive, continuous queries for streaming ETL and real-time analytics.

This instructor-led, live training in the UAE (online or onsite) is aimed at data scientists who wish to use the SMACK stack to build data processing platforms for big data solutions.

By the end of this training, participants will be able to:

• Implement a data pipeline architecture for processing big data.
• Develop a cluster infrastructure with Apache Mesos and Docker.
• Analyze data with Spark and Scala.
• Manage unstructured data with Apache Cassandra.

This instructor-led, live training in the UAE (online or onsite) is aimed at developers who wish to transform traditional architecture into a highly concurrent microservices-based architecture using Spring Cloud, Kafka, Docker, Kubernetes and Redis.

By the end of this training, participants will be able to:

• Set up the necessary development environment for building microservices.
• Design and implement a highly concurrent microservices ecosystem using Spring Cloud, Kafka, Redis, Docker and Kubernetes.
• Transform monolithic and SOA services to microservice based architecture.
• Adopt a DevOps approach to developing, testing and releasing software.
• Ensure high concurrency among microservices in production.
• Monitor microservices and implement recovery strategies.
• Carry out performance tuning.
• Learn about future trends in microservices architecture.