A Practical Introduction to Stream Processing Training Course

This course is intended for developers and data scientists who want to understand and implement artificial intelligence in their applications. Special focus is placed on data analytics, distributed AI, and natural language processing.

This instructor-led, live training in Malta (online or onsite) is aimed at developers who wish to integrate Apache Kafka with existing databases and applications for processing, analysis, etc.

By the end of this training, participants will be able to:

• Use Kafka Connect to ingest large amounts of data from a database into Kafka topics.
• Ingest log data generated by an application servers into Kafka topics.
• Make any collected data available for stream processing.
• Export data from Kafka topics into secondary systems for storage and analysis.

Apache Beam is an open source, unified programming model for defining and executing parallel data processing pipelines. It's power lies in its ability to run both batch and streaming pipelines, with execution being carried out by one of Beam's supported distributed processing back-ends: Apache Apex, Apache Flink, Apache Spark, and Google Cloud Dataflow. Apache Beam is useful for ETL (Extract, Transform, and Load) tasks such as moving data between different storage media and data sources, transforming data into a more desirable format, and loading data onto a new system.

In this instructor-led, live training (onsite or remote), participants will learn how to implement the Apache Beam SDKs in a Java or Python application that defines a data processing pipeline for decomposing a big data set into smaller chunks for independent, parallel processing.

By the end of this training, participants will be able to:

• Install and configure Apache Beam.
• Use a single programming model to carry out both batch and stream processing from withing their Java or Python application.
• Execute pipelines across multiple environments.

Format of the Course

• Part lecture, part discussion, exercises and heavy hands-on practice

Note

• This course will be available Scala in the future. Please contact us to arrange.

This instructor-led, live training (online or onsite) is aimed at engineers who wish to use Confluent (a distribution of Kafka) to build and manage a real-time data processing platform for their applications.

By the end of this training, participants will be able to:

• Install and configure Confluent Platform.
• Use Confluent's management tools and services to run Kafka more easily.
• Store and process incoming stream data.
• Optimize and manage Kafka clusters.
• Secure data streams.

Format of the Course

• Interactive lecture and discussion.
• Lots of exercises and practice.
• Hands-on implementation in a live-lab environment.

Course Customization Options

• This course is based on the open source version of Confluent: Confluent Open Source.
• To request a customized training for this course, please contact us to arrange.

This instructor-led, live training in Malta (online or onsite) introduces the principles and approaches behind distributed stream and batch data processing, and walks participants through the creation of a real-time, data streaming application in Apache Flink.

By the end of this training, participants will be able to:

• Set up an environment for developing data analysis applications.
• Understand how Apache Flink's graph-processing library (Gelly) works.
• Package, execute, and monitor Flink-based, fault-tolerant, data streaming applications.
• Manage diverse workloads.
• Perform advanced analytics.
• Set up a multi-node Flink cluster.
• Measure and optimize performance.
• Integrate Flink with different Big Data systems.
• Compare Flink capabilities with those of other big data processing frameworks.

In this instructor-led, live training in Malta, participants will learn about the technology offerings and implementation approaches for processing graph data. The aim is to identify real-world objects, their characteristics and relationships, then model these relationships and process them as data using a Graph Computing (also known as Graph Analytics) approach. We start with a broad overview and narrow in on specific tools as we step through a series of case studies, hands-on exercises and live deployments.

By the end of this training, participants will be able to:

• Understand how graph data is persisted and traversed.
• Select the best framework for a given task (from graph databases to batch processing frameworks.)
• Implement Hadoop, Spark, GraphX and Pregel to carry out graph computing across many machines in parallel.
• View real-world big data problems in terms of graphs, processes and traversals.

This instructor-led, live training in Malta (online or onsite) is aimed at data engineers, data scientists, and programmers who wish to use Apache Kafka features in data streaming with Python.

By the end of this training, participants will be able to use Apache Kafka to monitor and manage conditions in continuous data streams using Python programming.

Kafka Streams is a client-side library for building applications and microservices whose data is passed to and from a Kafka messaging system. Traditionally, Apache Kafka has relied on Apache Spark or Apache Storm to process data between message producers and consumers. By calling the Kafka Streams API from within an application, data can be processed directly within Kafka, bypassing the need for sending the data to a separate cluster for processing.

In this instructor-led, live training, participants will learn how to integrate Kafka Streams into a set of sample Java applications that pass data to and from Apache Kafka for stream processing.

By the end of this training, participants will be able to:

• Understand Kafka Streams features and advantages over other stream processing frameworks
• Process stream data directly within a Kafka cluster
• Write a Java or Scala application or microservice that integrates with Kafka and Kafka Streams
• Write concise code that transforms input Kafka topics into output Kafka topics
• Build, package and deploy the application

Audience

• Developers

Format of the course

• Part lecture, part discussion, exercises and heavy hands-on practice

Notes

• To request a customized training for this course, please contact us to arrange

This instructor-led, live training in Malta (online or onsite) is aimed at developers who wish to implement Apache Kafka stream processing without writing code.

By the end of this training, participants will be able to:

• Install and configure Confluent KSQL.
• Set up a stream processing pipeline using only SQL commands (no Java or Python coding).
• Carry out data filtering, transformations, aggregations, joins, windowing, and sessionization entirely in SQL.
• Design and deploy interactive, continuous queries for streaming ETL and real-time analytics.

In this instructor-led, live training in Malta (onsite or remote), participants will learn how to deploy and manage Apache NiFi in a live lab environment.

By the end of this training, participants will be able to:

• Install and configure Apachi NiFi.
• Source, transform and manage data from disparate, distributed data sources, including databases and big data lakes.
• Automate dataflows.
• Enable streaming analytics.
• Apply various approaches for data ingestion.
• Transform Big Data and into business insights.

In this instructor-led, live training in Malta, participants will learn the fundamentals of flow-based programming as they develop a number of demo extensions, components and processors using Apache NiFi.

By the end of this training, participants will be able to:

• Understand NiFi's architecture and dataflow concepts.
• Develop extensions using NiFi and third-party APIs.
• Custom develop their own Apache Nifi processor.
• Ingest and process real-time data from disparate and uncommon file formats and data sources.

In this instructor-led, live training in Malta, participants will learn how to use Python and Spark together to analyze big data as they work on hands-on exercises.

By the end of this training, participants will be able to:

• Learn how to use Spark with Python to analyze Big Data.
• Work on exercises that mimic real world cases.
• Use different tools and techniques for big data analysis using PySpark.

This instructor-led, live training in Malta (online or onsite) is aimed at data engineers, data scientists, and programmers who wish to use Spark Streaming features in processing and analyzing real-time data.

By the end of this training, participants will be able to use Spark Streaming to process live data streams for use in databases, filesystems, and live dashboards.

MLlib is Spark’s machine learning (ML) library. Its goal is to make practical machine learning scalable and easy. It consists of common learning algorithms and utilities, including classification, regression, clustering, collaborative filtering, dimensionality reduction, as well as lower-level optimization primitives and higher-level pipeline APIs.

It divides into two packages:

• spark.mllib contains the original API built on top of RDDs.

• spark.ml provides higher-level API built on top of DataFrames for constructing ML pipelines.

Audience

This course is directed at engineers and developers seeking to utilize a built in Machine Library for Apache Spark