Apache Kafka

• event streaming platform; enables users to collect, store, and process data to build real-time event-driven applications
• a single cluster is made up of brokers; broker handles producers and consumers and keeps data replicated in the cluster
• producer -> topic -> consumer

Zookeeper

• used to manage a cluster, track status of nodes, maintain a list of topics and messages

Demo projects

1. Simple Kafka setup using Docker and Python

See kafka-python directory

2. Python client applications which produce and consume messages from Confluent Kafka cluster

See confluent-kafka directory