Kafka – Event Streaming Platform

Apache Kafka is a distributed event streaming platform used by thousands of companies for high-performance data pipelines, streaming analytics, and data integration.

Key Features

• High Throughput: Millions of messages per second
• Durability: Persistent message storage
• Scalability: Horizontal cluster scaling
• Stream Processing: Kafka Streams API
• Connect: Integration framework

Installation

Install Kafka:

wget https://downloads.apache.org/kafka/3.6.0/kafka_2.13-3.6.0.tgz
tar -xzf kafka_2.13-3.6.0.tgz
cd kafka_2.13-3.6.0

# Start ZooKeeper
bin/zookeeper-server-start.sh config/zookeeper.properties &

# Start Kafka
bin/kafka-server-start.sh config/server.properties &

Usage Examples

Kafka operations:

# Create topic
bin/kafka-topics.sh --create --topic mytopic --bootstrap-server localhost:9092

# List topics
bin/kafka-topics.sh --list --bootstrap-server localhost:9092

# Produce messages
bin/kafka-console-producer.sh --topic mytopic --bootstrap-server localhost:9092

# Consume messages
bin/kafka-console-consumer.sh --topic mytopic --from-beginning --bootstrap-server localhost:9092

Benefits

Kafka handles real-time data at scale. Its durability and performance make it ideal for event-driven architectures and data pipelines.

Download Kafka