This project demonstrates a real-time streaming pipeline using:

• Kafka for log ingestion
• Spark Streaming for real-time processing
• Aerospike for high-speed data storage
• Prometheus and Grafana for monitoring

docker-compose up -d --build

• Kafka Topic: user-logs
• Prometheus: http://localhost:9090
• Grafana: http://localhost:3005
• Spark UI: http://localhost:4040

• Python dependencies for host scripts: kafka-python, faker, aerospike