Create real-time data platform with Kafka Confluent Paltform Multi broker on AWS machine
- AMI: Amazon Red Hat Linux 8 AMI (HVM), SSD Volume Type 64 bit
- Network: vpc by default + No precense subnet + Public IP Use subnet setting (enable)
- Storage: Root disk 40 GB + Add Volume 100GB Magnetic (delete on termination)
- Use security group created (ClsKsqlChong)
- Create or use new .pem key (copy this file in a secure place)
- Create new security group with the name: ClsKsqlChong
Type Protocol Port Source
All Traffic All 0 - 65535 Anywhere
SSH TCP 22 Anywhere
CUSTOM TCP TCP 2181 Anywhere
CUSTOM TCP TCP 2888 Anywhere
CUSTOM TCP TCP 3888 Anywhere
CUSTOM TCP TCP 9092 Anywhere
CUSTOM TCP TCP 8083 Anywhere
CUSTOM TCP TCP 8088 Anywhere- Install wget
- Install nano
- Install Java
- Install and Configure Kafka Confluent Multibroker
- Create,describe topic
- Testing producer messages to the topic
- Testing consumer messages from the topic
- Generate some data to topic
- List Kafka availables connectors
- Create streams, table
- Create source connector with JDBC to SQL SERVER instance