Kinesis Or Kafka

Kafka architecture

Pasted image 20230923115422.png

Multiple brokers which one brokers can contain multiple topics

AWS Kinesis architecture

Pasted image 20230923115505.png

Multi stream and each stream has different shard

Kafka or Kinesis

  • Kafka has higher through put of 30,000 messages per seconds whereas Kinesis can only max around 20,000 messages per second
  • Kinesis can have maximum of 3 machines per data-centers whereas kafka there is no limit
  • For Kinesis it's simpler as you don't need to manage cluster, for kafka you have to manage cluster, however amazon have Fully Managed Apache Kafka – Amazon MSK – Amazon Web Services now to support to manage kafka.

Pasted image 20230923115729.png

Overall if you can afford choose Kafka for a better performance