Why Kafka?
Problems
When multiple source systems want to send data to multiple target systems, earlier every source-target pair had to be integrated via their own integration mechanism.
This made the interaction between source and target system a big ball of mess which made it very complex to maintain.
It lead to lot of integrations to be written based on source system and target systems capabilities like protocol being used, data format, data schema changes etc.
An example shown below,
Solution
As many integrations becomes hard to maintain, this is where Kafka comes to rescue.
Kafka sits between these source and target systems, they can send and receive data from Kafka.
It helps in decoupling source and target systems direct coupling.
Last updated