Fan Out Pattern

Basics

• When a message needs to be send to multiple SQS queues, this pattern is useful.

• A message is first pushed to an SNS topic and SQS queues are subscribers to these topics.

• Due to this fan out pattern,

  • One can easily add new queues as subscriber

  • Application architecture becomes fully decoupled as there is no data loss.

  • Allows for data persistence, delayed processing and introduce retries.

• Make sure your SQS queue access policy allows for SNS to write.

• Cross-Region Delivery: SNS Topics in one region can send message to SQS queues in other region/s provided access has been given.

Diagram

SQS-SNS_Fanout

Applications

S3 and Fanout Pattern

• One of the application of this pattern, is for combination of event type and prefix, you can use only one S3 Event Rule.

• If you want to send the same S3 event to many SQS queues, use Fan-Out pattern.

S3 And Fanout Integration

SNS -> Kinesis Data Firehose -> S3 Kinesis

• SNS can be directly integrated to Kinesis Data Firehose.

• This integration can be leveraged to send data to S3.

SNS and S3 Integrations