Guaranteeing Delivery of Messages with AWS SQS

~ Oleg Dulin
Delivery Photo credit Ed Yourdon
Delivery
Photo credit Ed Yourdon

I wanted to scratch an itch and get feedback from the open-source community. So, I put together a little Github project that I like to call SQS-RetryQueue.

Amazon SQS can be utilized to guarantee delivery and processing of messages. This project serves the following purposes:

  1. Demonstrate an example of using AWS SQS with Java to send and receive messages.
  2. Provide an algorithm for retrying failed deliveries.
  3. Provide an approach to keeping SQS costs to a minimum while maintaining real-time processing of messages.
  4. Seek feedback on the approach from the open-source community

AWS SQS is priced by request. One of the goals should be to minimize costs.

Processing of messages can happen on either the server that sent the message or any other server subscribing to this queue. The goal is to begin processing messages as soon as possible.

Each receiver thread acts as follows:

wait on the monitor object for up to visibilityTimeout
while there are messages on the queue:
    receive message
    try processing message
    if processing was successful, delete the message

Sending a message then involves the following:

send the message
notify all receivers waiting on the monitor object

I wanted to get this out of the way for some time. So, here. it is! Any feedback is greatly appreciated.

Published by Oleg Dulin

Chief Software Architect, Staff+ Engineer, CTO, Trusted Advisor. I build PaaS and write my thoughts about it. All opinions are mine and do not represent my employers or customers.

3 thoughts on “Guaranteeing Delivery of Messages with AWS SQS”

  1. Pingback: Top Ten Differences Between ActiveMQ and Amazon SQS | The Dulin Report

  2. if the sender and the receiver are two different instances, then the step
    notify all receivers waiting on the monitor object

    wouldn’t work, would it?

  3. This entire approach in this post means that both instances can send and receive messages. If you have a use case where you need a sender on one instance and receive on another, this approach may not be adequate. I can address it separately in another post.

Comments are closed.