AWS SQS Dead Letter Queue Reprocessing script | Devops Junction

In this article let us see how to reprocess messages which are stuck in SQS Dead Letter Queue

At times, Even if your messages are valid, due to Lambda Concurrency or other issues your message lands in Dead Letter Queue

While there are a few ways to reprocess messages in the DLQ the most prominent way is to push the message back to the source SQS Queue

Once the message is back in the Source SQS queue it would get reprocessed by the Lambda Trigger or by other subscribers

In case you want to browse the Messages in SQS Queue including Dead Letter Queue, you can use our SQSCLI tool

 

SQSCLI – GUI Desktop App to Browse SQS Messages

Now let us go ahead and move our messages from Dead Letter Queue to the Source Queue or Original Queue

Here is the Python code that read messages one by one from the DLQ ( Dead Letter Queue) and push to the Original Queue

import boto
import boto3
import json
import time


DLQ="some-dead-letter-queue"
ORIGINALQ="some-original-queue"

sqs = boto3.resource('sqs')
DLqueue = sqs.get_queue_by_name(QueueName=DLQ)
PrimaryQueue = sqs.get_queue_by_name(QueueName=ORIGINALQ)

no_of_messages = DLqueue.attributes['ApproximateNumberOfMessages']


while int(no_of_messages) > 0 :
    print ("Number of messages in DLQ: " + no_of_messages)
    for message in DLqueue.receive_messages(MaxNumberOfMessages=10):
        print("\n")
        with open('messages.json', 'a') as outfile:
            outfile.writelines(message.body)

        #time.sleep(1)
        # Process the message
        try:
            response = PrimaryQueue.send_message(
                QueueUrl=PrimaryQueue.url,
                MessageBody=message.body,
                DelaySeconds=10)
            print(response)
            if response['ResponseMetadata']['HTTPStatusCode'] == 200:
                print("Message sent to primary queue, deleting from DLQ")
                message.delete()
        except Exception as e:
            print(e)
            print("Failed to send message to primary queue")
            exit(1)
    no_of_messages = DLqueue.attributes['ApproximateNumberOfMessages']

You have to update the DLQ and OriginalQ variables on the source code before executing it

DLQ="some-dead-letter-queue" 
ORIGINALQ="some-original-queue"

This script reads a message from SQS Dead Letter Queue one by one and pushes it to Source Queue

As it processes each message, it writes to a file named messages.json for backup and auditing.

This is just an initial version and this can be modified to suit your needs or expanded further

Feel free to share if you have better solutions in the comments section.

Cheers

Sarav AK

---