Site icon Mailtrap

All You Wanted to Know About Amazon SQS in 10 Minutes

Artur Hebda

When dealing with cloud-based infrastructure, it’s hard to pass by Amazon AWS. If you are considering Amazon SQS for improving your app architecture and need to quickly understand its basics and decide if it’s a fit for your project – you are in the right place. We will go over its basic functionality, main use cases, pricing, and common questions about this system and its integrations. 

How does Amazon SQS work?

Amazon Simple Queue Service (SQS) is a message queuing service that enables you to manage asynchronous communication between applications or microservices.

If you are new to queueing, imagine this process as a communication between two subscribers on a mobile network. There always should be at least two participants: the producer (the one who sends the message) and the consumer (the one who receives it). For direct communication (like a call) both participants must be online and available. 

Asynchronous communication is similar to messaging through a platform or even SMS. In this case, the producer sends a message and if the consumer is not available at the moment, this message will “wait” on an intermediary server. As soon as the consumer is available, the message is delivered and the consumer processes it (reads, forwards to another subscriber, etc.) 

With Amazon SQS, messages can be processed in two ways:

Unlike the conversation, queued messages need to be deleted after processing to avoid their duplication. 

This is a very simplified description of how message queueing works in SQS. To dig deeper, check out the basic Amazon SQS architecture in the official documentation. 

Here are several important terms to understand before you start working with Amazon SQS:

How much does Amazon SQS cost?

Similar to other Amazon services, SQS is charged according to the “pay for what you use” model. In fact, you pay for the invoked requests (1 request = 64 KB chunk of a payload). The first one million requests per month are free. Each following million requests will cost you $0.4 if you use standard queues or $0.5 if you use FIFO. 

It’s important to note that a single API call can have from 1 to 10 messages and is limited to a maximum total payload of 256 KB (4 chunks of 64 KB). Also, you need to remember, that you are charged for the outbound data transfer. 

To be able to predict Amazon SQS cost, you should estimate the number of requests for each type of queue per month as well as the data transfer volume. 

To get all the pricing details and calculate your possible expenses, refer to Amazon SQS pricing. 

How to configure the Amazon SQS queue

You can access Amazon SQS via AWS Management Console or integrate it via API. 

AWS Management Console 

Getting started with the AWS Management Console is easy and straightforward. Just assess SQS form the Services list on your AWS account homepage and follow the instructions. You will get explanations and hints from Amazon at every step. 

Creating a new queue via the AWS Management Console

Here you will be able to choose the type of the queue, make necessary configurations, and then start adding messages right away. Also, you can add metadata for each message, such as name, type, and value. 

Once your message is sent to a queue, it is ready for retrieval and processing by another application. To connect your app with the queue in this Amazon service, you will need to pass the queue URL (which you will find in the queue Details tab).

From the Queue Actions menu, you can request a message to view (by clicking  Start Polling for messages) and then delete it. 

Also, you can subscribe queue to the SNS topic and configure trigger from Lambda function right from the Queue Actions menu. 

For more details and further steps, refer to this guide. 

API integration

Amazon AWS provides SDKs for the most popular programming languages: C++, Go, Java, JavaScript, .Net, Node.js, PHP, Python, and Ruby. You can find them here. 

For SQS, the following automatic functionality is included:

To integrate SQS with your app via API, you will need to construct an Amazon SQS endpoint first, then make GET and POST requests, and interpret responses. For detailed instructions, refer to the SQS Developer Guide. 

When to use Amazon SQS

Amazon offers three services to enable communication between apps: SQS, MQ (Message Broker), and SNS (Simple Notifications Service). 

Amazon MQ is a managed message broker service for Apache ActiveMQ. Amazon recommends using it for migrating your messaging with existing applications to the cloud. 

Amazon SNS is a push notifications service, and using it in combination with SQS is one of the best practices. 

Amazon SQS will be a good fit for new applications built in the cloud. You can use it independently but it’s always a good idea to compose it with other Amazon services like Amazon EC2, Amazon EC2 Container Service (Amazon ECS), and AWS Lambda. In addition, you can store SQS messaging data in Amazon Simple Storage Service (Amazon S3) and Amazon DynamoDB.

In general, message queueing is used to provide application scalability and to decouple the complex back-end operations from the front-end output. 

Here is what else you can do with the queues in Amazon SQS:

And a couple of words about important Amazon SQS limits:

How to test Amazon SQS?

Once you have configured communication between your apps with Amazon SQS, you need to make sure that everything works correctly. Amazon suggests integrating SQS with their AWS X-Ray service, which collects details about your requests and provides options for their further analysis and optimization. 

The setup doesn’t involve much effort. And what is also important, you can use in AWS X-Ray in almost no cost. It has a perpetual free tier with the first 100,000 traces recorded and the first 1,000,000 traces retrieved or scanned each month.

With the integration of AWS X-Ray with Amazon SQS, you can trace messages transferred through an Amazon SQS queue. Tracing is implemented with the help of trace headers. There are two of them supported in Amazon SQS: 

In Amazon SQS, you can send a trace header automatically through the SendMessageBatch or SendMessage call. You will then find the traced queues on the service map in the X-Ray console (as downstream nodes).  

Source

Further, you need to retrieve the trace header and recover trace context. Those should be configured manually in three steps, as per Amazon documentation:

Here is an example of how you can implement Amazon SQS testing with Node.js:

// https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/SQS.html#receiveMessage-property
var params = {
  QueueUrl: "<QUEUE_URL>", // required
  AttributeNames: ["AWSTraceHeader"]
}
sqs.receiveMessage(params, function(err, data) {
  if (err) console.log(err, err.stack)
  else console.log(data)
  data.messages.forEach(function(message) {
    var headerString = message.Attributes["AWSTraceHeader"]
    if (headerString) {
      // comes from aws-xray-sdk-node
      var trace = AWSXRay.utils.processTraceData(headerString)
      trace.Root // trace id or null
      trace.Parent // id of the parent segment or subsegment or nil
      trace.Sampled // "0" means not sampled
    }
  })
})

And here is one more with Ruby: 

# https://docs.aws.amazon.com/sdkforruby/api/Aws/SQS/Client.html#receive_message-instance_method
response = client.receive_message({
  queue_url: "<QUEUE_URL>", # required
  attribute_names: ["AWSTraceHeader"]
})
response.messages.each do |message|
  header_string = message.attributes["AWSTraceHeader"]
  # comes from aws-xray-sdk
  trace_header = XRay::TraceHeader.from_header_string(header_string)
  trace_header.root # trace id or nil
  trace_header.parent_id # id of the parent segment or subsegment or nil
  trace_header.sampled # 0 means not sampled
end

For details, refer to the AWS X-Ray Developer Guide. 

Further steps 

Amazon offers an exhaustive list of services for cloud computing. You definitely benefit from combining them, and Amazon SQS is not an exception: it works perfectly fine with Amazon SNS (a distributed publish-subscribe system) and AWS Lambda (serverless computing platform). Also, you can integrate Amazon SES, SNS, and SQS to manage email notifications you send to clients as well as handling their replies and/or bounces. 

Compared to other services, Amazon is scalable and reliable and offers the most competitive prices on the market. 

Exit mobile version