Simple Queue Service (SQS)
5 minute read
AWS SQS is a fully managed distributed message queuing service. SQS is shipped with the LocalStack Community version and is extensively supported and tested.
Getting started
Trying to run the examples in the official AWS developer guide against LocalStack is a great place to start.
Assuming you have awslocal
installed you can also try the following commands:
$ awslocal sqs create-queue --queue-name sample-queue
{
"QueueUrl": "http://localhost:4566/000000000000/sample-queue"
}
$ awslocal sqs list-queues
{
"QueueUrls": [
"http://localhost:4566/000000000000/sample-queue"
]
}
$ awslocal sqs send-message --queue-url http://localhost:4566/000000000000/sample-queue --message-body test
{
"MD5OfMessageBody": "098f6bcd4621d373cade4e832627b4f6",
"MessageId": "74861aab-05f8-0a75-ae20-74d109b7a76e"
}
SQS Query API
The SQS Query API exposes SQS Queue URLs as endpoints and allows you to make HTTP requests directly against the Queue. LocalStack also supports the Query API.
LocalStack makes it easy to test SQS Query API calls without having to sign or add AUTHPARAMS
to your HTTP requests.
For example, you could send a SendMessage
command using a MessageBody
attribute with a simple curl command:
$ curl "http://localhost:4566/000000000000/sample-queue?Action=SendMessage&MessageBody=hello%2Fworld"
<?xml version='1.0' encoding='utf-8'?>
<SendMessageResponse xmlns="http://queue.amazonaws.com/doc/2012-11-05/"><SendMessageResult><MD5OfMessageBody>c6be4e95a26409675447367b3e79f663</MD5OfMessageBody><MessageId>466144ab-1d03-4ec5-8d70-97535b2957fb</MessageId></SendMessageResult><ResponseMetadata><RequestId>JU40AF5GORK0WSR75MOY3VNQ1KZ3TAI7S5KAJYGK9C5P4W4XKMGF</RequestId></ResponseMetadata></SendMessageResponse>
Adding the Accept: application/json
header will make the server return JSON:
curl -H "Accept: application/json" "http://localhost:4566/000000000000/my-queue?Action=SendMessage&MessageBody=hello%2Fworld"
{"SendMessageResponse": {"SendMessageResult": {"MD5OfMessageBody": "c6be4e95a26409675447367b3e79f663", "MessageId": "748297f2-4abd-4ec2-afc0-4d1a497fe604"}, "ResponseMetadata": {"RequestId": "XEA5L5AX16RTPET25U3TIRIASN6KNIT820WIT3EY7RCH7164W68T"}}}
Note
In previous releases, an empty HTTP request to a queue would return a<GetQueueUrlResponse>
.
This has since been aligned to the behavior of AWS, which returns a <UnknownOperationException/>
.
To run a GetQueueUrl
request, add the ?Action=GetQueueUrl&QueueName=<QueueName>"
query string to the URL.Configuration
Queue URLs
You can control the format of the generated Queue URLs by setting the environment variable SQS_ENDPOINT_STRATEGY
when starting LocalStack to one of the following values.
Value | URL format | Description |
---|---|---|
domain | <region>.queue.localhost.localstack.cloud:4566/<account_id>/<queue_name> | This strategy behaves like the SQS legacy service endpoints, and uses localhost.localstack.cloud to resolve to localhost. When using us-east-1 , the <region>. prefix is omitted. |
path | localhost:4566/queue/<region>/<account_id>/<queue_name> | An alternative that can be useful if you cannot resolve LocalStack’s localhost domain |
off | localhost:4566/<account_id>/<queue_name> | Currently the default for backwards compatibility. Since this format does not encode the region, you cannot query queues that exist in different regions with the same name. |
Enabling PurgeQueue errors
AWS only allows one call to PurgeQueue
every 60 seconds.
See the PurgeQueue API Reference.
LocalStack disables this behavior by default, but it can be enabled by starting LocalStack with SQS_DELAY_PURGE_RETRY=1
.
Enabling QueueDeletedRecently errors
AWS does not allow creating a queue with the same name for 60 seconds after it was deleted.
See the DeleteQueue API Reference.
LocalStack disables this behavior by default, but it can be enabled by starting LocalStack with SQS_DELAY_RECENTLY_DELETED=1
.
Disable CloudWatch Metrics Reporting
Sending, receiving, and deleting SQS messages will automatically trigger CloudWatch metrics. This AWS parity feature is enabled by default, but can be deactivated. Deactivating CloudWatch metrics can improve the performance of SQS message operations, but will disable any integration with CloudWatch such as triggering alarms.
Additionally, metrics about Approximate*
messages are send to CloudWatch by default once every minute, but the interval (in seconds) can also be configured using SQS_CLOUDWATCH_METRICS_REPORT_INTERVAL=120
.
SQS_DISABLE_CLOUDWATCH_METRICS=1
will disable all CloudWatch metrics for SQS (including Approximate*
metrics).
Developer endpoints
Our SQS implementation provides additional endpoints for developers under /_aws/sqs
, that allow you to inspect queues without side effects.
For instance, sometimes you want to peek into queues without actually performing a ReceiveMessage
operation.
Peeking into queues
The endpoint /_aws/sqs/messages
allows you to access all messages within a queue without triggering the visibility timeout our modifying access metrics.
This can be used, for example, in tests to wait until a particular message has arrived in the queue.
The endpoint is fully compatible with the ReceiveMessage
operation from the SQS API.
The endpoint returns all messages in the queue, and all attributes and system attributes by default.
It ignores any other parameters from the ReceiveMessage
operation, except the QueueUrl
.
Here are some examples of how to call the endpoint.
You can either call the endpoint with the query argument QueueUrl
, or use the path-based endpoint.
These two calls would be equivalent:
http://localhost:4566/_aws/sqs/messages?QueueUrl=http://queue.localhost.localstack.cloud:4566/000000000000/my-queue
http://localhost:4566/_aws/sqs/messages/us-east-1/000000000000/my-queue
XML response
You can call the endpoint directly to return the raw AWS XML response.
curl "http://localhost:4566/_aws/sqs/messages?QueueUrl=http://queue.localhost.localstack.cloud:4566/000000000000/my-queue"
import requests
response = requests.get(
url="http://localhost:4566/_aws/sqs/messages",
params={"QueueUrl": "http://queue.localhost.localstack.cloud:4566/000000000000/my-queue"},
)
print(response.text) # outputs the response XML
Example output:
<?xml version='1.0' encoding='utf-8'?>
<ReceiveMessageResponse xmlns="http://queue.amazonaws.com/doc/2012-11-05/">
<ReceiveMessageResult>
<Message>
<MessageId>6a736e5d-4997-4895-8c96-b65a2d7dd600</MessageId>
<MD5OfBody>5d41402abc4b2a76b9719d911017c592</MD5OfBody>
<Body>hello</Body>
<Attribute>
<Name>SenderId</Name>
<Value>000000000000</Value>
</Attribute>
<Attribute>
<Name>SentTimestamp</Name>
<Value>1672853965675</Value>
</Attribute>
<Attribute>
<Name>ApproximateReceiveCount</Name>
<Value>0</Value>
</Attribute>
<Attribute>
<Name>ApproximateFirstReceiveTimestamp</Name>
<Value>1672855121076</Value>
</Attribute>
<ReceiptHandle>SQS/BACKDOOR/ACCESS</ReceiptHandle>
</Message>
<Message>
<MessageId>173c5aee-503a-4249-90be-159e0d427b48</MessageId>
<MD5OfBody>7d793037a0760186574b0282f2f435e7</MD5OfBody>
<Body>world</Body>
<Attribute>
<Name>SenderId</Name>
<Value>000000000000</Value>
</Attribute>
<Attribute>
<Name>SentTimestamp</Name>
<Value>1672853968176</Value>
</Attribute>
<Attribute>
<Name>ApproximateReceiveCount</Name>
<Value>0</Value>
</Attribute>
<Attribute>
<Name>ApproximateFirstReceiveTimestamp</Name>
<Value>1672855121076</Value>
</Attribute>
<ReceiptHandle>SQS/BACKDOOR/ACCESS</ReceiptHandle>
</Message>
</ReceiveMessageResult>
<ResponseMetadata>
<RequestId>KR3H1IN3JQ4LO1592IMGK2JLH8HW3J0Y4LRY1TVW2SAFGZFVXJGI</RequestId>
</ResponseMetadata>
</ReceiveMessageResponse>
JSON response
If you prefer a JSON response, you can add the Accept: application/json
header.
curl -H "Accept: application/json" \
"http://localhost:4566/_aws/sqs/messages?QueueUrl=http://queue.localhost.localstack.cloud:4566/000000000000/my-queue"
import requests
response = requests.get(
url="http://localhost:4566/_aws/sqs/messages",
params={"QueueUrl": "http://queue.localhost.localstack.cloud:4566/000000000000/my-queue"},
)
print(response.text) # outputs the response XML
The output JSON will look something like this:
{
"ReceiveMessageResponse": {
"ReceiveMessageResult": {
"Message": [
{
"MessageId": "6a736e5d-4997-4895-8c96-b65a2d7dd600",
"MD5OfBody": "5d41402abc4b2a76b9719d911017c592",
"Body": "hello",
"Attribute": [
{
"Name": "SenderId",
"Value": "000000000000"
},
{
"Name": "SentTimestamp",
"Value": "1672853965675"
},
{
"Name": "ApproximateReceiveCount",
"Value": "0"
},
{
"Name": "ApproximateFirstReceiveTimestamp",
"Value": "1672855535794"
}
],
"ReceiptHandle": "SQS/BACKDOOR/ACCESS"
},
{
"MessageId": "173c5aee-503a-4249-90be-159e0d427b48",
"MD5OfBody": "7d793037a0760186574b0282f2f435e7",
"Body": "world",
"Attribute": [
{
"Name": "SenderId",
"Value": "000000000000"
},
{
"Name": "SentTimestamp",
"Value": "1672853968176"
},
{
"Name": "ApproximateReceiveCount",
"Value": "0"
},
{
"Name": "ApproximateFirstReceiveTimestamp",
"Value": "1672855535794"
}
],
"ReceiptHandle": "SQS/BACKDOOR/ACCESS"
}
]
},
"ResponseMetadata": {
"RequestId": "TF87187MUBXJHA39J4Y6OVQG57J51OEEMX62UWYBUQJKC8YVID3P"
}
}
}
Using an AWS client
Since the endpoint is compatible with the SQS ReceiveMessage
operation, you can use the endpoint as endpoint URL parameter in your AWS client call.
aws --endpoint-url=http://localhost:4566/_aws/sqs/messages sqs receive-message \
--queue-url=http://queue.localhost.localstack.cloud:4566/000000000000/my-queue
import boto3
sqs = boto3.client("sqs", endpoint_url="http://localhost:4566/_aws/sqs/messages")
response = sqs.receive_message(QueueUrl="http://queue.localhost.localstack.cloud:4566/000000000000/my-queue")
print(response)
Example output:
{
"Messages": [
{
"MessageId": "6a736e5d-4997-4895-8c96-b65a2d7dd600",
"ReceiptHandle": "SQS/BACKDOOR/ACCESS",
"MD5OfBody": "5d41402abc4b2a76b9719d911017c592",
"Body": "hello",
"Attributes": {
"SenderId": "000000000000",
"SentTimestamp": "1672853965675",
"ApproximateReceiveCount": "0",
"ApproximateFirstReceiveTimestamp": "1672854900237"
}
},
{
"MessageId": "173c5aee-503a-4249-90be-159e0d427b48",
"ReceiptHandle": "SQS/BACKDOOR/ACCESS",
"MD5OfBody": "7d793037a0760186574b0282f2f435e7",
"Body": "world",
"Attributes": {
"SenderId": "000000000000",
"SentTimestamp": "1672853968176",
"ApproximateReceiveCount": "0",
"ApproximateFirstReceiveTimestamp": "1672854900237"
}
}
]
}
Known limitations and differences to AWS
- The
ApproximateReceiveCount
attribute of a message will be reset to 0 when the message moves to a DLQ.