Optimizing Amazon Easy Queue Service (SQS) for pace and scale


Voiced by Polly

After a number of public betas, we launched Amazon Easy Queue Service (Amazon SQS) in 2006. Almost twenty years later, this totally managed service remains to be a basic constructing block for microservices, distributed techniques, and serverless purposes, processing over 100 million messages per second at peak instances.

As a result of there’s all the time a greater manner, we proceed to search for methods to enhance efficiency, safety, inner effectivity, and so forth. After we do discover a potential option to do one thing higher, we’re cautious to protect current conduct, and sometimes run new and outdated techniques in parallel to permit us to check outcomes.

As we speak I wish to inform you how we lately made enhancements to Amazon SQS to cut back latency, enhance fleet capability, mitigate an approaching scalability cliff, and cut back energy consumption.

Bettering SQS
Like many AWS companies, Amazon SQS is applied utilizing a group of inner microservices. Let’s give attention to two of them as we speak:

Buyer Entrance-Finish – The shopper-facing front-end accepts, authenticates, and authorizes API calls equivalent to CreateQueue and SendMessage. It then routes every request to the storage back-end.

Storage Again-Finish -This inner microservice is accountable for persisting messages despatched to straightforward (non-FIFO) queues. Utilizing a cell-based mannequin, every cluster within the cell accommodates a number of hosts, every buyer queue is assigned to a number of clusters, and every cluster is accountable for a mess of queues:

Connections – Outdated and New
The unique implementation used a connection per request between these two companies. Every front-end had to hook up with many hosts, which mandated using a connection pool, and likewise risked reaching an final, hard-wired restrict on the variety of open connections. Whereas it’s usually potential to easily throw {hardware} at issues like this and scale out, that’s not all the time one of the simplest ways. It merely strikes the second of fact (the “scalability cliff”) into the longer term and doesn’t make environment friendly use of assets.

After fastidiously contemplating a number of long-term options, the Amazon SQS workforce invented a brand new, proprietary binary framing protocol between the shopper front-end and storage back-end. The protocol multiplexes a number of requests and responses throughout a single connection, utilizing 128-bit IDs and checksumming to forestall crosstalk. Server-side encryption offers an extra layer of safety in opposition to unauthorized entry to queue knowledge.

It Works!
The brand new protocol was put into manufacturing earlier this 12 months and has processed 744.9 trillion requests as I write this. The scalability cliff has been eradicated and we’re already searching for methods to place this new protocol to work in different methods.

Efficiency-wise, the brand new protocol has diminished dataplane latency by 11% on common, and by 17.4% on the P90 mark. Along with making SQS itself extra performant, this transformation advantages companies that construct on SQS as effectively. For instance, messages despatched via Amazon Easy Notification Service (Amazon SNS) now spend 10% much less time “inside” earlier than being delivered. Lastly, as a result of protocol change, the present fleet of SQS hosts (a mixture of X86 and Graviton-powered situations) can now deal with 17.8% extra requests than earlier than.

Extra to Come
I hope that you’ve got loved this little peek contained in the implementation of Amazon SQS. Let me know within the feedback, and I’ll see if I can discover some extra tales to share.