Our great sponsors
-
SurveyJS
Open-Source JSON Form Builder to Create Dynamic Forms Right in Your App. With SurveyJS form UI libraries, you can build and style forms in a fully-integrated drag & drop form builder, render them in your JS app, and store form submission data in any backend, inc. PHP, ASP.NET Core, and Node.js.
-
opentracing-javascript
Discontinued OpenTracing API for Javascript (both Node and browser). 🛑 This library is DEPRECATED! https://github.com/opentracing/specification/issues/163
sqsiphon, installable via npm install @knockaway/sqsiphon, is a library we wrote to help us improve our webhook event ingestion. Initially, we were relying on the scalable nature of AWS Lambda to handle our incoming webhooks. This worked for a fair amount of time, but as we grew, so did the number of incoming webhook events we needed to process. This resulted in Lambda overloading our data stores. To solve the problem, we built out a system that receives the webhook events and puts them into an AWS SQS FIFO queue.
As we built out this new system, we looked at the current module ecosystem to determine if anything would fit our needs to keep this queue cleared. Unfortunately, the best module we could find, sqs-consumer, did not support FIFO queues and the issues indicated they did not have any intention to do so. Therefore, we wrote sqsiphon. Our library heavily borrows from the public interface of sqs-consumer, but is implemented in a much different way. In particular, sqsiphon leverages the Node.js event loop through setImmediate to build the polling mechanism instead of relying on SQS's "long polling" feature. Additionally, sqsiphon utilizes the JavaScript prototype to reduce reliance on expensive closures. As a bonus, OpenTracing support is built-in to sqsiphon so that applications built with it can be monitored effectively. The result is a library that can poll SQS extremely quickly while requiring very little overhead.
As we built out this new system, we looked at the current module ecosystem to determine if anything would fit our needs to keep this queue cleared. Unfortunately, the best module we could find, sqs-consumer, did not support FIFO queues and the issues indicated they did not have any intention to do so. Therefore, we wrote sqsiphon. Our library heavily borrows from the public interface of sqs-consumer, but is implemented in a much different way. In particular, sqsiphon leverages the Node.js event loop through setImmediate to build the polling mechanism instead of relying on SQS's "long polling" feature. Additionally, sqsiphon utilizes the JavaScript prototype to reduce reliance on expensive closures. As a bonus, OpenTracing support is built-in to sqsiphon so that applications built with it can be monitored effectively. The result is a library that can poll SQS extremely quickly while requiring very little overhead.
Related posts
- NestJS tip: how to change HTTP server timeouts
- 7 Frameworks, One SAML Jackson - Your Open Source Single Sign-On Solution
- Full Stack Web Development Concept map
- Don't go all-in Clean Architecture: An alternative for NestJS applications
- Utilizando Testcontainers para Testes de Integração com NestJS e Prisma ORM