awslogs VS sst

Using the official CLI (aws logs get-log-events) or https://github.com/jorgebastida/awslogs is pretty close to SSH-ing and grepping.

cw is a native executable targeting your OS, and not needed external dependencies such as pip and npm. Compared to awslogs which is famous helpful tool for CloudWatch Logs1, cw is written in golang and faster.

Its a command line tool but some people I know also use awslogs

Not a full solution, but when I was doing this I really got to love the awslogs utility:

https://github.com/jorgebastida/awslogs

It allows you to stream Cloudwatch logs from the command line, so you can grep them, save them to files, etc... (The web based Cloudwatch interface is terrible.)

Another suggestion is to try to modularize the core business logic in your lambda such that you separate the lambda-centric stuff from the rest of it. Obviously, though, if "the rest of it" is hitting other AWS services, you're going to hit the same testing roadblock.

Or you can try mocking, which may or may not provide much value for you. There's a python library for that, (moto), but it's not 100% up to date wrt AWS services/interfaces, last I had checked. Might be worth a try though.

https://github.com/spulec/moto

Cloudwatch Logs is pretty meh visually, but awslogs can give you a pretty good `tail -f`-like experience, and Insights is pretty good. Cloudwatch Metric Filters give you a 'StatsD'-like experience, in that you can log out a certain message or code and then use its appearance as a metric.

Check out https://github.com/jorgebastida/awslogs , you can define a `--start`, and it also has a `--watch`, and can be piped the `grep` or whatever you want. It's a pretty flexible tool.

📖 awslogs - a simple command-line tool for querying groups, streams, and events from Amazon CloudWatch logs.

We see some great results from using these in conjunction with frameworks such as SST or Serverless, and also some real spaghetti from people who organically proliferate 100’s of functions over time and lose track of how they relate to each other or how to update them safely across time and service. Buyer beware!

> But if you have a sufficiently large enough API surface, doing one lambda per endpoint comes with a lot of pain as well. Packaging and deploying all of those artifacts can be very time consuming, especially if you have a naive approach that does a full rebuild/redeploy every time the pipeline runs.

Yeah, thankfully SST [0] does the heavy lifting for me. I've tried most of the solutions out there and SST was where I was the happiest. Right now I do 1 functions per endpoint. I structure my code like url paths mostly, 1 stack per final folder, so that the "users" folder maps to "/users/*" and inside I have get/getAll/create/update/delete files that map to GET X/id, GET X, POST X, POST X/id, DELETE/id. It works out well, it's easy to reason about, and deploys (a sizable a backend) in about 10min on GitHub Actions (which I'm going to swap out probably for something faster).

I agree with the secrets/permissions aspect and I like that it's stupid-simple for me to attach secrets/permissions at a low level if I want.

I use NodeJS and startup isn't horrible and once it's up the requests as very quick. For my needs, an the nature of the software I'm writing, lambda makes a ton of sense (mostly never used, but when it's used it's used heavily and needs to scale up high).

[0] https://sst.dev

We will start by building a project with SST that provisions an API Gateway, a Lambda, and an S3 bucket. Once implemented, we'll look into testing for concurrent write conflicts or exceeding capacity limits.

I recently deployed a node websocket server using the SST Service construct. Until this point my stack had been functions and buckets. While I had no users 😢, I also had no costs 🤡.

Most my personal and side-business projects have very spiky load or just low load in general. Because of that I love using AWS Lambda as my backend since it scales to 0 and scales to whatever you have your limits set at.

I use SST [0] for my backend with NodeJS (TypeScript) and Vue (Quasar) for my frontend. For my database I use either Postgres or DynamoDB if the fit is right (Single Table Design is really neat). For Postgres I like Neon [1] though their recent pricing changes make it less appealing.

[0] https://sst.dev

[1] https://neon.tech

Yup. Entire core business product for a succeeding startup, though it's a small team of contributors (<10), and a much smaller platform team. Serverless backend started in 2018. Been a blessing in many regards, but it has its warts (often related to how new this architecture is, and of course we've made our own mistakes along the way).

I really like the model of functions decoupled through events. Big fan of that. It's very flexible and iterative. Keep that as your focus and it's great. Be careful of duplicating config, look for ways to compose/reuse (duh, but definitely a lesson learnt) and same with CI, structure your project so it can use something off-the-shelf like serverless-compose. Definitely monorepo/monolith it, I'd be losing my mind with 100-150 repos/"microservices" with a team this size. If starting now I'd maybe look at SST framework[0] because redeploying every change during development gets old fast

I couldn't go back to any other way to be honest, for cloud-heavy backends at least. By far the most productive I've ever been

Definitely has its warts though, it's not all roses.

[0] http://sst.dev

Along the way, you find an excellent framework, SST. Which is much faster than CDK and provides a better DX1. Here is how you then define your MultiPipelineStack.

SST is a powerful framework that simplifies the development of serverless applications. It offers a straightforward and opinionated approach to defining serverless apps using TypeScript. Built on top of AWS CDK, SST handles the complexity of setting up your serverless infrastructure automatically. SST is an open-source framework and is completely free to use.

SST is an open-source framework designed to facilitate the development and deployment of Serverless stacks on AWS. It operates under the hood by integrating with Amazon CDK. However, its primary benefit is in allowing us to concentrate on creating resources using familiar languages like TypeScript, treating them as Infrastructure as Code (IaC).