boto3
aws-lambda-power-tuning
boto3 | aws-lambda-power-tuning | |
---|---|---|
36 | 36 | |
8,703 | 5,163 | |
0.6% | - | |
9.7 | 8.7 | |
4 days ago | 4 days ago | |
Python | JavaScript | |
Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
boto3
-
Bug in std:shared_mutex on Windows
Former AWS here.
My literal job for the last part of my time at AWS was "help triage bugs in the AWS SDK." This is by far the best repro I've ever seen for such an in-depth event.
Most of the tickets you get in open ticket trackers are incomplete [ https://github.com/boto/boto3/issues/4011 ] nonsensical [ https://github.com/boto/boto3/issues/4018 ] or weird [ https://github.com/boto/boto3/issues/358 ].
-
AWS Boto3: Clients vs Resources - DynamoDB
Recently, my colleague brought up the difficulty of using the AWS SDK for Python (Boto3) while working with DynamoDB, especially the cumbersome mapping of AttributeValue objects on the Table operations. One of the easiest ways to get around this difficulty is to switch from the clients interface to the resources interface.
- Asynchronous Python lib to work with Amazon SQS
-
Beginning Python: Project Management With PDM
A majority of software in the modern world is built upon various third party packages. These packages help offload work that would otherwise be rather tedious. This includes interacting with cloud APIs, developing scientific applications, or even creating web applications. As you gain experience in python you'll be using more and more of these packages developed by others to power your own code. In this example I've decided to expand our math functionality with NumPy. pdm add is what's used to add dependencies like this to our project:
-
Creating RSS feeds for language/module specific AWS SDK updates
The updates could be parsed from the github repo's CHANGELOG files (ex: javascript, java, python). I'm picturing an RSS feed generated for a specific language and module (ex: python s3, javascript s3, java sqs)
-
Teaching boto3 to store floats and datetime objects in DynamoDB
This can be quite annoying because it makes you wonder why the high-level API isn't able to deal with these common data types. Part of the reason for this is most likely that floats in Python can be counter-intuitive, so Decimal is a better data type if you want numbers to behave as non-computer-scientists expect it. To learn more about these complexities, check out this discussion on GitHub about implementing float support in boto3 and the Python documentation on the subject. Additionally, DynamoDB has no native DateTime data type, so there is no straightforward mapping.
-
Interacting with Amazon S3 using AWS Data Wrangler (awswrangler) SDK for Pandas: A Comprehensive Guide
AWS Data Wrangler is a Python library that simplifies the process of interacting with various AWS services, built on top of some useful data tools and open-source projects such as Pandas, Apache Arrow and Boto3. It offers streamlined functions to connect to, retrieve, transform, and load data from AWS services, with a strong focus on Amazon S3.
-
Migrate 5 TB S3 bucket from one AWS account to another
Alternatively, you could create a Python script using either Boto3 or her asynchronous sister, aioBoto3 that will spin through the contents of the origin bucket and move it over to the destination.
-
Growing Outside of Work: My Journey with the Cloud Resume Challenge
Once my site was stood up, I needed to build out the user count API. Through the console, I set up a DynamoDB table and created a user count item. Getting my lambda to interface with AWS resources was a breeze with the Boto3 SDK. You can see my Python code that increments the user count whenever someone visits the site here. The key is the usage of the update_item method that comes from Boto3.
-
Logging code mess
If you want to get a feel for what kind of logging and how much logging is done in projects, boto3 is a very widely used SDK created by Amazon: https://github.com/boto/boto3
aws-lambda-power-tuning
-
Optimizing Costs in the Cloud: Embracing a FinOps Mindset
Sometimes, changing services, like opting for HTTP over REST API Gateway, leveraging tools like Lambda Powertuning to optimize functions, or reducing a CloudWatch log retention and changing log level, can lead to significant savings.
-
AWS SnapStart - Part 13 Measuring warm starts with Java 21 using different Lambda memory settings
In case of not enabling SnapStart for the Lambda function we observed that increasing memory reduces the warm execution time for our use case especially for p>90. As adding more memory to the Lambda function is also a cost factor, the sweet spot between cold and warm start time and cost is somewhere between 768 and 1204 MB memory setting for the Lambda function for our use case. You can use AWS Lambda Power Tuning for very nice visualisations.
-
How to enhance your Lambda function performance with memory configuration?
The aws lambda power tuning tool helps optimise the Lambda performance and cost in a data-driven manner. Let's try it out:
-
Controlling Cloud Costs: Strategies for keeping on top of your AWS cloud spend
For Lambda, a very useful tool to help optimise is the AWS Lambda Power Tuning tool, released by Alex Casalboni, Developer Advocate at AWS: https://github.com/alexcasalboni/aws-lambda-power-tuning
-
Best way to decrease latency (API <-> Lambda <-> Dynamodb)
Lambda memory affects not only the CPU performance and and host execution priority, but also network performance. Be wary though as the price scales linearly. You can use a tool like Lambda Power Tuning to find the sweet spot for your application. https://github.com/alexcasalboni/aws-lambda-power-tuning
-
How to optimize your lambda functions with AWS Lambda power tuning
This tool, which is open source and available here, takes the form of a Step Function that is deployed on your AWS account. The purpose of this Step Function is to run your lambda with different memory configurations several times and output a comparison in the form of a graph (or JSON) to try to find the optimal balance between cost and execution time. There are three possible optimization modes: cost, execution time, or a "balanced" mode where it tries to find a balance between the two.
-
Developers Journey to AWS Lambda
The AWS Documentation's Memory and Computing Power page is a good starting point. To avoid configuring it manually, it's worth checking out AWS Lambda Power Tuning, which will help you find the sweet spot.
-
Guide to Serverless & Lambda Testing — Part 2 — Testing Pyramid
Utilizing tools such as AWS X-Ray, AWS Lambda Power Tuning, and AWS Lambda Powertools tracer utility is recommended. Read more about it here.
-
Tunea tus funciones Lambda
Install the AWS SAM CLI in your local environment. Configure your AWS credentials (requires AWS CLI installed): $ aws configure Clone this git repository: $ git clone https://github.com/alexcasalboni/aws-lambda-power-tuning.git Build the Lambda layer and any other dependencies (Docker is required): $ cd ./aws-lambda-power-tuning $ sam build -u sam build -u will run SAM build using a Docker container image that provides an environment similar to that which your function would run in. SAM build in-turn looks at your AWS SAM template file for information about Lambda functions and layers in this project. Once the build has completed you should see output that states Build Succeeded. If not there will be error messages providing guidance on what went wrong. Deploy the application using the SAM deploy "guided" mode: $ sam deploy -g
-
AWS Serverless Production Readiness Checklist
Use AWS Lambda Power Tuning to balance cost and performance.
What are some alternatives?
terraform - Terraform enables you to safely and predictably create, change, and improve infrastructure. It is a source-available tool that codifies APIs into declarative configuration files that can be shared amongst team members, treated as code, edited, reviewed, and versioned.
json-schema-to-ts - Infer TS types from JSON schemas 📝
aws-cli - Universal Command Line Interface for Amazon Web Services
dynamodb-toolbox - A simple set of tools for working with Amazon DynamoDB and the DocumentClient
apache-libcloud - Apache Libcloud is a Python library which hides differences between different cloud provider APIs and allows you to manage different cloud resources through a unified and easy to use API.
middy - 🛵 The stylish Node.js middleware engine for AWS Lambda 🛵
boto - For the latest version of boto, see https://github.com/boto/boto3 -- Python interface to Amazon Web Services
aws-sam-cli - CLI tool to build, test, debug, and deploy Serverless applications using AWS SAM
Telethon - Pure Python 3 MTProto API Telegram client library, for bots too!
aws-graviton-getting-started - Helping developers to use AWS Graviton2 and Graviton3 processors which power the 6th and 7th generation of Amazon EC2 instances (C6g[d], M6g[d], R6g[d], T4g, X2gd, C6gn, I4g, Im4gn, Is4gen, G5g, C7g[d][n], M7g[d], R7g[d]).
google-api-python-client - 🐍 The official Python client library for Google's discovery based APIs.
failure-lambda - Module for fault injection into AWS Lambda