Get Scout setup in minutes, and let us sweat the small stuff. A couple lines in settings.py is all you need to start monitoring your apps. Sign up for our free tier today. Learn more →
Top 23 Python AWS Projects
-
devops-exercises
Linux, Jenkins, AWS, SRE, Prometheus, Docker, Python, Ansible, Git, Kubernetes, Terraform, OpenStack, SQL, NoSQL, Azure, GCP, DNS, Elastic, Network, Virtualization. DevOps Interview Questions
-
Scout Monitoring
Free Django app performance insights with Scout Monitoring. Get Scout setup in minutes, and let us sweat the small stuff. A couple lines in settings.py is all you need to start monitoring your apps. Sign up for our free tier today.
-
LocalStack
💻 A fully functional local AWS cloud stack. Develop and test your cloud & Serverless apps offline
LocalStack is a cloud service emulator that runs in a single container on your laptop or in your CI environment. With LocalStack, you can run your AWS applications or Lambdas entirely on your local machine without connecting to a remote cloud provider! Whether you are testing complex CDK applications or Terraform configurations, or just beginning to learn about AWS services, LocalStack helps speed up and simplify your testing and development workflow.
-
data-science-ipython-notebooks
Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.
-
Project mention: AWS-CLI: Master AWS from Your Terminal with This Universal CLI Tool | news.ycombinator.com | 2024-08-24
-
awesome-aws
A curated list of awesome Amazon Web Services (AWS) libraries, open source repos, guides, blogs, and other resources. Featuring the Fiery Meter of AWSome.
Awesome AWS
-
-
Project mention: Ask HN: How are you deploying your lambdas these days? | news.ycombinator.com | 2024-06-06
-
InfluxDB
Purpose built for real-time analytics at any scale. InfluxDB Platform is powered by columnar analytics, optimized for cost-efficient storage, and built with open data standards.
-
prowler
Prowler is an Open Source Security tool for AWS, Azure, GCP and Kubernetes to do security assessments, audits, incident response, compliance, continuous monitoring, hardening and forensics readiness. Includes CIS, NIST 800, NIST CSF, CISA, FedRAMP, PCI-DSS, GDPR, HIPAA, FFIEC, SOC2, GXP, Well-Architected Security, ENS and more
11. Prowler
-
-
serverless-application-model
The AWS Serverless Application Model (AWS SAM) transform is a AWS CloudFormation macro that transforms SAM templates into CloudFormation templates.
In my example I am using the Serverless Application Model (SAM) as my Infrastructure as Code tool. If you click on the links you can find more information about SAM and it's associated CLI (SAM CLI).
-
Project mention: Continuous Deployment applied to Authorization with IAM Identity Center and AWS IAM Access Analyzer – Part 1 | dev.to | 2024-08-04
AWS SDK for Python (Boto3): Boto3 makes it easy to integrate your Python application, library, or script with AWS services including Amazon S3, Amazon EC2, Amazon DynamoDB, and more.
-
-
Project mention: Recapping the AI, Machine Learning and Computer Meetup — August 15, 2024 | dev.to | 2024-08-15
As a data scientist/ML practitioner, how would you feel if you can independently iterate on your data science projects without ever worrying about operational overheads like deployment or containerization? Let’s find out by walking you through a sample project that helps you do so! We’ll combine Python, AWS, Metaflow and BentoML into a template/scaffolding project with sample code to train, serve, and deploy ML models…while making it easy to swap in other ML models.
-
-
While at it - I heartily recommend you to install aws-shell which boosts your aws cli productivity by providing graphical autocompletion, hints and shortcuts as shown in the image below. I only discovered it recently myself and it's definitely a game changer!
-
checkov
Prevent cloud misconfigurations and find vulnerabilities during build-time in infrastructure as code, container images and open source packages with Checkov by Bridgecrew.
Project mention: Automating Snowflake Resource Deployment using Terraform and GitHub Actions | dev.to | 2024-05-18The workflow also includes a step for infrastructure code scan to scan Terraform code. This uses Checkov action against infrastructure-as-code, open source packages, container images, and CI/CD configurations to identify misconfigurations, vulnerabilities, and license compliance issues.
-
Project mention: ScoutSuite: Open-Source Multi-Cloud Security Auditing Tool | news.ycombinator.com | 2024-08-14
-
-
dev-setup
macOS development environment setup: Easy-to-understand instructions with automated setup scripts for developer tools like Vim, Sublime Text, Bash, iTerm, Python data analysis, Spark, Hadoop MapReduce, AWS, Heroku, JavaScript web development, Android development, common data stores, and dev-based OS X defaults.
-
cloud-custodian
Rules engine for cloud security, cost optimization, and governance, DSL in yaml for policies to query, filter, and take actions on resources
Project mention: Cutting down AWS cost by $150k per year simply by shutting things off | news.ycombinator.com | 2024-01-22> The best optimization is simply shutting things off
This is the way.
A similar idea has been bouncing around in my mind for a while now. An ideal, turnkey system would do the following:
- Execute via Lambda (serverless).
- Support automated startup and shutdown of various AWS resources on a schedule influenced by specially formatted tags.
- Enable resources to be brought back up out of schedule when demand dictates.
- Operate as a TCP/HTTP proxy that can delay clients so that a given service can be started when it is dormant or, even better, the service isn't serverless but you want it to be. This can't work for everything, but perhaps enough things such that the need to run always on services is reduced.
Cloud Custodian [1] can purportedly do some of this, but I've been reluctant to learn yet another YAML-based DSL to use it.
So this is my "make things designed to be always-on serverless instead" project and the work AWS has done to make Java apps function on Lambda keeps me thinking about the potential to take things that 1) have a relatively long startup time and 2) are designed to be long running service loops, and find a way to force them into the serverless execution model.
[1] https://cloudcustodian.io/
-
-
image-super-resolution
🔎 Super-scale your images and run experiments with Residual Dense and Adversarial Networks.
-
Project mention: Show HN: Auto Wiki v2 – Turn your codebase into a Wiki now with diagrams | news.ycombinator.com | 2024-04-23
https://github.com/awslabs/gluonts is a great candidate for a sample wiki. It is an OSS lib, not great documentation, very hard to RTFM (unlike, say, sklearn which already has a great wiki), doubtful that awslabs would pay to produce.
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
Python AWS discussion
Python AWS related posts
-
How to accidentally create read-only DynamoDB items
-
Serverless AWS Account Watcher
-
Making the TPC-H dataset available in Athena using Airflow
-
AWS-CLI: Master AWS from Your Terminal with This Universal CLI Tool
-
dstack (K8s alternative) adds support for AMD accelerators on RunPod
-
ScoutSuite: Open-Source Multi-Cloud Security Auditing Tool
-
Show HN: RAG app example with self-hosted embedding and LLM services
-
A note from our sponsor - Scout Monitoring
www.scoutapm.com | 17 Sep 2024
Index
What are some of the best open-source AWS projects in Python? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | devops-exercises | 65,830 |
| 2 | LocalStack | 53,735 |
| 3 | data-science-ipython-notebooks | 27,142 |
| 4 | aws-cli | 15,332 |
| 5 | awesome-aws | 12,413 |
| 6 | phidata | 11,145 |
| 7 | chalice | 10,599 |
| 8 | prowler | 10,530 |
| 9 | docker-android | 9,423 |
| 10 | serverless-application-model | 9,323 |
| 11 | boto3 | 8,949 |
| 12 | ansible-for-devops | 8,327 |
| 13 | metaflow | 8,026 |
| 14 | Moto | 7,587 |
| 15 | aws-shell | 7,176 |
| 16 | checkov | 6,980 |
| 17 | ScoutSuite | 6,593 |
| 18 | aws-sam-cli | 6,508 |
| 19 | dev-setup | 6,107 |
| 20 | cloud-custodian | 5,364 |
| 21 | SAWS | 5,242 |
| 22 | image-super-resolution | 4,603 |
| 23 | gluonts | 4,546 |
Sponsored
Free Django app performance insights with Scout Monitoring
Get Scout setup in minutes, and let us sweat the small stuff. A couple lines in settings.py is all you need to start monitoring your apps. Sign up for our free tier today.
www.scoutapm.com