Python AWS

Open-source Python projects categorized as AWS

Top 23 Python AWS Projects

  • GitHub repo LocalStack

    💻 A fully functional local AWS cloud stack. Develop and test your cloud & Serverless apps offline!

    Project mention: Please fix the AWS Free Tier before somebody gets hurt | reddit.com/r/programming | 2021-06-05

    There's localstack, which runs in a docker container and will mock out a lot of the core services.

  • GitHub repo data-science-ipython-notebooks

    Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.

    Project mention: Beginner in Python for Data Science | reddit.com/r/learnpython | 2020-12-27

    data science ipython notebooks

  • GitHub repo aws-cli

    Universal Command Line Interface for Amazon Web Services

    Project mention: Downloading files from S3 with multithreading and Boto3 | news.ycombinator.com | 2021-04-10

    From my experience, I found it easier and more convenient to call aws-cli [1] than using Boto. Among other things, aws-cli is already multithreaded, in both download and upload. There is even a setting to tweak the number of parallel requests [2].

    [1] https://github.com/aws/aws-cli

    [2] https://docs.aws.amazon.com/credref/latest/refdocs/setting-s...

  • GitHub repo awesome-aws

    A curated list of awesome Amazon Web Services (AWS) libraries, open source repos, guides, blogs, and other resources. Featuring the Fiery Meter of AWSome.

    Project mention: There are 40,000+ quality AWS open source repositories on GitHub but are completely unorganized. I made a search engine and browser for all of them, all curated carefully with 1000+ filters. | reddit.com/r/sysadmin | 2021-06-06

    There is also https://github.com/donnemartin/awesome-aws

  • GitHub repo devops-exercises

    Linux, Jenkins, AWS, SRE, Prometheus, Docker, Python, Ansible, Git, Kubernetes, Terraform, OpenStack, SQL, NoSQL, Azure, GCP, DNS, Elastic, Network, Virtualization. DevOps Interview Questions

    Project mention: Questions you would get asked on an interview? | reddit.com/r/devops | 2021-01-28

    I think the link you're looking for is https://github.com/bregman-arie/devops-exercises

  • GitHub repo chalice

    Python Serverless Microframework for AWS

    Project mention: Any thoughts on chalice for websockets? | reddit.com/r/aws | 2021-03-31

    I use Chalice with websockets, and haven't had any issues. I had to make a slight hack to allow other non-websocket lambdas to be able to send messages on the websocket, but other than that it works without issue. There is a ticket with Chalice open about the sending messages from other Lambdas. https://github.com/aws/chalice/issues/1357

  • GitHub repo serverless-application-model

    AWS Serverless Application Model (SAM) is an open-source framework for building serverless applications

    Project mention: Build a GraphQL API with TypeScript, AWS AppSync, and CDK | dev.to | 2021-06-13

    Previously, I had used AWS SAM to build a CRUD app for notes. Recently, I wanted to learn more about the AWS Cloud Development Kit (CDK), GraphQL, and AWS AppSync, so I decided to refactor my original project.

  • GitHub repo boto3

    AWS SDK for Python

    Project mention: Deploy Django App on AWS Lambda using Serverless (Part 2) | dev.to | 2021-05-10

    automatically using AWS SDK

  • GitHub repo aws-shell

    An integrated shell for working with the AWS CLI.

    Project mention: MTurk Requester Notify-workers fails with endpoint URL error | reddit.com/r/mturk | 2021-06-03

    If so the region config is explained here: https://github.com/awslabs/aws-shell#configuration

  • GitHub repo aws-sam-cli

    CLI tool to build, test, debug, and deploy Serverless applications using AWS SAM

    Project mention: Build Serverless Applications using CDK and SAM | dev.to | 2021-05-21

    We were able to test our API and Lambda using the new Serverless Application Model integration with CDK! You can find all code on my GitHub. Be aware that this feature is in preview. Feel free to do more extensive testing. You can report bugs and submit feature requests to the SAM opensource repository.

  • GitHub repo dev-setup

    macOS development environment setup: Easy-to-understand instructions with automated setup scripts for developer tools like Vim, Sublime Text, Bash, iTerm, Python data analysis, Spark, Hadoop MapReduce, AWS, Heroku, JavaScript web development, Android development, common data stores, and dev-based OS X defaults.

    Project mention: MacOS Development workspace 2021 | dev.to | 2021-03-08

    donnemartin - dev setup

  • GitHub repo SAWS

    A supercharged AWS command line interface (CLI).

  • GitHub repo Moto

    A library that allows you to easily mock out tests based on AWS infrastructure.

    Project mention: Ask HN: Does anyone else find the AWS Lambda developer experience poor? | news.ycombinator.com | 2021-04-18

    Not a full solution, but when I was doing this I really got to love the awslogs utility:

    https://github.com/jorgebastida/awslogs

    It allows you to stream Cloudwatch logs from the command line, so you can grep them, save them to files, etc... (The web based Cloudwatch interface is terrible.)

    Another suggestion is to try to modularize the core business logic in your lambda such that you separate the lambda-centric stuff from the rest of it. Obviously, though, if "the rest of it" is hitting other AWS services, you're going to hit the same testing roadblock.

    Or you can try mocking, which may or may not provide much value for you. There's a python library for that, (moto), but it's not 100% up to date wrt AWS services/interfaces, last I had checked. Might be worth a try though.

    https://github.com/spulec/moto

  • GitHub repo ansible-for-devops

    Ansible for DevOps examples.

    Project mention: A Complete Step by Step Ansible Tutorial & Guide | reddit.com/r/devops | 2021-04-28

    link

  • GitHub repo cloud-custodian

    Rules engine for cloud security, cost optimization, and governance, DSL in yaml for policies to query, filter, and take actions on resources

    Project mention: How to optimize users manually turning on/off ec2's as needed for work? | reddit.com/r/aws | 2021-06-14
  • GitHub repo ScoutSuite

    Multi-Cloud Security Auditing Tool

    Project mention: Is there any way to efficiently audit security groups? | reddit.com/r/aws | 2021-04-28

    ScoutSuite is similar to prowler (runs faster though) but last time I used it it had less checks.

  • GitHub repo image-super-resolution

    🔎 Super-scale your images and run experiments with Residual Dense and Adversarial Networks.

    Project mention: Totally free and unlimited upscale or superresolution AI | reddit.com/r/ArtificialInteligence | 2021-06-03
  • GitHub repo checkov

    Prevent cloud misconfigurations during build-time for Terraform, Cloudformation, Kubernetes, Serverless framework and other infrastructure-as-code-languages with Checkov by Bridgecrew.

    Project mention: How to fix checkov failed item by its rules? | reddit.com/r/Terraform | 2021-05-10

    Have you tried posting an issue to https://github.com/bridgecrewio/checkov ?

  • GitHub repo bless

    Repository for BLESS, an SSH Certificate Authority that runs as a AWS Lambda function

    Project mention: How often should I rotate my SSH keys? | news.ycombinator.com | 2021-02-25
  • GitHub repo streamalert

    StreamAlert is a serverless, realtime data analysis framework which empowers you to ingest, analyze, and alert on data from any environment, using datasources and alerting logic you define.

    Project mention: Setting up lots of AWS alerts - make lots of SNS-SQS-Lambda chains? | reddit.com/r/devops | 2021-04-22
  • GitHub repo pacu

    The AWS exploitation framework, designed for testing the security of Amazon Web Services environments.

    Project mention: 🏹 Pacu: Framework de explotación de AWS #SeguridadOfensiva | reddit.com/r/u_esgeeks | 2021-05-11
  • GitHub repo AWS Data Wrangler

    Pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).

    Project mention: A guide to load (almost) anything into a DataFrame | reddit.com/r/Python | 2021-05-13

    Don't forget about https://aws-data-wrangler.readthedocs.io/

  • GitHub repo PynamoDB

    A pythonic interface to Amazon's DynamoDB

    Project mention: A minimalistic Python wrapper to AWS DynamoDB | news.ycombinator.com | 2021-05-30

    I've been pretty happy using PynamoDB, an ORM-like wrapper with a straightforward interface:

    https://github.com/pynamodb/PynamoDB

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2021-06-14.

Index

What are some of the best open-source AWS projects in Python? This list will help you:

Project Stars
1 LocalStack 30,614
2 data-science-ipython-notebooks 21,197
3 aws-cli 11,105
4 awesome-aws 9,377
5 devops-exercises 8,560
6 chalice 7,981
7 serverless-application-model 7,970
8 boto3 6,443
9 aws-shell 6,163
10 aws-sam-cli 5,577
11 dev-setup 5,443
12 SAWS 4,783
13 Moto 4,685
14 ansible-for-devops 4,635
15 cloud-custodian 3,613
16 ScoutSuite 3,252
17 image-super-resolution 2,830
18 checkov 2,798
19 bless 2,593
20 streamalert 2,539
21 pacu 2,121
22 AWS Data Wrangler 1,827
23 PynamoDB 1,594