Sonar helps you commit clean code every time. With over 225 unique rules to find Python bugs, code smells & vulnerabilities, Sonar finds the issues while you focus on the work. Learn more →
Top 23 Python GCP Projects
-
microservices-demo
Sample cloud-first application with 10 microservices showcasing Kubernetes, Istio, and gRPC.
This blog features the Skyramp fork of Google’s popular cloud-native microservices demo app, Online Boutique. Online Boutique is a web-based e-commerce app containing microservices that mimic real-world services, such as a product catalog, shopping cart, ad service, recommendation service, payment service, and others. The services use gRPC APIs by default, but Skyramp has also added support for REST and Thrift APIs.
-
prowler
Prowler is an Open Source Security tool for AWS, Azure and GCP to perform Cloud Security best practices assessments, audits, incident response, compliance, continuous monitoring, hardening and forensics readiness. Includes CIS, NIST 800, NIST CSF, CISA, FedRAMP, PCI-DSS, GDPR, HIPAA, FFIEC, SOC2, GXP, Well-Architected Security, ENS and more.
Task Role: Select or create a new role that has the necessary permissions. Ensure this role has the three required Prowler policies and an additional custom policy to put objects into the desired S3 bucket.
-
Mergify
Updating dependencies is time-consuming.. Solutions like Dependabot or Renovate update but don't merge dependencies. You need to do it manually while it could be fully automated! Add a Merge Queue to your workflow and stop caring about PR management & merging. Try Mergify for free.
-
If it was me I'd go with a docker-android container. You could go with Anbox or a virtual android machine, but if you just need to run one app, I think a docker container would be the way to go, since you could script it to open, run your app, and shut down.
-
checkov
Prevent cloud misconfigurations and find vulnerabilities during build-time in infrastructure as code, container images and open source packages with Checkov by Bridgecrew.
For your Dockerfiles, you can also scan them. There are lots of tools that can check your Dockerfiles. They will validate if Dockerfile is compliant with Docker best practices such as not using root user, making sure a health check exists, and not exposing the SSH port. You can use Snyk and Checkov.
-
Prowler and ScoutSuite are a good start for cloud stuff.
-
cloud-custodian
Rules engine for cloud security, cost optimization, and governance, DSL in yaml for policies to query, filter, and take actions on resources
You can start reading about it here.
-
ethereum-etl
Python scripts for ETL (extract, transform and load) jobs for Ethereum blocks, transactions, ERC20 / ERC721 tokens, transfers, receipts, logs, contracts, internal transactions. Data is available in Google BigQuery https://goo.gl/oY5BCQ
-
InfluxDB
Collect and Analyze Billions of Data Points in Real Time. Manage all types of time series data in a single, purpose-built database. Run at any scale in any environment in the cloud, on-premises, or at the edge.
-
OPAL
Policy and data administration, distribution, and real-time updates on top of Policy Agents (OPA, Cedar, ...) (by permitio)
One example of such an administration tool is OPAL, an open policy administration layer that works with OPA. OPAL tracks changes in external services and propagates the data to the OPA PDPs so the authorization requests can handle existing data and return faster results.
-
resoto
Resoto creates an inventory of your cloud ☁️, provides deep visibility 👀, and reacts to changes in your infrastructure 🌐.
Resoto: https://resoto.com/
-
opta
The next generation of Infrastructure-as-Code. Work with high-level constructs instead of getting lost in low-level cloud configuration.
-
-
dstack
dstack is an open-source framework for orchestrating GPU workloads across multiple cloud GPU providers. It provides a simple cloud-agnostic interface for development and deployment of generative AI models.
Project mention: Running dev environments and ML tasks cost-effectively in any cloud | /r/deeplearning | 2023-07-05Here's the repository with all the important links, including documentation, examples, and more: https://github.com/dstackai/dstack
-
zentral
Zentral is a high-visibility platform for controlling Apple endpoints in enterprises. It brings great observability to IT and makes tracking & reporting compliance much less manual.
-
-
energy-forecasting
🌀 𝗧𝗵𝗲 𝗙𝘂𝗹𝗹 𝗦𝘁𝗮𝗰𝗸 𝟳-𝗦𝘁𝗲𝗽𝘀 𝗠𝗟𝗢𝗽𝘀 𝗙𝗿𝗮𝗺𝗲𝘄𝗼𝗿𝗸 | Learn MLE & MLOps for free by designing, building and deploying an end-to-end ML batch system ~ 𝘴𝘰𝘶𝘳𝘤𝘦 𝘤𝘰𝘥𝘦 + 2.5 𝘩𝘰𝘶𝘳𝘴 𝘰𝘧 𝘳𝘦𝘢𝘥𝘪𝘯𝘨 & 𝘷𝘪𝘥𝘦𝘰 𝘮𝘢𝘵𝘦𝘳𝘪𝘢𝘭𝘴
Project mention: Full Stack 7-Steps MLOps Framework – Learn ML and MLOps by Building an ML System | news.ycombinator.com | 2023-09-26 -
-
Hey, are you aware of kcli? https://kcli.readthedocs.io/en/latest/
-
optscale
FinOps and cloud cost optimization platform combined with ML/AI developer tools to profile and instrument ML experiments and optimize their performance and cloud expenses.
Project mention: Profile and instrument ML experiments and optimize their performance expenses | news.ycombinator.com | 2023-09-27 -
ethereum-etl-airflow
Airflow DAGs for exporting, loading, and parsing the Ethereum blockchain data. How to get any Ethereum smart contract into BigQuery https://towardsdatascience.com/how-to-get-any-ethereum-smart-contract-into-bigquery-in-8-mins-bab5db1fdeee
-
bitcoin-etl
ETL scripts for Bitcoin, Litecoin, Dash, Zcash, Doge, Bitcoin Cash. Available in Google BigQuery https://goo.gl/oY5BCQ
-
Project mention: A user-built manual on using TPUs for AI and LLMs on Google Cloud | news.ycombinator.com | 2023-08-04
-
streamify
A data engineering project with Kafka, Spark Streaming, dbt, Docker, Airflow, Terraform, GCP and much more!
-
tfquery
tfquery: Run SQL queries on your Terraform infrastructure. Query resources and analyze its configuration using a SQL-powered framework.
-
Sonar
Write Clean Python Code. Always.. Sonar helps you commit clean code every time. With over 225 unique rules to find Python bugs, code smells & vulnerabilities, Sonar finds the issues while you focus on the work.
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
The latest post mention was on 2023-09-27.
Python GCP related posts
- Any better alternatives to fine-tuning GPT-3 yet to create a custom chatbot persona based on provided knowledge for others to use?
- Azure and M365 Secure Config Review
- Can I use AWS to build an adult website?
- Apim + function app & event grid
- BigQuery Remote Functions using Goblet
- Invoice granularity: Show different accounts/cost allocation tags on invoice
- Resoto | Graph-based Cloud Asset Inventory
-
A note from our sponsor - Sonar
www.sonarsource.com | 30 Sep 2023
Index
What are some of the best open-source GCP projects in Python? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | microservices-demo | 14,809 |
| 2 | prowler | 8,624 |
| 3 | docker-android | 6,140 |
| 4 | checkov | 5,920 |
| 5 | ScoutSuite | 5,656 |
| 6 | cloud-custodian | 4,951 |
| 7 | ethereum-etl | 2,661 |
| 8 | OPAL | 1,819 |
| 9 | resoto | 1,439 |
| 10 | opta | 908 |
| 11 | skyplane | 846 |
| 12 | dstack | 807 |
| 13 | zentral | 695 |
| 14 | PurplePanda | 568 |
| 15 | energy-forecasting | 508 |
| 16 | spotty | 479 |
| 17 | kcli | 419 |
| 18 | optscale | 390 |
| 19 | ethereum-etl-airflow | 369 |
| 20 | bitcoin-etl | 354 |
| 21 | tpu-starter | 331 |
| 22 | streamify | 325 |
| 23 | tfquery | 325 |
Write Clean Python Code. Always.
Sonar helps you commit clean code every time. With over 225 unique rules to find Python bugs, code smells & vulnerabilities, Sonar finds the issues while you focus on the work.
www.sonarsource.com