InfluxDB Platform is powered by columnar analytics, optimized for cost-efficient storage, and built with open data standards. Learn more โ
Top 23 Python GCP Projects
-
prowler
Prowler is an Open Source Security tool for AWS, Azure, GCP and Kubernetes to do security assessments, audits, incident response, compliance, continuous monitoring, hardening and forensics readiness. Includes CIS, NIST 800, NIST CSF, CISA, FedRAMP, PCI-DSS, GDPR, HIPAA, FFIEC, SOC2, GXP, Well-Architected Security, ENS and more
11. Prowler
-
Scout Monitoring
Free Django app performance insights with Scout Monitoring. Get Scout setup in minutes, and let us sweat the small stuff. A couple lines in settings.py is all you need to start monitoring your apps. Sign up for our free tier today.
-
-
Project mention: Recapping the AI, Machine Learning and Computer Meetup โ August 15, 2024 | dev.to | 2024-08-15
As a data scientist/ML practitioner, how would you feel if you can independently iterate on your data science projects without ever worrying about operational overheads like deployment or containerization? Letโs find out by walking you through a sample project that helps you do so! Weโll combine Python, AWS, Metaflow and BentoML into a template/scaffolding project with sample code to train, serve, and deploy ML modelsโฆwhile making it easy to swap in other ML models.
-
checkov
Prevent cloud misconfigurations and find vulnerabilities during build-time in infrastructure as code, container images and open source packages with Checkov by Bridgecrew.
Project mention: Automating Snowflake Resource Deployment using Terraform and GitHub Actions | dev.to | 2024-05-18The workflow also includes a step for infrastructure code scan to scan Terraform code. This uses Checkov action against infrastructure-as-code, open source packages, container images, and CI/CD configurations to identify misconfigurations, vulnerabilities, and license compliance issues.
-
Project mention: ScoutSuite: Open-Source Multi-Cloud Security Auditing Tool | news.ycombinator.com | 2024-08-14
-
cloud-custodian
Rules engine for cloud security, cost optimization, and governance, DSL in yaml for policies to query, filter, and take actions on resources
Project mention: Cutting down AWS cost by $150k per year simply by shutting things off | news.ycombinator.com | 2024-01-22> The best optimization is simply shutting things off
This is the way.
A similar idea has been bouncing around in my mind for a while now. An ideal, turnkey system would do the following:
- Execute via Lambda (serverless).
- Support automated startup and shutdown of various AWS resources on a schedule influenced by specially formatted tags.
- Enable resources to be brought back up out of schedule when demand dictates.
- Operate as a TCP/HTTP proxy that can delay clients so that a given service can be started when it is dormant or, even better, the service isn't serverless but you want it to be. This can't work for everything, but perhaps enough things such that the need to run always on services is reduced.
Cloud Custodian [1] can purportedly do some of this, but I've been reluctant to learn yet another YAML-based DSL to use it.
So this is my "make things designed to be always-on serverless instead" project and the work AWS has done to make Java apps function on Lambda keeps me thinking about the potential to take things that 1) have a relatively long startup time and 2) are designed to be long running service loops, and find a way to force them into the serverless execution model.
[1] https://cloudcustodian.io/
-
OPAL
Policy and data administration, distribution, and real-time updates on top of Policy Agents (OPA, Cedar, ...) (by permitio)
OPAL - https://opal.ac/
-
InfluxDB
Purpose built for real-time analytics at any scale. InfluxDB Platform is powered by columnar analytics, optimized for cost-efficient storage, and built with open data standards.
-
ethereum-etl
Python scripts for ETL (extract, transform and load) jobs for Ethereum blocks, transactions, ERC20 / ERC721 tokens, transfers, receipts, logs, contracts, internal transactions. Data is available in Google BigQuery https://goo.gl/oY5BCQ
Project mention: Blockchain transactions decoding: making wallet activity understandable | dev.to | 2023-10-27Event is a log entity which EVM smart contracts can emit during transaction execution. Events are very good at signalling that an some action has taken place on-chain. Applications can subscribe and listen to events to trigger some off-chain logic or they can index, transform and store events in some off-chain storage (look at The Graph protocol or Ethereum ETL).
-
flow-dashboard
[UNMAINTAINED] A goal, task & habit tracker + personal dashboard to focus on what matters
Project mention: Show HN: I made an open-source personal dashboard builder | news.ycombinator.com | 2024-08-16I made a tool for capturing data in JSON [0]. It's not in real-time but when I want to use my logs to troubleshoot something I have a local React project with https://github.com/plouc/nivo to make my own visualizations.
But for something you can host yourself I've used https://github.com/onejgordon/flow-dashboard in the past
[0] https://www.idiotlamborghini.com/strategies/weave
-
fixinventory
Fix Inventory consolidates user, resource, and configuration data from your cloud environments into a unified, graph-based asset inventory.
Project mention: Defenders think in lists. Attackers think in graphs. So attackers win | news.ycombinator.com | 2024-08-24There is also Fix Inventory, which is a graph-based security tool:
https://github.com/someengineering/fixinventory
I'm one of the people behind Fix Inventory. What scares a lot of developers away from graph-based tools is the graph query language. It has a steep learning curve, and unless you write queries every day, it's really cumbersome to learn.
We simplified that with our own search syntax that has all the benefits of the graph, but simplified a few concepts like graph traversal.
-
dstack
dstack is an open-source alternative to Kubernetes, designed to simplify development, training, and deployment of AI across any cloud or on-prem. It supports NVIDIA, AMD, and TPU.
Project mention: dstack (K8s alternative) adds support for AMD accelerators on RunPod | news.ycombinator.com | 2024-08-21Thatโs on our list! Weโre actually planning to expand our examples page with a lot more frameworks! Contributors are very welcome to our repo: https://github.com/dstackai/dstack/tree/master/examples/
-
-
optscale
FinOps, MLOps and cloud cost optimization tool. Supports AWS, Azure, GCP, Alibaba Cloud and Kubernetes.
Project mention: Profile and instrument ML experiments and optimize their performance expenses | news.ycombinator.com | 2023-09-27 -
-
runhouse
Dispatch and distribute your ML training to "serverless" clusters in Python, like PyTorch for ML infra. Iterable, debuggable, multi-cloud/on-prem, identical across research and production.
Project mention: Show HN: RAG app example with self-hosted embedding and LLM services | news.ycombinator.com | 2024-08-13 -
opta
The next generation of Infrastructure-as-Code. Work with high-level constructs instead of getting lost in low-level cloud configuration.
-
energy-forecasting
๐ ๐ง๐ต๐ฒ ๐๐๐น๐น ๐ฆ๐๐ฎ๐ฐ๐ธ ๐ณ-๐ฆ๐๐ฒ๐ฝ๐ ๐ ๐๐ข๐ฝ๐ ๐๐ฟ๐ฎ๐บ๐ฒ๐๐ผ๐ฟ๐ธ | ๐๐ฒ๐ฎ๐ฟ๐ป ๐ ๐๐ & ๐ ๐๐ข๐ฝ๐ for free by designing, building and deploying an end-to-end ML batch system ~ ๐ด๐ฐ๐ถ๐ณ๐ค๐ฆ ๐ค๐ฐ๐ฅ๐ฆ + 2.5 ๐ฉ๐ฐ๐ถ๐ณ๐ด ๐ฐ๐ง ๐ณ๐ฆ๐ข๐ฅ๐ช๐ฏ๐จ & ๐ท๐ช๐ฅ๐ฆ๐ฐ ๐ฎ๐ข๐ต๐ฆ๐ณ๐ช๐ข๐ญ๐ด
Project mention: Full Stack 7-Steps MLOps Framework โ Learn ML and MLOps by Building an ML System | news.ycombinator.com | 2023-09-26 -
-
-
-
streamify
A data engineering project with Kafka, Spark Streaming, dbt, Docker, Airflow, Terraform, GCP and much more!
-
-
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
Python GCP discussion
Python GCP related posts
-
dstack (K8s alternative) adds support for AMD accelerators on RunPod
-
ScoutSuite: Open-Source Multi-Cloud Security Auditing Tool
-
ScoutSuite
-
Automating Snowflake Resource Deployment using Terraform and GitHub Actions
-
Let's build your first ML app in Google Cloud Run
-
Any better alternatives to fine-tuning GPT-3 yet to create a custom chatbot persona based on provided knowledge for others to use?
-
Azure and M365 Secure Config Review
-
A note from our sponsor - InfluxDB
www.influxdata.com | 19 Sep 2024
Index
What are some of the best open-source GCP projects in Python? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | prowler | 10,530 |
| 2 | docker-android | 9,423 |
| 3 | metaflow | 8,055 |
| 4 | checkov | 6,980 |
| 5 | ScoutSuite | 6,593 |
| 6 | cloud-custodian | 5,364 |
| 7 | OPAL | 4,337 |
| 8 | ethereum-etl | 2,920 |
| 9 | flow-dashboard | 1,663 |
| 10 | fixinventory | 1,588 |
| 11 | dstack | 1,320 |
| 12 | gorilla-cli | 1,262 |
| 13 | optscale | 1,193 |
| 14 | skyplane | 1,000 |
| 15 | runhouse | 961 |
| 16 | opta | 906 |
| 17 | energy-forecasting | 853 |
| 18 | click-to-deploy | 728 |
| 19 | PurplePanda | 658 |
| 20 | redun | 513 |
| 21 | streamify | 508 |
| 22 | kcli | 496 |
| 23 | spotty | 492 |
Sponsored
Free Django app performance insights with Scout Monitoring
Get Scout setup in minutes, and let us sweat the small stuff. A couple lines in settings.py is all you need to start monitoring your apps. Sign up for our free tier today.
www.scoutapm.com