hydra
berglas
hydra | berglas | |
---|---|---|
14 | 37 | |
8,229 | 1,224 | |
1.6% | 0.1% | |
6.3 | 6.9 | |
21 days ago | 3 days ago | |
Python | Go | |
MIT License | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
hydra
- Hydra – a Framework for configuring complex applications
-
Show HN: Hydra - Open-Source Columnar Postgres
Nice tool, only unfortunate name, consider changing it. Already very well know security tool named hydra https://github.com/vanhauser-thc/thc-hydra been around since 2001. Then facebook went ahead and named their config tool hydra https://github.com/facebookresearch/hydra on top of it. Like we get it, hydra popular mythology but we could use more original naming for tools
-
Show HN: Hydra 1.0 – open-source column-oriented Postgres
This looks really impressive, and I'm excited to see how it performs on our data!
P.S., I think the name conflicts with Hydra, the configuration management library: https://hydra.cc/
-
Best practice for saving logits/activation values of model in PyTorch Lightning
I've been trying to learn PyTorch Lightning and Hydra in order to use/create my own custom deep learning template (e.g. like this) as it would greatly help with my research workflow. A lot of the work I do requires me to analyse metrics based on the logits/activations of the model.
-
[D] Alternatives to fb Hydra?
However, hydra seems to have several limitations that are really annoying and are making me reconsider my choice. Most problematic is the inability to group parameters together in a multirun. Hydra only supports trying all combinations of parameters, as described in https://github.com/facebookresearch/hydra/issues/1258, which does not seem to be a priority for hydra. Furthermore, hydras optuna optimizer implementation does not allow for early pruning of bad runs, which while not a deal breaker is definitely a nice to have feature.
-
Show HN: Lightweight YAML Config CLI for Deep Learning Projects
Do you hate the fact that they don't let you return the config file: https://github.com/facebookresearch/hydra/issues/407
-
Config management for deep learning
I kind of built this due to frustrations with Hydra. Hydra is an end to end framework, it locks you into a certain DL project format, it decides logging, model saving and a whole host of things. For example Hydra can do the same config file overwriting that I allow but you have to store the config file with the name config.yaml inside a specific folder. On top of that hydra doesn’t let you return the config file from the main function so you have to put all the major logic in the main function itself (link), the authors claim this is by design. I can find Hydra useful for a mature less experimental project. But in my robotics and ML research, I like being able to write code where I want and integrating it how I want, especially when debugging for which I think this package is useful. TLDR; If you just want the config file functionality use my package, if you want a complete DL project manager use Hydra. While hydra implements this config file functionality, it also adds a lot of restrictions to project structure that you might not like.
-
The YAML Document from Hell
For managing configs of ML experiments (where each experiment can override a base config, and "variant" configs can further override the experiment config, etc), Hydra + Yaml + OmegaConf is really nice.
https://hydra.cc/
I admit I don't fully understand all the advanced options in Hydra, but the basic usage is already very useful. A nice guide is here:
https://florianwilhelm.info/2022/01/configuration_via_yaml_a...
- Hydra - namestitev in osnovna uporaba
- Hydra - namestitevt in osnovna uporaba
berglas
-
How to deploy a Django app to Google Cloud Run using Terraform
Secret Manager: secure storage for sensitive data e.g passwords.
- How do you handle sensitive variables with a service-worker?
-
Increasing Your Cloud Function Development Velocity Using Dynamically Loading Python Classes
Google Secret Manager
-
Getting started using Google APIs: API Keys (Part 2)
API keys are easy to "leak" or compromise, so best to not only use the restrictions presented to you when you create them but physically protect them as well. Don't code them in plain-text, don't check them into GitHub, etc. Store them in a secure database or use a service like GCP Secret Manager.
-
Need some advice on API key storage
I've been looking at Google Secret Manager which sounds promising but I've not been able to find any examples or tutorials that help with the actual practical details of best practice or getting this working. I'm currently reading about Cloud Functions which also sound promising but again, I'm just going deeper and deeper into GCP without feeling like I'm gaining any useful insights.
-
Secure GitHub Actions by pull_request_target
In this post, I described how to build secure GitHub Actions workflows by pull_request_target event instead of pull_request event. Using pull_request_target, you can prevent malicious codes from being executed in CI. And by managing secrets in secrets management services such as AWS Secrets Manager and Google Secret Manager and access them via OIDC, you can restrict the access to secrets securely. To migrate pull_request to pull_request_target, several modifications are needed. And pull_request_target has a drawback that it's difficult to test changes of workflows, so it's good to introduce pull_request_target to repositories that require strong permissions in CI. For example, a Terraform Monorepo tends to require strong permissions for CI, so it's good to introduce pull_request_target to it.
-
Need Help with Deploying Directus on Google Cloud Platform (GCP)
If you want to make these secrets more secure and get versioning and access logs for them, you may want to switch to Secret Manager later on. They can still be exposed as environment variables to your code. It's a little more setup work, so start with the simple approach at the top.
-
Has anyone been able to implement the OpenAI API with a Firebase Function (which is needed for the env variable API Key)?
https://cloud.google.com/secret-manager https://aws.amazon.com/secrets-manager/
- Securely storing Social Security Numbers with Firebase?
- Dónde van las credenciales cuando voy a subir un código a la nube para correr 24/7?
What are some alternatives?
dynaconf - Configuration Management for Python ⚙
kubernetes-external-secrets - Integrate external secret management systems with Kubernetes
ConfigParser
helm-charts
python-dotenv - Reads key-value pairs from a .env file and can set them as environment variables. It helps in developing applications following the 12-factor principles.
kube-secrets-init - Kubernetes mutating webhook for `secrets-init` injection
python-decouple - Strict separation of config from code.
gitleaks - Protect and discover secrets using Gitleaks 🔑
django-environ - Django-environ allows you to utilize 12factor inspired environment variables to configure your Django application.
cocert - Split and distribute your private keys securely amongst untrusted network
classyconf - Declarative and extensible library for configuration & code separation
secrets-store-csi-driver-provider-gcp - Google Secret Manager provider for the Secret Store CSI Driver.