tsv-utils
sops
tsv-utils | sops | |
---|---|---|
10 | 150 | |
1,396 | 15,298 | |
0.0% | 2.2% | |
0.0 | 9.0 | |
over 1 year ago | 2 days ago | |
D | Go | |
Boost Software License 1.0 | Mozilla Public License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
tsv-utils
-
Frawk: An efficient Awk-like programming language. (2021)
If you need just csv/tsv parsing, you can also take a look at https://github.com/eBay/tsv-utils
-
Tracking SQLite Database Changes in Git
You might want to look at tsv-utils, or a similar project: https://github.com/eBay/tsv-utils
For the SQL part, but maybe a lot heavier, you can use one of the projects listed on this page: https://github.com/multiprocessio/dsq (No longer maintained, but has links to lots of other projects)
-
I feel like an idiot but… I need Excel help.
TSV is most often a better format than CSV. Localization, in particular, is a nightmare with CSV.
- Splitting CSV files at 3GB/s
-
Modernizing AWK, a 45-year old language, by adding CSV support
For anything down and dirty, what's wrong with -F'"'? For anything fancy there are plenty of things like the below.
eBay's TSV Utilities: Command line tools for large, tabular data files. Filtering, statistics, sampling, joins and more.
includes csv to tsv: https://github.com/eBay/tsv-utils
HT: https://simonwillison.net/
-
Dlang 2.098.0 released, now available on OpenBSD
As an example, eBay's tsv-utils took full advantage of the GC and performed better than existing programs that had been hand-optimized in C etc.
-
[OC]Tidy Viewer (tv) is a cross-platform csv pretty printer that uses column styling to maximize viewer enjoyment.
tsv-utils - Command line csv data manipulation toolkit. D
-
Changing Registry Key Value Based on Contents of TXT/CSV File
In the majority of cases you'll be better off with Tab Separated Values over Comma Separated Values. More info here.
-
Return 1 to N results from a large (19MM line) CSV
May well be overkill for your needs, but I'm a fan of tsv-utils It's fast and enormously flexible, and seems to me a "best of breed" toolset for data mining CSV files (that is what it was written for). https://github.com/eBay/tsv-utils
sops
-
Pico.sh – Hacker Labs
My script just sets up default .sops.yaml for https://github.com/getsops/sops
You can further edit .sops.yaml(eg have multiple of them) and decide how you split secrets in your directory tree to further customize who can decrypt the secrets.
It works pretty well for prod/dev splits, etc
-
Encrypting your secrets with Mozilla SOPS using two AWS KMS Keys
Mozilla SOPS (Secrets OPerationS) is an open-source command-line tool for managing and storing secrets. It uses secure encryption methods to encrypt secrets at rest and decrypt them at runtime. SOPS supports a variety of key management systems, including AWS KMS, GCP KMS, Azure Key Vault, and PGP. It's particularly useful in a DevOps context where sensitive data like API keys, passwords, or certificates need to be securely managed and seamlessly integrated into application workflows.
-
An opinionated template for deploying a single k3s cluster with Ansible backed by Flux, SOPS, GitHub Actions, Renovate, Cilium, Cloudflare and more!
Encrypted secrets thanks to SOPS and Age
-
Tracking SQLite Database Changes in Git
We do the exact same thing to keep track of some credentials we use sops[1] and AWS KMS to separate credentials by sensitivity, then use the git differ to view the diffs between the encrypted secrets
Definitely not best practice security-wise, but it works well
[1] https://github.com/getsops/sops
-
The Twelve-Factor App
For anyone new to SOPS like I was - https://github.com/getsops/sops
- Storing and managing private keys
-
Show HN: Shello – Wrangle Environment Variables
I've found this is largely solved by strictly separating plain config and secrets, and then having secrets pull from GCP secret manager / vault / whatever.
You can then commit all the config (including the secret identifiers) and it all just works so long as you're authenticated with your secret storage system.
We do this for the live configuration as well in line with Gitops and find it to work well.
If you don't want to use a cloud secret manager you can also use something like https://github.com/getsops/sops to commit the encrypted secrets safely
-
Check your secrets into Git [video]
Basically, the simpler the better --just encrypt your secrets and check them in to version control.
We use SOPS[0] for this, and have found it to be pretty nice.
[0]: https://github.com/getsops/sops
-
How to secure secrets of docker-compose stacks with git?
The answer is that secrets shouldn't be stored in the git repo at all, but somewhere safe like a password manager or Mozilla's SOPS which people seem to love.
-
Is it safe to commit a Terraform file to GitHub?
Unfortunately, the SOPS project is in some sort of a limbo state and there has been quite a long period with limited maintenance and unclear position from Mozilla. Despite the project being accepted into the CNCF, it's still unclear what will happen with it going forward.
What are some alternatives?
dextool - Suite of C/C++ tooling built on LLVM/Clang
sealed-secrets - A Kubernetes controller and tool for one-way encrypted Secrets
structured-text-tools - A list of command-line tools for manipulating structured text data
Vault - A tool for secrets management, encryption as a service, and privileged access management
csvtk - A cross-platform, efficient and practical CSV/TSV toolkit in Golang
age - A simple, modern and secure encryption tool (and Go library) with small explicit keys, no config options, and UNIX-style composability.
q - Quick and dirty debugging output for tired programmers. ⛺
git-crypt - Transparent file encryption in git
goawk - A POSIX-compliant AWK interpreter written in Go, with CSV support
terraform-provider-sops - A Terraform provider for reading Mozilla sops files
zsv - zsv+lib: tabular data swiss-army knife CLI + world's fastest (simd) CSV parser
vault-secrets-operator - Create Kubernetes secrets from Vault for a secure GitOps based workflow.