lakeFS
Git
Our great sponsors
lakeFS | Git | |
---|---|---|
48 | 285 | |
4,066 | 49,964 | |
2.5% | 2.0% | |
9.8 | 10.0 | |
4 days ago | 6 days ago | |
Go | C | |
Apache License 2.0 | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
lakeFS
-
A Step-by-Step Guide to Implementing Data Version Control
# Download the LakeFS binary wget https://github.com/treeverse/lakeFS/releases/latest/download/lakefs # Make the binary executable chmod +x lakefs # Initialize LakeFS with S3 as the storage backend ./lakefs init --backend s3 --s3-gateway-endpoint --s3-region --s3-force-path-style --s3-access-key --s3-secret-key
-
Jujutsu: A Git-compatible DVCS that is both simple and powerful
Might want to look at purpose built tools for that such as lakeFS (https://github.com/treeverse/lakeFS/)
* Disclaimer: I'm one of the creators/maintainers of the project.
-
Data diffs: Algorithms for explaining what changed in a dataset (2022)
Might want to checkout lakeFS: https://github.com/treeverse/lakeFS
(full disclosure: I'm one of the creators)
-
Transactions in Spark / Delta lake?
Take a look at https://github.com/treeverse/lakeFS -
- LakeFS – Version Control for Big Data
- DuckDB <3 LakeFS
- We built an open-source project (3.1K stars on GitHub) for data version control
-
How are you incrementally testing your data pipelines as you develop them?
I mean if you're ready to adopt a new framework into your ecosystem this is one of the major usecases for LakeFS.
- Git-for-Data
- LakeFS: Git-like versioning for object stores
Git
- GitHub Git Mirror Down
- Four ways to solve the "Remote Origin Already Exists" error.
-
So You Think You Know Git – Git Tips and Tricks by Scott Chacon
Boy, I can't find this either (but also, the kernel mailing list is _really_ difficult to search). I really remember Linus saying something like "it's not a real SCM, but maybe someone could build one on top of it someday" or something like that, but I cannot figure out how to find that.
You _can_ see, though, that in his first README, he refers to what he's building as not a "real SCM":
https://github.com/git/git/commit/e83c5163316f89bfbde7d9ab23...
- Maintain-Git.txt
-
Git Commit Messages by Jeff King
Here is the direct link, as HN somehow removes the query string: https://github.com/git/git/commits?author=peff&since=2023-10...
- Git commit messages by Jeff King
- My favourite Git commit (2019)
-
Do we think of Git commits as diffs, snapshots, and/or histories?
I understand all that.
I'm saying, if you write a survey and one of the possible answers is "diff", but you don't clearly define what you mean by "diff", then don't be surprised if respondents use any reasonable definition that makes sense to them. Ask an ambiguous question, get a mishmash of answers.
The thing that Git uses for packfiles is called a "delta" by Git, but it's also reasonable to call it a "diff". After all, Git's delta algorithm is "greatly inspired by parts of LibXDiff from Davide Libenzi"[1]. Not LibXDelta but LibXDiff.
Yes, how Git stores blobs (using deltas) is orthogonal to how Git uses blobs. But while that orthogonality is useful for reasoning about Git, it's not wrong to think of a commit as the totality of what Git does, including that optimization. (Some people, when learning Git, stumble over the way it's described as storing full copies, think it's wasteful. For them to wrap their heads around Git, they have to understand that the optimization exists. Which makes sense because Git probably wouldn't be practical if it lacked that optimization.)
The reason I'm bringing all this up is, if you're trying to explain Git, which is what the original article is about, then it's very important to keep in mind that someone who is learning Git needs to know what you mean when you say "diff". Most people who already know Git would tend to gravitate toward the definition of "diff" that you're assuming (the thing that Git computes on the fly and never stores), but people who already know Git aren't the target audience when you're teaching Git.
---
[1] https://github.com/git/git/blob/master/diff-delta.c
-
The State of Merging Technology
Didn't Git have a new default merge strategy, `ort` https://github.com/git/git/blob/master/Documentation/RelNote... ?
-
The bash book to rule them all
Yes, but you are referring to standalone scripts, not functions defined within a Bash script.
Compare for example the following helper code used for git command completion inside Bash and inside PowerShell.
Bash: https://github.com/git/git/blob/master/contrib/completion/gi...
What are some alternatives?
dvc - 🦉 ML Experiments and Data Management with Git
scalar - Scalar: A set of tools and extensions for Git to allow very large monorepos to run on Git without a virtualization layer
delta - An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs
PineappleCAS - A generic computer algebra system targeted for the TI-84+ CE calculators
git-lfs - Git extension for versioning large files
Subversion - Mirror of Apache Subversion
Ory Kratos - Next-gen identity server replacing your Auth0, Okta, Firebase with hardened security and PassKeys, SMS, OIDC, Social Sign In, MFA, FIDO, TOTP and OTP, WebAuthn, passwordless and much more. Golang, headless, API-first. Available as a worry-free SaaS with the fairest pricing on the market!
vscode-gitlens - Supercharge Git inside VS Code and unlock untapped knowledge within each repository — Visualize code authorship at a glance via Git blame annotations and CodeLens, seamlessly navigate and explore Git repositories, gain valuable insights via rich visualizations and powerful comparison commands, and so much more
MLflow - Open source platform for the machine learning lifecycle
linux - Linux kernel source tree
duf - Disk Usage/Free Utility - a better 'df' alternative
chromebrew - Package manager for Chrome OS [Moved to: https://github.com/chromebrew/chromebrew]