VFSForGit
nixpkgs
VFSForGit | nixpkgs | |
---|---|---|
24 | 975 | |
5,944 | 15,753 | |
0.4% | 2.8% | |
4.7 | 10.0 | |
about 2 months ago | 3 days ago | |
C# | Nix | |
MIT License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
VFSForGit
-
Debian Git Monorepo
It's not only Windows that uses Git at Microsoft, but Sharepoint and Office (which includes the on-prem version of SharePoint). In terms of repo size Windows and Office are similar. I was part of the team that migrated Sharepoint from a Perforce clone to Git and helped build the tooling to allow Office to move as well. VFS for Git [1] and Scalar [2] are really good pieces of software.
[1] - https://github.com/microsoft/VFSForGit
[2] - https://github.com/microsoft/scalar
-
Serving a Website from a Git Repo Without Cloning It
Congratulations! That means you basically figured out how the clone procedure works and found a way to do so just in a partial way (also in an unsafe way). But it is a cool idea, nonetheless.
Also check out the Scalar [1] project and its predecessor, GVFS [2], both from Microsoft to manage their monorepo via a VFS layer.
[1]: https://github.com/microsoft/scalar
[2]: https://github.com/microsoft/VFSForGit
-
We Put Half a Million Files in One Git Repository, Here's What We Learned (2022)
VFS for Git is still Open Source: https://github.com/microsoft/VFSForGit
Microsoft's blog posts have indicated a move to use something as close to off-the-shelf git as possible, though. They say they've stopped using VFS much and are instead more often relying on sparse checkouts. They've upstreamed a lot of patches into git itself, and maintain their own git fork but the fork distance is generally shrinking as those patches upstream.
-
Why SQLite Does Not Use Git
https://github.com/microsoft/VFSForGit
better than it used to, with the caveat that git in particular is not and has never claimed to be good at versioning blobs.
-
π πΎ Oxen.ai - Blazing Fast Unstructured Data Version Control, built in Rust
Oh dear you're not going to like this.
-
He is very conservative...
Itβs virtualised file system: https://github.com/microsoft/VFSForGit, only downloads what you actually use. Same thing in every large company, but different implementations.
-
FYI: LLVM-project repo has exceeded GitHub upload size limit
This is where something like VFSForGit[0] helps out. Instead of cloning the entire repo, it creates a virtual file system and fetches objects on demand. MSFT uses it internally for the Windows source tree (which now exceeds 300GB).
[0]: https://github.com/microsoft/VFSForGit
-
Created a Small Program To Display Upcoming Assignments On My Desktop
There's also a performance consideration. Not excluding /bin/ or /obj/ folders means dependencies are being tracked as well, and sometimes dependencies themselves are bigger than the program's source code itself. This is commonly the case with node projects, as the node_modules folder can balloon to hundreds of megabytes. They should never be tracked in git due to the nature of how git's internal database works. For e.g. if you delete a dependency because it's no longer needed, you can never fully reclaim that disk space (at least for the master branch) as git will need to keep the binary data stored in its internal tracking database because a previous commit in the master branch has captured the data. As you make more branches, git needs to store the data required to reconstruct your repo to a different state when you switch branches. When a branch has changes measured in the kilobytes, check out is very manageable, but when the differences balloon to many MBs due to the presence of heavy binary files, then checkout between different branches/commits can get very slow. Though, this happens anyway when source code data eventually reaches a certain threshold, beyond the hundreds of megabytes, it's made unnecessarily worse by including any binary files. It's one of the reasons Microsoft created VFS for git: https://github.com/microsoft/VFSForGit.
- Meta releases Sapling, a new way of using source control
-
Software for managing config files
You mean like VFSforGit? Or the successor for that called Scalar? This has been a solved problem. Microsoft moved their entire Windows codebase to git. There have been a ton of huge improvements to performance as a result of that. And the above two plugins are easily better ways to deal with what you're referring to without resulting to dead tech.
nixpkgs
-
Nix: The Breaking Point
I don't think so. The article is probably intended for the Nix community, so the author doesn't need to convince HN that something is going on. If as an outsider you are interested then you need to look into it yourself, the community has no obligation to make their internal conflicts legible to the outside world.
As an outsider myself, it certainly looks like something is going on as more than 20 Nixpkg maintainers left in a week: https://github.com/NixOS/nixpkgs/issues?q=label%3A%228.has%3...
- Maintainers Leaving
-
Air Force picks Anduril, General Atomics to develop unmanned fighter jets
https://github.com/NixOS/nixpkgs/commits?author=neon-sunset
-
Eelco Dolstra's leadership is corrosive to the Nix project
I see two signers in the top 6 displayed on https://github.com/NixOS/nixpkgs/graphs/contributors
-
3rd Edition of Programming: Principles and Practice Using C++ by Stroustrup
For a single file script, nix can make the package management quite easy: https://github.com/NixOS/nixpkgs/blob/master/doc/languages-f...
For example,
```
- NixOS/nixpkgs: There isn't a clear canonical way to refer to a specific package
-
NixOS Is Not Reproducible
Yes, Nix doesn't actually ensure that the builds are deterministic. In fact it works just fine if they aren't. There are packages in nixpkgs that aren't reproducible: https://github.com/NixOS/nixpkgs/issues?q=is%3Aopen+is%3Aiss...
-
The xz attack shell script
I'm not familiar with Bazel, but Nix in it's current form wouldn't have solved this attack. First of all, the standard mkDerivation function calls the same configure; make; make install process that made this attack possible. Nixpkgs regularly pulls in external resources (fetchUrl and friends) that are equally vulnerable to a poisoned release tarball. Checkout the comment on the current xz entry in nixpkgs https://github.com/NixOS/nixpkgs/blob/master/pkgs/tools/comp...
-
Debian Git Monorepo
NixOS uses a monorepo and I think everyone's love it.
I love being able to easily grep through all the packages source code and there's regularly PRs that harmonizes conventions across many packages.
Nixpkgs doesn't include the packaged software source code, so it's a lot more practical than what Debian is doing.
https://github.com/NixOS/nixpkgs
-
From xz to ibus: more questionable tarballs
In this specific case, nix uses fetchFromGitHub to download the source archive, which are generated by GitHub for the specified revision[1]. Arch seems to just download the tarball from the releases page[2].
[1]: https://github.com/NixOS/nixpkgs/blob/3c2fdd0a4e6396fc310a6e...
[2]: https://gitlab.archlinux.org/archlinux/packaging/packages/ib...
What are some alternatives?
scalar - Scalar: A set of tools and extensions for Git to allow very large monorepos to run on Git without a virtualization layer
asdf - Extendable version manager with support for Ruby, Node.js, Elixir, Erlang & more
EdenSCM - A Scalable, User-Friendly Source Control System. [Moved to: https://github.com/facebook/sapling]
Home Manager using Nix - Manage a user environment using Nix [maintainer=@rycee]
juicefs - JuiceFS is a distributed POSIX file system built on top of Redis and S3.
git-lfs - Git extension for versioning large files
git - A fork of Git containing Microsoft-specific patches.
easyeffects - Limiter, compressor, convolver, equalizer and auto volume and many other plugins for PipeWire applications
dvc - π¦ ML Experiments and Data Management with Git
spack - A flexible package manager that supports multiple versions, configurations, platforms, and compilers.
mvfs - ClearCase file system
waydroid - Waydroid uses a container-based approach to boot a full Android system on a regular GNU/Linux system like Ubuntu.