Git archive generation meets Hyrum's law

This page summarizes the projects mentioned and recommended in the original post on news.ycombinator.com

Our great sponsors
  • WorkOS - The modern identity platform for B2B SaaS
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • SaaSHub - Software Alternatives and Reviews
  • zephyr

    Primary Git Repository for the Zephyr Project. Zephyr is a new generation, scalable, optimized, secure RTOS for multiple hardware architectures.

  • I think that's uncharitable. Almost no one realized these things were being generated. We all assumed that links to github's "releases" were just links to files because they look like links to files! Here's one to Zephyr 3.2.0: https://github.com/zephyrproject-rtos/zephyr/archive/refs/ta...

    You pull that and get a tarball that is presented to the world as an "official release". Looks like a file. Acts like a file. It's a file.

    So now your package manager or reproducible build engine or whatever needs a reference to the "official source code release", and what do you point it to? That file, obviously. It's right there on the "release" page for the download. And of course you checksum it for security, because duh.

    Then last week all of a sudden that file changed! Sure, it has the same contents. But the checksum that you computed in good faith based on the official release tarball doesn't match!

    If there's a misunderstanding here, it's on github and not the users. They can't be providing official release tarballs if they won't guarantee consistency. "As documented", this feature was a huge footgun. That's bad.

  • SIG-rules-authors

    Governance and admin for the rules authors Special Interest Group

  • In this case, it seems that GitHub was asked about it. From the thread linked in the article:

    > After a fruitful exchange with GitHub support staff, I was able to confirm the following (quoting with their permission):

    >> I checked with our team and they confirmed that we can expect the checksums for repository release archives, found at /archive/refs/tags/$tag, to be stable going forward. That cannot be said, however, for repository code download archives found at archive/v6.0.4.

    >> It's totally understandable that users have come to expect a stable and consistent checksum value for these archives, which would be the case most of the time. However, it is not meant to be reliable or a way to distribute software releases and nothing in the software stack is made to try to produce consistent archives. This is no different from creating a tarball locally and trying verify it with the hash of the tarball someone created on their own machine.

    >> If you had only a tag with no associated release, you should still expect to have a consistent checksum for the archives at /archive/refs/tags/$tag.

    > In summary: It is safe to reference archives of any kind via the /refs/tags endpoint, everything else enjoys no guarantees.

    (posted 4 Feb 2022)

    https://github.com/bazel-contrib/SIG-rules-authors/issues/11...

    There's even a million linked PRs and issues where people went around and specifically updated their code to point to the URLs that were, nominally, stable.

    I suspect that the GH employee who made these comments just misunderstood how these archives were being generated, or the behavior was depending on some internal implementation detail that got wiped away at some point. But if an employee at a big-ass company publicly says "yeah that's supported" to employees at another big-ass company, people are gonna take it as somewhat official.

  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

    WorkOS logo
NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts