Github Checksums ..

[Ryan Schmidt]

On Apr 16, 2016, at 9:55 AM, Rainer Müller wrote:

> On 2016-04-16 02:54, Brandon Allbery wrote:
>> Yes, that's what I meant. You want to point to archives, because they
>> don't change; tarballs will be regenerated on the fly by github, so they
>> do not have fixed checksums and you would have to either make "fake"
>> accesses to them every so often so github thinks they are still in use
>> and won't remove and regenerate them, or update the checksums every week
>> or so for the latest generated tarball. Neither one is worth the effort.
> 
> I don't think checksums for GitHub tarballs change anymore. Was there
> any recent case where it happened? I don't know their implementation
> details, but even a simple 'git archive' generates the same reproducible
> tarball...
> 
> The tarball changes based on whether you download them from
>  https://github.com/Z3Prover/z3/tarball/z3-4.4.1
> as compared to
>  https://github.com/Z3Prover/z3/archive/z3-4.4.1.tar.gz
> The reason is that the top-level directory inside the tarball is named
> differently.
> 
> My interpretation of that statement in the ticket is that the GitHub
> port group will fetch a different file, and checksums need to be updated
> in the Portfile for that.

Neither "tarball" nor "archive" downloads are particularly unstable. Their contents are deterministic and can be successfully verified with checksums.

The port was submitted without using the github portgroup and using an "archive" download. I requested it be switched to using the github portgroup and using a "tarball" download since that is what the github portgroup wants to do and there is no reason to override it since "tarball" and an "archive" downloads are nearly identical. There is one difference: the name of the directory it extracts into. That difference is enough to change the checksums. Of course you must "sudo port clean --all z3" to delete the previously downloaded file with the old checksums.

"tarball" and "archive" download checksums can change in the unusual circumstance that the developer has deleted the tag and recreated it from a different commit. Developers should not do that, but sometimes do. It's happened to me with mongo-tools. In these cases, we educate the developers on the problem this action causes, and hope they don't do it again in the future.

"tarball" checksums can also change if a project has moved from one GitHub owner to another, because the name of the enclosing directory is ${owner}-${project}-${commit}, which includes the name of the project's GitHub owner. This probably doesn't happen a lot for any individual project, but given how many ports we have, it does happen rather a lot on the whole and is annoying. I did not realize until I checked into it just now that "archive" downloads to not seem to have this problem; I am not sure exactly how the directory name is assembled by GitHub but it does not appear to contain the organization name. This would finally be a justification for adding support for "archive" downloads to the github portgroup and making it the default, requested here:

https://trac.macports.org/ticket/40518