GitHub post-extract shenanigans

[Ryan Schmidt]

Thanks for finding this answer; I wouldn't have remembered.

The github portgroup's post-extract shenanigans are necessary when extracting an automatically-generated GitHub tarball, because the directory name inside those tarballs contains the abbreviated git commit hash, which is not a piece of information we have in most Portfiles, nor do we want developers to have to add this and keep it updated every time they update the port. We want the resulting directory name to be predictable based on information already available in the Portfile (like name and version).

The shenanigans are not needed when github.tarball_from is anything other than the default "tags" (which will in future be renamed to "tarball") or "archive" (which is currently unsupported but see https://github.com/macports/macports-ports/pull/2587 for work on adding that). The conditions in the "if" statement don't state that, but are instead overly complicated. I would like to fix this, but the difficulty of sorting out exactly what all those conditions were trying to do, and ensuring I don't break something by changing it, has prevented me from devoting the needed time to this.

Note that we also have a PR to change MacPorts base behavior to automatically create a properly-named symlink to whatever got extracted, which should among other things make the github portgroup shenanigans unnecessary. (See https://github.com/macports/macports-base/pull/55)

\r



On Oct 17, 2018, at 01:22, Aaron Madlon-Kay wrote:

> After poking around some more I think I can answer my own question:
> 
> We don't use the `extract.mkdir yes` solution because it extracts to
> worksrcdir, but per the comment in the github 1.0 portgroup (linked in
> previous message) the port may want to set worksrcdir to a
> subdirectory of the extracted directory. So for it to be a viable
> solution, we would need yet another variable like `extract.target` to
> indicate where the result of extraction should be.
> 
> Maybe a flag to simply disable the github 1.0 portgroup's shenanigans
> would be more appropriate.
> 
> -Aaron
> 
> On Wed, Oct 17, 2018 at 12:33 PM Aaron Madlon-Kay wrote:
>> 
>> Hello all.
>> 
>> The github 1.0 portgroup does some post-extract shenanigans to
>> normalize the result of extracting the distfile: e.g. a GitHub project
>> `me/myproject` will, as retrieved from GitHub, extract to
>> `me-myproject-0123abc`; this is moved to `myproject` via globbing:
>> https://github.com/macports/macports-ports/blob/46c352bcc820ef80909e27a57d2f5eb71569abc6/_resources/port1.0/group/github-1.0.tcl#L81
>> 
>> ### My question
>> 
>> I am wondering why the above is done rather than doing the following,
>> which extracts directly to the desired `worksrcpath`:
>> 
>> extract.mkdir yes
>> extract.post_args-append --strip-components=1
>> 
>> Example extract invocation (simplified):
>> 
>> cd "/path/to/worksrcdir" && /usr/bin/gzip -dc
>> '/path/to/distfile.tar.gz' | /usr/bin/tar -xf - --strip-components=1
>> 
>> This seems to nicely solve the problem without relying on the format
>> of the tarball root. Is there a reason we are not using this solution?
>> 
>> ### Background
>> 
>> The following confluence of issues led me to the above solution:
>> 
>> 1. The golang portgroup has specific requirements for the worksrcdir
>> related to the GOPATH (long story)
>> 2. The github portgroup post-extract manipulations require that the
>> worksrcdir be already present, or that the result of extraction
>> matches the expected format
>> 3. When `github.tarball_from` is `release` the result of extraction is
>> unpredictable, so it needs to be handled by the port
>> 4. The github portgroup's post-extract block can't be overridden or
>> pre-empted, so additional post-extract blocks can't resolve this
>> conflict
>> 
>> The `extract.mkdir yes` solution ensures that the worksrcdir is
>> correct regardless.
>> 
>> An alternate solution might involve a flag to enable or disable the
>> github portgroup's post-extract manipulations, or a new variable to
>> specify the expected result of extraction independently of worksrcdir.
>> But these seem inelegant if the `extract.mkdir yes` solution is
>> viable.
>> 
>> Thanks,
>> Aaron