fetch.type git & GitHub submodules (was: [133168] trunk/dports/sysutils)

[Rainer Müller]

On 2015-03-04 22:27, Mojca Miklavec wrote:
>> I agree with you, creating the distfiles from VCS would be possible.
>>
>> There could be a target to be run on 'port mirror' that downloads and
>> creates a tarball if a non-default fetch.type is used. That alone would
>> reduce multiple downloads and even makes port development faster.
>>
>> However, for end-users, there is the problem that we would need to
>> distribute checksums for these tarballs (or rely on signatures only?).
> 
> Of course we would have to distribute the checksums in that case, like
> for any other port. What exactly is considered a "problem" here?

There are two options:

a) the maintainer generates the tarball locally, uploads it to the main
mirror and also adds an additional checksum to the Portfile before
committing it

b) tarballs are generated automatically on the server after the Portfile
was committed


For a), you would have to trust the maintainer about the contents of the
generated tarball. Also, why would we even need anything in the Portfile
how this source tarball was generated? Furthermore, an additional
infrastructure for uploading files would be needed. We are low on
infrastructure resources already, I don't think it would be a good idea
to start it this way.


I would prefer b) for the simple fact that this ensures that the
maintainer did not modify any of the files and would be closer to our
existing distfiles mirroring. The infrastructure changes would be small
if it can be integrated into what the existing 'port mirror' does.
However, checksums for the generated tarball are definitely not known at
the time the Portfile is committed.

One solution for this would be to add an additional file in the port
directory after tarball generation that holds the checksums. Or, the
generated tarballs are also signed by the job that generated them. With
the signature it is possible to verify that this is the intended file
without distributing any additional checksum through other channels.


In general, note that generating a tarball might include timestamps,
usernames, and other metadata. Generating it multiple times, locally by
the maintainer and once again on the server, will not always give the
same results. Although that would be the closest to what we do for
distfiles at the moment, combining the checksum in Portfile from a)
*and* the automatic generation on the server from b) is not possible.

Rainer