[MacPorts] #54800: Set up a private server for archives that aren't distributable to speed up Travis CI builds
MacPorts
noreply at macports.org
Thu Nov 29 00:31:15 UTC 2018
#54800: Set up a private server for archives that aren't distributable to speed up
Travis CI builds
-----------------------------+---------------------
Reporter: yan12125 | Owner: admin@…
Type: enhancement | Status: new
Priority: Normal | Milestone:
Component: server/hosting | Version:
Resolution: | Keywords:
Port: |
-----------------------------+---------------------
Comment (by ryandesign):
Replying to [comment:8 ryandesign]:
> * On the buildbot server, set up a new directory outside of the rsync
directory to store nondistributable archives.
Done. I've used a separate script to manually populate this directory with
all current archives from all builders. After buildbot is modified to
upload nondistributable archives there, I can run the separate script once
more to sync everything up, then run another script to remove
distributable archives from this directory, leaving only the
nondistributable ones.
> * Change mpbb so that it uploads nondistributable archives to there. (We
don't need to worry about cleaning up old nondistributable archives; the
existing [ticket:56181 cleanup script] can probably be used for that.)
https://github.com/macports/mpbb/pull/9
Buildbot configuration changes are also needed to supply the new arguments
to mpbb.
https://github.com/macports/macports-infrastructure/pull/8
> * At NameCheap, make 2 new hostnames on macports.org for this: one for
the origin (on the buildbot server) and one for the CDN.
Done.
Still to do: automating the renewal of the Let's Encrypt SSL certificates.
> * On the buildbot server, configure nginx to serve the new directory via
that new origin hostname (and also a local-network-only hostname for use
by the buildbot), and to allow access only from MaxCDN IP addresses (and
localhost).
Done.
> * At MaxCDN, add a new zone for this with the new CDN hostname and
configure the zone to allow access only from Travis IPs.
Done.
> * Modify Travis config to also find packages at the new CDN hostname by
adding an entry to archive_sites.conf.
Still to do. Another possibility is setting `archive_site_local` in
macports.conf.
> * Modify buildbot config to also find packages at the new local-network-
only hostname by adding an entry to archive_sites.conf.
The problem is that mpbb has its own separate copy of macports.conf, which
references its own separate copy of sources.conf. However, I don't see any
way to reference a separate copy of archive_sites.conf. Using
`archive_site_local` in macports.conf is an alternative; this is included
in https://github.com/macports/mpbb/pull/9. However, it means that the
standard set of archive sites is still used; I would like to find a way to
disable that.
> * Maybe the preceding 2 happen in mpbb; if so, do we have a way in mpbb
to differentiate whether we're running on Travis or buildbot?
mpbb sets up macports.conf in mpbb-checkout. Travis doesn't use mpbb-
checkout, so at present these changes will only affect buildbot. I've
added arguments so that the archive site URLs can be specified when
calling mpbb-checkout, so that Travis and Buildbot could request different
URLs.
--
Ticket URL: <https://trac.macports.org/ticket/54800#comment:10>
MacPorts <https://www.macports.org/>
Ports system for macOS
More information about the macports-tickets
mailing list