Regarding github CI

Joshua Root jmr at macports.org
Thu Sep 7 08:44:18 UTC 2023


I have wanted to add some testing to the CI and buildbot for some time, 
but there are a few complications to consider. The first and simplest 
barrier is that the current release of MacPorts errors out if you try to 
run tests on a port that doesn't explicitly enable them. Related to that 
is the fact that many (most?) ports don't have any meaningful tests to run.

There are unreleased changes that start to address this. I've added one 
basic test that can be run for all ports, which at the moment just 
checks if they are using `supported_archs noarch` appropriately. More 
similarly simple and generic tests can be added, e.g. checking binaries 
for broken linking. Ports that don't set `test.run yes` will run only 
the built-in tests, while those that do will also run their custom tests 
as defined in the Portfile.

The other problem is that some projects have, shall we say, 
comprehensive test suites. This is undoubtedly a good thing for those 
projects! But as a distro building thousands of ports, we have to have a 
relatively tight limit on how long tests can take to run. There's 
currently no way of differentiating a CI-appropriate test phase from an 
unreasonably long-running one. Ideally we might have the option of 
defining a short test phase that is run on CI, and a longer one that can 
be run locally by maintainers and other interested parties if desired.

- Josh


More information about the macports-dev mailing list