Xcode and macOS SDK requirement (was: Re: GSoC 2018: Interested in Improving startup item code)

[Ryan Schmidt]

On Feb 23, 2018, at 03:59, Clemens Lang wrote:

> 
> Most ports will build fine. Some will not. GUI software is a good candidate
> for failure.
> 
> The GSoC task would be to provide a method to declare this implicit dependency
> in Portfiles so that we could stop showing this warning for every installation,
> recommend to only install the command line tools and bail out if a port
> requires Xcode but it is not installed.
> 
> This could probably be done together with rule adaptions for trace mode to hide
> Xcode if a dependency was not declared, so port maintainers had a simple way to
> test whether their port uses Xcode.

My understanding is that the work that has already been committed to master by Jeremy Sequoia and which will be released in MacPorts 2.5 goes the other way, making the command line tools unnecessary and requiring only Xcode. This should work for all ports. (Some ports may have bugs where they don't build without the command line tools; those bugs will need to be fixed.) But I understand that Xcode is large and it would be nice to avoid installing it when it is not required.

The current error message in MacPorts base, which states that most ports will fail to build because Xcode is not installed, dates from the days of Xcode 3.x and earlier when the command line tools were part of the same installer that installed Xcode. The error message also does not consider that a binary of the port might be installed, which would not require Xcode nor the command line tools.

Ports that need Xcode would include those that want to build an Xcode project. Most of those ports probably do so using the xcode portgroup, so if we add any option to MacPorts base to indicate that a port needs Xcode, that option could be set in the xcode portgroup to fix most affected ports.

We also already have the xcodeversions portgroup, by which ports can indicate that on a particular macOS version they require a particular minimum Xcode version. The hypothetical "Xcode required" option should be set in that portgroup as well. Though probably this portgroup has outlived its usefulness, and most of the time when a port specifies that it needs a minimum Xcode version, what it actually needs is a minimum macOS SDK version, so that's probably what ports should be specifying instead.

Because the macOS SDK is in Xcode, ports which expect a macOS SDK to exist would also currently require Xcode, whether or not they use an Xcode project. (Ports that build using cmake often use an SDK, for example.) Apple recommends developers use the latest macOS SDK, even if they want to target older systems. By setting the macOS deployment target to the minimum OS version, and using the latest SDK, developers can write code that uses new OS capabilities when running on newer OS versions, while still allowing the program to run (without those new capabilities) on older OS versions. This assumes that the software is built with the latest SDK, which requires the latest version of Xcode, which requires a very recent version of macOS, and is then distributed to users who might run it on earlier OS versions. This does not match how MacPorts works, where we always build on the same major version of macOS on which the program will run (and for most open source software that is *not* specific to Apple platforms, there are good reasons why we do that).

We already have the situation where some ports require a newer macOS SDK in order to build. For example, libsdl2 builds and runs on Snow Leopard and later, but only if the 10.7 SDK is present, and no Snow Leopard-compatible version of Xcode includes that. I have manually placed the 10.7 SDK onto the 10.6 buildbot workers, which solves that specific instance of the problem, but not the general problem.

We also already have many ports that require a newer compiler than is available in the latest Xcode for an old version of macOS. We already have a solution for this, in the form of the compiler.blacklist option and the compiler_blacklist_versions portgroup which extends that option's meaning. Ports that require C++14 capabilities, for example, can use "compiler.blacklist {clang < 602}"; if the clang version is less than 602, then MacPorts installs and uses a newer Clang to build that port.

MacPorts base already has the configure.sdk_version option, but it dates from a time when a single Xcode version would include multiple macOS SDK versions. These days, only the latest SDK is in Xcode. What we need is a way to specify a *minimum* SDK version. (We might consider making the syntax flexible enough to allow a port to also specify a *maximum* SDK version. It might be possible that Apple introduces a change in a new SDK that is incompatible with some existing software.) This could be a new option, or it could be a change to the existing option, which would work similar to the way that compiler.blacklist works when the compiler_blacklist_versions portgroup is included. If that were done, then libsdl2 could specify "configure.sdk_version {>= 10.7}". A hypothetical port that required at least the 10.8 SDK, but not the 10.13 SDK or newer, could specify "configure.sdk_version {>= 10.8 < 10.13}".

Currently, if configure.sdk_version is set to a version of the SDK that is not present on disk, an SDK is not used. I would like to change this so that if an acceptable SDK is not present, then it is installed using MacPorts. I have nearly finished developing the necessary ports for this but have not tested them yet. I expect a portgroup to be created as well, to implement the preceding new configure.sdk_version behavior. We are probably not allowed to distribute the SDKs, so it will not be very convenient for users to use, but they should be able to follow the instructions and get it to work, which is better than what we have now where a port either fails to build or gives an error message that installation is not possible. And for ports that are distributable, they'll just be able to receive our pre-built binaries.