Phase Out Xcode Dependency GSOC Project Update

[Satryaji Aulia]

To say it more explicitly:

Besides removing the messages revolving Xcode, the latest changes in
macports-base now **force** MacPorts to use CommandLineTools instead
of Xcode.app for common tools like make, clang etc. This is to make every
build reproducible and force ports to explicitly declare Xcode dependency
if they need it by setting `use_xcode yes` or be in the `xcode 1.0` PortGroup.
We haven't discussed whether or not the `xcodeversion` PortGroup should be
included. We also **restrict** Xcode in tracemode if not needed. An exception
if is CLT isn't installed.

Within the next week there should be 2 more PRs about some problems
that we've encountered surrounding this change:

1. `system -W` calls not respecting DEVELOPER_DIR
About 60 ports use system -W with varying degree to call `/usr/bin/make`,
`/usr/bin/gnumake`, etc. which doesn't respect the macports environment
variables. These binaries will then resolve to Xcode's make, gnumake etc
which we don't want. Xcode-independent ports like libnetpbm that do this
will now fail to install in tracemode since we restrict Xcode when not
needed.
As Josh (jmroot) said, it should be the caller's responsibility to set
the environment if they're using `system`, but I'm working on an alternative
solution besides of patching those 60 something ports that use system -W.
If we go down the patch route, here's an example patch by Clemens. [1]

2. sdkroot pointing to Xcode's sdkroot instead of CLT's
Some ports explicitly call ${sdkroot} which point to Xcode's SDK when CLT's
can be used instead. Some ports like gmp fail in tracemode because of this.
The solution is to modify the proc which sets the sdkroot variable to point to
CLT's SDK by running xcrun --show-sdk-path with DEVELOPER_DIR set
and moving it to the top (previously, xcrun was the final fallback). [2]
The proc should be functionally the same with maybe a slight performance hit.

Please voice your concerns/input if any. Thanks.

[1] https://paste.macports.org/fc84fb351e73
[2] https://github.com/macports/macports-base/blob/31e0c5619836ebfc23c812b2d9f65d3c98938efa/src/port1.0/portconfigure.tcl#L500-L511

- Sat

On Tue, Jul 16, 2019 at 6:41 AM Satryaji Aulia <satraul at gmail.com> wrote:
>
> To update from last month about phasing out Xcode,
>
> 1. What has changed from last time
> The warnings that Xcode should be installed has been removed.
> DEVELOPER_DIR is now universally exported for every command (build,
> destroot, autotools, etc.).
>
> 2. The effects of the changes
> Right now, we’re testing to use see if the changes work as expected. [1]
> Ports like iTerm2 now intendedly fail because it calls `xcodebuild` without
> specifying Xcode dependency. The solution would be to add `use_xcode yes`
> to the Portfile, or add the `xcodeversion` portgroup to the Xcode-dependent
> list.
> Ports like libnetpbm unintendedly fail in tracemode because it calls
> `system -W something /usr/bin/gnumake` which problem was described in
> pull#134 but not solved yet. [2]
>
> More problems/solutions are being discovered, any input is appreciated.
> Thanks.
>
> [1]
> https://docs.google.com/spreadsheets/d/1NYmJsedVFLn7PeYFB08gCimqYANsbv7xwd78k4yaQPs/edit#gid=0
> [2] https://github.com/macports/macports-base/pull/134
>
> - Sat
>
> Satryaji Aulia <satraul at gmail.com> wrote:
>
> > Hi all,
> >
> > I'd like to write about what we've accomplished so far in this project,
> > with my mentors Marcus (mcalhoun) and Clemens (cal/neverpanic).
> > The project's focus is to provide a smoother experience of using MacPorts
> > without Xcode.
> >
> > 1. Opened PR for better handling of Xcode dependency [1]
> > In addition to the port 1.0 PortGroup, maintainers can declare Xcode
> > dependency using a new Portfile option use_xcode. Unless ports declare
> > Xcode dependency:
> > - Prefer Command Line Tools instead of Xcode by setting the DEVELOPER_DIR
> > environment.
> > - Hide the Xcode installation when building with trace mode, so MacPorts
> > can fail builds for ports that actually need Xcode but are not declared as
> > such. This will help maintainers identify Xcode dependency.
> > Also, error out when users try to install Xcode-dependent ports without
> > installing Xcode.
> >
> > 2. New command `port bump` to easily bump checksums of a port [2,3]
> > More features are planned such as @version, --livecheck, and Git actions
> >
> > Please check it out, any input is appreciated. Shoutout to my mentors (and
> > Mojca for reminding me to write to the mailing list).
> >
> > Thank you.
> >
> > [1] https://github.com/macports/macports-base/pull/130
> > [2] https://github.com/macports/macports-base/pull/120
> > [3] https://github.com/macports/macports-base/pull/127