[MacPorts] #58779: base: ports cannot use SDK inside of Developer dir unless using Xcode build type

MacPorts noreply at macports.org
Thu Aug 1 05:50:38 UTC 2019 

#58779: base: ports cannot use SDK inside of Developer dir unless using Xcode build
type
---------------------+-----------------------
  Reporter:  Ionic   |      Owner:  (none)
      Type:  defect  |     Status:  new
  Priority:  Normal  |  Milestone:
 Component:  base    |    Version:  2.5.99
Resolution:          |   Keywords:  tracemode
      Port:          |
---------------------+-----------------------

Comment (by Ionic):

 > Yes, the existence of /usr/lib/libxcselect.dylib (xcode-select)
 indicates macOS >= 10.9 which we assume is the earliest macOS version that
 moved CLT from / to /Library/Developer/CommandLineTools/.

 Which is fine... I mean, we do want to get rid of needing the full Xcode
 suite, hence your GSoC project.

 The current approach fails if both CLT and Xcode are installed, though, at
 least for older OS versions.

 Let me quote from #58770:

 {{{
 :info:build /usr/bin/clang -arch x86_64 -o bits.os -c -Os -O2 bits.c
 :info:build xcrun: error: invalid active developer path
 (/Applications/Xcode.app/Contents/Developer), missing xcrun at:
 /Applications/Xcode.app/Contents/Developer/usr/bin/xcrun
 }}}

 What essentially happened here, as far as I can tell, is that the system
 clang binary was invoked, which... is actually a shim at /usr/bin/clang.
 This shim uses libxcselect.dylib, which then probably calls xcrun to get
 the actual tool location.

 With Xcode installed, xcode-select -p returns a Developer dir path of
 /Applications/Xcode.app/Contents/Developer. Since tracemode hides that,
 xcrun won't be able to see the xcrun binary bundled in there, assume that
 that path is wrong and fail to execute tools.

 I see two viable options:

   1. Always allow access to the Xcode-based Developer directory if it
 exists. That probably wouldn't be too bad since it's what base has always
 done so far. If that directory doesn't exist, xcode-select -p SHOULD use
 /Library/Developer/CommandLineTools anyway so whitelisting a non-existent
 path shouldn't cause trouble.
   2. Override the Developer dir path via the DEVELOPER_DIR environment
 variable if we don't want to use Xcode (i.e., if `use_xcode` is `no`).
 Example:
      {{{
 root at nopileos~#  xcrun --sdk macosx10.9 -f clang
 /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang
 root at nopileos~#  DEVELOPER_DIR=/Library/Developer/CommandLineTools xcrun
 --sdk macosx10.9 -f clang
 /Library/Developer/CommandLineTools/usr/bin/clang
      }}}

 Option two would mimic your original intention more closely, but requires
 deeper changes to base code since we'd now always have to set
 DEVELOPER_DIR if using Xcode is not explicitly requested.

 ----

 > I just found out that there are many ports (~150) that depend on qmake,
 so I think we should consider a larger solution. Ccing my project mentors.

 Well, ports that use qmake(5) usually also use the qmake(5)-1.0
 PortGroups, so it would be easy to just change the two as a workaround.
 There are some ports that don't use the PGs, but they probably should do
 so anyway.

 This said, we probably don't want to use yet another workaround anyway.
 Such a workaround would essentially use option one and be put into every
 port (or PortGroup) that uses the system clang compiler, which we also do
 by default most of the time. Not really a good option.

 ----

 > There's one problem. I have Xcode-10.3 installed with CLT, but it did
 not install include files into /use/include.

 Uh, that sounds really, really odd. Not having /usr/include would
 essentially make CLT useless. Xcode bundles SDKs anyway which include the
 command line tools, but a standalone CLT package without the proper header
 files would defeat the whole point of this exercise.

 Do you have the shims in /usr/bin (e.g., /usr/bin/clang)?

 Did they move the headers around by any chance?
 /Library/Developer/CommandLineTools/usr/include exists traditionally (on
 my 10.9 machine at least), but that only ships some C++ headers as far as
 I can tell. Not sure why they wouldn't have been installed in /usr/include
 as well together with all the other stuff, but I guess it's because they
 are specifically used by clang. /usr/include/c++ includes a 4.2.1
 directory, which smells like Apple-GCC at first, but:
 {{{
 root at nopileos~#  DEVELOPER_DIR=/Library/Developer/CommandLineTools xcrun
 g++ --version
 Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-
 gxx-include-dir=/usr/include/c++/4.2.1
 Apple LLVM version 6.0 (clang-600.0.57) (based on LLVM 3.5svn)
 Target: x86_64-apple-darwin13.4.0
 Thread model: posix
 }}}

 Funnily enough, clang seems to be lying:

 {{{
 root at nopileos~#  DEVELOPER_DIR=/Library/Developer/CommandLineTools xcrun
 g++ -E -x c++ - -v < /dev/null
 Apple LLVM version 6.0 (clang-600.0.57) (based on LLVM 3.5svn)
 Target: x86_64-apple-darwin13.4.0
 Thread model: posix
  "/Library/Developer/CommandLineTools/usr/bin/clang" -cc1 -triple x86_64
 -apple-macosx10.9.0 -E -disable-free -disable-llvm-verifier -main-file-
 name - -mrelocation-model pic -pic-level 2 -mdisable-fp-elim -masm-verbose
 -munwind-tables -target-cpu core2 -target-linker-version 241.9 -v
 -resource-dir /Library/Developer/CommandLineTools/usr/bin/../lib/clang/6.0
 -stdlib=libc++ -fdeprecated-macro -fdebug-compilation-dir /var/root
 -ferror-limit 19 -fmessage-length 224 -stack-protector 1 -mstackrealign
 -fblocks -fobjc-runtime=macosx-10.9.0 -fencode-extended-block-signature
 -fcxx-exceptions -fexceptions -fdiagnostics-show-option -fcolor-
 diagnostics -vectorize-slp -o - -x c++ -
 clang -cc1 version 6.0 based upon LLVM 3.5svn default target x86_64-apple-
 darwin13.4.0
 ignoring nonexistent directory "/usr/include/c++/v1"
 #include "..." search starts here:
 #include <...> search starts here:
  /Library/Developer/CommandLineTools/usr/bin/../include/c++/v1
  /usr/local/include
  /Library/Developer/CommandLineTools/usr/bin/../lib/clang/6.0/include
  /Library/Developer/CommandLineTools/usr/include
  /usr/include
  /System/Library/Frameworks (framework directory)
  /Library/Frameworks (framework directory)
 End of search list.
 # 1 "<stdin>"
 # 1 "<built-in>" 1
 # 1 "<built-in>" 3
 # 175 "<built-in>" 3
 # 1 "<command line>" 1
 # 1 "<built-in>" 2
 # 1 "<stdin>" 2

 }}}

 I also noticed that one should better not specify --sdk when overriding
 DEVELOPER_DIR to the CLT location, because more fun stuff seems to happen
 in that case:

 {{{
 root at nopileos~#  DEVELOPER_DIR=/Library/Developer/CommandLineTools xcrun
 --sdk macosx10.9 g++ --version
 Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-
 gxx-include-dir=macosx10.9/usr/include/c++/4.2.1
 Apple LLVM version 6.0 (clang-600.0.57) (based on LLVM 3.5svn)
 Target: x86_64-apple-darwin13.4.0
 Thread model: posix
 }}}

 ... which, again, is a lie:

 {{{
 root at nopileos~#  DEVELOPER_DIR=/Library/Developer/CommandLineTools xcrun
 --sdk macosx10.9 g++ -E -x c++ - -v < /dev/null
 Apple LLVM version 6.0 (clang-600.0.57) (based on LLVM 3.5svn)
 Target: x86_64-apple-darwin13.4.0
 Thread model: posix
  "/Library/Developer/CommandLineTools/usr/bin/clang" -cc1 -triple x86_64
 -apple-macosx10.9.0 -E -disable-free -disable-llvm-verifier -main-file-
 name - -mrelocation-model pic -pic-level 2 -mdisable-fp-elim -masm-verbose
 -munwind-tables -target-cpu core2 -target-linker-version 241.9 -v
 -resource-dir /Library/Developer/CommandLineTools/usr/bin/../lib/clang/6.0
 -stdlib=libc++ -fdeprecated-macro -fdebug-compilation-dir /var/root
 -ferror-limit 19 -fmessage-length 224 -stack-protector 1 -mstackrealign
 -fblocks -fobjc-runtime=macosx-10.9.0 -fencode-extended-block-signature
 -fcxx-exceptions -fexceptions -fdiagnostics-show-option -fcolor-
 diagnostics -vectorize-slp -o - -x c++ -
 clang -cc1 version 6.0 based upon LLVM 3.5svn default target x86_64-apple-
 darwin13.4.0
 ignoring nonexistent directory "/usr/include/c++/v1"
 #include "..." search starts here:
 #include <...> search starts here:
  /Library/Developer/CommandLineTools/usr/bin/../include/c++/v1
  /usr/local/include
  /Library/Developer/CommandLineTools/usr/bin/../lib/clang/6.0/include
  /Library/Developer/CommandLineTools/usr/include
  /usr/include
  /System/Library/Frameworks (framework directory)
  /Library/Frameworks (framework directory)
 End of search list.
 # 1 "<stdin>"
 # 1 "<built-in>" 1
 # 1 "<built-in>" 3
 # 175 "<built-in>" 3
 # 1 "<command line>" 1
 # 1 "<built-in>" 2
 # 1 "<stdin>" 2

 }}}

 Also note how clang doesn't show the "Configured with:" line at all when
 invoking it as "clang++", so that output can probably be mostly ignored:

 {{{
 root at nopileos~#  DEVELOPER_DIR=/Library/Developer/CommandLineTools xcrun
 --sdk clang++ --version
 Apple LLVM version 6.0 (clang-600.0.57) (based on LLVM 3.5svn)
 Target: x86_64-apple-darwin13.4.0
 Thread model: posix
 }}}

 Sometimes it feels like all of this stuff is held together by band aid.

-- 
Ticket URL: <https://trac.macports.org/ticket/58779#comment:8>
MacPorts <https://www.macports.org/>
Ports system for macOS

More information about the macports-tickets mailing list