[MacPorts] #68329: py311-scipy @1.10.1_0+gfortran+openblas not building on Sonoma apple silicon
MacPorts
noreply at macports.org
Sat Oct 21 14:59:18 UTC 2023
#68329: py311-scipy @1.10.1_0+gfortran+openblas not building on Sonoma apple
silicon
--------------------------+-----------------------
Reporter: quintusdias | Owner: michaelld
Type: defect | Status: assigned
Priority: Normal | Milestone:
Component: ports | Version: 2.8.1
Resolution: | Keywords:
Port: py311-scipy |
--------------------------+-----------------------
Comment (by markmentovai):
I do think that it’s possible to fix this for Xcode 15.0, and in the
process, fix a latent bug that may not have been appreciated until now.
This can be done in a way that’s not a hack and isn’t wrong for future
versions of Xcode.
We don’t have the full source for the new `ld`, but some of its source
code is shared with `dyld`:
{{{
mark at arm-and-hammer zsh% ld -v 2>&1 | head -1
@(#)PROGRAM:ld PROJECT:dyld-1015.7
}}}
The `duplicate LC_RPATH` messages are in the shared source that we can
see, and come from [https://github.com/apple-oss-
distributions/dyld/blob/18d3cb0f6b46707fee6d315cccccf7af8a8dbe57/mach_o/Header.cpp#L618
mach_o::Header::validSemanticsRPath]. There, you can see the logic that
determines whether this situation is a warning or an error: the condition
is based on `enforceNoDupRPath`, which relies on [https://github.com
/apple-oss-
distributions/dyld/blob/18d3cb0f6b46707fee6d315cccccf7af8a8dbe57/mach_o/Policy.cpp#L246
mach_o::Policy::enforceNoDuplicateRPaths], which is in turn based on the
[https://github.com/apple-oss-
distributions/dyld/blob/18d3cb0f6b46707fee6d315cccccf7af8a8dbe57/mach_o/Policy.cpp#L40
SDK version]. `enforceNoDuplicateRPaths` is not supposed to become a hard
error until `Platform::Epoch::fall2024`, which would be next year’s batch
of OS releases. So why are we seeing this as a hard error today, in fall
2023?
The SDK used to link a module is embedded in its `LC_BUILD_VERSION` load
command, and can be configured by the `-platform_version` option to `ld`.
The compiler driver will normally pass rational values to the linker using
this option. For example, here’s what Xcode 15.0.1 clang-1500.0.40.1 does:
{{{
mark at arm-and-hammer zsh% clang -dynamiclib lib.c -o liblib.dylib
-Wl,-rpath,/tmp/lib -Wl,-rpath,/tmp/lib -Wl,-ld_classic -v 2>&1 | grep /ld
| tr ' ' '\n' | grep -A 3 platform_version
-platform_version
macos
14.0.0
14.0
mark at arm-and-hammer zsh% otool -l liblib.dylib | grep -B 1 -A 7
LC_BUILD_VERSION
Load command 8
cmd LC_BUILD_VERSION
cmdsize 32
platform 1
minos 14.0
sdk 14.0
ntools 1
tool 3
version 907.0
mark at arm-and-hammer zsh% otool -l liblib.dylib | grep -B 1 -A 2 LC_RPATH
Load command 11
cmd LC_RPATH
cmdsize 24
path /tmp/lib (offset 12)
Load command 12
cmd LC_RPATH
cmdsize 24
path /tmp/lib (offset 12)
}}}
And even though I’ve managed to embed two identical `LC_RPATH` load
commands (by using `ld_classic`), I can link against this module with
“only” a warning:
{{{
mark at arm-and-hammer zsh% clang main.c -L. -llib && echo succeeded || echo
failed
ld: warning: duplicate LC_RPATH are deprecated ('/tmp/lib')
succeeded
}}}
But repeat with MacPorts gcc:
{{{
mark at arm-and-hammer zsh% gcc-mp-13 -dynamiclib lib.c -o liblib.dylib
-Wl,-rpath,/tmp/lib -Wl,-rpath,/tmp/lib -Wl,-ld_classic -v 2>&1 | grep
collect2 | tr ' ' '\n' | grep -A 3 platform_version
-platform_version
macos
14.0.0
0.0
mark at arm-and-hammer zsh% otool -l liblib.dylib | grep -B 1 -A 7
LC_BUILD_VERSION
Load command 8
cmd LC_BUILD_VERSION
cmdsize 32
platform 1
minos 14.0
sdk n/a
ntools 1
tool 3
version 907.0
mark at arm-and-hammer zsh% clang main.c -L. -llib && echo succeeded || echo
failed
ld: duplicate LC_RPATH '/tmp/lib' in '/private/tmp/liblib.dylib'
clang: error: linker command failed with exit code 1 (use -v to see
invocation)
failed
}}}
`man ld` documents `-platform_version`:
{{{
-platform_version platform min_version sdk_version
This is set to indicate the platform, oldest supported
version of
that platform that output is to be used on, and the SDK that
the
output was built against. platform is a numeric value as
defined
in <mach-o/loader.h>, or it may be one of the following
strings:
• macos
[…]
Specifying a newer min or SDK version enables the linker to
assume features of that OS or SDK in the output file. The
format
of min_version and sdk_version is a version number such as
10.13
or 10.14
}}}
gcc passing `-platform_version macos 14.0.0 0.0` is causing the SDK to not
be properly recorded (the “n/a” in `otool -l` output), which is
inadvertently triggering no-duplicate-LC_RPATH enforcement that we
wouldn’t ordinarily see until next year.
I can show that simply adding the proper SDK version to the module allows
it to be linked against even though it contains multiple identical
`LC_RPATH` load commands:
{{{
mark at arm-and-hammer zsh% gcc-mp-13 -dynamiclib lib.c -o liblib.dylib
-Wl,-rpath,/tmp/lib -Wl,-rpath,/tmp/lib -Wl,-ld_classic
-Wl,-platform_version,macos,14.0.0,14.0
ld: warning: duplicate -rpath '/tmp/lib' ignored
ld: warning: passed two min versions (14.0, 14.0.0) for platform macOS.
Using 14.0.0.
mark at arm-and-hammer zsh% otool -l liblib.dylib | grep -B 1 -A 7
LC_BUILD_VERSION
Load command 8
cmd LC_BUILD_VERSION
cmdsize 32
platform 1
minos 14.0
sdk 14.0
ntools 1
tool 3
version 907.0
mark at arm-and-hammer zsh% clang main.c -L. -llib && echo succeeded || echo
failed
ld: warning: duplicate LC_RPATH are deprecated ('/tmp/lib')
succeeded
}}}
So where does the `0.0` that gcc uses come from? Well, unfortunately, it’s
a hard-code at [https://github.com/gcc-
mirror/gcc/blob/3b18fd28c83ac90bf408389c003ed25d93438210/gcc/config/darwin.h#L286
gcc’s gcc/config/darwin.h]. This is new in [https://github.com/gcc-
mirror/gcc/commit/032b5da1fc781bd3c23d9caa72fb09439e7f6f3a gcc
032b5da1fc78] (2023-07-13). The older method, using the older
`-macosx_version_min` (or newer synonym for it, `-macos_version_min`),
doesn’t record this invalid SDK value.
{{{
mark at arm-and-hammer zsh% ld-classic -syslibroot
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.sdk/
-dylib -platform_version macos 14.0.0 0.0 -o liblib.dylib lib.o -lSystem
-rpath /tmp/lib -rpath /tmp/lib
ld: warning: duplicate -rpath '/tmp/lib' ignored
mark at arm-and-hammer zsh% otool -l liblib.dylib | grep -B 1 -A 7
LC_BUILD_VERSION
Load command 8
cmd LC_BUILD_VERSION
cmdsize 32
platform 1
minos 14.0
sdk n/a
ntools 1
tool 3
version 907.0
mark at arm-and-hammer zsh% ld-classic -syslibroot
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.sdk/
-dylib -macos_version_min 14.0.0 -o liblib.dylib lib.o -lSystem -rpath
/tmp/lib -rpath /tmp/lib
ld: warning: duplicate -rpath '/tmp/lib' ignored
mark at arm-and-hammer zsh% otool -l liblib.dylib | grep -B 1 -A 7
LC_BUILD_VERSION
Load command 8
cmd LC_BUILD_VERSION
cmdsize 32
platform 1
minos 14.0
sdk 14.0
ntools 1
tool 3
version 907.0
}}}
So the easy thing to do here would be to revert gcc 032b5da1fc78 (using
`-macos_version_min` instead of `-macosx_verison_min` as the latter will
now issue deprecation warnings) which infers the SDK version based on
`-syslibroot` and records it properly. Longer-term, gcc should calculate
the proper SDK version and pass it to `-platform_version` itself.
Under no circumstance should gcc be passing an SDK value of `0.0`. Apple
has taken to enabling and disabling features based on the SDK that a
module was linked against, and by providing an invalid version in this
spot, gcc-linked modules are very likely to continue to experience these
sorts of unexplained disturbances.
--
Ticket URL: <https://trac.macports.org/ticket/68329#comment:22>
MacPorts <https://www.macports.org/>
Ports system for macOS
More information about the macports-tickets
mailing list