ld64 circular dependency and bootstrapping thoughts

[Ken Cunningham]

the ld64 update is looking quite ready to commit. Jeremy has approved it. I’ve been over it on all the systems I have, 10.4 up, and it looks OK to me everywhere I have used it. The ld64 test suite has been enabled, and anyone is welcome to test it out. 

Anyone with comments or concerns, please speak up now. 

In my review, it appears that all systems will have an ld64 at least as recent as what they have now after the upgrade.

Xcode 9 and 10  users will move (by default) to the new ld64-latest for new installs. Existing installs remain on the +ld64_xcode variant, as forcing variants is (AFAIK) not possible, otherwise we’d have done it these past years for ld64 already.

Xcode 11 will still default to ld64 +ld64_xcode as that ld64 will still be newer than the one in MacPorts.


In the process of going through this, I’ve learned that 10.4 PPC (and Tiger) can likely upgrade to at least ld64-127, which will be nice for them. Leopard Intel can probably go up to ld64-236 and likely ld64-274 is possible too. Maybe newer, who knows? 10.6.8 and up will be all on the newest ld64-450, which is perhaps something of a miracle, really.

the bootstrapping issue, where there is a manual step to get to ld64-latest on all systems < 10.12, will have to remain for now. Nobody will be set back.

I remain interested in an automatic bootstrapping method whereby a bootstrap version of a port can be replaced by a newer version once the bootstrap version is installed.


Best,

Ken



> On Feb 2, 2020, at 3:39 PM, Ken Cunningham <ken.cunningham.webuse at gmail.com> wrote:
> 
> Touch complicated, hope I explained it clearly. Probably best not to read at half-time today :> Watch J-Lo instead...
> 
> 
> After a number of years at @274,  ld6 @450 can now build with MacPorts. It matches Xcode 10.2.  ld64 @450 supports the latest TAPI-requiring SDKs, new features, etc, and all our Intel systems (10.6+ at least) would do well to move to that version.
> 
> The WIP is here, and seems pretty close to done, and does pretty well on the test suite, other than a bootstrapping issue I'm outlining with this question:
> 
> <https://github.com/macports/macports-ports/pull/6262>
> 
> I would appreciate any potential testers to exercise this port prior to us committing it, if there are any interested users out there.
> 
> 
> There has been a bootstrapping issue with ld64 for years, but until now it only involved 10.6.8, and so was not fixed to date. Now it will involve < 10.12, so we should look at it. The essence is:
> 
> 
> 1. ld64 > 274 requires libtapi. 
> 2. libtapi uses c++17 features, so only builds with Xcode clangs > 802 ( ie Xcode 9) and macports-clang > 5.0. This does not appear immediately simple to downgrade.
> 3. This means all systems < 10.12 will have to build libtapi with a macports-clang version. 
> 4. But macports-clang-* requires ld64 as a runtime dependency (it's hardcoded into the compiler to search for it in ${prefix}/bin/ld ). (NB this does not have to be ld64-latest.)
> 5. Ergo circular dependency.
> 
> 
> ld64 uses variants to select the last version your toolchain can build   (10.4=97, 10.5 and 10.6=127, 10.7 to 10.11 will be =274 due to libtapi, 10.12 and up=450). -- and the "ld64-latest" version is the one you get with no variants enabled. This works, but doesn't lend itself easily to an automated bootstrap, and requires a manual step.
> 
> ie to get the latest ld64 on a given system, you would need to do what 10.6.8 does now (NB adding in the new ld64_274 variant).
> 
> sudo port -v install ld64
> sudo port -v -n upgrade --enforce-variants ld64 -ld64_97 -ld64_127  -ld64_236 -ld64_274 -ld64_xcode
> 
> So, not bootstrap-friendly... if we want to automate getting all systems onto ld64-latest, we'll need to come up with something.
> 
> 
> Possible Solution:
> 
> We could make a new port, ld64-bootstrap, that symlinks in the last ld64 that the users Xcode toolchain can build.
> 
> Then change the runtime ld64 dependency in the clang ports (and the gcc ports as well, I guess) to a path dependency so that ld64-bootstrap would satisfy it:
> 
> depends_run-append path:bin/ld:ld64
> 
> the ld64 port would depend on ld64-bootstrap.
> 
> That's all pretty easy.
> 
> 
> The issue seems to be getting ld64-bootstrap to be preferentially installed if "${prefix}/bin/ld"  does not exist, and then getting ld64 to replace it if "${prefix}/bin/ld" does exist.
> 
> I am not sure how to go about doing that at present.
> 
> Ideas welcome.
> 
> Ken
> 
> 
> 
> PS - anticipating the libtapi issue, I have left it possible to build libtapi with gcc against ${prefix}/lib/libgcc which avoids this, but this has potential stdlib issues of course. And still doesn't fix 10.6.8.
> 
> 
> 
> 
> 
>