standard way to require c++11?

[Jeremy Huddleston Sequoia]

> On Apr 20, 2015, at 23:51, Mihai Moldovan <ionic at macports.org> wrote:
> 
>>> I don't know any port that fails to build with libstdc++ but is
>>> happy with libc++ offhand.
>> llvm-3.7 and webkit-gtk are two such ports that I can point to off
>> the top of my head.
>> 
>> By "libstdc++" I am of course referring to the *system* libstdc++. I
>> apologize if that was not clear.
> 
> I was talking about explicitly needing libc++, not C++11 support. I assume webkit-gtk and llvm-3.7 only need libc++ by extension because they are using C++11 features. Naturally, system libstdc++ won't cut it for these, but there's no immediate reason why they wouldn't in theory compile with a more recent libstdc++.

Yes, but that's not a supported configuration, and I don't plan on putting any effort into trying to support that.

Note that pretty much *ALL* of MacPorts C++ ports need to be compiled using the same C++ runtime.


>> Yes, but then you'd need to use the FSF GCC compilers from MacPorts
>> to build all of your C++ ports in order for them to interoperate, and
>> those ports would all have a new dependency on the libgcc port.
>> 
>> Furthermore, nobody has bothered to update the gcc ports to use the
>> Apple gcc driver driver, so the use of multiple -arch options pretty
>> much blocks +universal which is a big issue for ports like wine that
>> have heavy dependencies and are i386.
> 
> Yes, that would be adding a new dependency on libgcc and FSF GCC for all C++ ports. But so does using libc++ on 10.6.

But the libc++ runtime that we install on 10.6 is signifigantly different in installation method than the libstdc++ runtime that is installed by libgcc.  It is designed to be compatible with future OS versions.  The port just installs a root, and activation installs the root.

> Implicitly at least. Fortunately that doesn't add dependencies on 10.7 and higher.
> 
> For specific ports, yes. In general, muniversal might take of that. But it doesn't make a lot of sense to change our ports base to muniversal, if we can salvage the current state by using clang. Which implies libc++. Gotcha.

What does muniversal have to do with anything?


> On 15.04.2015 10:45 AM, Mojca Miklavec wrote:
>> On Wed, Apr 15, 2015 at 9:42 AM, Mihai Moldovan wrote:
>>> On 14.04.2015 10:02 AM, Jeremy Huddleston Sequoia wrote:
>>>> Because the above is completely entirely true.  You can have
>>>> multiple versions of C++ runtimes in the same process (even
>>>> libc++ and libstdc++).  The issue arrises when you try to pass
>>>> objects between them.  For example, assume libA links libstdc++
>>>> and libB links another copy of libstdc++ as well as libA.  If
>>>> libA exports C++ API that libB consumes, then it will be
>>>> interacting with them through the wrong runtime.
>>> 
>>> Re-iterating: is this not the case for multiple copies of libc++?
>> 
>> I don't know enough to answer, but at the moment we don't have
>> multiple copies of libc++ anywhere. (Yet. I'm not sure what C++1z
>> will bring.)
> 
> On 15.04.2015 12:55 PM, Jeremy Huddleston Sequoia wrote:
>> It is certainly be the case for multiple copies of the libc++
>> runtime.  But we're not dealing with multiple copies of libc++.
>> However, you are considering multiple copies of libstdc++.
> 
>> How would you end up with two different versions of libc++?
> 
> Yes, I was explicitly thinking of C++1z support, which would (as far as I can tell) potentially lead to multiple copies of libc++.

Why would that potentially lead to multiple copies of libc++?

> Ok, so that's a problem. Not an immediate one, but maybe coming up some time later. I realize this is slightly off-topic for *this thread*, but I'd like to understand when and why this is a problem in general (multiple copies of some C++ runtime.)
> I haven't yet understood it completely. Maybe I'm too stupid to do so in general, but I'm still very fuzzy on that aspect.

Each runtime manages its internal data structures independently of the other.  If you allocate an object in one runtime and release it in another, you may mess up accounting or worse because the second runtime does not own the memory of the object you are trying to free.

> On 15.04.2015 12:55 PM, Jeremy Huddleston Sequoia wrote:
>> Do you have evidence of any such issues? If so, that is a bug that
>> should be filed. If you notice, our headers are heavily annotated
>> with '#ifdef __BLOCKS__' around such cases.
> 
> No. No hard evidence as such. I can only remember seeing this happen before with... the wine ports IIRC? That's why we blacklisted FSF GCC on the wine* ports quite a while ago.

If so, it's a bug and deserves a radar.


> On 15.04.2015 12:55 PM, Jeremy Huddleston Sequoia wrote:
>> What the system libraries link against is irrelevant.  It is the
>> interfaces provided by the system libraries that matters.
> 
> So... uhm, to know if things could potentially go awry, I'd need to check the API if functions take C++ objects directly and guess what these functions might do with the objects?

You could just dump global symbols and see if any of them are mangled C++ symbols.  If they are, then they are exposing C++ APIs.  If not, then you're safe and the C++ usage is all internal and not across an API boundary.

For example, LTO is a perfect example.  Run 'nm /opt/local/libexec/llvm-3.7/lib/libLTO.dylib' and look at the output.  You will notice that it makes heavy usage of C++ (by using llvm), but it provides a pure C API.  Thus, clients of libLTO don't have to be using the same C++ runtime as libLTO.

> On 15.04.2015 12:55 PM, Jeremy Huddleston Sequoia wrote:
>> Yes, it most certainly is a system runtime.  We've been shipping
>> libc++ with OS X since Lion (10.7).  It was just made the default
>> C++ runtime with Mavericks (10.9).
> 
> I've been too unspecific. When I say "system runtime", I mean the runtime that other system libraries providing a C++ ABI are linking against.

Are there any?  I can't think of any off the top of my head...


>>> It is not. A user must edit macports.conf multiple time while
>>> bootstrapping the MacPorts system for libc++, (re-) build
>>> clang/llvm ports multiple times to make them use the correct
>>> stdlib, will not be able to use binary archives etc.
>> 
>> This is only true for 10.5. It works automatically on 10.6 and later
>> as soon as the user changes three lines in macports.conf (before
>> installing anything). The complicated bootstraping procedure is only
>> needed for 10.5. 10.6 and later is a piece of cake.
> 
> Yes, the procedure isn't too difficult to carry out, but we will need to make users aware that they need to change their MacPorts runtime and rebuild all ports. I will add references to http://trac.macports.org/wiki/LibcxxOnOlderSystems and https://trac.macports.org/wiki/Migration to the error message of my two ports. A C++11 PortGroup (or whatever) shall also include them to make users aware that they need to switch the MacPorts C++ runtime and rebuild their ports.
> 
> My general criticism wasn't that the instructions are too difficult to follow, though.
> What I mean is that *new* installations of MacPorts on 10.6 to 10.8 should be very rare by now, if even existent. Instead, users will have likely been using MacPorts for quite a while now and accumulated quite an amount of installed ports. As such, a rebuild of all installed ports (well... actually only those that use C++, but we currently have no way of telling which port uses what exactly, so the easiest way is still rebuild *all* installed ports) is a time-consuming process. I tried avoiding this and failed. Sorry.
> 
> 
>> (It would also be nice if someone could provide "bootstrapped"
>> version of MacPorts for 10.5 with all the binaries that are needed
>> to continue from there.)
> 
> With clang etc. precompiled and linked against libc++? Mh, yeah, that's a nice idea. We currently have nothing like this, though. One would probably still need to edit the header file and exchange libgcc manually, though.
> 
> 
>>> The needed compile time alone is inconvenient for users. And users
>>> will need to actually *know* about having to change options and
>>> maybe even re-compile all ports/bootstrap somehow. My goal was to
>>> provide C++11 support transparently, without needing to change the
>>> macports.conf or rebuilding the whole system.
>> 
>> Of course we need to solve two different problems:
>> 
>> (a) Trying to patch the ports just enough to allow users to use C++11
>> software on "default" MacPorts installation with libstdc++. Taking
>> into account all the limitations.
> 
> I don't know. Jeremy is in favor of dropping this altogether. That's what my ports currently (after his requested changes) do. It looks like libstdc++ is dead and a C++11 PortGroup/WhatEver should just bail out for libstdc++.
> 
> 
>> (b) I've been suggesting for a long time already that we should set
>> up three new buildbots to build packages against libc++ on
>> 10.6-10.8. The lack of a buildbot is a showstopper for me, preventing
>> me from globally switching to libc++ (on an old OS). And as more and
>> more software is now shifting to C++11, the situation will just get
>> much worse with time. If we start supporting libc++ in such a way
>> that: - buildbots will build packages against libc++ - users will be
>> able to download a pre-configured MacPorts dmg (including the proper
>> configuration) then it will no longer be a problem for users to
>> switch to libc++. Of course that doesn't mean turning off the old
>> buildbots right away. We need sufficient testing first and we'll
>> certainly need quite a bit of adapting of the Portfiles before we
>> are able to provide a smooth user experience.
> 
> Yes, but also see Ryan... someone has to prepare base for this.
> 
> 
>> If you are trying to address (a), that's still a very valuable
>> effort. I was just pointing out that some of your assumptions (like
>> inability to use clang) were not entirely correct.
> 
> I give up on that. I accept libstdc++ as being unusable for ports requiring C++11 and instead will suggest switching to libc++ for any ports that break because they now rely on C++11.
> 
> 
>> A more problematic aspect of that approach is that users might
>> install certain libraries (like libpng, wxWidgets, ...) and then try
>> to build their own software (like gnuplot) against libraries in
>> MacPorts without switching to libc++. So users installing the libc++
>> flavour of MacPorts would have to know what they are doing.
> 
> Oh, they certainly will have to. But it looks like more and more software is switching to using C++11, so anyone who doesn't switch to libc++ is screwed.
> Sadly I don't see a way to properly document how users are supposed to build C++ software outside of MacPorts. Even worse -- if they don't specify -stdlib=libc++, their software may still build and run successfully... sometimes. There's nothing MacPorts can do about users choosing to install software manually (aside documentation *everyone* is guaranteed to see, but there's no such thing...)
> 
> 
> 
>>> [C++1z]
>> 
>> Don't open another can of worms ;) We have enough problems already.
>> It's probably good to keep those future problems in mind, but we
>> should solve the current problems first.
> 
> On 15.04.2015 12:55 PM, Jeremy Huddleston Sequoia wrote:
>> If the system libc++ runtime is not compatible with requirements of
>> future versions of C++, then I suspect expert users will do what they
>> already do now.  They can use the +replacemnt_libcxx variant of
>> libcxxabi and libcxx to build roots that they can use darwinup to
>> install, replacing the system runtimes.  It's certainly not ideal and
>> will also potentially cause codesigning issues, but that's one
>> option.  Another option would be as you suggest, moving to having
>> MacPorts provide its C++ runtime, but that would also mean MacPorts
>> always using its own toolchain instead of the one provided by Xcode.
>> I don't think we want to do that (other than in the fallback cases we
>> do now).
> 
> I'm just afraid that we will run into a similar situation again once there is C++1z. And an even worse one... as a C++1z-aware libc++ has to be "backported" somehow to older systems. What concerns me most is that this will not be as clean-cut as switching to a completely different C++ runtime (libstdc++ -> libc++.)

I use the +replacemnt_libcxx variant of the libcxx and libcxxabi ports to replace Lion's libc++ with that from llvm-3.6.  This is certainly not for everyone because it if risk-prone.  It also requires manually installing the roots after an updgrade as an extra level of precaution to make sure Joe User doesn't shoot his foot off on accident.  Thus, this isn't really something for Joe User (which is who we're trying to help out here).

> Replacing the libc++ as provided by Apple with a newer one sounds risky...

Yep.

> and we currently have no way of determining whether a new (replaced) or old one is being used.

Yep.  That's why we don't actually replace it but rather build a root that the user can install themselves.

> At least when comparing to the current C++11 situation with an C++11-incompatible system libstdc++, configure.cxx_stdlib is a very handy way to make sure C++11 is supported. => libc++? C++11 supported. libstdc++? unsupported [if not overriding/using a newer libstdc++, but we already determined that this is an unsupported scenario by MacPorts in this thread.]
> 
> N.B.: oh, "darwinup" looks interesting. I haven't come across that so far.
> 
> 
> Mihai
> 
> _______________________________________________
> macports-dev mailing list
> macports-dev at lists.macosforge.org
> https://lists.macosforge.org/mailman/listinfo/macports-dev