standard way to require c++11?

[Mihai Moldovan]

[This message is merging Mojca's and Jeremy's answers]


On 14.04.2015 09:11 AM, Mojca Miklavec wrote:
> On Tue, Apr 14, 2015 at 8:18 AM, Mihai Moldovan wrote:
>> On 14.04.2015 07:44 AM, Mojca Miklavec wrote:
>>> On Tue, Apr 14, 2015 at 3:54 AM, Mihai Moldovan wrote: Could 
>>> something like that be added to the compiler_blacklist
>>> PortGroup? I believe that pure C++11 projects need consistent
>>> handling and it would be very handy to allow a keyword like
>>> "compiler.c++11" or "compiler.<something> c++11" to replace all
>>> of the logic above inside a port.
>> 
>> Yes, I should make that a PortGroup, iff it turns out to be good 
>> enough to be one.
> 
> In my opinion this could/should be part of compiler_blacklist. No 
> real need for a separate PortGroup.

Whatever makes most sense, really. The implementation is the same, only
details differ (using a variable like "use_c++11" in compiler_blacklist
vs. just including "PortGroup cxx11 1.0".)


>>> I don't fully understand that part. What if someone sets libc++ 
>>> globally? And: won't that pull in libstdc++ from gcc?
>> 
>> .>= 10.9: libstdc++? error
>> .>= 10.9: libc++? continue
>>
>> .<= 10.8: libstdc++? continue
>> .<= 10.8: libc++? error
>> 
>> This by itself is not pulling in anything yet. What I wanted to 
>> express is that I assume using libstdc++ on >= 10.9 an error and 
>> using libc++ on <= 10.8 an error.
> 
> While I do understand "10.9: libstdc++? error", I disagree with "<= 
> 10.8: libc++? error" for at least two reasons: - A bunch of my ports 
> explicitly switch to libc++ (just because they need C++11) - Users 
> can set to use libc++ globally

On 14.04.2015 10:02 AM, Jeremy Huddleston Sequoia wrote:
> Yes, I actually have libc++ on all of my SL+ machines except a couple
> VMs I keep around for testing the default configuration.  IMO, it
> seems far easier to live on libc++ on SL than libstdc++ because so 
> many ports these days require it.

What are they actually requiring? libc++ or C++11? I don't know any port
that fails to build with libstdc++ but is happy with libc++ offhand.


>> If someone on <= 10.8 set libc++ globally... tough luck. He'll get 
>> an error when trying to install the port.
> 
> Which is stupid. Users would set libc++ globally precisely for the 
> reason you are trying to address: to avoid issues with the lack of 
> C++11 support in libstdc++. And then you are ruling out exactly the 
> users who care about C++11 most.

It's not stupid if you consider my point that libstdc++ is well able to
compile C++11 code if using the FSF GCC in MacPorts. That will leave the
problem if choking on Apple's headers that make use of their "Blocks"
extension, though. (Which is practically all Cocoa/Carbon stuff, I
guess?) For that reason (and "+universal"), clang + libc++ are indeed
the superior choice. I agree. There are problems with that approach as
well, though. More to that later in this message...


>> But given that, in the case of hardcoding configure.cxx_stdlib to 
>> another C++ runtime than the autodetected/default value, the 
>> selected C++ runtime not matching the system C++ runtime and this 
>> potentially leading to a variety of subtle to not so subtle bugs, 
>> erroring out looked like the best thing to do.

On 14.04.2015 10:02 AM, Jeremy Huddleston Sequoia wrote:
> What do you mean by "the system runtime"?  There is no "system 
> runtime".  There are two runtimes available on the system, an ancient
> libstdc++ kept around for binary compatibility, and a more modern
> libc++.

With "system runtime", I mean the runtime that gets linked by Apple's
libraries. For instance:

/usr/lib/libobjc.dylib:
        /usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current
version 228.0.0)
        /usr/lib/libauto.dylib (compatibility version 1.0.0, current
version 1.0.0)
        /usr/lib/libc++abi.dylib (compatibility version 1.0.0, current
version 48.0.0)
        /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current
version 120.0.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
version 1197.1.1)

Anything that links against -lobjc will also pull-in the "system
runtime" libc++. This is a problem, because we cannot provide an libobjc
alternative in MacPorts, so you may end up with two different versions
of libc++ instead. Is that any better?


> Jeremy did extensive testing of that configuration and I'm running a
>  copy of MacPorts with those changes as well. I hardly experienced
> any problems (other than initial build failures that had to be
> polished out every now and then). I believe that this should become
> the default setting in not too distant future if we want to keep
> MacPorts working on older systems.

On 14.04.2015 10:02 AM, Jeremy Huddleston Sequoia wrote:
> To be fair, I did most of the work in MacPorts only after extensive 
> testing of the configuration and transition by Apple in Mountain Lion
> (or maybe it was Mavericks).  On newer OSX versions, we have an added
> benefit of having libstdc++ on top of libc++abi.  We don't have that
> benefit on SL, but other than that, installing the libcxx port on SL
> gets you a pretty modern and well tested C++ runtime that is forwards
> compatible with the system libc++ on Lion and later.

What does mean "forward compatible" here? Does it mean that programs
linked against libc++ will continue to work if you upgrade libc++ (which
is AFAIK also the case with libstdc++), or does it mean that you can mix
multiple libc++ versions in the same process? If I read "port info
libcxx" correctly, the latter case doesn't work even with libc++.


>> No, libstdc++ is explicitly designed to allow multiple, 
>> incompatible libstdc++ version being in use at the same time.

On 14.04.2015 10:02 AM, Jeremy Huddleston Sequoia wrote:
> I think you misunderstand.  This is not entirely true.

> Why do people report problems then?

> Because the above is completely entirely true.  You can have multiple
> versions of C++ runtimes in the same process (even libc++ and
> libstdc++).  The issue arrises when you try to pass objects between
> them.  For example, assume libA links libstdc++ and libB links
> another copy of libstdc++ as well as libA.  If libA exports C++ API
> that libB consumes, then it will be interacting with them through the
> wrong runtime.

Re-iterating: is this not the case for multiple copies of libc++?


>>>> In my experiments with a 10.6 VM, mp-clang-3.4 -std=c++11 
>>>> -stdlib=libstdc++ chokes on #include <type_traits>.
>>> 
>>> I would suspect some misconfiguration. I successfully compiled 
>>> many C++11 projects with mp-clang-3.4 (and libc++ of course).
>> 
>> On OS X lower than 10.9? I can take out my VM for a test spin 
>> "tonight" and see if I can reproduce the issue (as a "minimal 
>> testcase" without MacPorts.)
> 
> Maybe I didn't express it properly (or maybe I didn't read your 
> statement carefully enough in the first place). Yes, you can compile
>  "#include <type_traits>" on < 10.9 with clang++-mp-3.4, but not with
>  the system libstdc++ and I'm not sure how you would compile against
>  libstdc++ as shipped by libgcc/gcc-4.9.

Ok. We have a consensus here. clang and libstdc++ on < 10.9 doesn't work.
It potentially could, but it's: complicated. You can *link* against
libstdc++ as shipped by libgcc/gcc-4.9 just fine, but you
cannot easily *compile* against it. As far as I know this is the case
because clang uses a hardcoded list of system STL header locations,
which will make it use system-libstdc++ headers, but not
MacPorts-libstdc++ headers. This problem could probably be solved by
patching clang (but it's not elegant or very sane.)


> But the idea is that you *can* compile that if you use libc++. My 
> main point was that there should be no need to blacklist clang.

I believe you. But I didn't *want* to use libc++ on < 10.9, because
that's not the "system runtime". Maybe that's a bad idea, see my
comments and questions above.


>>>> Hence, I blacklist all clang versions on 10.8-.
>>>> 
>>>> This will leave us with quite a mess. Now all compilers are 
>>>> blacklisted on older systems. Great.
>>>> 
>>>> Not a big deal, though. We can set compiler.fallback to 
>>>> macports-gcc-4.9 and port will use this specific compiler, 
>>>> given all others are blacklisted.
>>>> 
>>>> With that, compiling C++11 code on 10.8- works great (I've 
>>>> tested it in a 10.6 VM) -- and the binaries even run 
>>>> correctly.
>>> 
>>> I still believe that if the code was moved to a PortGroup, one 
>>> would have to allow building with clang against libc++.
>> 
>> Maybe... I mean, coupled with Jeremy's efforts getting libc++ to 
>> work on 10.5 and 10.6.
> 
> It already works (perfectly) on 10.6. His recent efforts were 
> addressing 10.5 and I'm not sure if the work is finished by now, but
>  yes, 10.5 might be doable.
> 
>> That procedure isn't exactly automatic, though.
> 
> It is on 10.6.

On 14.04.2015 10:02 AM, Jeremy Huddleston Sequoia wrote:
> 'sudo port -v -s install libcxx' is fairly automatic ;)
> 
> If you want to configure MacPorts to use the compiler by default, 
> that isn't automatic but is fairly easy with steps laid out at: 
> http://trac.macports.org/wiki/LibcxxOnOlderSystems

It is not. A user must edit macports.conf multiple time while
bootstrapping the MacPorts system for libc++, (re-) build clang/llvm
ports multiple times to make them use the correct stdlib, will not be
able to use binary archives etc.

About the same thing will need to be done on 10.7 and 10.8, too (well,
setting configure.cxxstdlib and rebuilding everything. The bootstrapping
part can be omitted there.)

The needed compile time alone is inconvenient for users. And users will
need to actually *know* about having to change options and maybe even
re-compile all ports/bootstrap somehow. My goal was to provide C++11
support transparently, without needing to change the macports.conf or
rebuilding the whole system.

Maybe that's just not possible.


>> And even then -- won't you be mixing libc++ and libstdc++ when 
>> linking anything system-related that happens to be written in C++ 
>> and links libstdc++?
> 
> In your case one would be mixing libstdc++ shipped by apple and 
> libstdc++ as shipped by gcc. From what I understand from fellow 
> macports developers/maintainers mixing different libstdc++ is worse.

On 14.04.2015 10:02 AM, Jeremy Huddleston Sequoia wrote:
> It's worse only in that it's less obvious.  If you screw up 
> accidentally mixing libc++ and libstdc++ binaries, you'll get a 
> linker error.  If you screw up mixing two different libstdc++ based 
> binaries, you'll probably link ok since the symbol mangling will be 
> the same, and you'll only notice odd errors at runtime.

Will linking to both libc++ and libstdc++ *always* lead to a linker error?

What if, say, you have a binary that links -lc++ -lc++abi -lobjc on 10.8
and below? Will this also result in a linker error?


>> My idea is: stay with libstdc++ on "legacy platforms" and avoid the
>> problems created by mixing C++ runtimes, messing with user systems
>> in a non-interactive way and creating a new buildslave 
>> infrastructure. Why would you not use libstdc++, if it works *and* 
>> can be used -- together with GCC -- to compile C++11 code? (Yes, 
>> I'm avoiding your "PPC problem". I'm seriously sorry GCC is not 
>> building there, but there's not much I can do about that.)

On 14.04.2015 10:02 AM, Jeremy Huddleston Sequoia wrote:
> It can't.  You'd be using *differnet* libstdc++ runtimes.

It can for my ports. In hindsight, that's probably only working by
chance, because the ports in question do not use anything Apple-specific
and thus do not pull in the system libstdc++.

So it turns out it's not a general solution.


> I wasn't talking about PPC here at all (and I believe that gcc 
> actually compiled successfully in the meantime***). We are still 
> talking about 10.6-10.8 with x86_64.

I just remembered GCC wasn't working for you on PPC at all. That would
have been another "corner case" which my solution didn't address.


Phew. That was quite a stretch, sorry.

I'll try to summarize the situation as far as I've understood. Please
correct me if needed.

- There is no way to transparently provide C++11 support on OS X < 10.9.
Users will need to switch their MacPorts installation to libc++ by
rebuilding all ports.
- Even doing that may lead to situations where C++11 is used but also
Apple's libraries or frameworks linked at the same time. If libstdc++ is
being (implicitly) linked by that, linking fails. These ports will be
uninstallable on a pre-10.9-libc++ system.
- Following my original idea of using libstdc++ on pre-10.9 systems will
potentially link different versions of libstdc++ at the same time and
not lead to linker errors, but most likely crashes or other undesired
behavior at run time.
- OS X >= 10.9 is fine regarding C++11.
- C++14 support comes "free of charge" on 10.9 and 10.10 with XCode 6
and if using MacPorts' libc++ and clang-3.4 or higher on 10.8 and below.

That doesn't sound too nice.


I have to add yet another thing, though. We'll be running into a similar
issue on 10.9 and 10.10 (and (10.10)++ and ...) once C++1z (the next
standard) is released and gains widespread usage. How can we provide
C++1z support, if the system libc++ does not? Install our own version of
libc++ and rebuild MacPorts against that?
At some point, this will *also* lead to using different libc++ runtimes.



Mihai

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 884 bytes
Desc: OpenPGP digital signature
URL: <https://lists.macosforge.org/pipermail/macports-dev/attachments/20150415/57bbeb65/attachment.sig>