Host Versus MacPorts lib[std]c++ [was: Re: Fortran recipe]

[Jeremy Huddleston Sequoia]

On Aug 28, 2013, at 9:58, Michael Dickens <michaelld at macports.org> wrote:

> The latest version of gnuradio (devel or next; the 3.7.1 release is
> imminent) contains a fix which allows those ports to compile using
> MacPorts GCC (C / C++, not Fortran). When I use
> configure.compiler=macports-gcc-4.X, the resulting gnuradio binaries
> (libraries and executables) link with MacPorts' libstdc++ while, by
> default (without setting the compiler suite globally), all of gnuradio's
> dependencies use Apple's clang or llvm and hence link with the host
> libstdc++. In this setup, most of gnuradio's "make test" fails (for more
> info see, e.g., <
> http://stackoverflow.com/questions/4697859/mac-os-x-and-static-boost-libs-stdstring-fail
>> ).  If I rebuild, as it turns out, just boost and cppunit using the
> same MacPorts GCC (or, it probably could be different since they now use
> the common libgcc/libstdc++), "make test" now works.  

That makes sense.  Assuming those are the complete set of C++ API being used here, that would get them to all use the same C++ runtime.

> Interestingly: If
> I revert back to the original boost and cppunit (using Apple's
> libstdc++), and use "install_name_tool" to change the version of
> libstdc++ in the already-installed libboost* and libcppunit* to
> MacPorts' libstdc++, then everything works again -- this is probably not
> a generic solution to the issue, but it is curious that it works.

That makes sense as well, for the same reason.  The issue is the runtime being used.  MP's libstdc++ is binary compatible with the host libstdc++ runtime, so you just forced everything in your process to use MP's rather than the host's.  Going the other way (having your app use the host's) might work if it doesn't use any newer features from the newer libstdc++, but there's certainly no guarantee.

> Compiling gnuradio with GCC48 results in better executing code than with
> GCC4X (X <= 4; using GCC48 is best right now), including Apple's GCC
> compilers.  gnuradio does not currently make use of clang for optimizing
> assembly code, so all clang compilers are blacklisted.  Hence, I would
> love to be able to use GCC48 as the default compiler for gnuradio.  But, 
> 
> I can create tests (e.g., in gnuradio's CMake script, or even in the
> gnuradio Portfile) to check for the "multi-libstdc++" condition and
> provide feedback to the user ("You need to rebuild boost and cppunit
> ..."),

I'd rather not do that.  I'd prefer to get the best experience for users without having a port suggest they go and do something that's not really supported.

My goal here is to try and get all ports off of libsdtc++ as provided by libgcc.  It should be there for users of the gcc compilers, but ports should cooperate with eachother well, and to do that, we need a single C++ runtime.

> but is there a way for port to do the work automatically -- sort
> of like "enforce variants" except more along the lines of "enforce
> compiler suite"?

Strictly speaking, we just need to ensure that all interoperating C++ ports use the same runtime.  In practice, almost all C++ applications either use boost or use another library that uses boost.  Since boost and all its recursive dependents need to use the same C++ runtime, that essentially covers 99.9% of C++ ports.  It's easier to just say "all ports" need to use the same C++ runtime and special case where that isn't an easy option (mkvtoolnix is such an example).

>  Somehow, I don't think so just yet.  I think this is
> one reason why all of the various +gcc4X options were added to various
> ports (e.g., fftw-3, atlas, lapack, octave), along with using the for
> Fortran.

Yes, I understand that some of these cases are for "newer faster better", and that made sense when Apple was stuck on GPL-2 gcc-4.2 and llvm was still maturing, but on Lion+, we have a very mature version of clang provided by Xcode 4.6, and Tiger through Snow Leopard intel users can use macports-clang-3.3 if they want a mature compiler that is compatible with the host.

If you really want a "newer faster better" gcc option, then you can certainly use gcc for C code (following the same recipe as the fortran recipe, but setting configure.cc), but NOT for C++ code.  This is because of link time tricks to use the host's libSystem for the libgcc_s runtime and only use the runtime provided by the libgcc port for functionality not in the host's libSystem (eg: emutls).

That being said, if you do go down this road, please place a comment in the Portfile indicating why clang is not a viable option and link to a bug report at llvm.org (or a radar number for a report filed at bugreport.apple.com).  It would be nice to include performance metrics to back up such a decision as well (in the Portfile if filed via radar, or in the bugzilla entry is fine if filed at llvm.org).  clang and llvm are very mature at this point.

Yet another reason to not use the gcc ports is the assembler.  They are using the legacy asembler from cctools which does not support AVX.  llvm-based compilers (including dragonegg if using the integrated-as.specs) use the newer assembler.  One option is to have the gccXX ports use 'clang -c -x assembler' instead of as, but that requires someone who isn't afraid of GPL3 cooties to fix (or if nobody does, I'll have to write a bash script wrapper which is probably sub-ideal).

See https://trac.macports.org/ticket/37846#comment:6

--Jeremy


> 
> I value your thoughts. - MLD
> 
> On Sun, Aug 25, 2013, at 09:07 PM, Jeremy Huddleston Sequoia wrote:
>> On Aug 25, 2013, at 13:53, Chris Jones <jonesc at hep.phy.cam.ac.uk> wrote:
>>> On 25 Aug 2013, at 7:22pm, Christoph Deil <Deil.Christoph at gmail.com> wrote:
>>>> On Aug 25, 2013, at 10:44 AM, Mojca Miklavec <mojca at macports.org> wrote:
>>>>> On Sun, Aug 25, 2013 at 5:16 PM, Jeremy Huddleston Sequoia wrote:
>>>>>> Seeing as how many developers don't understand the problems surrounding mixing multiple versions of the C++ runtime in a single process, I doubt that users understand those problems.  I think the root port needs to be simplified significantly.
>>>>>> 
>>>>>> Does root use any C++ APIs exposed by the host or other ports?  Does root expose any C++ APIs?
>>>>> 
>>>>> I don't understand much about those problems either.
>>>> 
>>>> I also don't understand the issues, or how to debug / check them.
>>>> 
>>>> I am using a non-public gamma-ray astronomy data analysis package built on top of ROOT.
>>>> After a lot of trial and error I did get it to compile with Macports gcc, I never managed to compile it with Apple gcc or (Apple or Macports) clang.
>>>> So at least I can write code and check that it compiles on my Mac  … actually running the software sometimes works and sometimes mysteriously crashes.
>>>> 
>>>> Probably this is a very specialised use case, if the Macports GCC variants are removed from the ROOT port I'll just build ROOT from source myself, no problem if that is the right thing to do to avoid "C++ runtime mix errors" (whatever that means).
>>> 
>>> The issue is in essence the gcc compilers use one c++ runtime library (libstdc++), the other compiles, clang etc. use another (libc++). 
>> 
>> That's not exactly correct.  There are three runtimes we are concerned
>> with: host libstdc++, host libc++, and MacPorts libstdc++ (libgcc port)
>> 
>> Here's the breakdown:
>> 
>> gcc-*            : host libstdc++
>> *llvm-gcc-4.2    : host libstdc++
>> apple-gcc-*      : host libstdc++
>> macports-gcc-*   : MP libstdc++
>> dragonegg-*      : MP libstdc++
>> clang < 163      : host libstdc++
>> clang >= 163     : host libstdc++ or host libc++
>> macports-clang-* : host libstdc++ or host libc++
>> 
>>> Mixing these is a bad idea, so if you where to compile root using gcc, you should in theory compile *all* ports that that uses with it as well. This is not done. Maybe the crashes you refer to are to do with this …
>>> 
>>> Removing the gcc variants is thus a good idea to avoid this. I have a version under test which does this, uses the fortran recipe JH posted earlier in this thread, and fixes the cocoa variant by selecting the compiler in a better way. 
>> 
>> This solves the primary issue of mixing the MP runtime and the host
>> runtime.  We will still need to deal with host libc++ vs host libstdc++,
>> but those problems are much more obvious (build failure).
>> 
>>> Note that for those users that *really* want to use gcc, they will always be able to do something like
>>> 
>>>> sudo port install root configure.compiler=macports-gcc-4.8
>>> 
>>> So all is not lost, but by doing so you are accepting whatever the consequences might be.
>> 
>> And such expert users might want to set configure.compiler globally after
>> they first bootstrap it.
> _______________________________________________
> macports-dev mailing list
> macports-dev at lists.macosforge.org
> https://lists.macosforge.org/mailman/listinfo/macports-dev

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4145 bytes
Desc: not available
URL: <http://lists.macosforge.org/pipermail/macports-dev/attachments/20130828/3baee569/attachment-0001.p7s>