thread_local storage on 10.6.8 (and earlier) with clang-7.0

Ken Cunningham ken.cunningham.webuse at gmail.com
Fri Dec 7 20:14:21 UTC 2018


I suppose the symbols in libclang_rt.10.4.a are hidden because they are set to be hidden in the build:


cmake/Modules/CompilerRTDarwinUtils.cmake
266:  set(CFLAGS "-fPIC -O3 -fvisibility=hidden -DVISIBILITY_HIDDEN -Wall -fomit-frame-pointer")


But I don't at this point understand how that is supposed to be useful if the symbols are hidden. How can you use them?

Ken




On 2018-12-07, at 11:29 AM, Ken Cunningham wrote:

> 
> 
> As per the details in the previous post below that I sent a few weeks ago, I have clang/llvm thread_local support working nicely on 10.6.8 and earlier locally.  This is becoming a common issue now.
> 
> On 10.6.8 and earlier thread_local is implemented using existing llvm infrastructure that is in place for other OS systems that needed emulated TLS. I enabled it and forced it to be used for 10.6 and earlier on macOS. This is exactly the same emulated_tls that gcc uses.
> 
> It works perfectly well when linked against macports-libstdc++ (currently the default) as the supporting objects are built into libgcc. So from that point of view it could roll out right now, for stock users who are not using libc++.
> 
> That is really quite simple enough, and I should probably put that into a PR and get it on the PR list soon.
> 
> 
> 
> However I also have thread_local working when linking against libc++, and it would be my preference to fully fix this and share the solution to all before rolling it out.
> 
> The issue is the "___emutls_get_address" function, which is called from libc++abi.dylib, and is built into clang_rt:
> 
> $ nm /opt/local/libexec/llvm-5.0/lib/clang/5.0.1/lib/darwin/libclang_rt.10.4.a  | grep emu
> /opt/local/libexec/llvm-5.0/lib/clang/5.0.1/lib/darwin/libclang_rt.10.4.a(emutls.c.o):
> 0000000000000000 T ___emutls_get_address
> 00000000000001a0 t _emutls_init
> 0000000000000210 d _emutls_init_once.once
> 00000000000001c0 t _emutls_key_destructor
> 0000000000000220 d _emutls_mutex
> 00000000000002f8 b _emutls_num_object
> 0000000000000300 b _emutls_pthread_key
> 
> 
> 
> These objects appear to be included in the built executables,  but are apparently NOT visible to libc++abi.dylib at runtime (you can see the small "t" indicating they are local only, and generate a runtime error when you try to run it.
> 
> $ nm 4 | grep emu
> 0000000100007070 t ___emutls_get_address
> 0000000100009480 d ___emutls_v._ZGVZL18thread_with_accessPvE23counting_function_local
> 0000000100009400 d ___emutls_v._ZL15counting_static
> 0000000100009420 d ___emutls_v._ZN12_GLOBAL__N_128counting_anonymous_namespaceE
> 0000000100009460 d ___emutls_v._ZZL18thread_with_accessPvE23counting_function_local
> 00000001000094a0 d ___emutls_v.__tls_guard
> 0000000100009440 d ___emutls_v.counting_extern
> 0000000100007200 t _emutls_init
> 00000001000094c0 d _emutls_init_once.once
> 0000000100007220 t _emutls_key_destructor
> 00000001000094d0 d _emutls_mutex
> 0000000100009538 b _emutls_num_object
> 0000000100009540 b _emutls_pthread_key
> 
> 
> $ ./4
> info: testing pthread_create
> dyld: lazy symbol binding failed: Symbol not found: ___emutls_get_address
>  Referenced from: /usr/lib/libc++abi.dylib
>  Expected in: flat namespace
> 
> dyld: Symbol not found: ___emutls_get_address
>  Referenced from: /usr/lib/libc++abi.dylib
>  Expected in: flat namespace
> 
> Trace/BPT trap
> 
> 
> 
> 
> Adding -Wl,-flat_namespace does not make the symbols visible.
> 
> 
> If I try to fix it like this by forcing the symbol to be exported, I get a clue:
> 
> $ clang++ -std=c++11 -stdlib=libc++ -Wl,-exported_symbol,___emutls_get_address  -o 4 4.cpp
> ld: warning: cannot export hidden symbol ___emutls_get_address from /opt/local/libexec/llvm-5.0/lib/clang/5.0.1/lib/darwin/libclang_rt.osx.a(emutls.c.o)
> 
> 
> 
> 
> 
> Things DO work however if I build emutls.c directly into libc++abi.dylib, however, using a slightly modified version of libcxxabi that includes the emutls.c source file in the build:
> 
> $ nm libc++abi.kenspecial.20181206.dylib | grep emutls_
> 0000000000020054 T ___emutls_get_address
> 0000000000025458 s ___emutls_t._ZN10__cxxabiv112_GLOBAL__N_15dtorsE
> 000000000002a270 d ___emutls_v._ZN10__cxxabiv112_GLOBAL__N_111dtors_aliveE
> 000000000002a290 d ___emutls_v._ZN10__cxxabiv112_GLOBAL__N_15dtorsE
> 000000000002a2b0 d _emutls_get_index.once
> 00000000000201c8 t _emutls_init
> 00000000000201ea t _emutls_key_destructor
> 000000000002a2c0 d _emutls_mutex
> 000000000002a360 b _emutls_num_object
> 000000000002a368 b _emutls_pthread_key
> 
> 
> and with this modified libc++abi.dylib, all the symbols are found, things work correctly, and all the tests that should pass in the llvm "testit" suite do pass.
> 
> 
> SO here is my question:
> 
> is there some simple way I am missing to force the symbols from emutls.c to be made visible in the executables built by clang using libclang_rt.10.4.a that I am not seeing? If so, I would  need to change very little here, other than making those symbols visible, however that is done.
> 
> OR -- if there is no way -- I can roll emutls.c into libc++abi.dylib. It's a tiny bit tricky, and might well need to be manual steps. You have to build clang-5.0 first with thread support enabled as below, and then use that to build the new libc++abi.dylib. And once that new libc++abi.dylib is installed, everything works.
> 
> 
> Ken
> 
> 
> 
> 
> 
> 
> 
> On 2018-10-05, at 10:50 AM, Ken Cunningham wrote:
> 
>> With a couple of very minor modifications, recent versions of clang will support thread_local storage including support for complex destructors on 10.6.8 using the same emutls.c system that gcc uses to support it. I'll attach the (amazingly simple) patches below. It picks up the emutls.c support from libgcc and libstdc++:
>> 
>> $ nm  /opt/local/lib/libgcc/libstdc++.6.dylib | grep cxa_thread
>> 0000000000001e16 T ___cxa_thread_atexit
>> 
>> $ nm /opt/local/lib/libgcc/libgcc_s.1.dylib | grep emu
>> 000000000000bd56 T ___emutls_get_address
>> 000000000000bec7 T ___emutls_register_common
>> 
>> 
>> This works when we use the MacPorts alternate c++11 support Marcus came up with, using
>> 
>> -stdlib=macports-libstdc++
>> 
>> and it seems to me to pass all the tests of how thread_local should work that I can throw at it. So this seems very usable to me at present.
>> 
>> 
>> 
>> 
>> However, when using libc++ (my preferred setup) I'm still having a couple of issues getting cxa_thread_atexit and emutls_get_address to build into libc++abi and show up at runtime.
>> 
>> Using RJVB's libcxx Port, I have built libc++abi more in the LINUX way, with cxa_thread_atexit.cpp included (that was fairly easy), but it doesn't find the symbols in 
>> 
>> /opt/local/libexec/llvm-7.0/lib/clang/7.0.0/lib/darwin/libclang_rt.osx.a
>> 
>> that it needs to find and dies at runtime. I think it's possibly a visibility thing.
>> 
>> If there is anyone out there with any skills and interest in working on this please speak up and I'll bring you fully up to speed offline regarding my libc++/libc++abi build efforts
>> 
>> Best,
>> 
>> Ken
>> 
>> 
>> Patches for llvm/clang-7.0 to enable thread_local support using -stdlib=macports-libstdc++:
>> 
>> ==========================
>> --- a/include/llvm/ADT/Triple.h.orig	2018-10-02 17:38:10.000000000 -0700
>> +++ b/include/llvm/ADT/Triple.h	2018-10-02 17:38:58.000000000 -0700
>> @@ -682,7 +682,7 @@
>> 
>>  /// Tests whether the target uses emulated TLS as default.
>>  bool hasDefaultEmulatedTLS() const {
>> -    return isAndroid() || isOSOpenBSD() || isWindowsCygwinEnvironment();
>> +    return isAndroid() || isOSOpenBSD() || isWindowsCygwinEnvironment() || isMacOSXVersionLT(10, 7);
>>  }
>> 
>>  /// @}
>> ==========================
>> --- a/tools/clang/lib/CodeGen/ItaniumCXXABI.cpp.orig	2018-10-02 18:31:17.000000000 -0700
>> +++ b/tools/clang/lib/CodeGen/ItaniumCXXABI.cpp	2018-10-02 18:32:35.000000000 -0700
>> @@ -2255,7 +2255,7 @@
>>  const char *Name = "__cxa_atexit";
>>  if (TLS) {
>>    const llvm::Triple &T = CGF.getTarget().getTriple();
>> -    Name = T.isOSDarwin() ?  "_tlv_atexit" : "__cxa_thread_atexit";
>> +    Name = (T.isOSDarwin() && !T.isMacOSXVersionLT(10, 7)) ?  "_tlv_atexit" : "__cxa_thread_atexit";
>>  }
>> 
>>  // We're assuming that the destructor function is something we can
>> --- a/tools/clang/lib/Basic/Targets/OSTargets.h.orig	2018-10-02 17:14:10.000000000 -0700
>> +++ b/tools/clang/lib/Basic/Targets/OSTargets.h	2018-10-02 17:14:41.000000000 -0700
>> @@ -93,7 +93,7 @@
>>    this->TLSSupported = false;
>> 
>>    if (Triple.isMacOSX())
>> -      this->TLSSupported = !Triple.isMacOSXVersionLT(10, 7);
>> +      this->TLSSupported = !Triple.isMacOSXVersionLT(10, 4);
>>    else if (Triple.isiOS()) {
>>      // 64-bit iOS supported it from 8 onwards, 32-bit device from 9 onwards,
>>      // 32-bit simulator from 10 onwards.
>> ==========================
>> 
> 



More information about the macports-dev mailing list