[MacPorts] #41343: python extension module builds missing -I/opt/local/include

[MacPorts]

#41343: python extension module builds missing -I/opt/local/include
--------------------+----------------------------------------
 Reporter:  nad@…   |      Owner:  macports-tickets@…
     Type:  defect  |     Status:  new
 Priority:  Normal  |  Milestone:
Component:  ports   |    Version:  2.2.1
 Keywords:          |       Port:  python27 python33 python34
--------------------+----------------------------------------
 When building third-party Python packages (i.e. those not in the Python
 standard library), the MacPorts configuration is causing C extension
 modules to be compiled without `-I/opt/local/include` but linked with
 `-L/opt/local/lib`.  This definitely causes problems for packages that
 contain C modules that make use of OS X libraries that are shadowed in
 MacPorts.  A good example are the openssl libraries, libssl and libcrypto,
 which are shipped with OS X and with newer versions provided in MacPorts.

 The problem is caused by the port files only including
 `-I/opt/local/include` in `CPPFLAGS` but not in `CFLAGS`.  For Python that
 is not enough because the value of `CPPFLAGS` does not influence extension
 modules built by Python's Distutils.  There are a few long-open issues
 about this behavior, for example http://bugs.python.org/issue4010, but
 that's the way things have worked for a long time and continue to.

 If you examine the logs of various py* ports which include C extension
 modules, you can see the difference between the compile and link compiler
 calls.  This could affect ports in subtle ways if there is a library
 shadowing situation.  The py*-openssl port seems to be vulnerable though I
 have not verified that it causes problems.

 This issue came up while helping debug a segfault a user was seeing with a
 third-party Python package when built with a MacPorts Python.
 (https://bitbucket.org/tiran/backports.pbkdf2/issue/1/running-benchmarkpy-
 on-os-x-1085-using).  It was determined that the root cause was this issue
 in that the package was being compiled with system-provided openssl
 include files but linked with MacPorts openssl libs.  Here is a simplified
 demo of the problem:

 {{{
 $ /opt/local/bin/python2.7 -c 'import
 sysconfig;print(sysconfig.get_config_var("CFLAGS"))'
 -fno-strict-aliasing -fno-common -dynamic   -pipe -Os -fwrapv   -DNDEBUG
 -g -fwrapv -O3 -Wall -Wstrict-prototypes
 $ /opt/local/bin/python2.7 -c 'import
 sysconfig;print(sysconfig.get_config_var("CPPFLAGS"))'
 -I. -IInclude -I./Include -I/opt/local/include -I/opt/local/include/db46
 $ /opt/local/bin/python2.7 -c 'import
 sysconfig;print(sysconfig.get_config_var("LDFLAGS"))'
   -isysroot / -L/opt/local/lib -Wl,-headerpad_max_install_names
 -L/opt/local/lib/db46

 $ /opt/local/bin/pip-2.7 install
 https://bitbucket.org/tiran/backports.pbkdf2/get/a6791ee674c7.zip
 [...]
     building 'backports.pbkdf2._pbkdf2' extension
     /usr/bin/clang -fno-strict-aliasing -fno-common -dynamic -pipe -Os
 -fwrapv -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes
 -I/opt/local/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7
 -c backports/pbkdf2/_pbkdf2.c -o
 build/temp.macosx-10.8-x86_64-2.7/backports/pbkdf2/_pbkdf2.o
     backports/pbkdf2/_pbkdf2.c:22:15: warning: 'ERR_peek_last_error' is
 deprecated [-Wdeprecated-declarations]
         errcode = ERR_peek_last_error();
                   ^
     /usr/include/openssl/err.h:274:15: note: 'ERR_peek_last_error'
 declared here
     unsigned long ERR_peek_last_error(void)
 DEPRECATED_IN_MAC_OS_X_VERSION_10_7_AND_LATER;
                   ^
     backports/pbkdf2/_pbkdf2.c:27:5: warning: 'ERR_clear_error' is
 deprecated [-Wdeprecated-declarations]
         ERR_clear_error();
         ^
     /usr/include/openssl/err.h:278:6: note: 'ERR_clear_error' declared
 here
     void ERR_clear_error(void )
 DEPRECATED_IN_MAC_OS_X_VERSION_10_7_AND_LATER;
          ^
 [...]
     31 warnings generated.
     /usr/bin/clang -bundle -undefined dynamic_lookup -isysroot /
 -L/opt/local/lib -Wl,-headerpad_max_install_names -L/opt/local/lib/db46
 build/temp.macosx-10.8-x86_64-2.7/backports/pbkdf2/_pbkdf2.o -lcrypto -o
 build/lib.macosx-10.8-x86_64-2.7/backports/pbkdf2/_pbkdf2.so

 [...]
 $ /opt/local/bin/pip-2.7 uninstall backports.pbkdf2

 $ export CFLAGS="$(/opt/local/bin/python2.7 -c 'import
 sysconfig;print(sysconfig.get_config_var("CFLAGS"))')
 -I/opt/local/include"
 $ echo $CFLAGS
 -fno-strict-aliasing -fno-common -dynamic -pipe -Os -fwrapv -DNDEBUG -g
 -fwrapv -O3 -Wall -Wstrict-prototypes -I/opt/local/include
 $ /opt/local/bin/pip-2.7 install --user
 https://bitbucket.org/tiran/backports.pbkdf2/get/a6791ee674c7.zip
 [...]
     building 'backports.pbkdf2._pbkdf2' extension
     /usr/bin/clang -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fno-
 strict-aliasing -fno-common -dynamic -pipe -Os -fwrapv -DNDEBUG -g -fwrapv
 -O3 -Wall -Wstrict-prototypes -I/opt/local/include
 -I/opt/local/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7
 -c backports/pbkdf2/_pbkdf2.c -o
 build/temp.macosx-10.8-x86_64-2.7/backports/pbkdf2/_pbkdf2.o
     /usr/bin/clang -bundle -undefined dynamic_lookup -isysroot /
 -L/opt/local/lib -Wl,-headerpad_max_install_names -L/opt/local/lib/db46
 -fno-strict-aliasing -fno-common -dynamic -pipe -Os -fwrapv -DNDEBUG -g
 -fwrapv -O3 -Wall -Wstrict-prototypes -I/opt/local/include
 build/temp.macosx-10.8-x86_64-2.7/backports/pbkdf2/_pbkdf2.o -lcrypto -o
 build/lib.macosx-10.8-x86_64-2.7/backports/pbkdf2/_pbkdf2.so

 }}}

 I have not looked at the older, non-current python ports to see if they
 are configured and behave the same but chances are that they do.  It is
 also quite possible that other py*-* ports have unknowingly run into this
 problem in the past and have worked around it with patches to the package
 port rather than fixing the root problem in the python* port.

-- 
Ticket URL: <https://trac.macports.org/ticket/41343>
MacPorts <http://www.macports.org/>
Ports system for OS X