Problem with Fortran compilation when numpy.distutils is used

[Peter Danecek]

Hi all,

here as short follow up on this issue (the problems was described earlier, see below), as I have now a clearer idea what is happening. There seems to be no other python port (Python Portgroup) using numpy.distutils to compile Python extensions written in Fortran. 

Now I understand better that the difference between building manually and using Macports and what is causing the problem. I have found solutions to the problem, but I am not sure if it would be okay.

It is caused by the Python Portgroup which is setting by default the environment variable LDFLAGS to pass the architecture, (see http://guide.macports.org/#reference.portgroup.python.keywords). If this environment variable is set, the `setup.py` script (or more precisely `numpy.distutils.fcompiler`) fails to determine the correct linker flags ("-undefined dynamic_lookup -bundle" is missing) and to link the binary extension file (*.so) from fortran sources. This affects only fortran code.

One solution I came up with is to set `python.add_archflags no`. Is this appropriate? I am not exactly sure why this setting of LDFLAGS is done the first place, but I assume it could be related to universal builds. The port is `universal_variant no` so is it okay just to disable?

Thanks!
~petr



On 10 Jan 2014, at 19:03, Peter Danecek <Peter.Danecek at bo.ingv.it> wrote:

> 
> Hi all,
> 
> I am trying to update one of the ports i maintain (py-obspy) and encounter some problem I do not really understand. Maybe someone can help.
> 
> Background:
> The obspy packages includes compiled extension, both C and Fortran. The C extensions are build with Xcode clang and Fortran with gcc (by variants). This is true for both the version currently available in MacPorts (@ 0.8.4) and the new version, i.e. 0.9.0. One difference is that now the `setup.py` script uses numpy.distutils to be more portable. 
> 
> Now when then `python setup.py build` is invoked by Macports, it fails at the linking step of the Fortran extension.
> 
> --- snip ---
> /opt/local/bin/gfortran-mp-4.8 -Wall -arch x86_64 build/temp.macosx-10.8-x86_64-2.7/build/src.macosx-10.8-x86_64-2.7/build/libtau_Darwin_64bit_py27module.o build/temp.macosx-10.8-x86_64-2.7/build/src.macosx-10.8-x86_64-2.7/fortranobject.o build/temp.macosx-10.8-x86_64-2.7/opt/local/var/macports/build/_Users_petr_sandbox_MyPorts_ports_python_py-obspy-devel/py27-obspy-devel/work/obspy-0.9.0/obspy/taup/src/emdlv.o build/temp.macosx-10.8-x86_64-2.7/opt/local/var/macports/build/_Users_petr_sandbox_MyPorts_ports_python_py-obspy-devel/py27-obspy-devel/work/obspy-0.9.0/obspy/taup/src/libtau.o build/temp.macosx-10.8-x86_64-2.7/opt/local/var/macports/build/_Users_petr_sandbox_MyPorts_ports_python_py-obspy-devel/py27-obspy-devel/work/obspy-0.9.0/obspy/taup/src/ttimes_subrout.o -L/opt/local/lib/gcc48/gcc/x86_64-apple-darwin12/4.8.2 -lgfortran -o build/lib.macosx-10.8-x86_64-2.7/obspy/lib/libtau_Darwin_64bit_py27.so
> Undefined symbols for architecture x86_64:
>  "_PyArg_ParseTupleAndKeywords", referenced from:
>      _f2py_rout_libtau_Darwin_64bit_py27_ttimes in libtau_Darwin_64bit_py27module.o
>  "_PyCObject_AsVoidPtr", referenced from:
>      _initlibtau_Darwin_64bit_py27 in libtau_Darwin_64bit_py27module.o
>      _F2PyCapsule_AsVoidPtr in fortranobject.o
> [… more symbols …]
> --- snap ---
> 
> I understand why this command fails (the symbols are from python library). But what puzzles me is what determines this set of flags. When the C extensions are compiled the following flags are used. 
> 
> --- snap ---
> /usr/bin/clang -bundle -undefined dynamic_lookup -L/opt/local/lib -Wl,-headerpad_max_install_names -L/opt/local/lib/db46 -arch x86_64 -arch x86_64 build/temp.macosx-10.8-x86_64-2.7/opt/local/var/macports/build/_Users_petr_sandbox_MyPorts_ports_python_py-obspy-devel/py27-obspy-devel/work/obspy-0.9.0/obspy/mseed/src/libmseed/fileutils.o [… more object files …] -o build/lib.macosx-10.8-x86_64-2.7/obspy/lib/libmseed_Darwin_64bit_py27.so
> --- snip ---
> 
> In understand that C (with Clang) and Fortran (with gcc) can have different flags and behave different. But the other think is that when I try to reproduce the build process manually executing in the work directory: 
> 
>  sudo env F90=/opt/local/bin/gfortran-mp-4.8 /opt/local/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7 setup.py --no-user-cfg build
> 
> --- snip ---
> /opt/local/bin/gfortran -Wall -Wall -undefined dynamic_lookup -bundle build/temp.macosx-10.8-x86_64-2.7/build/src.macosx-10.8-x86_64-2.7/build/libtau_Darwin_64bit_py27module.o build/temp.macosx-10.8-x86_64-2.7/build/src.macosx-10.8-x86_64-2.7/fortranobject.o build/temp.macosx-10.8-x86_64-2.7/opt/local/var/macports/build/_Users_petr_sandbox_MyPorts_ports_python_py-obspy-devel/py27-obspy-devel/work/obspy-0.9.0/obspy/taup/src/emdlv.o build/temp.macosx-10.8-x86_64-2.7/opt/local/var/macports/build/_Users_petr_sandbox_MyPorts_ports_python_py-obspy-devel/py27-obspy-devel/work/obspy-0.9.0/obspy/taup/src/libtau.o build/temp.macosx-10.8-x86_64-2.7/opt/local/var/macports/build/_Users_petr_sandbox_MyPorts_ports_python_py-obspy-devel/py27-obspy-devel/work/obspy-0.9.0/obspy/taup/src/ttimes_subrout.o -L/opt/local/lib/gcc48/gcc/x86_64-apple-darwin12/4.8.2 -lgfortran -o build/lib.macosx-10.8-x86_64-2.7/obspy/lib/libtau_Darwin_64bit_py27.so
> --- snap ---
> 
> The script behaves "correctly" or at least uses a flags set which builds. So I would need to understand what exactly makes the difference here and where the flags are influenced.
> 
> Any useful hints here? Thanks!
> ~petr
>