[MacPorts] #20938: ffmpeg/ffmpeg-devel fails to build on 10.6

[MacPorts]

#20938: ffmpeg/ffmpeg-devel fails to build on 10.6
-------------------------------+--------------------------------------------
 Reporter:  gonhidi@…          |       Owner:  devans@…           
     Type:  defect             |      Status:  assigned           
 Priority:  Normal             |   Milestone:                     
Component:  ports              |     Version:  1.8.0              
 Keywords:  snow-leopard       |        Port:  ffmpeg ffmpeg-devel
-------------------------------+--------------------------------------------

Comment(by macports@…):

 The solution provided above (just switch off mmx for 32-bit machines) is
 heavy-weight and less than ideal. We can do a little better.

 ffmpeg, when compiled through macports for i386 (rather than 86x64)
 generates a compiler error when compiling the instructions in
 h264_h_loop_filter_luma_mmx2
 h264_h_loop_filter_chroma_mmx2
 h264_h_loop_filter_chroma_intra_mmx2
 The same sort of error in each case which I interpret to mean that gcc
 believes it is being forced to allocate more MMX registers than are
 available.

 This bug report implies that the problem is in swscale_template.c
 (specifically in code swizzling between RGB and RGBA formats), but I have
 no such problem using the most recent XCode (the one released about two
 weeks ago) and the most recent ffmpeg code.
 I do, however, as I said, get this error message at the three functions I
 listed above.

 Since I need to have a (reasonably fast) ffmpeg on a Core Duo (NOT Core 2
 Duo) machine, I stuck a quick hack into the source to get this to work.
 Basically the problem appears to be that gcc (or at least Apple's version
 of it, so this may be an issue with the LLVM backend) is being dumb about
 not reusing MMX registers from the inlined assembly, even though it is
 obvious that they can be reused. I don't know enough gcc to understand
 what it is thinking (or even how to generate assembly so I can see what it
 is doing along the way) so I did a quick hack with functionPtrs to force
 the damn code to be compiled into separate functions. It's obviously less
 than ideal --- better would be to tell gcc just not to inline the relevant
 functions --- but I could not figure out how to get that to work. The net
 says use __attribute(noinline))__ but every variant I tried of that gave a
 compiler error.

 This involves some minor changes to the file h264dsp_mmx.c
 I'm sorry --- I don't know how to do patching or any of the other stuff
 that you guys use; so I just included the raw h264dsp_mmx.c file as an
 attachment. As you can see, what I have done is trivial --- forced some
 inlined code to be broken into separately compiled functions to prevent
 register overload.

 IMHO the ideal path going forward is
 (a) install my patch ASAP to get an immediate speedboost
 (b) improve my patch by anyone who understands gcc to tell it to just not
 inline the functions I jump to via procPtr. What we want is separate
 compilation of the relevant function without paying the procPtr overhead
 (c) anyone who understands gcc assembly and the linkages between gcc and
 llvm should try to spend a little time figuring out WHY gcc/LLVM is
 complaining about register spillage. This really looks like a (serious)
 bug, and is probably worth bringing to the attention of someone inside
 Apple who handles the compilers.

-- 
Ticket URL: <https://trac.macports.org/ticket/20938#comment:31>
MacPorts <http://www.macports.org/>
Ports system for Mac OS