+universal help

[Ryan Schmidt]

On Jan 28, 2010, at 00:28, Scott Haneda wrote:

> Trac of patch file has been posted: 
> http://trac.macports.org/ticket/23453

Committed.


>>> Not using MacPorts, I just figured out how to get this to built out universal.  There are two files in the source
>>>  conf-cc
>>>  conf-ld
>> 
>> Yes, sorry, I was going to suggest to you that you modify conf-cc and conf-ld but I hadn't quite gotten everything to work yet.
> 
> No problem, in a way I am glad you did not, as I had to trudge through it :)  I looked at those files before, and noticed they had random comments below line #1.  I always wondered how that worked.
> 
> I explored the source and saw there are `head -n1` lines that read those two files in, which now makes sense to me.  From there, I figured to check the difference in gcc/cc
> 
>    $which cc
>      /usr/bin/cc
>    $which gcc
>      /usr/bin/gcc
> 
> Learning those are just links to gcc-4.2
> 
>    lrwxr-xr-x   1 root  wheel   7 Jan 22 03:06 /usr/bin/cc -> gcc-4.2
>    lrwxr-xr-x   1 root  wheel   7 Jan 22 03:06 /usr/bin/gcc -> gcc-4.2
> 
> And I also found:
> 
>    -rwxr-xr-x  1 root  wheel   97392 May 18  2009 /usr/bin/gcc-4.0
> 
> Not sure why 4.0 is kept around, but at least I know there is that option available just in case some day, it may be needed.
> 
> I thought to point this out in case it would be helpful to anyone else in this same "configure no" scenario.

Right.

Different versions of GCC are default on different versions of Mac OS X:

Mac OS X 10.3: GCC 3.3
Mac OS X 10.4: GCC 4.0
Mac OS X 10.5: GCC 4.0
Mac OS X 10.6: GCC 4.2

Most software builds with any version of GCC. Some require a specific version of GCC. That's why Tiger and Leopard still ship with GCC 3.3, and why Snow Leopard still ships with GCC 4.0 -- if some old software isn't ready for the new version of GCC yet, it can use the older one.

In MacPorts, ${configure.cc} represents the suggested compiler to use for the current version of Mac OS X. MacPorts automatically sets the CC variable to ${configure.cc} so most autoconf-based ports should just work and pick up this compiler.


> Ahhh, ok, I get it.  So ${configure.universal_archs} and ${configure.universal_cflags} are sort of dynamic variables, in that, at install time of MacPorts, they will get set to sane defaults for that architecture/flags.  I need not really think about adding in the arch's, especially not to add in archs for example, for ppc or ppc64, since in my case, I am on Intel.  However, someone on PPC, when they installed, they would have entirely different defaults in macports.conf, and they would inherit those into those variables.

${configure.universal_cflags} is not set at install time; it's set every time you run the port command. Same with all other variables in the port. In this case, ${configure.universal_cflags} variable you can access in a portfile is set by MacPorts from the value of the universal_cflags setting in macports.conf. MacPorts does set this value differently in macports.conf at install time based on your OS version. But the user can change it at any time.


> This actually clears up the last bits of confusion I had with my thread the other day on configuring and setting up macports.conf on a new install.
> 
> And I am quite please to be able to show:
>    $port installed | wc -l
>          45
> 
> So 44 ports installed (line one does not count) on this new system install.
> 
> I am a little confused about:
> 
>    $port installed | grep -v 'universal'
>    The following ports are currently installed:
>      autoconf @2.65_1 (active)
>      autoconf213 @2.13_1 (active)
>      curl-ca-bundle @7.19.7_1 (active)
>      mysql5 @5.1.42_0 (active)
>      mysql5-server @5.1.42_0 (active)
>      p5-locale-gettext @1.05_2 (active)
>      perl5 @5.8.9_0 (active)
> 
> I never explicitly installed any of those, (sans Mysql) so they came along with Apache2, or php5, or some other software.  Am I just getting lucky that the universal builds are working with these?  Or is it that as long as those are all the *same* arch, I am fine?

Build dependencies don't need to be universal, generally. Library and runtime dependencies do.

autoconf (and the older version autoconf213 which is needed by php5) are build dependencies only so it doesn't matter if they're universal.

curl-ca-bundle does not install compiled software; it only installs a text file. Therefore it has no universal variant.

Same with mysql5-server, which installs only directories and plists.

p5-locale-gettext is a perl module, using the perl5 portgroup. The perl5 portgroup specifies "universal_variant no"; presumably there is some problem endemic to perl modules preventing them from building universal, so it's turned off for all of them at the source.

Perhaps the reason is that perl5.8 itself could not build universal until ten days ago (see r62847). Perhaps it's now possible to build perl modules universal as well. Someone should test that.

Note that perl5 is a stub port, doing nothing more than depending on perl5.8 or perl5.10 where the actual software is installed. Since perl5 installs no files, it has no universal variant.


>    $file `which mysql5`
>      /opt/local/bin/mysql5: Mach-O 64-bit executable x86_64
> 
> Is this the relevant reason for MySql not being able to have a universal variant:
>  http://bugs.mysql.com/bug.php?id=47360
> 
> I also found this ticket:
>  http://trac.macports.org/changeset/62287
> 
> Since it does not appear there is going to be a timely fix on MySql's part for this, at least, in reading over the bugs.mysql.com thread, would taking the suggestion to use lipo and stuff them together be an option, or does lipo add more build time than is desired, that being the only downside I can estimate.
> 
> Just curious, as it does not effect me at this time.

We could try the muniversal portgroup for the mysql5 port. However I would prefer the developers of MySQL fix this properly upstream.

It also works fine if you just disable the embedded server, which most people don't need. That's the +no_embedded_server variant. Then you can use the +universal variant. The embedded server is built by default solely for the benefit of amarok. I am inclined to not build the embedded server by default, and change the +no_embedded_server variant into an +embedded_server variant, so that mysql5 can build universal by default.


>> memtester also doesn't support selecting a different build_arch in macports.conf (portfile variable: ${configure.build_arch}). This is to be expected, since that also happens in a way that assumes a configure-based script; ports that don't use the default universal variant don't by default get build_arch support either. The solution is similar, and the pre-made architecture flag is in the variable ${configure.cc_archflags}.
> 
> In regards to the lack of a configure script, why would a developer chose omit it?  How exactly are configure scripts made?  I am assuming that they are not hand coded, but perhaps tuned by hand at some point?  Looking over a few, they look more machine written than hand written.  That being the case, I fail to see why one wold chose to not have a configure step.

Most configure scripts are machine-generated by a program called autoconf. The developer writes an automake file called configure.am, which gets turned into configure.in, which gets turned into configure, which is what the user runs. autoconf is a very complex program by now with tons of cruft in it; you've seen how large and unwieldy the configure scripts it generates get. Some developers can't tolerate autoconf's bloat, complexity and general not-the-best-way-of-doing-things-ness and either roll their own configure scripts, use a competing product (e.g. cmake), or if their software is simple enough (few files to compile, few dependencies) they can get away with just writing a Makefile and forgoing the configure script entirely. Except that then in MacPorts we have to manually handle CC, CXX, CPP, CFLAGS, CPPFLAGS ,CXXFLAGS, LDFLAGS and other variables, since MacPorts only passes them to the configure script; if there isn't a configure script, then the variables don't get passed anywhere, and if there's a homegrown configure script, it may not respect these variables.


>>> My guess would be to add:
>>> platform darwin 10 {
>>>  # Reinplace foo to alter the above two files
>>> }
>> 
>> This has nothing to do with Snow Leopard specifically so there's no call for a platform darwin 10 section.
> 
> Yeah I was sort of stumbling along at this point, using `mtr` and a grep for 'configure no' through all the ports.  I still do not entirely understand the platform "phase"?

It's not a phase; it's basically a variant that MacPorts auto-selects for you based on the current OS.

> but am gathering, it is how you do specific things for specific platforms.  So darwin 10 == 10.6 and darwin 9 == 10.5 etc etc.

Yes.

> If there is ever call for special scenarios just for that OS release, this is where you would do it?

Yes.

> For example, `mtr` needs "configure.env-append LIBS=-lresolv" for 10.6, but apparently does not need it for lesser versioned OS's?

Presumably that's correct. Looking through the log, I added this in r56011 specifically to fix Snow Leopard, as reported on the list:

http://lists.macosforge.org/pipermail/macports-users/2009-August/016147.html

> Or I must have this wrong, as `port info mtr` lists "darwin_10" as a variant, implying I can apply that to any version OS I desire.  So yeah, still a little muddled in this area.

You should not try to select +darwin_10 on any other platform. In fact, MacPorts will prevent you from doing so:

$ sudo port install mtr +darwin_10
Password:
Warning: Implicit variants should not be explicitly set or unset. darwin_10 will be ignored.
--->  Computing dependencies for mtr
--->  Fetching mtr
^C