On Jun 11, 2008, at 07:06, Rainer Müller wrote:
> Ryan Schmidt wrote:
>> Does universal mean "it works at full performance on all
>> architectures"? In this case, ports like this that install no
>> architecture-specific files should be modified to have an empty
>> universal variant selected by default.
>>
>> Or does universal mean "it has more than one architecture of compiled
>> software"? In this case, no-arch ports should be modified to turn off
>> the universal variant to indicate that a universal build of this port
>> is not applicable. I believe this definition of universal has been
>> used in several other no-arch ports before.
>
> I see "universal" as "can be compiled to work on multiple
> architectures". If +universal is missing, I assume that this is never
> going to work on multiple architectures with just one build. But maybe
> we can change this to give even more information.
>
> I think we need some way to distinguish the four cases for the user:
> 1) Arch independent
> 2) Can be build universal
> 3) Builds always universal (because the build system does that)
I think we should strive not to have this situation... if the port
builds universal by default, an effort should be made to change the
build so that it builds non-universal by default, and universal only
when the universal variant is chosen. I know Anthony did this for
many ports. For my ports that build universal by default I did not do
this (sleepwatcher, and I just found out adtpro), and I should go
back and fix this.
> 4) Does not build universal
>
> We could add all ports not building universal at the moment to 4), but
> move most of them to 2) once we have a working lipo method.
>
> Maybe we could add a 'default_variants +noarch' for 1) to indicate
> that?
> Or set 'supported_archs noarch' which stores this somewhere in the
> registry (see also below).
I would be in favor of "supported_archs noarch" and have MacPorts
automatically handle it from there.
>> [...]
>> I proposed awhile ago that we need more fine-grained control over the
>> universal variant: not just the ability to turn the default universal
>> variant on and off, but if we turn it off, to indicate why we do so:
>> is it because it's a no-arch port, because the universal variant does
>> not work, etc. It would even be good to indicate when a universal
>> build has been tested and is known to work. Also the addition of a
>> second type of universal variant which employs the build-for-arch-
>> arch-then-lipo strategy, instead of the build-all-at-once strategy we
>> use now which is incompatible with some ports (openssl, cairo,
>> probably others).
>>
>> http://lists.macosforge.org/pipermail/macports-dev/2007-June/
>> 001868.html
>
> I agree that we need the lipo strategy and we already have a merge
> proc
> which is able to lipo to independent builds together. Your proposed
> universal.method sounds fine to me.
Then later, I would want to rename "livecheck.check" to
"livecheck.method" for consistency.
>> Now we allow users to configure which architectures are included in a
>> universal build. This complicates matters immensely. For example the
>> wine port turns off the universal variant because it cannot be built
>> universal for ppc and i386 because wine runs on Intel processors
>> only. But it could possibly be built universal for i386 and x86_64.
>> How can we modify port syntax to allow the one type of universal
>> build but disallow the other?
>>
>>
>> Sometimes source code is not available, so we have some ports which
>> install from binaries, not from source. Sometimes these binaries are
>> architecture-specific, thus no universal binary is possible (e.g.
>> blender, oracle-instant-client). Sometimes these binaries are
>> universal, thus the ports are always universal (empty universal
>> variant, enabled by default, e.g. isightcapture) -- at least that had
>> been the case until we allowed users to configure the architectures.
>> Now, the set of architectures requested by the user might not match
>> the set of architectures provided by the binary distribution.
>
> Why is blender distributed as binary at all? Source code would be
> available.
I tried updating blender to the latest version a couple months back.
I read the compilation instructions several times but couldn't get
what they wanted me to do. It's confusing.
> oracle-instant-client does not seem to be open source, so there is no
> source available at all (which brings up again that we should record
> licenses in Portfiles).
Might not be a bad idea.
Oracle also doesn't want anyone redistributing their files, I think,
so we need a way to indicate in the portfile that the files should
not be mirrored on distfiles.macports.org. (Manually emailing William
doesn't sound like a great solution; he'd have to maintain a manual
list which would be better stored in the portfiles themselves.)
> But this does not offer some way to specify that a port builds
> universal
> by default.
>
>> Many ports work fine as 2-architecture (ppc and i386) universal
>> binaries but do not work as 4-architecture (ppc, i386, ppc64, x86_64)
>> universal binaries (e.g. apr, and therefore everything depending on
>> apr (like subversion and apache)). There is no existing syntax to
>> indicate this in the port.
>>
>>
>> For the sake of ports like apr and isightcapture and others in the
>> same boat, we need a way to indicate in each port which architectures
>> are supported. MacPorts would by default assume that all ports can be
>> built for all four architectures, but individual ports could add
>> lines to change this. For example, apr could add "supported_archs
>> i386 ppc" until their bug is fixed. wine could add "supported_archs
>> i386 x86_64". This could solve a number of problems. 1) If the user
>> requests the universal variant but the list of supported
>> architectures doesn't include all the architectures the user
>> requested in their universal_archs definition in macports.conf, a
>> warning can be issued informing the user. 2) If a port isn't
>> available at all on the user's local architecture (for example, the
>> user is on ppc and tries to install wine which only supplies i386 and
>> x86_64) port can issue an error and halt. Then we could eliminate
>> from the portfile the code that currently checks the architecture and
>> warns the user and exits.
>
> supported_archs sounds great.
"supported_archs" may be too wordy... maybe "archs" is enough. After
all, we have "platforms", not "supported_platforms".
> I was also thinking some time ago about extending the platforms
> statement to include version information. Something like:
> platforms {darwin >=8}
Maybe.
For now I'm making use of ${os.major} inside a "platform macosx"
block. Quite often when I need to employ a patch, the criteria is
"Mac OS X 10.x and earlier" and not merely "Mac OS X 10.x exactly".
> In combination with supported_archs this would let us get rid of these
> custom checks in pre-fetch blocks.
>
>> I proposed last month that we need a way (e.g. in "port installed")
>> to see for which architectures a given universal binary was built:
>>
>> http://lists.macosforge.org/pipermail/macports-dev/2008-May/
>> 005314.html
>>
>> When a port is installed, we need to record somewhere what
>> architectures were requested. Ideally this should happen regardless
>> of whether the universal variant was used or not. If it wasn't used,
>> then the requested architecture is just "i386" or "ppc".
>
> I agree here, we should store for which archs a port was build and
> installed. Maybe we should even allow the same version to be installed
> with different architectures as we do for different variants.
Yes please.
> It would be a possibility to add the architectures to the variants
> as we
> use it for the platforms.
I like it. Always include the architectures in the selected variants.
I'd go one further and say always include the OS -- always include
darwin_8 on Tiger, for example. Recording the arch and OS makes it
clear that users who upgrade from one OS to the next or from one arch
to another should reinstall. It also gets closer to something I've
wanted: a string that uniquely identifies what was installed. Port
name + version + variants (now including OS and archs) should do it.
I'm thinking about if and when we have binary packages. With this
enhancement, we would no longer have the current situation that e.g.
"cairo at 1.6.4_1+macosx" built on Panther is a different complement of
software than it is on Tiger.
You know, for completeness, we should probably also record somewhere
the arch of the machine used to build the port. In an ideal world it
shouldn't matter, but it's a fact that for some software a universal
binary built on Intel gives you a different result than a universal
binary built on PowerPC.
Someone suggested in another thread we should store the Xcode version
and the complete OS version. Why not. Let's store more info now, then
if we need it later it'll already be there.
Maybe we don't want to store all this in the variants though. Maybe
we can invent a new place to store this information.
>> [...]
>> MacPorts could check the architectures in each installed Mach-O file
>> to see if it matches the list of architectures requested by the user.
>> If not, a warning could be issued. We may need to be a little
>> selective here and not issue a fatal error at the first opportunity.
>> There are some ports where mismatched architectures might be ok. For
>> example, graphviz +gui installs graphviz from source but adds a GUI
>> from a binary, and the GUI is PowerPC-only so far.
>
> Would be something for the post-destroot, where we also compress man
> pages. I am not sure how expensive it is to check a file for its
> supported architectures, but running 'file' on each binary and library
> could be slow, don't you think?
Yes, it is a bit slow. I already have a PHP script that identifies
what arch(s) actually got installed for a given port, for me to test
my own port tree. Still, I think it's an important bit of data that
we should capture and store.
My script actually runs "lipo -info" (not "file") on each file to get
the archs. Possibly the speed of identifying files can be improved by
integrating the minimal required functionality into MacPorts (in Tcl
or C) so that we don't have to execute a subprocess for each
installed file.