apache2 location

[Ryan Schmidt]

This is a mega-reply to many previous messages in this thread.



On Feb 24, 2009, at 16:35, Scott Haneda wrote:

> A few questions... how come the apacche2 does not warn me of the  
> violation, I see the "violate" in the port file, but as far as I  
> can tell, there is nothing when installing it to tell me?  I will  
> confirm once more on a install, but i am pretty sure I did not see it.

It does warn you, as you noted in a later message:

Warning: portname requests to install files outside the common  
directory structure!

> If I modify the port to put apache in www, I believe that to be the  
> correct place based on the above url, is this a huge undertaking  
> that is going to require a lot of discussion to not break thing?   
> Every path is going to need to change.

Apache does not belong in ${prefix}/www. That directory is for web  
site data (htdocs, cgi-bin, etc.)

> To me it is worth it to follow the guidelines of MacPorts, but this  
> may be one that is so engrained in MacPorts, it might not want to  
> change.

I'm not opposed to changing apache2 (and apache and apache20) to  
conform better to the MacPorts directory layout. apache2 is  
maintained by James Cox who hasn't been very active with MacPorts  
recently so we will have to take some initiative to decide what  
exactly we want to change, file a ticket, attach a new portfile  
proposal, and if James doesn't reply in awhile, we can commit it.  
apache20 is maintained by Blair Zajac and is supposed to be similar  
to apache2 in all respects except the version, so we should be able  
to make the same changes there. apache is maintained by me with  
openmaintainer. It currently has two different modes of installation  
(two different possible layouts); this should be changed to just a  
single layout that agrees with whatever we choose for the apache2  
layout.

One possible reason we might want separate directories for the  
different apaches (${prefix}/apache2, ${prefix}/apache20, ${prefix}/ 
apache) is to allow simultaneous installation of multiple versions.  
However this is not even possible today; the manpages still conflict.  
We could go to considerable effort to make all three ports install  
into ${prefix} but with version-specific names for each file, or we  
could admit that probably nobody needs to have both apache (1) and  
apache2 at the same time and just let them conflict with one another.

> My feeling is, the sooner the better, there are already a handful  
> of blogs out there, which instructions and hard paths in their  
> instructions pointing to the current location.  The sooner we put  
> it where MacPorts recommends, the better the long term usability is  
> going to be.

We have our own documentation in the MAMP wiki page which should be  
canonical and up to date with the current state of the ports,  
whatever that may be. Users should always look there first.



On Feb 24, 2009, at 19:03, Scott Haneda wrote:

> Warning: portname requests to install files outside the common  
> directory structure!
> ( That to me sound serious, ominous, and something we should strive  
> to solve for new users )

The wording could be improved to sound less-ominous. It is merely a  
warning to the user that this port does not install (all) files in  
the locations the user might otherwise have come to expect from  
MacPorts. The user can run "port contents apache2" to find out where  
the files got installed.

> It generates an error, which is something people have to learn to  
> ignore or research about why the error happened.  For a user who is  
> not a port contributor, this error is off putting to them.

It is not an error; it is a warning.

> Now I know what the error is about.  There are a lot of people I  
> try to get to use ports, and they all tell me they tried in the  
> past but could not get past errors, maybe this is one of those issues.

If users are encountering issues with ports, they should subscribe to  
macports-users and tell us about them so that we can fix them. Or  
they should file tickets in Trac.



On Feb 24, 2009, at 21:13, Mike Savory wrote:

> I think you are confusing destroot.violate_mtree
>
> For which the docs say:
>
>> This means that the port installed files outside of their normal  
>> locations in ${prefix}. These could be files totally outside of $ 
>> {prefix}, which could cause problems on your computer, or files  
>> inside of ${prefix} that are not in a standard location. Use port  
>> contents portname to see the location for all files that were  
>> installed by a given port.
>
>
> And the apache2 port which actually does install all its files in  
> $prefix in a normal Mac OSX install, as the command "port contents  
> apache2" shows
> For most people $prefix is /opt/local/

I don't think Scott is confusing these things. The documentation is  
correct. Violating the mtree can occur not only by installing files  
completely outside of ${prefix} (which apache2 does not do) but also  
by installing into directories inside ${prefix} which are  
nonstandard. apache2 installs to ${prefix}/apache2 which is not one  
of the standard directory specified by "man porthier", hence it is  
considered an mtree violation.

> So what you are actually suggesting is changing the default layout
>     htdocsdir:     ${datadir}/htdocs
>
> to read in macports
>     htdocsdir:     ${prefix}/www
>
> to comply with the standard layout?

First of all, the document root should not be ${prefix}/www but $ 
{prefix}/www/htdocs (there will be other things in ${prefix}/www  
other than the document root -- for example the cgi-bin directory).  
But a discussion of where apache2 and other webserver and web app  
ports should look for and put their web files is a totally separate  
issue, which I began a discussion of 2 weeks ago in this thread:

http://lists.macosforge.org/pipermail/macports-dev/2009-February/ 
007388.html


> I believe the
> destroot.violate_mtree yes
> in the portfile is primarily  to allow the platform Darwin to  
> install files outside the prefix.

No, apache2 does not install files outside of ${prefix} (except for  
the launchd plist in /Library/LaunchDaemons, which is not considered  
an mtree violation; it is a normal and necessary part of allowing Mac  
OS X to start processes at system startup time).


On Feb 24, 2009, at 22:25, Scott Haneda wrote:

> I have a feeling I am going to stand alone on this one, which is  
> fine, I have no intention of pushing it, this was just to find out  
> why this was chosen,

You don't stand alone on this issue, I don't think there's a need to  
argue the benefits of putting the software into the standard  
directories, we just need to consider what the exact directory layout  
should be and then change the port to put the files there.

> and why it was not caught as in violation of suggested layouts.  
> There is even a patch where the violate_mtree option was added in  
> trac.  Not that long ago either, where did apache 2 install prior  
> to that?

As far as I know, the apache2 port has always installed to ${prefix}/ 
apache2. The mtree checking code was added relatively recently to  
MacPorts base. Prior to that, no checking was done. Ports could  
install files wherever on the user's system and the user would never  
know unless they looked at "port contents". Now that checking is  
done, a warning is printed at install time if a port installs to a  
nonstandard location. It reads like this:

Warning: violation by /opt/local/apache2
Warning: apache2 violates the layout of the ports-filesystems!
Warning: Please fix or indicate this misbehavior (if it is intended),  
it will be an error in future releases!

As a result, "destroot.violate_mtree" was added to the port to  
indicate that this violation of the standard directory structure was  
intentional (i.e. was not an accident). Now the warning reads as you  
originally noted:

Warning: apache2 requests to install files outside the common  
directory structure!



On Feb 24, 2009, at 22:41, Scott Haneda wrote:

> But yes, I am suggesting it be in /opt/local/etc or wherever is  
> most appropriate according to the guidelines.  Let's just say, if I  
> were writing this port and it had never been written before, I  
> would have seen those guidelines, and put it elsewhere.  Is this to  
> mean, I am reading those guidelines wrong?

You are reading the guidelines correctly. In this case, the default  
upstream apache2 software simply by default installs into ${prefix}/ 
apache2, and no effort was undertaken in the port to change this. We  
could change this now.

>> I believe the
>> destroot.violate_mtree yes
>> in the portfile is primarily  to allow the platform Darwin to  
>> install files outside the prefix.
>
> I very much think you are correct as well.  That makes sense, as  
> there are a few cases, where you just may have to do this, and you  
> need that warning, in order to know you are doing something ASU may  
> kill.

Ports should not install things that Apple Software Update may  
overwrite. The whole point of having a separate MacPorts prefix is to  
isolate MacPorts-installed software from Apple-installed software.

> This is apache2, it should not generate an alert.  There is nothing  
> funky about it.

Currently, because apache2 installs to a nonstandard location, a  
warning is printed.

> Maybe MacPorts should then dismiss the warning except when things  
> are installed outside of prefix, then this makes sense.  They  
> should also update the docs to allow anything to go in /opt/local.

No, it is also incorrect (or at least, non preferable) for a port to  
install into a nonstandard directory within ${prefix}. apache2  
currently does this, hence a warning is (and should continue to be)  
printed.

> What I can say, is I am more than a new ports user, still beginner,  
> but if I were a totally new user, this would be a hang-up to me,  
> maybe most just let it go and ignore the warning, I tend to not be  
> that person, as these emails probably illustrate :)

Could you suggest an alternate wording of the message that would make  
it more clear to you that the port is deliberately installing files  
into nonstandard places and that you as a user should simply be aware  
of that fact?

A suggestion I made some time ago was that each error or warning  
MacPorts can print should be rather short (no more than one line),  
and then it should print a URL to a wiki page where the error can be  
described in more detail. Each such error page should, I think, be  
divided into two sections: 1) what a user should do about this  
message, and 2) what a port author should do about this message. What  
do you think about this idea?


On Feb 25, 2009, at 08:11, Chris Janton wrote:

> hmmm - I'm comparing my Mac OS X 10.3 system, my Centos 5 system,  
> and an Ubuntu system.
>
> Yeah, well, the Ubuntu system is just too weird (now I know why I  
> chose Centos ;-) they name httpd apache2...
>
> The executable files could be split amongst /usr/sbin (apachectl,  
> httpd) and /usr/bin (htdigest, htpasswd)

Of course MacPorts will not (should not) install anything into  
directories /usr/sbin or /usr/bin or anywhere else that Apple puts  
files. MacPorts should install such files to ${prefix}/sbin and $ 
{prefix}/bin.

> The configuration files could be put in /opt/local/etc/httpd

That sounds reasonable. But both apache (1) and apache2 are called  
"httpd" so we may want to use ${prefix}/etc/apache2 and ${prefix}/etc/ 
apache instead so that a user who has tried more than one of apache  
and apache2 will not be confused. An apache (1) configuration file  
would not be appropriate for use with apache2 and vice versa. The  
same goes for whatever directory we choose for apache modules --  
these are not compatible between versions.

> The htdocs directory could be put in /opt/local/www, but some could  
> make that case that it should actually be /opt/local/var/www - at  
> this point it probably becomes "whose distribution is the master/ 
> overlord" - Unix politics at its finest ;-)

"man porthier" says we use ${prefix}/www. It mentions ${prefix}/www/ 
cgi-bin but no other directories (notably it doesn't mention $ 
{prefix}/www/htdocs which I would like it to mention).

> Of course, why would you put your web site assets into the /opt/ 
> local tree? not like anyone is going to do that.
> Same goes for MySQL databases as well, but that's a different issue.

True, but there are many web app ports that need to install web files  
somewhere, and it seems a good place to use is ${prefix}/www/htdocs/$ 
{name} (if the web app is designed to go directly into the document  
root) or ${prefix}/www/${name} (if the web app is designed to go  
outside the document root; then symlinks can be placed in ${prefix}/ 
www/htdocs/${name} as appropriate, or an apache configuration snippet  
can be included in a separate file). Users who are doing their own  
web site development can still reconfigure how their web server works  
if desired, and can use Alias directives to direct certain URLs to  
MacPorts-installed web files in ${prefix}/www even if their  
DocumentRoot has been changed.

> So does "fixing" the apache2 port to behave like "real" systems  
> simply mean the port should add symlinks in sbin and bin (wherever  
> you put them) that point to the binaries in /opt/local/apache2?

That would not cure the mtree violation, which I think is the point  
of the discussion.

> Please see MySQL5 - they provide the symlinks. Want an oddball path  
> or 2?
>
> ls -al /opt/local/bin/mysqladmin
> lrwxr-xr-x  1 root  admin  28B  2 Sep  2007 /opt/local/bin/ 
> mysqladmin@ -> ../lib/mysql5/bin/mysqladmin

Yes, the mysql5 layout is convoluted, but it represents perhaps an  
extreme case of what you will get if you try to shoehorn a big  
package into an existing directory structure. Either you put  
everything in ${prefix}/mysql5, which violates the mtree, or you go  
to great lengths to put it into the standard directories, which  
results in what mysql5 does. Well, maybe it goes a bit overboard. It  
probably won't be so bad for apache2.



On Feb 25, 2009, at 16:05, Adam Byrtek wrote:

> Scott, I think that in principle you are right. On the other hand I
> can understand that many people are used to the current layout and if
> the change is accepted, some reasonable migration strategy will have
> to be prepared. Such migration shouldn't involve any manual work
> and/or data loss.

If we rearrange the layout of apache2, you should not necessarily  
expect it to handle any migration of config files or data for you.  
You should expect to have to do so manually. It would not be proper  
for MacPorts to move files that are not registered to a port.

The new port could check whether, say, the httpd.conf exists at the  
location where the old port looked for it, and in that case, print a  
message explaining the new layout to the user and recommend they move  
their files.

Any apache2 modules (for example php5) would have to be rebuilt by  
the user so that they get installed into the new module directory.  
The apache2 port cannot do this for you. It might be possible for  
apache2 to figure out which modules you have installed and tell you  
which ports you have to rebuild, though that would take a little work.



On Feb 25, 2009, at 16:51, Scott Haneda wrote:

> Fully agree on that front.  It was why I wondered if this was too  
> engrained in how it was done, it just may not be worth it.  But,  
> then again, the move should be pretty simple, ports is sort of  
> designed by nature to fiddle with paths and move stuff around.

It is not designed to move files that are not registered to a port.  
The apache config file httpd.conf and any web data you have installed  
are not registered to the apache2 port so it cannot move those files  
for you.



On Feb 25, 2009, at 17:40, Adam Byrtek wrote:

> I guess the best first step would be to raise this on macports-dev
> mailing list and get acceptance from the gurus :)

Since the issue began being discussed here on macports-users I think  
it's fine to continue here.



On Feb 26, 2009, at 01:35, Bill Hernandez wrote:

> I just like going to /opt/local/apache2 and finding everything  
> related to apache in one spot.
>
> I remember when I did one of my first installs and began trying to  
> track down the logfiles to mysql and pgsql. It took me some time to  
> locate them due to what I considered to be some inconsistencies.  
> Now that I have done this a few times, it doesn't matter where the  
> stuff lives. It is obvious that I was only considering the  
> convenience, and not the practicality.
>
> /opt/local/var/db/mysql5/hostname.local.err
> /opt/local/var/log/postgresql83/postgres.log
>
> Apache on the other hand was very easy to locate.
>
> /opt/local/apache2/logs/access_log
> /opt/local/apache2/logs/error_log
> /opt/local/apache2/logs/mod_rewrite.log
>
> Now that I more or less know where things live, it doesn't matter,  
> but in the beginning it was not easy tracking down  all the logs  
> and conf files.

"port contents foo" always tells you what files are installed by port  
foo so that is a good way to learn.