New to MacPorts, stumped

Sat Aug 10 19:30:35 UTC 2019

Welcome to MacPorts!

On Aug 10, 2019, at 09:12, Gerben Wierda wrote:

> I am building a new Mac Mii Server, replacement for one running High Sierra with Server.app that still includes Mail & Web. I want the new server still to run Server.app for DNS, device management, profile management, Open Directory (required for some other stuff) so I will be adding MacPorts mail & web (could be difficult as there already is an Apache server for profile manager and device manager) to a macOS Mojave Server.
> 
> A stretch objective is to connect postfix and dovecot to OpenDirectory for authentication.

Steve Smith has recently been adding ports to MacPorts that try to replicate the configuration that was formerly available in macOS Server. The ports of this type that are currently available are:

https://ports.macports.org/port/clamav-server/
https://ports.macports.org/port/dns-server/
https://ports.macports.org/port/macos-vpn-server/
https://ports.macports.org/port/mail-server/

Maybe these will be useful to you.

> Question: Why is so much owned in /opt owned by group unbound?
> 
> ./sources/rsync.macports.org/macports/release/tarballs:
> total 327312
> -rw-r--r--   1 root  wheel    12680818 Aug 10 07:36 PortIndex
> -rw-r--r--   1 root  wheel         512 Aug 10 07:48 PortIndex.rmd160
> drwxr-xr-x  29 root  unbound       928 Oct  3  2018 base
> -rw-r--r--   1 root  wheel    85608960 Oct  3  2018 base.tar
> -rw-r--r--   1 root  wheel         512 Oct  3  2018 base.tar.rmd160
> drwxr-xr-x  62 root  unbound      1984 Aug 10 11:20 ports
> -rw-r--r--   1 root  wheel    69277184 Aug 10 07:48 ports.tar
> -rw-r--r--   1 root  wheel         512 Aug 10 07:48 ports.tar.rmd160
> 
> ./sources/rsync.macports.org/macports/release/tarballs/base:
> total 1352
> -rw-r--r--    1 root  unbound  111011 Oct  3  2018 ChangeLog
> -rw-r--r--    1 root  unbound   99338 Aug 22  2017 Doxyfile.in
> -rw-r--r--    1 root  unbound    3281 May  6  2018 HACKING
> -rw-r--r--    1 root  unbound    1556 May  6  2018 LICENSE
> -rw-r--r--    1 root  unbound    7079 May 28  2018 Makefile.in
> drwxr-xr-x    5 root  unbound     160 Jul  5  2018 Mk
> -rw-r--r--    1 root  unbound   36452 Jul  5  2018 aclocal.m4
> -rwxr-xr-x    1 root  unbound      44 May 28  2018 autogen.sh
> drwxr-xr-x    7 root  unbound     224 Oct  3  2018 config
> 
> I’m stumped and I don’t understand that group ownership. Can anybody explain?
> 
> Gerben Wierda
> Chess and the Art of Enterprise Architecture
> Mastering ArchiMate
> Architecture for Real Enterprises at InfoWorld
> On Slippery Ice at EAPJ

The short answer is that we should fix this, but the user/group of those files is not important and you can ignore that some of them have an unexpected group.

Now for the long answer.

We have an automated build system that builds our ports using the Buildbot continuous integration software. This software runs under a macOS user called "buildbot" which has an associated macOS group called "buildbot".

The same server is also responsible for running our mprsyncup script which populates the rsync server with new ports and base files and creates the ports.tar and base.tar archives from them.

The server runs mprsyncup periodically via launchd. Buildbot is not currently involved. But I had a goal to one day move mprsyncup into our Buildbot setup. It was probably for that reason that I coded the mprsyncup launchd plist to launch mprsyncup using the "buildbot" user and group (not realizing all of the implications this would have). And because mprsyncup runs as "buildbot", the files and directories it creates are also owned by "buildbot".

For each file and directory in a tar archive, the tar program stores the name of the user and group that owned it. You can see this by listing the tarball verbosely:

$ tar -tvf /opt/local/var/macports/sources/rsync.macports.org/release/tarballs/base.tar | head
drwxr-xr-x  0 buildbot buildbot    0 Oct  3  2018 base/
-rw-r--r--  0 buildbot buildbot  106 Aug 22  2017 base/.gitattributes
-rw-r--r--  0 buildbot buildbot  930 May 28  2018 base/.gitignore
-rw-r--r--  0 buildbot buildbot  212 May 28  2018 base/.mailmap
-rw-r--r--  0 buildbot buildbot   82 Aug 22  2017 base/.travis.yml
-rw-r--r--  0 buildbot buildbot 111011 Oct  3  2018 base/ChangeLog
-rw-r--r--  0 buildbot buildbot  99338 Aug 22  2017 base/Doxyfile.in
-rw-r--r--  0 buildbot buildbot   3281 May  5  2018 base/HACKING
-rw-r--r--  0 buildbot buildbot   1556 May  5  2018 base/LICENSE
-rw-r--r--  0 buildbot buildbot   7079 May 28  2018 base/Makefile.in

When you tell MacPorts to sync or selfupdate and it gets a new ports.tar or base.tar and extracts it, it's running tar as root. When running as root, tar tries to fix the ownership of the extracted files to match what they were on the original system. If your Mac has a "buildbot" user and group, then the tar program would make sure that the extracted files are owned by that user and group.

But most users don't have a "buildbot" user and group. In that case, the tar program falls back on the numeric user and group IDs that are also recorded in the file:

$ tar --numeric-owner -tvf /opt/local/var/macports/sources/rsync.macports.org/release/tarballs/base.tar | head
drwxr-xr-x  0 500    505         0 Oct  3  2018 base/
-rw-r--r--  0 500    505       106 Aug 22  2017 base/.gitattributes
-rw-r--r--  0 500    505       930 May 28  2018 base/.gitignore
-rw-r--r--  0 500    505       212 May 28  2018 base/.mailmap
-rw-r--r--  0 500    505        82 Aug 22  2017 base/.travis.yml
-rw-r--r--  0 500    505    111011 Oct  3  2018 base/ChangeLog
-rw-r--r--  0 500    505     99338 Aug 22  2017 base/Doxyfile.in
-rw-r--r--  0 500    505      3281 May  5  2018 base/HACKING
-rw-r--r--  0 500    505      1556 May  5  2018 base/LICENSE
-rw-r--r--  0 500    505      7079 May 28  2018 base/Makefile.in

It just so happens that, because of the order in which I created users and groups on our server, its "buildbot" user ended up with a UID of 500 and the "buildbot" group got a GID of 505. When tar extracts this archive on your system that doesn't have a "buildbot" user or group, it will give the extracted files a UID of 500 and a GID of 505. On your system, UID 500 is probably the first macOS user you created when you initially set up your computer. And because of the order in which you created groups, it just so happens that GID 505 got assigned to the "unbound" group.

You've probably already noticed that this description doesn't match your output. You showed that your files only have an unexpected group. They don't have an unexpected user. Why did only the GID get changed but the UID didn't? ...Unfortunately my investigative skills fail me at this point. I don't know why. But what you see is consistent with what I see on other systems, so there's nothing wrong on your system that you would need to investigate.

I said above that the unexpected GID doesn't matter, but it is unexpected, and other users have pointed this out before. ports.tar contains not only Portfiles and patch files, but also sample configuration files and other files that a port might copy to elsewhere on the user's disk. Users find it unusual that a sample configuration file for one program might appears to be "owned by" a completely different program. So while it does no harm in the sense that nothing should stop working because of this, it does cause confusion and for that reason we should fix it.

The above describes the way our server currently works, which I set up in late 2016. Prior to that, we used servers set up by someone else. I found a base.tar created by that system, and its contents were set to be owned by user and group ID 0 (i.e. root/wheel). Maybe I should modify our server to do this too, but this might require running mprsyncup as root, which would be less secure. Or we might employ the fakeroot program to do it. That would only help with the tar archives; it wouldn't help with the individual files which are also on the server. But users have been instructed for years to use the tar archives instead of the individual files, so that's probably ok.

In any case, we should probably change MacPorts base to invoke tar using its `-o` flag to tell it not to attempt to restore ownership (at least when extracting ports.tar and base.tar, and perhaps we should consider this also when a port extracts its distfiles).