Architectural proposal

[Rainer Müller]

Michelangelo wrote:
> Hello,

Hello Michelangelo,

> I've been thinking about this proposal for the whole day; William has  
> suggested me to write it here, to hear directly from you, to hear your  
> feedbacks and, most thing important for me, your objections.
> Ah! I was about to forget, my name's Michelangelo De Simone, I'm very  
> pleased to meet you all.:)

I read your proposal careful to make sure I fully understand it. But I 
added comments on items where I think this can be solved in other ways 
or are already possible.

> Synopsis
> MacPortsDaemon (“MaPoD”) is an additional layer above the existing  
> MacPorts utilities and API that provides asynchronous access to  
> MacPorts functionalities (and new ones) to upper layer clients. It’s  
> based on a pluggable architecture.

So this should be a daemon running in the background. I don't like the 
idea of running compile jobs in the background. Also, I don't see real 
advantages from it, see comments below.

> Visionary scenarios
> Mike (limited user) wants to use an Adium version built on his own, to  
> accomplish that he launches a MacPorts/MaPoD client, search for Adium  
> in the search box, once he has found it he chooses to build and  
> install it.
> The client confirms the action to Mike and giving him a notification  
> that his version of Adium will be delivered directly into his ~/ 
> Applications folder once the build process is over. The client becomes  
> immediatly available for new commands.

So this is mostly a request for running MacPorts as non-root. This could 
really be improved, for example with an easier installation. Anders 
already made some additions to make it possible. But currently it needs 
./configure flags which are not obvious.

> Amanda (limited user) already built Cyberduck but she needs it no more  
> and then she decides to uninstall it. To accomplish that Amanda  
> launches a MacPorts/MaPoD client, search for installed apps, once she  
> has found Cyberduck she select it and mark it for removal. Her client  
> confirms the action and becomes immediatly available for new commands.

Comments below.

> John (administrator on his own system) decides that Mike and Amanda  
> need Gimp, so he decides to build and install it in order to let  
> Amanda and Mike use it. To do that he launches a MacPorts/MaPoD  
> client, search for Gimp; once he has found it he tells the client to  
> install Gimp system wide, for users to use it. His client confirms the  
> action and becomes immediatly available for new commands.

Already possible. If you add /opt/local/bin to the system wide PATH, 
software installed by MacPorts is available for all users.

> Amanda (limited user) needs Gimp, so she starts looking for it with  
> her MacPorts/MaPoD client and asks the client to build and install it.  
> Client notifies Amanda that Gimp is already being built because of  
> John choice to deploy it system wide and that she we’ll be able to use  
> in some time; MacPorts/MaPoD will notify her with a Growl notification  
> when Gimp will be ready to be used.

I don't think this will happen often. Most Mac OS X installations are 
used with a single user only and especially not with multiple users at a 
time.

How many times will a user request software to be installed while a 
system-wide build is running?

Also, why should a user not be allowed to compile his/her own version 
(maybe even newer)?

> Mike (limted user) asks himself how to remove objects and “stuff”  
> produced by MacPorts/MaPoD and finds out that there’s no reason to be  
> concerned about because MacPorts/MaPoD mantains itself scheduling  
> maintance operations at the right time.

Who wants jobs kicking in at random time and removing stuff? Also, there 
is no stuff left from a successful build. Users may decide to keep stuff 
from unsuccessfull builds in order to try to fix it.

> Amanda (limted user) worries about tree sync and tries to manually  
> update it using her client but it notifies her that the latest sync  
> has been done automatically by MacPorts/MaPoD itself yesterday night.

Install a cronjob. No need for a daemon to accomplish this task.

> MaPoD Architecture Overview
> MaPoD aims to provide MacPorts additional features applying de- 
> coupling among actual and future software components. See pic here: http://snipurl.com/22dyy

Do you really think anyone wants a web interface for compiling stuff? I 
doubt that. I also doubt that anyone will write additional clients - 
for what purpose?

> The whole architecture of MaPoD shall be developed as a system service  
> (LaunchDaemon), letting the clients at the upper layer request  
> services asynchronously  (“Build Adium, notify me when you are done”);  
> users could also benefit from scheduled/automatic maintance operations  
> such as self-update, application upgrades, tree syncs, objects cleanup.

Leaving out the daemon thing here, you are requesting an API for these 
task. I think you are right; currently the API does not export these 
functionality.

> MaPoD Core ‘s task is to accomplish basic functionalities, wrapping  
> those already provided by MacPorts’ port and adding them mechanism  
> such scheduling, permission managment, event notifications (“Adium  
> succesfully built by John”). No user on the system should sudo to  
> build an app for his own use.

As noted above, using MacPorts as non-root needs improvement, but can 
already be done.

> MaPoD Interface is the real glue between MaPoD/MacPorts and GUI.  
> Clients and MaPoD are completely loosly coupled because MaPod  
> Interface provides a neutral way for them to communicate.
 >
> Third party MacPorts developers could develop plugins to extend core  
> functionalities (“Build Adium, notify me when you are done with a  
> Growl notification” or “Build Adium, notify me when you are done with  
> an email to jdoe at apple.com”).

What kind of plugins do you think of? Adding a general hook to the API 
"run-when-ready" would be enough, I think.

> Conclusions
> With the appropriate choices any user on the same system, even limited  
> ones, would see a full-featured MacPorts “distribution”; any user will  
> have the possibility to search, install (from source or normal  
> binary), remove his application with no touch to MacPorts at all.  
> System administrator would also benefit from a fine grained permission  
> system (“Do not let Mike build Adium” or “John can build and install  
> only between 3pm and 9pm”). MacPorts installation would be “piloted”  
> by MaPoD itself, so there will be no need to sudo-run it for  
> occasional users. It’s clean, it’s “secure” (you know, Santa believes  
> in security...:)).

Add cronjobs, set permissions using chmod... No need to engineer a full 
new system? Why would anyone prohibit to install a specific port? Users 
can always build them outside MacPorts or install a binary. Also see the 
comment above about multi-user installations. As noted, non-root is 
possible.

> Whis this architecture possibilities are endless: how long would be  
> needed to develop a plugin to mirror the tree among multiple nodes?  
> How long to develop a web client and an adapter to manage the same  
> build on multiple nodes?

Mirror the tree? Run rsync and done? I don't get the point here. I 
already commented on a web interface above.

If you want to build software for use on multiple nodes, compile it on 
one and create archives from it (already possible with MacPorts) and 
distribute it in your preferred way. Or provide archives to be installed 
on the clients.

> Obviuosly MacPorts core should never be used by users/clients,  
> preferring MaPoD way to accomplish tasks.

A user should never call the API manually also, what's the point? Users 
use clients like port on the command line or a GUI.

I don't see any advantage from compiling in the background with a 
daemon. A GUI could provide queued builds which fits your request for 
"giving control back immediately."

So, after all I don't see any real improvements for MacPorts from your 
proposal to add another layer.

To achieve most parts of your proposal, install MacPorts system-wide and 
let the systems administrator compile software every user needs. Now 
every user itself can decide to also install MacPorts on his/her own in 
his/her home directory and install software there. No need to engineer 
new stuff.

I don't think this needs to be synchronized, as I doubt anybody will 
install software at the same time as the system administrator does. Th 
whole point of installing software yourself is that you need another 
version or some other variants. And this synchronization is the only 
point which can't be achieved here. But is it really needed?

My opinion is, what MacPorts would really need is a good API to be used 
by clients. The command line client should use the same API as any GUI 
client would use. This would need separation of interface and logic.

And I see your request was merely a good GUI client which provides 
queuing, growl notifications and all the functionality MacPorts command 
line interface does. No need for a daemon to achieve this.

Rainer