Dependencies on non-default variants

[Craig Treleaven]

> On Apr 2, 2019, at 5:40 PM, Ryan Schmidt <ryandesign at macports.org> wrote:
> 
> On Mar 25, 2019, at 08:48, Craig Treleaven wrote:
> 
>> On Mar 24, 2019, at 4:05 PM, Mojca Miklavec wrote:
>> 
>>> I now checked the first two random MythTV ports, which basically boils down to:
>>> 
>>> require_active_variants qt5-mysql-plugin mariadb55
>>> require_active_variants p${perl5.major}-dbd-mysql mariadb
>>> require_active_variants ${pymodver}-mysql mariadb55
>>> 
>>> out of which only the second one provides the wrong default variant,
>>> and that port doesn't have the maintainer. Neither do any of the mysql
>>> packages have any maintainer at the moment.
>>> 
>>> My first question is: why exactly does p5-dbd-mysql need a different
>>> default variant?
>> 
>> Myth requires a database backend and I’ve chosen mariadb 5.5.  I was pushed to add variants for mysql and different db versions.  That would have been a nightmare to support.  The perl dbd-mysql modules have to know where to find the db socket (AIUI) and therefore need a different variant depending on which database/version they are connecting to.  I’ve documented the required variants in the cookbook instructions on the MythTV wiki.
>> 
>> If I change the default variant for p5-dbd-mysql to suit me, I just push the problem to someone else.  I have no idea how many others are using this port and depend on it connecting to MySQL 5.7 by default.
> 
> It seems arbitrary and unnecessary for the MythTV ports to be imposing a requirement on MariaDB 5.5. If MythTV works with other versions or forks of MySQL, it should allow the user to choose that. It would not be surprising if the user would need to configure MythTV to tell it what socket path to use, and the MythTV ports might want to give the user some guidance on that, for example in the port notes ("port notes php73-mysql" shows an example of that) or in the wiki page.
> 
This is water long under the bridge.  Over the years, I’ve answered hundreds of emails and postings about MythTV on Mac.  I have been the de facto Mac support person for the project for a long time.  A significant fraction of the issues have dealt with database problems.  While allowing multiple databases would be a good theory it would have increased the support load…on me.  I selfishly chose to standardize on one database at a particular version.  Truth be told, I made the similar choice for a web server (apache).  

> Ideally, all of the ports that support MySQL would have the same set of variants, to allow the user to choose which version of which fork of MySQL to use, and we would decide on the same default MySQL variant to use in all those ports. When Oracle bought MySQL, some Linux distributions switched their default MySQL fork to MariaDB, and I think MacPorts wanted to do that as well. But a problem occurred when MariaDB 10.0 was released and the mariadb-10.0 port was created, which is that we could not decide what the variant name for using that version of MariaDB should be. For all prior versions of MySQL and its forks, the variant name was the same as the port name (e.g. "mysql57") but a dash is not legal in variant names, so "mariadb-10.0" could not be used as a variant name. (MacPorts would interpret that as "enable the variant 'mariadb' and disable the variant '10.0'".) "mariadb10.0" couldn't be used either because periods were not legal in variant names either. And "mariadb100" looks strange; the user likely does not know that they are supposed to infer a period between the two zeroes. As of MacPorts 2.5.0 periods are allowed in variant names now, so the variant name "mariadb10.0" is now legal and someone who cares about MySQL could pursue updating the variants of all the MySQL-using ports.
> 

Strangely enough, I filed enhancement tickets 5 years ago suggesting that database variants be harmonized for a couple of ports that you maintain:

https://trac.macports.org/ticket/44484
https://trac.macports.org/ticket/44481

> 
>> More generally, a relatively common issue on MacPorts is X11 versus Quartz. Say the support library (such as gtk2) defaults to one (like +x11) but your package really needs the other.  Changing the default messes up others.  BTW, this would be a compelling use of installation statistics.  If we determined that, say, 80% of gtk2 installs are +quartz, then it would be a no-brainer to change the default.
> 
> We still recommend that the user should decide whether to use +x11 or +quartz before installing any ports; if the user has already installed ports, the user should uninstall them if switching from +x11 to +quartz or vice-versa. There's currently no functionality in MacPorts base that would automate uninstalling and reinstalling all ports. So I don't see how we could switch the default variants of ports from +x11 to +quartz without breaking existing users' installations.
> 
In my made-up example, 80% of users had the non-default variant selected.  At some point, would not the convenience of the great majority outweigh some easily fixed breakage for the minority of users?
 
> 
>> I believe there is a very, very old ticket on this issue but I can’t find it right now.
> 
> Do you mean https://trac.macports.org/ticket/126 ? I'm on record there and elsewhere opposing adding support for dependencies on variants. I understand how in extremely limited specific cases it could be useful, but I don't see how it can work if we consider the problem just a little further.
> 
> Currently, mythtv-core.28 requires p5.26-dbd-mysql with the mariadb variant. If the user does not already have p5.26-dbd-mysql installed, we could possibly improve MacPorts so that it automatically installs p5.26-dbd-mysql with the mariadb variant when mythtv-core.28 is installed. But what if the user has already installed p5.26-dbd-mysql with a different variant such as mariadb10_2? Presumably they did so for a reason: Presumably they already have a MariaDB 10.2 server set up and they want to use it from Perl. If they now try to install mythtv-core.28, they can't, unless they forgo being able to access their MariaDB 10.2 server from Perl 5.26. That restriction shouldn't exist.
> 
Seems like a use for the interactive feature added to base a few years ago.  If the user has a conflicting variant installed, we could explain the situation and ask the user’s decision.  By ‘explain’, we could check if the conflicting variant is used by another active and requested port and show  what might be broken by switching to the now-desired variant.

> 
>>>> And if we can implement a workaround, why can’t we share it with this other page?
>>> 
>>> Because I don't want to implement "build is both working and broken at
>>> the same time" functionality in buildbot.
>>> 
>> I never said to report that the build is “working”.  
>> 
>> I suggested that we report that the build was not attempted which is exactly what happened.
> 
> Currently, when our buildbot encounters a port that requires dependencies with non-default variants, what happens is that a build for that port is scheduled, and then that build fails because the dependencies are not installed with the required variants. I agree with Mojca that we should not spend a great deal of time and effort implementing special support in a web app for identifying and reporting this specific type of failure.
> 
> Instead, we might implement a solution similar to what we already do for graveyard and obsolete ports, which is that we don't schedule a build for those ports at all. Difficulty #1 would be in extracting the list of required and forbidden variants from the Portfile: Since there is no MacPorts base support for this, they would be expressed in terms of the active_variants 1.1 portgroup; we may have to parse the Portfile code directly. Difficulty #2 would be in identifying in advance which variants the dependencies would be installed with.
> 
> More generally, I have previously argued that MacPorts should have a keyword that can be set from Portfiles or portgroups to indicate that a port is expected to fail to build; the port or portgroup could set this conditionally based on whatever criteria it needs to (OS version, Xcode version, variants of dependencies, etc.). If we had this, it would be trivial for the buildbot to check that variable before scheduling a build. For some reason, others didn't seem to like this proposal and I don't think any code was written for it.

Yes, I have saved emails from August 2016 where there was discussion of exactly these issues.  Sometimes we seem to get bogged down waiting for years for a pure and elegant solution when a simple and expedient solution would solve much of the pain right now.  OTOH, I don’t have the time or skills to contribute to base so it is easy for me to lob in suggestions.

Craig