Maintainer Abuse

[Fred Wright]

On Thu, 29 Dec 2022, Joshua Root wrote:
> On 2022-12-29 15:59 , Fred Wright wrote:
>> 
>> Twice recently I've had changes made to ports I maintain without respecting 
>> the maintainer timeout (and not for any urgent security-related reasons). 
>> The first was py-serial, where the change was merged without waiting for 
>> the maintainer timeout.  And just now I see that someone abused their write 
>> access to bypass the PR mechanism entirely for a gpsd update, so that I 
>> wasn't even notified of the change.  And I've had good reason to hold off 
>> on updating gpsd, due to its missing dependency on asciidoctor, which is 
>> currently broken on some platforms due to the insistence on tying it to a 
>> broken version of ruby, which I've actually been working on fixing.
>> 
>> Is this now the Wild West?

> Sorry you've been put out by these commits. Both of these ports are marked as 
> openmaintainer, which according to the project policy [1] means that minor 
> changes are allowed without obtaining the maintainer's permission first. That 
> certainly isn't carte blanche to do whatever you want, but it does mean that 
> pushing changes directly isn't necessarily against the rules.

My understanding is that only time-critical changes, or stuff in the 
"cleanup" category, should be pushed without maintainer input.

> The definition of a minor update is left somewhat vague, but can probably be 
> thought of as synonymous with low-risk. I would say anything beyond simple 
> bugfixes, and certainly anything that changes the API or ABI, should be run 
> by the maintainer first. And as the policy says, the committer is responsible 
> for ensuring that the changes work properly. If you push a change to someone 
> else's port, you should consider yourself "on the hook" for fixing anything 
> that breaks as a result.

More on "minor" and "low risk" below.

> When in doubt, run it by the maintainer.
>
> I'm not familiar enough with gpsd to say whether the recent update was minor 
> or not. Marius, please work with Fred to resolve any issues that it may have 
> caused.
>
> If the change to py-serial you're referring to was mine of Dec 13, that was 
> part of a mass update to adopt a new feature in MacPorts 2.8.0, which only 
> touched openmaintainer and nomaintainer ports. IMO it was well within the 
> definition of a minor change.

That isn't the change I was referring to (but see below); I was referring 
to 66c7d25d427.  I had no objection to adding the py310 subport (though 
checking still would have been appropriate), but I strongly object to the 
removal of the py34-py36 subports.  And no change that removes a 
capability can be considered "low risk" by any stretch of the imagination.

I'm a firm believer in maximum compatibility, and one of the things I like 
about MacPorts is its support for a wide range of OS versions (though 
adequate testing is often lacking).  But for some reason, that same 
courtesy isn't extended to older Python versions, and some even think that 
it's a good idea to engage in rampages of ethnic cleansing of older Python 
subports.

One shouldn't look to the python.org folks for guidance on this issue; 
those are the same folks that (initially) expected everyone to immediately 
migrate to Python 3, and explicitly discouraged making code simultaneously 
compatible with both Python 2 and Python 3 at all.  But Python versions 
can't be mixed within a single execution of a single program, so a given 
program and all its imported modules have to be compatible with at least 
one Python version.  Similarly, every module needs to have at least one 
Python version compatible with itself and every program that uses it. 
This creates a massive interconnected web of Python code that needs to be 
simultaneously compatible with at least one Python version.  Denying 
simultaneous Python 2/3 compatibility was tantamount to expecting the 
majority of all Python code in the entire world to be converted in 
lockstep from Python 2 to Python 3, which was utterly insane (particularly 
given that a lot of that code is maintained by volunteers 
working in their spare time).

Once the pitchfork-carrying mobs appeared at the gates, the python.org 
folks relented and provided documentation and recommendations for 
"polyglot" code that works with both Python 2 and Python 3.  They also 
extended the support period for Python 2.7, but even the extended period 
was woefully inadequate.  There are substantial incompatibilities between 
Python 2 and Python 3, and it takes real work to fix existing Python 2 
code to handle Python 3.  I know of at least one project that was still 
fixing bugs related to Python 3 many months after dropping support for 
Python 2, so that running it with Python 2 wasn't an available workaround.

Personally, I now try to write all *new* Python code so that it's 
compatible with both, and I've fixed a substantial amount of my existing 
code to do so, but I still have a substantial amount of code that hasn't 
yet been fixed for Python 3, so my default Python is still 2.7.

On another ML, there was a brief discussion related to Python versions on 
Google App Engine.  Someone who was instrumental in the development of 
both Python itself and Google App Engine posted this:
------------------------------------------------------------------------
In 2020 a Googler told me that 2.7 was going to stay "for the foreseeable
future". When the end really comes, I plan to just move to Azure. :-)
------------------------------------------------------------------------

The "newer may be brokener" principle isn't limited to Python 2 vs. Python 
3.  Thanks to the ill-conceived PEP 475, any program expecting a simple, 
straightforward interaction between signals and select() or poll() is 
broken in Python >=3.5.  Even though the rationale for PEP 475 is 
inapplicable to select/poll, its approach was inflicted on them anyway. 
In Python >=3.5, in order for a signal to unblock a select/poll, it needs 
to use either one approach that's somewhat more complicated and also has a 
bug requiring a kludgy workaround that may not be 100% reliable, or 
another approach that's more complicated and poorly documented.  Hence, 
certain programs that don't want to contend with that mess shouldn't use 
anything later than Python 3.4, and Python 3.4 joins Python 2.7 in the 
list of versions that need to be kept around for the forseeable future to 
avoid forcing breakage on users' code.

There may be other such cases; these are just the ones I'm aware of, and 
the python.org folks have a very cavalier attitude toward breaking things. 
The only way to avoid both foreseen and unforeseen problems is to avoid 
removing versions altogether.  Version removal is like a war game - the 
only way to win is not to play.

Some folks like to obsess over avoiding anything that doesn't get 
"security updates".  But this concern is largely overblown, especially in 
the EOL case.  A security fix is only relevant if it applies to a feature 
that a given user or piece of software is actually using.  And for a 
"mature" version (which probably applies to anything EOLed), it's quite 
likely that all the bugs in the widely used features have already been 
discovered, and the remaining ones are in relatively obscure features that 
are far less likely to be used in any given case.

That being said, if anyone wants to avoid EOLed software *themselves*, 
they're perfectly welcome to do so, but they shouldn't be forcing their 
judgment of what's appropriate on everyone else.

Regarding maintainability, a port with significant patches becomes *more* 
maintainable when it's EOLed, since it's no longer necessary to reconcile 
local patches with upstream changes.

Regarding 46ab9a2e798 - it would be best to defer adding 'any' to 
'platforms' anywhere until "port lint" is fixed to stop treating that as 
an error.

> If you would like your permission to be required for all changes to these 
> ports, the openmaintainer tag can be removed from the maintainers option.

That would be a possibility, though I didn't think it was supposed to be 
necessary.

On Thu, 29 Dec 2022, Marius Schamschula wrote:

> As I’m the one to blame for the update to gpsd, let me explain:
>
> 1) gpsd is a dependency of Stellarium which needs a major update. Two 
> maintainers were working on this last night and found that the default 
> dependency tree asked for Python 2.7 and py27-serial (more on that 
> later)

Other variants have been available for quite some time (thanks to Michael
Dickens).

> 2) MacPorts currently supports py37, py38, py39 and py310. As of Sunday, 
> that’ll be py38, py39, 310 and py311. There’s a reason for that: 
> upstream support. Python 2.7 should not be used, never mind, the default 
> anymore.

I completely disagree - see above regarding Python versions.

Although "port upgrade" tries to avoid breaking existing installs by 
preferring current installed variants to new (possibly changed) default 
variants, that only works when the old variants are actually available. 
Removing a variant altogether when it was formerly the default is 
extremely rude.  There needs to be a substantial minimum time (probably 
measured in months) between when a default variant is made not the default 
and when the old default variant is removed (if ever).  And a suitable 
deprecation warning is warranted as well.

> As far as py-serial: the name is confusing.  For a personal project 
> (weewx) I had created a local version (pyserial is the upstream name) as 
> PyPi has a ’serial’ package that does serialization rather than serial 
> port support. I only figured out this redundancy when trying to install 
> gpsd.

I agree that the naming is confusing.  I didn't create the port - I just 
took it over from nomaintainer.  Perhaps someone thought that py-pyserial 
smacked too much of the Department of Redundancy Department.  But changing 
it now probably isn't worth the potential additional confusion.

BTW (as documented in the Portfile comments), gpsd's dependency on 
py-serial is "soft", in the sense that it's mostly fine without it, with 
just a couple of its programs having reduced capability in that case. 
But since py-serial is so lightweight, it didn't seem worth making it a 
variant.

Fred Wright