Package managers and package versioning

[Jason Liu]

On Fri, Oct 2, 2020 at 9:00 PM Ryan Schmidt <ryandesign at macports.org> wrote:

>
> On Oct 2, 2020, at 17:42, Lothar Haeger wrote:
>
>> Instead of creating separate copies of perl for each version, it would've
>> probably been smarter to fix the limitation in MacPorts that made this
>> workaround necessary, i.e. its inability to maintain and install all but
>> the latest version of a port. RPM can do it, DEB can do it, MSI can do it,
>> nothing unusual in the grand scheme of package managers in general to be
>> able to choose a specific version to install. Just MacPorts did not
>> implement it yet and when the necessity arose a seemingly simple workaround
>> was chosen instead of solving the underlying problem.
>>
>
> I have no familiarity with rpm, deb, msi, or other package managers so I
> cannot say whether or how they allow the user to select which version of a
> package to install.
>

Many modern package management systems give both package makers and users
the ability to specify version numbers, both for packages and for
dependencies. I call them "modern", because either they have only existed
for, say, 10 years or less, and were created from the very beginning with
version-choosing capabilities, or if they have been around for around 20
years or more, then they gained the ability one of two ways. Either the
ability was added as part of a complete rewrite of the package management
system, or it was added as a major new feature.

One of the most visible consequences of this ability is demonstrated by how
dependencies are specified. Giving users the ability to choose which
version of a package to install could very easily result in dependency
hell. Thus, to prevent this, dependencies also need to be able to have
version numbers specified:

   - RPM Guide: Specifying the Version of the Dependencies
   <https://docs.fedoraproject.org/en-US/Fedora_Draft_Documentation/0.1/html/RPM_Guide/ch-advanced-packaging.html#id809995>
   - ServerFault: RPM: Set Required: somepackage >= 0.5.0 AND sompackage <
   0.6.0
   <https://serverfault.com/questions/411444/rpm-set-required-somepackage-0-5-0-and-somepackage-0-6-0>

This results in package management systems with spec files that have
dependencies specified like this:

Requires: somepackage >= 0.5.0, somepackage < 0.6.0

I'm speaking of the user being able to specify an arbitrary version. If
> you're instead thinking that the port maintainer would specify a list of
> valid versions or something, that might be more feasible, but still not
> without some of the above problems.
>

APT has the ability to do this. <https://askubuntu.com/a/92021/1047868> APT
is a package management system used by Debian, Ubuntu, and other flavors of
Linux that use .deb-based packages. To repeat what I just linked to, using
apt-get, a user can install a specific version of a package:

apt-get install apache2=2.2.20-1ubuntu1

YUM can also do this.
<https://www.thegeekdiary.com/centos-rhel-how-to-install-a-specific-version-of-rpm-package-using-yum/>
YUM is a package management system used by Red Hat Enterprise Linux,
CentOS, and other flavors of Linux that use RPM-based packages. To repeat
what I just linked to, using yum:

yum install firefox-31.5.3-3.el7_1.x86_64

NPM and Yarn, which are PMSes for JavaScript packages, were created with
package versioning capabilities as a feature right from their very
beginnings:

   - Yarn: Version ranges
   <https://classic.yarnpkg.com/en/docs/dependency-versions/#toc-version-ranges>
   - NPM/Greenkeeper: Introduction to Version Ranges
   <https://blog.greenkeeper.io/introduction-to-version-ranges-e0e8c6c85f0f>
   - NPM: Specifying dependencies and devDependencies in a package.json file
   <https://docs.npmjs.com/specifying-dependencies-and-devdependencies-in-a-package-json-file>

This results in the possibility of restricting dependencies to very
specific version numbers, like this:

1.4.0-1.5.2 || >=1.5.6 <2.4.0 : Matches version 1.4.0 through 1.5.2, and
1.5.7, but not 1.5.3, 1.5.4, or 2.4.0

I can't currently remember which one at the moment, but I seem to vaguely
recall that the syntax for a certain package manager looked something like:

<pms> install apache2 @2.2.20-1ubuntu1

which, it seems, looks very close to a syntax that would fit MacPorts. For
example, perhaps something like this might be a nice syntax if such a
feature were to be added to MacPorts:

port install vlc @5.0.6 +jack +svg +shout

and in portfiles, something like this for dependency version ranges might
make sense:

depends_lib port:boost >=1.67.0 +cmake_scripts +python37|python38
-mpich_devel \
            port:openexr >=1.7.4_3,<2.7.1_2

(this example also includes a capability on my personal wish-list: the
ability to specify a dependency to be installed with/without a certain set
of variants, but without needing to resort to things like using the
active_variants PortGroup or require_active_variants)

As for MacPorts, it's not that we haven't implemented it because we're
> lazy. It's because, besides being an unimaginably large amount of work in
> rearranging our code to do it, I have absolutely no idea how it would be
> accomplished without providing the user with unlimited opportunities to
> create broken combinations of port versions, which would generate an
> unlimited number of bug reports that we would then need to respond to, and
> my goal in MacPorts is to reduce, not increase, the likelihood that users
> would find something broken or need to contact us to help troubleshoot it.
>

I agree, it would be a monumental amount of work. In the worst-case
scenario, it would require a complete rewrite of MacPorts, including
redesigning the entire server infrastructure as well as a top-to-bottom
rewrite of the client software. At a minimum, it would probably require
significantly changing how the MacPorts "port" command works. For example,
I have so far submitted two versions of my MaterialX port: version 1.37.1
and 1.37.2. If I were to do a "port install materialx @1.37.1" (even though
the current version in MacPorts is 1.37.2), then the MacPorts client
software would first need to somehow go out onto the internet and fetch the
portfile for MaterialX version 1.37.1. If you're interested how other PMSes
accomplish this, I could go into further details.

However, I disagree that it would provide unlimited opportunities to create
broken combinations of port versions. This "unlimited"-ness would be
prevented by restricting dependencies to a range of version numbers (or
just a single version number). In reality, it would most likely reduce the
number of bug reports, since you would be able to cap the maximum version
of a dependency that a package was compatible with. Right now, MacPorts
doesn't have that ability.

-- 
Jason Liu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.macports.org/pipermail/macports-dev/attachments/20201003/8428ce46/attachment-0001.htm>