The State of Rust in MacPorts Today

Tue Dec 13 17:32:13 UTC 2022

This is a tangential, so please forgive me if this seems as if it is
the wrong time to bring this up, but I seem to have some Rustaceans
who may know more about this than I.

I was recently seeing if there might be a way to improve upon the
FFmpeg ports (there are currently three: ffmpeg, fffmpeg-devel and
ffmpeg-upstream, though two of them are at the same version presently)
to reduce the number of dependencies. While I was able to make a go of
it successfully and included an attempt at a Portfile here:
https://trac.macports.org/ticket/66424# in the comments Ken suggested
that since the current MacPorts for FFmpeg have rust as a dependency,
that brings in a flood of other dependencies.

To be honest, I am unsure why FFmpeg would require rust (it doesn't in
my builds from upstream's repository if cloning from source nor in my
Portfile using a versioned release), and it seems as if such things
may be better separated into a variant, but even in the occasion where
rust is considered required, does rust really have that many
dependencies? I guess it is the Kolmogorov complexity reduction spirit
in me, but am I crazy for thinking that a dependency audit and
minimizing such things might be worthwhile (probably for more than
merely FFmpeg, perhaps even the rust MacPort itself)?

Thank you for any insights into a rather unrelated matter.

On Tue, Dec 13, 2022 at 5:21 PM Kirill A. Korinsky via macports-dev
<macports-dev at lists.macports.org> wrote:
>
> Chris,
>
> Clearly some thought has to be given to how to ensure the dependency tree does not get too long. We don’t want, when a new OS comes out for it to have to build tens of rust versions, just to ultimately bootstrap the last one. That might just be keeping the first bootstrap port, mrustc new enough at all times such that the list is kept manageable.
>
>
> Unfortunately mrust supports to build rust up to 1.54.
>
> As soon as upstream of mrust is updated compiler to something never, I'll update the port and short the tree.
>
> --
> wbr, Kirill
>
> On 13. Dec 2022, at 18:16, Christopher Jones <jonesc at hep.phy.cam.ac.uk> wrote:
>
> Hi,
>
> On 13 Dec 2022, at 5:07 pm, Kirill A. Korinsky via macports-dev <macports-dev at lists.macports.org> wrote:
>
> David,
>
> the idea is creating a dependency chain:
>
> Port rust (version 1.66) depends on rust-1.65 to be build;
> Port rust-1.65 depends on rust-1.64 to be build;
> Port rust-1.64 depends on rust-1.63 to be build;
> ...
> Port rust-1.56 depends on rust-1.55 to be build;
> Port rust-1.55 depends on rust-1.54 to be build;
> Port rust-1.54 depends on mrsutc to be build.
>
> :)
>
> When someone would like to add rust 1.67, he need to add port rust-1.66 which should be used as bootstrap compiler.
>
> I hate this way, but it is the only way to bootstrap it from scratch.
>
>
> Clearly some thought has to be given to how to ensure the dependency tree does not get too long. We don’t want, when a new OS comes out for it to have to build tens of rust versions, just to ultimately bootstrap the last one. That might just be keeping the first bootstrap port, mrustc new enough at all times such that the list is kept manageable.
>
> Chris
>
>
>
> When mrust had support new rust, we may cut the tree by removing a lot of unused ports.
>
>
> --
> wbr, Kirill
>
> On 13. Dec 2022, at 17:53, David Gilman <davidgilman1 at gmail.com> wrote:
>
> The work on mrustc is novel but I don't think it solves the issues we have here. On modern systems MacPorts uses bootstrap compilers provided by Rust upstream. MCL's bootstrap compilers are for older systems.
>
> To update rust, my understanding is that you have to do the usual work of rebasing patches (my PR), but you also have to provide the binaries for older systems which I could not provide.
>
>
> On Tue, Dec 13, 2022, 11:07 AM Kirill A. Korinsky via macports-dev <macports-dev at lists.macports.org> wrote:
>>
>> Folks,
>>
>> From the third hand we may build our own bootstrap chain of rust from scratch.
>>
>> Or almost.
>>
>> We have a https://ports.macports.org/port/mrustc/details/ which is able to bootstrap 1.54 rust on x86_64 and arm64.
>>
>> Unfortunately support of i386 isn't yet finished at upstream. I plan to fix it, but it requires time and availability of hardware to test it :)
>>
>> I do have a commits which implements rust bootstrap by cahin: mrustc -> rust 1.54 -> rust 1.55 -> rust 1.56; I can start to open PRs to move step-by-step and in month we'll have the last rust via this chain.
>>
>> --
>> wbr, Kirill
>>
>> On 13. Dec 2022, at 16:49, Christopher Jones <jonesc at hep.phy.cam.ac.uk> wrote:
>>
>> Hi,
>>
>> In my opinion, hosting and maintaining these ‘bootstrap’ compilers outside the macports infrastructure was a poor choice, for all the reasons you mention below. I thought this at the time it was done, and even more so now.
>>
>> Personally, I would suggest you think about a change to how the rust compiler is package, to mirror a bit how things are done with gcc and clang. Namely, move to a model where the version is part of the port name, e.g. the current one would be called something like rust-1.61.
>>
>> The main reason for doing this, is adding a new version would that not remove the previous version, and thus you could simple use it as the bootstrap compiler. So with the above, when you add rust-1.62 that would simple configure itself to bootstrap using the macports rust-1.61 port.
>>
>> Yes, this will require some work to set up. You will need to make all the various rust versions installable along side each other, so some tweaking of the install prefix would be needed.
>>
>> One thing I would do differently though to how gcc/clang do things is I would try and have a single rust port file, that implements all the versions as sub-ports. I suspect most of what each needs can then just be shared , such that what needs to be different for each sub-port is actually not that much.
>>
>> Regarding how users of rust then use these ports, there are a couple options
>>
>> 1. Add a shim port ‘rust’ which simply installs sym-links etc. to the ‘current best version’ that mimics the current installation, i.e. in the main prefix. If done well, users should then be blind to the changes above.
>> 2. Users that want an older rust could explicitly depend on and use a specific versioned rust-N
>>
>> For me, this approach makes a lot more sense than the current way these bootstrap compilers are maintained.
>>
>> cheers Chris
>>
>>
>> On 13 Dec 2022, at 2:57 pm, Herby G <herby.gillot at gmail.com> wrote:
>>
>> Hello all,
>>
>> Right now, Rust in MacPorts is severely out of date. It's about 5 versions behind the current release, which at the moment is at 1.65.0. In comparison, MacPorts Rust is currently at 1.61.0.
>>
>> As a core language underlying a lot of other ports, many of these ports cannot be updated to their latest versions because these versions require current versions of Rust. At the time of this writing, 156 ports are being built using Rust ( https://ports.macports.org/port/rust/details/ ), some quite heavily used by the community, including projects like `git-delta`, `bat` and `fd`.
>>
>> MarcusCalhoun-Lopez's PR here ( https://github.com/macports/macports-ports/pull/14277 ) heavily rewrote the Rust port to run on older systems, and was very much celebrated and endorsed. However, as a result of this PR, the Rust port became a lot more complicated, and also introduced a new critical bootstrap compiler (referenced in the Rust portgroup here: https://github.com/macports/macports-ports/blob/2d39b30a32fcf0f5e1cff04f172e9d55ae08ba48/_resources/port1.0/group/rust-1.0.tcl#L140), which is being hosted in MarcusCalhoun-Lopez's personal Github account ( https://github.com/MarcusCalhoun-Lopez/rust/releases ).  Marcus did try to ask about a more official location to host the bootstrap compiler in a macports-dev thread: https://lists.macports.org/pipermail/macports-dev/2022-April/044243.html , but ultimately per the  responses he decided to just host it in his personal account himself.
>>
>> Since this massive change to the Rust port at 1.60.0, it's only seen one update since then to 1.61.0 ( https://github.com/macports/macports-ports/commit/8431ccb48eec4824736eca51f643523356091cd6 )
>>
>> David Gilman opened a PR recently attempting to update Rust to 1.64.0 ( https://github.com/macports/macports-ports/pull/16329 ), but Gilman doesn't have access to update the bootstrap compiler, because as of right now, only MarcusCalhoun-Lopez knows how to build it, and also it's hosted in Calhoun's Github account as mentioned prior.
>>
>> We need to figure out a more sustainable approach for this bootstrap compiler, including how it can be built, and hosting it somewhere where a small set of MacPorts maintainers can build and update it so that we can get MacPorts Rust back on track.  As things are today, only MarcusCalhoun-Lopez has all the pieces required to update this port, and there's been no word from him for months now as the Rust port has fallen further and further behind. Being such a critical core language port, it may make sense to create a repo within the MacPorts Github organization where a set of maintainers can host and update the Rust bootstrap compiler.
>>
>>
>>
>
>
>