GSOC Ideas
Mojca Miklavec
mojca at macports.org
Fri Feb 8 12:34:24 UTC 2019
Dear Marcus,
Thank you very much for the extensive list of fresh ideas :)
Just a few general explanations.
One general rule is that the suggested ideas should be something that
a mentor is able to implement himself in cca. 3 weeks. There might be
exceptions if you meet a really brilliant student (but the program
should also be able to work for students with basic programming
skills, who never worked on any big project before, who have never
seen or heard about Tcl before, who have never been using MacPorts
before). One of the super important goals of the program is to
introduce students to opensource and the project in particular, and
keep them engaged with the project. If the tasks are too challenging
for the students, they would likely drop out of the program and never
return. If we give them something that they can comprehend, they are
more likely to stick with the project.
I'm not too familiar with the base development, so in many cases I'm
unable to judge how much effort a certain task would be, I just wanted
to point out that we should not ask students to solve problems which
we are not able to solve ourselves. If the student gets stuck at some
point, we definitely need to be able to help them continue. I hope
that some others will provide further feedback about the suggested
projects for the base.
Also, in the past we were often stuck with the task being to complex,
then maybe implemented 60-80%, never rigorously tested, and
consequently never merged. This is bad and demotivating for both the
student and the project. I would say that ideally the first official
merge of the student's code should not be later than after 4 weeks, so
that it can get proper testing etc.
On Thu, 7 Feb 2019 at 14:12, Marcus Calhoun-Lopez wrote:
>
> [1] https://trac.macports.org/wiki/SummerOfCode#Projects
>
> 1. Update Orphaned Ports
> I originally became interested in MacPorts development because I wanted to install certain software, and the MacPorts version was (slightly) out of date.
Same for me. Maybe we should stress that out more clearly on the website.
> Fixing and/or updating ports is an excellent introduction to the MacPorts systems (phases, variants, PortGroups, etc.).
> It forces you to read the documentation (but only as an absolute last resort) and examine commit histories.
> This would not be the entire GSOC project, but it could be used as part of the application process and/or a supplement to the main project.
Agreed. Actually, I would list this under "ideas for ports" section.
For some super complex software, or for stuff like "setting up the
whole ecosystem to allow reasonable packaging of npm or ruby packages"
or some other less trivial piece of software that could be a full-time
work for the summer.
> 2. Tweak qt5 PG
> Currently, the qt5 PG defines three procedures to declare dependencies on Qt components
> (https://github.com/macports/macports-ports/blob/000ec3f12e02aa6c3159bd1608cd8607eb9e18f0/_resources/port1.0/group/qt5-1.0.tcl#L572).
> These should almost certainly be `options`.
I agree that there's a lot in Qt5 that could be done, and it could
certainly fill the whole summer. Or KDE.
> 3. Make blacklisting MacPorts compilers easier.
> Currently, the compiler_blacklist_versions PG makes blacklisting ranges of compilers very easy, but it only works for Xcode compilers.
> It would be nice, for example, to be able to have something like `compiler.blacklist-append {macports-clang < 6.0}`.
> See https://trac.macports.org/ticket/56093.
Agreed. Maybe not enough work for the full summer(?), but it could be
one part of the task.
> 4. Add support for x86_64h architecture.
> x86_64h is a lot like x86_64, except it allows compiler to take advantage of the Haswell microarchitecture.
> Unlike other architectures in MacPorts, x86_64h and x86_64 libraries can be linked together.
> As a side note, GCC does not support `-arch x86_64h`, but Xcode and MacPort Clang do.
> See https://github.com/openssl/openssl/pull/6497 for a discussion on why OpenSSL did NOT want to add support for x86_64h.
This is completely out of my own expertise, so I cannot comment about it.
(I've never seen anyone explicitly requesting this, but I also have
absolutely no idea how much performance improvement this could bring.)
> 5. Allow for multiple runs of each phase.
> There are times when it is convenient to run configure/build/destroot more than once.
> For example:
> *) cargo depends on the port cargo-stage1, whose only purpose is to help build cargo.
> Instead, one could run configure/build and use the resulting binary without installing another port.
> *) fluidsynth depends on a subproject, which has no use beyond aiding the build process.
> The subproject does not inherit the MacPorts settings.
> It would be nice to configure/build the subproject properly (from a MacPorts point of view).
> See https://github.com/macports/macports-ports/pull/3583.
> *) 75% of the muniversal PG is to get multiple runs of the configure/build/destroot phases.
This would be super useful for cross-compilers. For MinGW32-w64 we now
ship three versions of gcc (two of them are bootstrapping versions
that need to be thrown away) and in fact we have a cyclic dependency.
However in that case one first needs to (a) build and activate a
working bootstrapping compiler, (b) install some other packages [crt,
newlib. etc.], (c) build and activate the compiler again, (d) install
more packages, (e) build and activate the final version of the
compiler. I suspect this use case is more complicated than the one you
described above.
Again, I have absolutely no idea how much work is needed and whether
this can realistically be done by the student.
> 6. Allow a variant to more elegantly become “undefaulted.”
> When it was decided that support for the FLTK toolkit should no longer be the default in octave, the (previously default) variant fltk was renamed fltk_toolkit.
> There is another port (I cannot remember which one) that hacks together another solution to avoid renaming a previously default variant.
> Is there a better solution?
This would be an awesome feature. I don't know how much work is needed
/ how difficult it is to implement.
> 7. Find a way to make mass revision increases easier.
> There are times when upgrading a library means increasing the revision of ALL of its dependencies (e.g. readline, icu, and libgcc).
> Finding all of these ports can be tricky and error prone.
> -) Some ports do not have an explicit dependency.
> -) Some ports do not explicitly set a revision variable.
> -) If there are sub-ports, then there might be multiple revision variables.
> Personally, I have a script that partially automates this process, but it is far from ideal.
> Is there a better way?
This would be a super useful feature, but do you know how you would
implement it yourself in a way that is not so far from ideal (assuming
you had sufficient amount of time)?
> 8. Find a solution to the longstanding issue of std::locale being broken in GCC.
> See https://trac.macports.org/ticket/15653
This is completely outside of my expertise, so I cannot comment on it.
> 9. Prevent `port reclaim` from removing build dependencies.
> See https://lists.macports.org/pipermail/macports-users/2019-January/thread.html#46344
This sounds doable by the student. (Maybe not on the super top of our
priority list, but still useful.)
Btw: Homebrew now runs the cleanup by default after each port
installation. I would say that for MacPorts this is somewhat too
dangerous, in particular on the plenty of cases where we don't provide
the binaries.
What I would also like to see is that after uninstalling a particular
port or upgrading a port that now requires different dependencies,
MacPorts would tell you that you can probably remove some other
unrequested dependencies. I don't know how difficult that would be to
implement.
> 10. Improve the finding of dependencies.
> Finding dependencies can sometimes be difficult:
> *) Read the project documentation, which is sometimes incomplete.
> *) Read the code, but dependencies can be quite well hidden within the build system or the code itself.
> *) Run the port command in trace mode, which is excellent, but not 100%.
> *) Wait for the bug reports.
> Can this be improved?
> See https://trac.macports.org/wiki/SummerOfCode#dependencies-gen.
The project description on the wiki sounds ok to me. (I wouldn't know
how to mentor or implement it myself, but that's a different topic.)
Clemens apparently had a very clear idea how to do it (it's outside of
my expertise).
> 11.Give Portfile better access to CFLAGS, CXXFLAGS, etc.
> Currently, CFLAGS and the like are set late in the process (arch flags added, -pipe added, etc.).
> Of course, there are good reasons for this.
> Unfortunately, this means that there is no Portfile access to these values.
> This causes some ugly workarounds such as duplication of code from the base or `configure.cmd printenv` and `configure.post_args {>> Makefile.macports.inc}`.
> Is there a way to give the Portfile access to most of these values?
> See https://lists.macports.org/pipermail/macports-dev/2018-November/thread.html#39694
> See https://github.com/macports/macports-base/blob/5e17a692f991a55a7f450e40ce47ab6fe0f2da57/src/port1.0/portconfigure.tcl#L1367
[unable to comment]
> 12. Simplify compilers and mpi PGs with recent base changes.
> Recently, significant changes were made in the base code concerning compiler selection.
> Is there a way to use these changes to simplify the PortGroups that manage compiler selection via variants?
> See https://github.com/macports/macports-base/pull/88
> See https://trac.macports.org/ticket/55139
I cannot judge the complexity and suitability of this one either.
> 13. Buildbot & Continuous Integration Improvements
> I am afraid these are well outside of my knowledge area, but here are a few issues that have arisen lately.
> *) https://lists.macports.org/pipermail/macports-dev/2018-November/039651.html
How is this related to our CI? It's just that the user was surprised
to see that his commit did not end up in his MacPorts installation,
but that has more to do with our release cycles that our CI.
> *) https://lists.macports.org/pipermail/macports-dev/2018-April/thread.html#38023
> *) https://lists.macports.org/pipermail/macports-dev/2018-July/thread.html#39148
This is one of the most burning issue in our CI.
The idea could be addressed (improving our CI infrastructure to
process PRs on buildbot all the way down to 10.6 or even 10.5/10.4),
but it would have to be a brilliant student. It's a bad idea to ask a
student to do something for which you have no idea how to implement
yourself, unless you get someone outstanding.
Zero King was an example of a super skilled student (we knew him from
before GSOC) and we tried to ask him to implement testing of pull
requests on our own infrastructure instead of working on Travis, but
he was afraid of the zillion (mostly security-related) issues related
with that, and went with Travis (for which we are really thankful,
even if not perfect, as it still simplified our PR processing a lot).
It could be that someone would step up and set up virtualisation of
multiple OS versions in a few days, then do some minor fixes in the
buildbot configuration to fire up those VMs with libvirt; something
that might take 2 weeks to finish. I would put this idea on the page,
albeit with a warning sign that only skilled students should attempt
to do it.
> *) https://lists.macports.org/pipermail/macports-dev/2019-February/040070.html
The student cannot be responsible for fixing the lack of disk space on
the machine in Ryan's cellar :)
OK, Ryan did recently suggest some ideas for auto-cleanup, but I'm not
even sure that we know outselves what would be the best solution. I'm
not sure if it's a good idea to ask a student to work on this.
Mojca
More information about the macports-dev
mailing list