Documentation overhaul

[Ryan Schmidt]

> On Oct 10, 2016, at 2:44 PM, Rainer Müller <raimue at macports.org> wrote:
> 
> On 2016-10-10 01:23, Ryan Schmidt wrote:
>> Thanks for your interest, Marcel, but do you feel you would be able
>> to lead such an effort, since you seem to be new to MacPorts? I've
>> been with MacPorts for around 10 years and have thought of rewriting
>> the documentation for several years so I feel I might be able to do
>> it. I'm just not doing it right this second because I'm busy trying
>> to move us to new hosting. And also because we don't yet know how how
>> we will work on GitHub, so we can't fully document that yet.
> 
> Our documentation has been in a dire state for years now. As I mentioned
> before, the new help system was started by me about 7 years ago. Even
> back then we were already discussing to convert/rewrite the guide. Not
> much has changed since then.
> 
> Most of the time the man page project was in limbo because nobody was
> writing the actual man pages. I also lost motivation after a while and
> eventually Clemens picked it up and finished it.
> 
> Now here is Marcel offering his help with the documentation. I do not
> want to turn down his offer or stop him from contributing in any way.
> 
> I do not think a lot of experience is required to write good
> documentation. Especially when we are talking about reorganizing the
> existing documents. Being only a user could help to write the
> documentation from the view of a user without diving too deep into
> internals.
> 
> Moving away from Docbook, converting the existing content or evaluating
> other tools would be a lot of work that could be done by anyone.

Note that the docbook xsl can be automatically converted to asciidoc or other formats using pandoc; that's what I used for a quick test.

In the guide directory, run this script:


#!/bin/bash

OUTPUT_FMT=asciidoc
OUTPUT_EXT=asciidoc
OUTPUT_DIR=asciidoc

mkdir -p $OUTPUT_DIR
find xml -name '*.xml' | xargs -n 1 -I % sh -c "CMD="\""pandoc --from docbook --to $OUTPUT_FMT --standalone --output=\$(echo % | sed -E 's,xml/(.*)\.xml,$OUTPUT_DIR/\1.$OUTPUT_EXT,') %"\"" && echo \$CMD && \$CMD"


But manual cleanup will probably be required.


>> My plan is to read all documentation, maybe even print it all out on
>> paper (guide, wiki, web site), cut it up, group related information
>> together, remove duplication (like the three different sets of
>> install documentation we currently have), remove superfluous
>> language, simplify, and also add new documentation for things that
>> currently aren't documented, like newer portgroups. Also review the
>> organization and content of the documentation of other projects that
>> I consider well done, such as svnbook.org. If you know of other good
>> documentation I should review let me know.
> 
> One of the most important steps would be to split the documentation
> clearly into:
> 
> * Installation
> + MacPorts Quickstart/Cheat Sheet
> * Using MacPorts
> * Writing Portfiles
> * Contributing
> + List of Terms/Glossary
> * Management Policies
> * Internals for base Developers
> 
>  * = already exists
>  + = needs to be written
> 
> Overall:
> make it more concise, less text, examples instead of explanations.
> 
> Maybe split installation instructions by macOS versions so users only
> see relevant instructions.

Yes, I did this for the "new web site" I built a few years back, and intend to use that strategy on whatever new web site we end up with. I intend for this to be on the web site, not in the guide. The guide can contain alternate installation instructions, like building from source.

Or, if we prefer, all installation instructions, even dmg/pkg installation instructions, by OS version, can be in the guide, and the web site can be reduced to just providing links to the downloads and to the installation instructions. That could be fine, and I do like the idea of having all documentation together in one place. In my previous "new web site", I used include files and if/then logic so that any given text only had to be written once. There is overlap in installation instructions for different OS versions, and I would not want to have to manually keep that content synchronized between multiple OS versions. I didn't know whether asciidoc supported include files or conditional logic.


> It would also be good to have some conditionals to mark
> paragraphs/sections by version. Then we could write documentation for
> trunk while the website keeps displaying the stable versions (or let you
> choose).
> 
> To me, svnbook.org is old, outdated and the HTML version on the web
> definitely does not feel modern.

I'm not referring at all to the presentation of the svnbook.org site. I intend to use Twitter Bootstrap for CSS styling across the new web presence.

Rather, I was referring to the content of the book, which I consider to be very well written and well organized, and I'm seeking other examples of well written and well organized project documentation. Come to think of it, I also liked what I read of Apple's Swift 2 book; I think I read the first 2/3 of it. Though, both of those are written to be books, even printed, whereas our documentation could be more page-focused, with shorter pages focused on more specific topics.


Regarding organization, I'm undecided about how to proceed at a high level. Should there be a single guide, that targets everyone: users, port maintainers, committers, base developers? Or should it be split up in some way, so that there is user documentation which is separate from developer documentation?

It's natural for a user to experience a progression from just using MacPorts, to contributing a port update or two, to becoming committer, to hacking on base. I understand that progression, being a programmer, and having gone through it myself, and from that standpoint, it's nice to have all the documentation in one place. On the other hand, there are plenty of people who aren't like me, and who just want to install software, and don't want to learn MacPorts internals, and I don't want to scare them away with documentation that keeps sidetracking them with information they don't require.

Maybe it can be handled on a page by page basis, where each page indicates its audience. For example: a page for users about how to upgrade a port. Then, a link at the bottom to a page for users about how to test and submit an update to a new version of a port. Then, a link at the bottom of that page to a page for developers about how to accept such an update and test and commit it.


>> My plan is to do this after redoing the web site. I intend to keep
>> the installation instructions that have to do with the default
>> installation using the macOS Installer on the web site, so rewriting
>> that will be a good beginning to rewriting the documentation.
> 
> I know you have been working on the new website for quite a while. But
> you also do the buildbot and other management related tasks. I would
> even say chances are high that this will be delayed further...

I did spend time on a "new web site" back in 2012.

I liked several aspects of that site: Twitter Bootstrap styling; more concise text; installation instructions per OS version; a page for each port.

But I built the site using nodejs, and in retrospect, I'm not sure that was the best idea. The rapid pace of node development leads to frequent backwards-incompatible module updates; I doubt the site I developed in 2012 still runs with today's version of node, and I'm not very interested in diving into that to make it work again, and to commit to keeping it working in the future.

I would rather focus efforts on building a site that has similar styling and content, but can be hosted on GitHub Pages; I've come to understand the benefits of the simplicity of just using static web pages, generated by an engine like Jekyll and a template system like Liquid, which is what GitHub Pages uses. I've gained some experience with Jekyll and Liquid, having built several GitHub Pages sites for the recent macOS forge transition. The only difficulty is the ports search feature, but there are some options, including JavaScript search and Google search which I want to explore.


> If Marcel has time right now, I think he should just start with the
> documentation.