Convert alias documentation to Sphinx/RST

[krader1961]

This is a proof of concept change to implement the documentation using
Sphinx/RST markup. It includes documentation for the alias special
builtin and the basename command builtin. The sole purpose is to
establish that using Sphinx/RST is viable by providing concrete examples
for examination.

I've deliberately chosen to include the documentation build artifacts
in the commit. This means that for now the CI builds, and anyone else
building from source, doesn't need to have Sphinx installed to build the
documentation. This does mean that some care is needed to always run make man and make html before committing any change to the documentation. TBD
is whether a trigger of some sort will be needed to ensure the two sets
of files are kept in sync.

Related #507

[krader1961]

You can see the generated web site here: https://www.skepticism.us/ast/html/

Here is a screenshot of man basename:

Compare the basename DocOpt source fed to optget() with the reStructuredText version:

https://github.com/att/ast/blob/master/src/cmd/ksh93/docs/basename.1
https://raw.githubusercontent.com/krader1961/ast/cmd-documentation/src/cmd/ksh93/docs/basename.rst

[krader1961]

Force pushed an updated commit because I decided not to include the generated HTML; only the man pages for now.

[siteshwar]

This will be a major change after beta release. Shall we cut out separate branch for next stable release ?

[krader1961]

Shall we cut out separate branch for next stable release ?

Yes, that is my preferred practice. Development always happens on master. Stable releases get their own branch. When necessary a fix is applied to master then cherry-picked onto a stable branch.

[siteshwar]

Is it possible to use meson instead of Makefiles ?

[krader1961]

Maybe but probably not worth the trouble. The makefile was autogenerated by the sphinx-quickstart command to set things up. If we decide to generate man files as part of meson install we can just do run_command(['make', 'man']) from a meson.build file.

[siteshwar]

It looks like a step in the right direction.

@kdudka Any thoughts on this ?

[krader1961]

FYI, I've made more changes that support generating cmd --help output from the Sphinx generated man page. Here's an asciinema recording showing the difference between the current optget() based output and the new scheme for alias --help: https://asciinema.org/a/VuN6U9tXviWoZSt7EwLSaioQ1

[krader1961]

@siteshwar Take a look at the diff of the second commit I just pushed. The first is the same as the one you've already reviewed. It's commit ID has changed due to rebasing. The second commit shows where I'm headed. Specifically, replacing optget() with getopt_long() and generating cmd --help output from the Sphinx generated man page rather than a DocOpt style string.

Eventually we should be able to construct the ksh(1) man page from the man pages for each builtin and man pages for the other major subsections. Much like zsh does. Sometimes you want the zshall man page. But lots of time you want something much more focused. Such as the documentation for a specific builtin and don't want to have to wade through tons of irrelevant text.

I'll work on creating a man function that gives priority to the ksh specific man pages. That way you'll be able to type man alias, for example, and see just the man page for the ksh alias builtin.

[krader1961]

I'll work on creating a man function that gives priority to the ksh specific man pages.

Done. If you install from this branch you can type man alias or man basename to view the man pages for those ksh builtins.

[kdudka]

I would prefer not to store generated documentation in git but have it available in release tarballs. It is unclear to me how the commit named Switch b_alias() from optget() to getopt_long() appeared in a pull request named Convert alias documentation to Sphinx/RST. Otherwise looks OK.

[siteshwar]

btw it's causing CI to fail with these messages:

--- stderr ---
<E> builtins[33]: alias should show usage info on unrecognized options
<E> expect: |Usage: alias|
<E> actual: |alias: line 1: _ksh_print_help: not found

alias: Unknown flag '--this-option-does-not-exist'|
<E> builtins[50]: alias --man should show documentation
<E> expect: |NAME alias - |
<E> actual: |alias: line 1: _ksh_print_help: not found alias: Unknown flag '--man' |
<W> builtins[-1]: error_count = 2
-------

[krader1961]

I would prefer not to store generated documentation in git but have it available in release tarballs.

Agreed. The current state of affairs is just a stepping stone intended to make it easier to transition to Sphinx without requiring everyone doing builds to have Sphinx installed. We should probably make Sphinx a hard dependency once all the man pages have been converted but leave it a soft dependency till then.

It is unclear to me how the commit named Switch b_alias() from optget() to getopt_long() appeared in a pull request named Convert alias documentation to Sphinx/RST.

That commit illustrates the logical next step. We don't want three versions of each command's documentation; e.g., in ksh.1, alias.1 and alias.rst. Eliminating the DocOpt text in alias.1 means we can no longer use the AST optget() function. Instead switch to the widely used getopt_long() and show how doing so it is still possible to support alias --help.

btw it's causing CI to fail with these messages

Yep. I was just ignoring those errors until we agreed to move forward with converting the docs to Sphinx and reStructuredText.

[krader1961]

FYI, I haven't merged this because, after fixing the easy test failures, there were two test failures that were inexplicable. At least in as much as they only occurred on Linux but not BSD systems. Too, the error message was inconclusive. The problem turns out to be due to a difference in the behavior of getopt_long(). Specifically, whether global var optind must be reset to one or zero before starting a new arg scan. On BSD systems it is sufficient to assign one to optind. On Linux it must be set to zero. Which is a weak argument for retaining the AST optget() function since it doesn't suffer from that platform specific behavior. But in my opinion the problems with that API exceed the benefits many times over.

[krader1961]

I have merged this after fixing the test failures. It also, as a side-effect, results in the dirs, popd and pushd functions being enabled by default as a consequence of making man a ksh function. See #835. That side-effect is being debated and might result in those specific functions being demoted (i.e., not enabled by default) or removed completely from the project.