How to access man pages as structured content

Question

I'm building a resource that references man pages, and I'm wondering if anyone knows of a way to access man pages as structured data? My current approach is to do a lot if REGEXing, but this is tedious and prone to errors.

I'm not an expert on *nix, but what I understand about man pages is that they are basically text files with a particular syntax that is parsable by the man command. This makes me a little skeptical that there might be an easy way to, say, access a list of the options or flags. But maybe there's a way to do it that I don't know.

Answer 1

You might peek at how the fish shell builds its completions from the man pages in particular how __fish_complete_man works. An easier option assuming groff might be to emit HTML and then use one of the multitude of HTML parsers out there to get what you want:

$ groff -T html -mdoc xpquery.1 | xpquery -p HTML '//p[b="xpquery"][2]' -
<p style="margin-left:17%;"><b>xpquery</b>
[<b>−E </b><i>encoding</i>]
[<b>−n </b><i>namespace</i>]
[<b>−p </b><i>method</i>]
[<b>−S </b><i>xpath-subquery</i>]
[<b>−t </b><i>timeout</i>] <i>xpath-query
file-or-url ..</i></p>
$ 

That's a man page rendered as HTML and then selected on using XPath to obtain the list of flags in the SYNOPSIS section; using CSS selectors might be more hip these days. However, the HTML generated is not very structured.

Answer 2

Man pages are roff formatted text files, which unfortunately is one of the most irritating to parse formatting languages around right now. You may have some luck pulling out what you want by looking through the documentation for whatever roff implementation your system uses (it will almost always be one of nroff, troff, or groff). The format itself is somewhat complicated (it was designed in an era when minimalism was the epitome of good software design), but the structure of most man pages is reasonably consistent, so it shouldn't be too hard.

In addition to that, you have a couple of other options. The first is to look at the sources for the man pages in the software's source distribution. Most software these days compiles their man pages to the roff sources parsed by the man program from some other format, most of which are far easier to parse than roff macros. The other is to look into tools (including groff) which convert man pages to other formats, as suggested in thrig's answer.

Answer 3

Check out manServer which is a perl script from 1999 that parses man pages in a simple way and presents them as web pages. It has all the perl code you need to interpret the troff source.