19

I'm running into weird behavior when trying to grep a man page on macOS. For example, the Bash man page clearly has an occurrence of the string NAME:

$ man bash | head -5 | tail -1
NAME

And if I grep for name I do get results, but if I grep for NAME I don't:

$ man bash | grep 'NAME'
$ man bash | grep NAME

I've tried other uppercase words that I know are in there, and searching for SHELL yields nothing whereas searching for BASH yields results.

What's going on here?

Update: Thanks for all the answers! I thought it worth adding the context in which I ran into this. I wanted to write a bash function to wrap man and in cases where I've tried to look up the man page for a shell builtin, jump to the relevant section of the Bash man page. There might be a better way, but here's what I've got currently:

man () {
  case "$(type -t "$1")" in
    builtin)
      local pattern="^ *$1"

      if bashdoc_match "$pattern \+[-[]"; then
        command man bash | less --pattern="$pattern +[-[]"
      elif bashdoc_match "$pattern\b"; then
        command man bash | less --pattern="$pattern[[:>:]]"
      else
        command man bash
      fi
      ;;
    keyword)
      command man bash | less --hilite-search --pattern='^SHELL GRAMMAR$'
      ;;
    *)
      command man "$@"
      ;;
  esac
}

bashdoc_match() {
  command man bash | col -b | grep -l "$1" > /dev/null
}
  • What operating system are you using? I'm sure the accepted answer is correct but IO couldn't reproduce this on my Arch Linux box. man bash | grep NAME works as expected. – terdon Jun 15 '17 at 14:40
  • @terdon I'm on macOS. I get this behavior with Bash 3.2 and 4.4.5 – ivan Jun 15 '17 at 16:10
  • Just as an aside: if you detect a builtin, you can just use the bash help command to get its information. – Joe Jun 16 '17 at 23:50
  • @Joe The problem is I often find the help results leave too much out. Check out help complete vs the complete section in man bash, for instance. – ivan Jun 17 '17 at 2:43
33

If you add a | sed -n l to that tail command, to show non-printable characters, you'll probably see something like:

N\bNA\bAM\bME\bE

That is, each character is written as X Backspace X. On modern terminals, the character ends up being written over itself (as Backspace aka BS aka \b aka ^H is the character that moves the cursor one column to the left) with no difference. But in ancient tele-typewriters, that would cause the character to appear in bold as it gets twice as much ink.

Still, pagers like more/less do understand that format to mean bold, so that's still what roff does to output bold text.

Some man implementations would call roff in a way that those sequences are not used (or internally call col -b -p -x to strip them like in the case of the man-db implementation (unless the MAN_KEEP_FORMATTING environment variable is set)), and don't invoke a pager when they detect the output is not going to a terminal (so man bash | grep NAME would work there), but not yours.

You can use col -b to remove those sequences (there are other types (_ BS X) as well for underline).

For systems using GNU roff (like GNU or FreeBSD), you can avoid those sequences being used in the first place by making sure the -c -b -u options are passed to grotty, for instance by making sure the -P-cbu options is passed to groff.

For instance by creating a wrapper script called groff containing:

#! /bin/sh -
exec /usr/bin/groff -P-cbu "$@"

That you put ahead of /usr/bin/groff in $PATH.

With macOS' man (also using GNU roff), you can create a man-no-overstrike.conf with:

NROFF /usr/bin/groff -mandoc -Tutf8 -P-cbu

And call man as:

man -C man-no-overstrike.conf bash | grep NAME

Still with GNU roff, if you set the GROFF_SGR environment variable (or don't set the GROFF_NO_SGR variable depending on how the defaults have been set at compile time), then grotty (as long as it's not passed the -c option) will use ANSI SGR terminal escape sequences instead of those BS tricks for character attributes. less understand them when called with the -R option.

FreeBSD's man calls grotty with the -c option unless you're asking for colours by setting the MANCOLOR variable (in which case -c is not passed to grotty and grotty reverts to the default of using ANSI SGR escape sequences there).

MANCOLOR=1 man bash | grep NAME

will work there.

On Debian, GROFF_SGR is not the default. If you do:

GROFF_SGR=1 man bash | grep NAME

however, because man's stdout is not a terminal, it takes it upon itself to also pass a GROFF_NO_SGR variable to grotty (I suppose so it can use col -bpx to strip the BS sequences as col doesn't know how to strip the SGR sequences, even though it still does it with MAN_KEEP_FORMATTING) which overrides our GROFF_SGR. You can do instead:

GROFF_SGR=1 MANPAGER='grep NAME' man bash

(in a terminal) to have the SGR escape sequences.

That time, you'll notice that some of those NAMEs do appear in bold on the terminal (and in a less -R pager). If you feed the output to sed -n l (MANPAGER='sed -n /NAME/l'), you'll see something like:

\033[1mNAME\033[0m$

Where \e[1m is the sequence to enable bold in ANSI compatible terminals, and \e[0m the sequence to revert all SGR attributes to the default.

On that text grep NAME works as that text does contain NAME, but you could still have problems if looking for text where only parts of it is in bold/underline...

  • 2
    Wow, pretty interesting to see the legacy of physical tele-type there. Twice as much ink => bold. Makes perfect sense – ivan Jun 14 '17 at 12:28
  • 1
    I'm loving sed -n l as a substitute for od. – Tom Hale Aug 20 '17 at 6:35
13

If you look at any manual page, you'll notice that the headers are in bold. This is achieved through formatting them with control characters. To be able to grep like you're wanting to, these have to be stripped out.

The col utility may be used for this:

$ man bash | col -b | grep 'NAME'

The -b option has the following description on OpenBSD:

Do not output any backspaces, printing only the last character written to each column position. This can be useful in processing the output of mandoc(1).


Linux the col manual (on Ubuntu) doesn't have the last sentence in there (but it works in the same way).

On Linux, unsetting the MAN_KEEP_FORMATTING environment variable (or setting it to an empty string) may also help, and will allow you to grep without passing the output of man through col -b.

  • I think (as in I tested this on an Arch and an Ubuntu system) that on Linux this isn't necessary, or not anymore. On both systems, the NAME in the bash manual is just NAME, no \b. – terdon Jun 15 '17 at 16:14
  • @terdon I didn't spot the mentioning of macOS first, so I assumed a wrongly configured Linux system was a possibility. I have now trimmed off the Linux bits. – Kusalananda Jun 15 '17 at 16:18
  • You missed nothing, I asked the OP what OS they're using because I couldn't reproduce on Linux, they said macOS and I just added it now. And I wasn't implying you were wrong, for all I know there are Linux distributions out there where the MAN_KEEP_FORMATTING variable works exactly as you say it does. I just wanted to point out that that isn't the case always. – terdon Jun 15 '17 at 16:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.