Non-printing escape sequence: when?

Question

Recently, I have been playing around a lot with color in the terminal and, therefore, with escape sequences, too. I've read the relevant parts of the Bash manpage along with numerous helpful pages on the Net.

I've got most of what I want working; a nice color Bash prompt, for example. That said, I am still somewhat confused over when I should use (or need to use) the "non-printing escape sequence" characters. Those would be \[ and \].

If I don't use them in PS1 when defining my prompt then my prompt most definitely does not display properly. If I do use them, everything is fine. Good.

But, outside of PS1, they don't seem to operate the same way. For example, to make scripts more readable I have defined a variable $RGB_PURPLE which is set via a simple function c8_rgb(). The end result is that the variable contains the value \[\e[01;38;05;129m\] which turns on a bold purple foreground color.

When I use this variable in PS1, it does what I expect. If I use it via printf or echo -e it "half" works. The command printf "${RGB_PURPLE}TEST${COLOR_CLR}\n" (where COLOR_CLR is the escape sequence to reset text properties) results in the following display: \[\]TEST\[\] where everything except the first \[ and final \] are displayed in purple.

Why the difference? Why are these brackets printed instead of being processed by the terminal? I would have expected them to be treated the same when printed as part of the prompt as when printed by other means. I don't understand the change.

It seems, empirically, that these characters must be used inside the prompt definition, while they shouldn't be used in pretty much every other case. This makes it difficult to use a common function, like my c8_rgb() function mentioned above, to handle escape sequence generation and output since the function cannot know whether its result will be in a prompt config or someplace else.

And a quick related question: are echo -e and printf essentially the same with regards to outputting escape sequences? I typically use printf, but is there any reason to favor one over the other?

Can anyone explain this apparent subtle difference? Are there any other oddities I should be aware of when using escape sequences (usually just for color) in the terminal? Thanks!

Answer 1

The "non-printing escape sequence" is needed when using non-printing characters in $PS1 because bash needs to know the position of the cursor so that the screen can be updated correctly when you edit the command line. Bash does that by counting the number of characters in the $PS1 prompt and then that's the column number the cursor is in.

However, if you place non-printing sequences in $PS1, then that count is wrong and the line can be messed up if you edit the command line. Hence the introduction of the \[ and \] markers to indicate that the enclosed bytes should not be counted.

Answer 2

I found a sufficient solution by using octal escape sequences throughout, and by using the readline forms of the delimiter characters. PS1, echo -e and printf all do the right thing with exactly the same strings this way. I tested this in Bash 4 on OSX and Bash 3 on Linux.

I got the idea for \001 and \002 from an answer here: https://superuser.com/questions/301353/escape-non-printing-characters-in-a-function-for-a-bash-prompt?rq=1

Example:

$ TextGreen='\001\033[0;32m\002'
$ TextReset='\001\033[0m\002'
$ SomeString="${TextGreen}prompt_stuph${TextReset} \$ "
$ #
$ # now all of the following work as expected
$ #
$ PS1="$SomeString"
$ printf "$SomeString"
$ echo -e "$SomeString"

Answer 3

Something like ␛[01;38;05;129m, where the first character ␛ is the ASCII escape character (U+0027), is a terminal escape sequence. It instructs the terminal to start displaying bold, blinking text in color 129. \e is bash syntax for the escape character (inside $'…', in PS1, in echo -e and in printf).

\[ and \] are not terminal escape sequences, they're bash prompt escape sequences. They're interpreted by bash, not sent to the terminal. Their purpose is to tell bash that what's in between are non-printing characters, so that the width of the prompt is really the number of characters not inside \[…\]. Bash needs to know the width of the prompt to calculate the cursor position in the line editor.

Your function should not output \[…\] if it's meant to be used outside prompt strings. Include \[ and \] directly in the prompt string.