3

TL;DR - I know how to overwrite lines of output normally but none of the methods I've used previously (e.g. printf '\e[1A\e[2K') or have found online seem to work when the line being overwritten is longer than width of terminal (e.g. a line-wrap was triggered). Disabling line-wrap effectively truncates the displayed text. Is there some other trick or tool or some way to handle this scenario that I'm missing?

I have a bash script that I share between my desktop (Fedora) and my phone (Termux on Android). Functionality-wise, I don't have any issues and everything works as expected. But script is fairly lengthly and the terminal output is a bloody mess. Recently, I learned that I could use ANSI Escape Codes from bash to overwrite a previous line of output and significantly clean up the output from my script, while still having an idea of its progress and seeing whatever error if it encounters one. My understanding of these is still pretty basic but from my testing it seems that printf recognizes \e as the start of an ASCII escape and then based on this, ESC[#A "moves cursor up # lines" and ESC[2K "erase the entire line".

Anyway, where I ran into an issue was that I was expecting all but the last line to be overwritten and instead multiple other lines were still being displayed. Initially, I thought this was due to some Termux bug but I later was able to confirm is due to the size (width) of the terminal (I am able to recreate the issue in gnome-terminal by resizing my window or by increasing the length of the text). Basically, what I am seeing is that if the line of output that I wish to overwrite is longer than the width of the terminal, then it appears that the line "wraps" the remaining text onto a new line and the overwrite only replaces the wrapped portion of the text.

Here is a snippet that recreates the issue I'm having in my script:

# create an array with variable-length texts to simulate status messages
arrStatusTexts=(  );
for i in {10..200..10}; do
    arrStatusTexts+=("$(printf '%*s\n' $i ' ' | tr ' ' X)");
done
 
# print out status at each step, overwriting output of each previous step as we go
echo "";
echo "--------------------";
echo "Steps of process"
echo "--------------------";
for ((i=0; i < ${#arrStatusTexts[@]}; i++)); do
    [[ 0 != "$i" ]] && printf '\e[1A\e[2K';
    printf 'Step %s of %s: %s\n' "$(($i + 1))" "${#arrStatusTexts[@]}" "${arrStatusTexts[$i]}";
done

From my desktop terminal window, before running the above, I get:

$ stty size
34 135

Edit: From both my desktop and from termux, my TERM variable shows as:

$ echo "$TERM"
xterm-256color

What I would like to see as the final output after running the above for loop is something like:

Step 20 of 20: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

What I actually see after for loop completes is:

Step 13 of 20: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Step 14 of 20: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Step 15 of 20: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Step 16 of 20: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Step 17 of 20: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Step 18 of 20: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Step 19 of 20: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Step 20 of 20: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Basically it is overwriting steps 1-12 correctly because those are smaller than the width of my terminal window. But for steps 13 onward, the length of each line "wraps" and only the wrapped portion is cleared.


Edit 2022-Sep-01: Found more oddities. Based on an answers here and here, using the \e[6n sequence, I was attempting to get the cursor position prior to each printf statement in order to see if that would somehow be useful.

altering the above to:

# from: https://unix.stackexchange.com/a/183121/379297
function pos () {
    local CURPOS
    read -sdR -p $'\E[6n' CURPOS
    CURPOS=${CURPOS#*[} # Strip decoration characters <ESC>[
    echo "${CURPOS}"    # Return position in "row;col" format
}
export -f pos;
arrCursorPos=(  );
for ((i=0; i < ${#arrStatusTexts[@]}; i++)); do
    # get cursor position
    arrCursorPos+=("$(pos)");
    printf 'Step %s of %s: %s\n' "$(($i + 1))" "${#arrStatusTexts[@]}" "${arrStatusTexts[$i]}";
done
for ((i=0; i < ${#arrCursorPos[@]}; i++)); do
    echo "cursor pos $(($i + 1)): '${arrCursorPos[$i]}'";
done

The output I got for the status messages is the same as above but here's the output of the 2nd array which holds cursor positions:

cursor pos 1: '19;1'
cursor pos 2: '20;1'
cursor pos 3: '21;1'
cursor pos 4: '22;1'
cursor pos 5: '23;1'
cursor pos 6: '24;1'
cursor pos 7: '25;1'
cursor pos 8: '26;1'
cursor pos 9: '27;1'
cursor pos 10: '28;1'
cursor pos 11: '29;1'
cursor pos 12: '30;1'
cursor pos 13: '31;1'
cursor pos 14: '33;1'
cursor pos 15: '34;1'
cursor pos 16: '34;1'
cursor pos 17: '34;1'
cursor pos 18: '34;1'
cursor pos 19: '34;1'
cursor pos 20: '34;1'

At first, I thought this was not working correctly because I was getting the same pos for indexes 15-20. After clearing screen (Ctrl+L) and rerunning a few times I saw different output

cursor pos 1: '11;1'
cursor pos 2: '12;1'
cursor pos 3: '13;1'
cursor pos 4: '14;1'
cursor pos 5: '15;1'
cursor pos 6: '16;1'
cursor pos 7: '17;1'
cursor pos 8: '18;1'
cursor pos 9: '19;1'
cursor pos 10: '20;1'
cursor pos 11: '21;1'
cursor pos 12: '22;1'
cursor pos 13: '23;1'
cursor pos 14: '25;1'
cursor pos 15: '27;1'
cursor pos 16: '29;1'
cursor pos 17: '31;1'
cursor pos 18: '33;1'
cursor pos 19: '34;1'
cursor pos 20: '34;1'

And realized what is going on. For the last few array elements, it is getting to the last row (in my case 34 - as is reported in col1 by stty size). At that point, any new lines that are output cause the displayed text to scroll and I still end up on the last line (34). So this method does seem like it might be a reliable way of keep track of initial cursor position.

I also tried an alternate approach (suggested here, here, and here involving exec < /dev/tty and stty. Using the function here and modifying the snippet to:

function extract_current_cursor_position () {
    export $1
    exec < /dev/tty
    oldstty=$(stty -g)
    stty raw -echo min 0
    echo -en "\033[6n" > /dev/tty
    IFS=';' read -r -d R -a pos
    stty $oldstty
    eval "$1[0]=$((${pos[0]:2} - 2))"
    eval "$1[1]=$((${pos[1]} - 1))"
}
export -f extract_current_cursor_position;
arrCursorPos=(  );
for ((i=0; i < ${#arrStatusTexts[@]}; i++)); do
    # get cursor position
    extract_current_cursor_position pos1
    arrCursorPos+=("${pos1[0]} ${pos1[1]}");
    printf 'Step %s of %s: %s\n' "$(($i + 1))" "${#arrStatusTexts[@]}" "${arrStatusTexts[$i]}";
done
for ((i=0; i < ${#arrCursorPos[@]}; i++)); do
    echo "cursor pos $(($i + 1)): '${arrCursorPos[$i]}'";
done

Rerunning this after clearing screen, I got:

cursor pos 1: '10 0'
cursor pos 2: '11 0'
cursor pos 3: '12 0'
cursor pos 4: '13 0'
cursor pos 5: '14 0'
cursor pos 6: '15 0'
cursor pos 7: '16 0'
cursor pos 8: '17 0'
cursor pos 9: '18 0'
cursor pos 10: '19 0'
cursor pos 11: '20 0'
cursor pos 12: '21 0'
cursor pos 13: '22 0'
cursor pos 14: '24 0'
cursor pos 15: '26 0'
cursor pos 16: '28 0'
cursor pos 17: '30 0'
cursor pos 18: '32 0'
cursor pos 19: '32 0'
cursor pos 20: '32 0'

This one seems like it would work too. Though I am not sure why the extract_current_cursor_position function subtracts 2 from the y and 1 from the x values. I would probably need to look into that or remove it the subtractions

I still have to look into ncurses options (e.g. tput). I did already check that the ncurses package is at least available on Termux but I'll come back and fill in more info as I test more.


The obvious changes that I could make are:

1. Don't change my script and just live with the multi-line clutter like I've been doing. But I would strongly prefer to fix my output, unless it becomes too much work.

2. Shorten all the status texts and reduce printing variables in output till everything is smaller than the smallest screen's width (stty size on my phone reports 17 48 so e.g. limit everything to 48 chars per line). While I don't mind some refactoring and putting in more effort, the idea of doing hundreds of text changes seems very tedious and teaches me nothing about what is actually going on. I would rather save this as a last resort for if there turns out not to be a better way. That and this would probably lose meaningful info or require me making other compromises to how things are displayed.

3. Same as 2 but use print -v msg to put the output in a variable then print truncated text e.g. printf '%s\n' "${msg:0:48}". Same issues as with 2 but I would also likely lose some meaningful info.

4. Similar to above but rather than truncating, just keep track of the previous message's length and divide by terminal width to determine the number of printf '\e[1A\e[2K'; statements I should use. Still a bit of work but less than if I had to edit every message and should give a better end result.

Curious if there is an easier way to go about this that I am missing. Maybe some way way to "grep" text that was already printed and clear to a specific offset from current position (adding a unicode char or asterisk or something to beginning of each status line would be pretty easy search/replace). Or some command or builtin that I am unaware of? My google fu hasn't turned up any obvious solutions besides what I listed above.

I don't need a fully POSIX-compliant solution; just one that works in bash with standard tools (python/perl/awk one-liners are all fair game... but rewriting my entire bash script into one of those is not). Considering this domain usually deals with desktop questions, I'm not expecting familiarity with Termux but if the solution requires a graphical session (e.g. won't work on ssh) or a tool only available on x86_64 architectures (Termux uses ARM builds) then it probably won't work. Most common bash/linux tools seem to work fine there tho. My desktop currently has bash 5.1.8 (and I will be upgrading to Fedora 36 soon) and Termux has the ARM version of bash 5.1.16.

8
  • 3
    "TBH, my code is a mess and refactoring everything while still making it intelligible would be a lot of work." -- I stopped reading here. If you're not willing to put in the work, why are you asking us to? Commented Aug 31, 2022 at 20:45
  • 1
    "… print truncated text …" – Maybe you can do this easily: setterm -linewrap off. Commented Aug 31, 2022 at 21:00
  • @glennjackman I admit I phrased that poorly. But I never said I was not willing to put ANY effort in. I have spent quite a bit of time researching and testing this already before even posting here. I rephrased my text above to hopefully reflect that the issue is not that I am unwilling to work at it still, but rather that I don't like the idea of losing info and making hundreds of tedious text changes where I learn nothing without first investigating if there is a better way. If you don't want to help that's your call. But there is a technical question independent from the state of my code.
    – zpangwin
    Commented Aug 31, 2022 at 23:04
  • @KamilMaciorowski that seems to work both on my desktop and in termux, provided that one doesn't mind the current line not showing that text that would have been wrapped (e.g. effectively truncated in terms of what you can see). I'll keep playing with it and see if I can get one that handles while still displaying but appreciate you pointing me at that
    – zpangwin
    Commented Aug 31, 2022 at 23:33
  • 1
    It looks like you're trying to reinvent terminfo and possibly (n)curses. For "erase to end of line", which is easier: tput el or printf '\033[K'? Commented Sep 1, 2022 at 22:31

3 Answers 3

2

About half-way through writing a solution to your question I realised that this is why (n)curses terminal virtualisation is so much easier: it hides all the nasty terminal dependant details (well most of them, well some of them). Using terminfo directly is painful - but better than raw escape sequences.

This is bash code. It shouldn't be hard to rewrite as sh though.

# Output a printf style format string and arguments and return the cursor
# to the beginning of the line. DO NOT use newline `\n`.
#
lineOut() {
    local rows cols len lines

    # Number of rows/columns on the terminal device
    rows=$(tput lines)
    cols=$(tput cols)

    # Output
    printf "$@"

    # How many lines we wrote
    len=$(printf "$@" | unexpand -a | wc -m)
    lines=$(( len / cols ))

    if tput am
    then
        # Cursor does not wrap when writing to the last column
        len=$(( len - (cols * lines) ))
        [[ $len -eq 0 ]] && (( lines-- ))
    fi

    # Move up the necessary number of lines to column 1
    printf '\r'
    for (( ; lines > 0; lines-- )); do tput cuu1; done
}


# Populate the arrStatusTexts (demo only) to simulate status messages
arrStatusTexts=()
for i in {10..200..10}; do
    arrStatusTexts+=("$(printf '%*s\n' "$i" ' ' | tr ' ' X)")
done

# Output
for ((i=0; i < ${#arrStatusTexts[@]}; i++)); do
    lineOut 'Step %s of %s: %s' "$(($i + 1))" "${#arrStatusTexts[@]}" "${arrStatusTexts[$i]}";
    sleep 1
done
printf '.\n'

The terminfo codes used by tput are documented in man 5 terminfo. Running with a terminal type that does not have the necessary sets of control codes (try export TERM=dumb, for example) the solution degrades cleanly.


While working on a solution I found u7 (user7), which happens to trigger the "What is your cursor position?" question of the terminal:

# Magic to read the current cursor position (origin 1,1)
tput u7; read -t1 -srd'[' _; IFS=';' read -t1 -srd'R' y x

It's no longer relevant to the solution proposed here but it may come in useful as a bonus.

3
  • Thanks! This is almost exactly what I was looking for, so I am happy to be the dummy per our previous discussion lol ;-) That manpage link you gave has a lot of good info not in man tput too (I hadn't even seen the tput am or tput cuu1 options before). There are a few tweaks I'll have to make, for example, if I break from the loop early due to error condition or some such, it was overwriting the message from the loop so I might need to add more newlines before printing errors. But that wasn't in-scope for my question anyway so this is great. Thanks again!
    – zpangwin
    Commented Sep 3, 2022 at 5:13
  • @zpangwin if you start doing a lot of cursor movements consider switching it off while moving (civis and cnorm) to remove flicker Commented Sep 3, 2022 at 7:07
  • 1
    Dude. DUDE. Your "Magic to read the current cursor position (origin 1,1)" just earned you a beer if you're ever out in Kansas City. No, it's NOT relevant to OP's question. Which works out fine; HIS question was only tangentially relevant to MY question: "How the heck do I read the bloody CURRENT CURSOR POSITION!?" (and I'll totally cop to the fact that the tack I'd BEEN taking wasn't even kinda sorta remotely close). Good gods man, thank you for thinking to include that little afterthought!
    – NerdyDeeds
    Commented Jan 28, 2023 at 18:03
0

One method is to record where the cursor was, then jump back to that point as need be. This could be done manually with escape sequence wrangling, but that's more work. This method may not play well with scrolling text, though Curses does have a scrollok call to influence what happens when something prints off the end of the window. Probably depends on what you want to see in the scrollback verses simply having text in a non-scrolled window; maybe play around with the alternate screen to show one thing there and something else afterwards?

#!/usr/bin/env perl
use strict;
use warnings;
# or you could do it the hard way with XTerm Control Sequences: ask the
# terminal where the cursor is, manually parse the result, etc
use Curses;
initscr;

my ( $y, $x );    # where we started printing from to jump back to

move int rand(4), 0;    # put the cursor somewhere (start point)
getyx $y, $x;           # record this

my $i = 0;
while ( ++$i < 10 ) {
    clrtobot;       # maybe subsequent messages are shorter? clear all below
    addstring $i x ( $COLS + int rand 8 );    # noise that wraps a line
    refresh;
    sleep 1;
    move $y, $x;                              # back to where we started from
}

endwin;
4
  • For the record/replay cursor pos, I've made several attempts from bash but haven't had any luck thus far. Not sure if using wrong escapes or there's some other issue like scrollback / script vs interactive terminal / etc (debug values I've printed seem to always have the same position before/after). I've been continuing to experiment and now have a few long-winded bash functions that work but require keeping track of previous input string or \# of lines (after being wrapped). If I could figure out how to mark / recall based on cursor coordinates in bash, that would probably work better
    – zpangwin
    Commented Sep 1, 2022 at 17:27
  • As far as the perl code you provided.. Unfortunately, I'm not very proficient with perl outside of simple one-liners. Maybe I need to learn perl but some parts I don't really get, even after reading thru a few times and searching a couple things. So I'm still largely lost. Not even really clear if this is intended as a demo or a callable script but I'm guessing more of a demo bc of things like rand and sleep and it not seeming to take any passed args. But as it is, I really have no idea which parts are key or how to convert it to a perl one-liner / etc that I could call from my bash script
    – zpangwin
    Commented Sep 1, 2022 at 17:38
  • At least without curses, this is never going to work. (I can't say for certain whether it would work with curses; it's possible.) But using raw escape sequences, the problem is that you can only query things like cursor position in relation to the physical window coordinates, and both wrapping and scrolling throw that off.
    – FeRD
    Commented Sep 30, 2023 at 23:42
  • For example, say I have an 80x24 terminal window. I grab the starting cursor position, output a line of 120 character text (without newline), and then grab the ending cursor position. If I'm on the first line of the terminal, $start will be 1, 1, and $end will be 2, 41. So far, so good. But if I'm on line 24, $start will be 24, 1, and $end will be 24, 41. The fact that the terminal wrapped and scrolled between those two measurements, moving the first set of coordinates up the screen, gets lost.
    – FeRD
    Commented Sep 30, 2023 at 23:46
0

An old question but.... If combine the \r already mentioned with printf's %s precision options gives you a space paded fixed width field that you can overwrite as much as you like

This is not how I'd do it, but you specifically wanted something that didn't require lots of updates to your code. I'd actually generally do a "msg()" function that wrote trimmed output to the screen, and untrimmed to a log.. but for discussions sake:

cols=$( tput cols )
eol=$'\r'
printf() {
    command printf "%-${cols}.${cols}s${eol:-\n}" "$( command printf "${1%\\n}" "${@:2}" )"
}

# print out status at each step, overwriting output of each previous step as we go
echo "";
echo "--------------------";
echo "Steps of process"
echo "--------------------";
echo
for ((i=0; i < ${#arrStatusTexts[@]}; i++)); do

    eol=$'\r'
    (( i % 5 )) || eol=$'\n'
    printf 'Step %s of %s: %s\n' "$(($i + 1))" "${#arrStatusTexts[@]}" "${arrStatusTexts[$i]}";
done
echo

Given you can get the screen rows and columns easily enough, you could just print to $(( $rows - 2 )) or whatever too...

#print_at [col] [row] [normal printf params...]
printf_at() {
    printf "\E[%d;%dH" "$1" "$2"
    shift 2
    eval "printf \"$@\""
}

As safer version, which requires preformating the string

# printf_at [col] [row] "$( printf fmt vars.... )"
printf_at() {
    printf "\E[%d;%dH%b" "$1" "$2" "${@:2}"
}

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .