TL;DR - I know how to overwrite lines of output normally but none of the methods I've used previously (e.g. printf '\e[1A\e[2K'
) or have found online seem to work when the line being overwritten is longer than width of terminal (e.g. a line-wrap was triggered). Disabling line-wrap effectively truncates the displayed text. Is there some other trick or tool or some way to handle this scenario that I'm missing?
I have a bash script that I share between my desktop (Fedora) and my phone (Termux on Android). Functionality-wise, I don't have any issues and everything works as expected. But script is fairly lengthly and the terminal output is a bloody mess. Recently, I learned that I could use ANSI Escape Codes from bash to overwrite a previous line of output and significantly clean up the output from my script, while still having an idea of its progress and seeing whatever error if it encounters one. My understanding of these is still pretty basic but from my testing it seems that printf
recognizes \e
as the start of an ASCII escape and then based on this, ESC[#A
"moves cursor up # lines" and ESC[2K
"erase the entire line".
Anyway, where I ran into an issue was that I was expecting all but the last line to be overwritten and instead multiple other lines were still being displayed. Initially, I thought this was due to some Termux bug but I later was able to confirm is due to the size (width) of the terminal (I am able to recreate the issue in gnome-terminal
by resizing my window or by increasing the length of the text). Basically, what I am seeing is that if the line of output that I wish to overwrite is longer than the width of the terminal, then it appears that the line "wraps" the remaining text onto a new line and the overwrite only replaces the wrapped portion of the text.
Here is a snippet that recreates the issue I'm having in my script:
# create an array with variable-length texts to simulate status messages
arrStatusTexts=( );
for i in {10..200..10}; do
arrStatusTexts+=("$(printf '%*s\n' $i ' ' | tr ' ' X)");
done
# print out status at each step, overwriting output of each previous step as we go
echo "";
echo "--------------------";
echo "Steps of process"
echo "--------------------";
for ((i=0; i < ${#arrStatusTexts[@]}; i++)); do
[[ 0 != "$i" ]] && printf '\e[1A\e[2K';
printf 'Step %s of %s: %s\n' "$(($i + 1))" "${#arrStatusTexts[@]}" "${arrStatusTexts[$i]}";
done
From my desktop terminal window, before running the above, I get:
$ stty size
34 135
Edit: From both my desktop and from termux, my TERM variable shows as:
$ echo "$TERM"
xterm-256color
What I would like to see as the final output after running the above for loop is something like:
Step 20 of 20: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
What I actually see after for loop completes is:
Step 13 of 20: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Step 14 of 20: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Step 15 of 20: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Step 16 of 20: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Step 17 of 20: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Step 18 of 20: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Step 19 of 20: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Step 20 of 20: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Basically it is overwriting steps 1-12 correctly because those are smaller than the width of my terminal window. But for steps 13 onward, the length of each line "wraps" and only the wrapped portion is cleared.
Edit 2022-Sep-01: Found more oddities. Based on an answers here and here, using the \e[6n
sequence, I was attempting to get the cursor position prior to each printf
statement in order to see if that would somehow be useful.
altering the above to:
# from: https://unix.stackexchange.com/a/183121/379297
function pos () {
local CURPOS
read -sdR -p $'\E[6n' CURPOS
CURPOS=${CURPOS#*[} # Strip decoration characters <ESC>[
echo "${CURPOS}" # Return position in "row;col" format
}
export -f pos;
arrCursorPos=( );
for ((i=0; i < ${#arrStatusTexts[@]}; i++)); do
# get cursor position
arrCursorPos+=("$(pos)");
printf 'Step %s of %s: %s\n' "$(($i + 1))" "${#arrStatusTexts[@]}" "${arrStatusTexts[$i]}";
done
for ((i=0; i < ${#arrCursorPos[@]}; i++)); do
echo "cursor pos $(($i + 1)): '${arrCursorPos[$i]}'";
done
The output I got for the status messages is the same as above but here's the output of the 2nd array which holds cursor positions:
cursor pos 1: '19;1'
cursor pos 2: '20;1'
cursor pos 3: '21;1'
cursor pos 4: '22;1'
cursor pos 5: '23;1'
cursor pos 6: '24;1'
cursor pos 7: '25;1'
cursor pos 8: '26;1'
cursor pos 9: '27;1'
cursor pos 10: '28;1'
cursor pos 11: '29;1'
cursor pos 12: '30;1'
cursor pos 13: '31;1'
cursor pos 14: '33;1'
cursor pos 15: '34;1'
cursor pos 16: '34;1'
cursor pos 17: '34;1'
cursor pos 18: '34;1'
cursor pos 19: '34;1'
cursor pos 20: '34;1'
At first, I thought this was not working correctly because I was getting the same pos for indexes 15-20. After clearing screen (Ctrl+L
) and rerunning a few times I saw different output
cursor pos 1: '11;1'
cursor pos 2: '12;1'
cursor pos 3: '13;1'
cursor pos 4: '14;1'
cursor pos 5: '15;1'
cursor pos 6: '16;1'
cursor pos 7: '17;1'
cursor pos 8: '18;1'
cursor pos 9: '19;1'
cursor pos 10: '20;1'
cursor pos 11: '21;1'
cursor pos 12: '22;1'
cursor pos 13: '23;1'
cursor pos 14: '25;1'
cursor pos 15: '27;1'
cursor pos 16: '29;1'
cursor pos 17: '31;1'
cursor pos 18: '33;1'
cursor pos 19: '34;1'
cursor pos 20: '34;1'
And realized what is going on. For the last few array elements, it is getting to the last row (in my case 34 - as is reported in col1 by stty size
). At that point, any new lines that are output cause the displayed text to scroll and I still end up on the last line (34). So this method does seem like it might be a reliable way of keep track of initial cursor position.
I also tried an alternate approach (suggested here, here, and here involving exec < /dev/tty
and stty
. Using the function here and modifying the snippet to:
function extract_current_cursor_position () {
export $1
exec < /dev/tty
oldstty=$(stty -g)
stty raw -echo min 0
echo -en "\033[6n" > /dev/tty
IFS=';' read -r -d R -a pos
stty $oldstty
eval "$1[0]=$((${pos[0]:2} - 2))"
eval "$1[1]=$((${pos[1]} - 1))"
}
export -f extract_current_cursor_position;
arrCursorPos=( );
for ((i=0; i < ${#arrStatusTexts[@]}; i++)); do
# get cursor position
extract_current_cursor_position pos1
arrCursorPos+=("${pos1[0]} ${pos1[1]}");
printf 'Step %s of %s: %s\n' "$(($i + 1))" "${#arrStatusTexts[@]}" "${arrStatusTexts[$i]}";
done
for ((i=0; i < ${#arrCursorPos[@]}; i++)); do
echo "cursor pos $(($i + 1)): '${arrCursorPos[$i]}'";
done
Rerunning this after clearing screen, I got:
cursor pos 1: '10 0'
cursor pos 2: '11 0'
cursor pos 3: '12 0'
cursor pos 4: '13 0'
cursor pos 5: '14 0'
cursor pos 6: '15 0'
cursor pos 7: '16 0'
cursor pos 8: '17 0'
cursor pos 9: '18 0'
cursor pos 10: '19 0'
cursor pos 11: '20 0'
cursor pos 12: '21 0'
cursor pos 13: '22 0'
cursor pos 14: '24 0'
cursor pos 15: '26 0'
cursor pos 16: '28 0'
cursor pos 17: '30 0'
cursor pos 18: '32 0'
cursor pos 19: '32 0'
cursor pos 20: '32 0'
This one seems like it would work too. Though I am not sure why the extract_current_cursor_position
function subtracts 2 from the y and 1 from the x values. I would probably need to look into that or remove it the subtractions
I still have to look into ncurses options (e.g. tput
). I did already check that the ncurses package is at least available on Termux but I'll come back and fill in more info as I test more.
The obvious changes that I could make are:
1. Don't change my script and just live with the multi-line clutter like I've been doing. But I would strongly prefer to fix my output, unless it becomes too much work.
2. Shorten all the status texts and reduce printing variables in output till everything is smaller than the smallest screen's width (stty size
on my phone reports 17 48
so e.g. limit everything to 48 chars per line). While I don't mind some refactoring and putting in more effort, the idea of doing hundreds of text changes seems very tedious and teaches me nothing about what is actually going on. I would rather save this as a last resort for if there turns out not to be a better way. That and this would probably lose meaningful info or require me making other compromises to how things are displayed.
3. Same as 2 but use print -v msg
to put the output in a variable then print truncated text e.g. printf '%s\n' "${msg:0:48}"
. Same issues as with 2 but I would also likely lose some meaningful info.
4. Similar to above but rather than truncating, just keep track of the previous message's length and divide by terminal width to determine the number of printf '\e[1A\e[2K';
statements I should use. Still a bit of work but less than if I had to edit every message and should give a better end result.
Curious if there is an easier way to go about this that I am missing. Maybe some way way to "grep" text that was already printed and clear to a specific offset from current position (adding a unicode char or asterisk or something to beginning of each status line would be pretty easy search/replace). Or some command or builtin that I am unaware of? My google fu hasn't turned up any obvious solutions besides what I listed above.
I don't need a fully POSIX-compliant solution; just one that works in bash with standard tools (python/perl/awk one-liners are all fair game... but rewriting my entire bash script into one of those is not). Considering this domain usually deals with desktop questions, I'm not expecting familiarity with Termux but if the solution requires a graphical session (e.g. won't work on ssh) or a tool only available on x86_64 architectures (Termux uses ARM builds) then it probably won't work. Most common bash/linux tools seem to work fine there tho. My desktop currently has bash 5.1.8 (and I will be upgrading to Fedora 36 soon) and Termux has the ARM version of bash 5.1.16.
setterm -linewrap off
.tput el
orprintf '\033[K'
?