2

I am looking for a way to find and remove the common initial whitespace from a text stream or file. I do not want to remove all leading whitespace (that would be a simple sed 's/^[[:space:]]*//'). Only the amount that is common to all but blank lines.

Example:

printf '  some text\n\n   some more text\n' | the_awesome_command_or_script

should print

some text

 some more text

Progress:

I know that one can use tools like awk or a shell while loop to first loop over all lines and count the initial whitespace and then one can delete the whitespace with a dynamically generated sed command.

The awk script to count the whitespace could look similar to this

awk 'BEGIN { amount = 0 }
     /^[^[:space:]]/ { print 0; exit }
     /^$/{ next }
     /^[[:space:]]/ { amount = match($0, "[^[:space:]]") - 1 }
     END { print amount }'

But then I need a temp file and my script would look like this:

generate_some_text | cat > tempfile
amount=$(above_awk_script < tempfile)
sed "s/^[[:space:]]\{$amount\}//" < tempfile
rm tempfile

Questions:

Is there a tool that is better suited for that job? Can I modify the script to get rid of the tempfile?

Reality:

I am trying to improve my mailcap entry for text/html if copiousoutput is requested: Currently it is text/html; elinks -no-home -dump %s; nametemplate=%s.html; copiousoutput; but as you might have guessed I want to get rid of some initial whitespace. Maybe I am just thinking overly complicated and there is a really simple solution for that?

6
  • You have to do two passes on your input file anyway since you first need to compare all lines to find the one with minimal leading blank characters and then remove that amount to all lines. So you have to either rely on a temporary file or "multiplex" your pipe to be able to read twice the same thing from it.
    – lgeorget
    Apr 27 '16 at 11:02
  • I know tee and the zsh option multios which can multiplex pipes for me. But how can I get $amount from one "end" of this "pipe tree" to the sed at the other "end"? As far as I know the shell evals the line as a whole and I will never get variables that I assign in one subshell into the main shell.
    – Lucas
    Apr 27 '16 at 11:28
  • 1
    What if a line has one leading tab and another line has three leading spaces ? Is one tab less "white space" than three spaces or is it the other way around ? Apr 27 '16 at 11:33
  • 1
    @don_crissti maybe pipe it through expand -i -t 4 (or -t 8 or whatever) to convert initial tabs to a consistent number of spaces first? or maybe unexpand --first-only -t4 or similar to convert initial spaces to tabs.
    – cas
    Apr 27 '16 at 11:58
  • don_chrissti: The current implementation I give above treats all ws chars equally. One could adopt the awk script to handle one tab like eight spaces or whatnot. But that is not the important point for my "reality" use case. Let's assume that we don't have "mixed indent" so either they are all spaces or all tabs. EDIT: But now that cas has solved this problem via a simple prepossessing this assumption is not necessary any longer.
    – Lucas
    Apr 27 '16 at 12:02
2

Since it's either all spaces or all tabs you could pipe it to

sed 'H;$!d;g;: m;/\n[^\n[:blank:]]/!s/\n[^\n]/\n/g;t m;s/.//'

That's gnu sed (I don't think other seds support [\n]). It works by appending each line to the Hold buffer and then deleting it if it's not the last one ($!). On the last line it copies the hold space content over the pattern space via g (the content of the pattern space starts with a \newline now).
It then deletes the first character on each line (s/\n[^\n]/\n/g) if no line in the pattern space starts with a non-blank (/\n[^\n[:blank:]]/!). After each successful substitution it branches back to label m. If there's at least one line in the pattern space that starts with a non-blank it just removes the leading newline from the pattern space (s/.//) and then auto-prints it.

0

if you are only concerned with the first line of the output then limit what sed looks at by only addressing line 1:

printf ' some text\n\n some more text\n' |sed '1s/^[ \t]*\([^ \t]\+.*\)$/\1/'

this will ignore any white space at the beginning then match from anything that isn't white space plus the rest of the line and only on line 1.

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.