140

I would like to remove all leading and trailing spaces and tabs from each line in an output.

Is there a simple tool like trim I could pipe my output into?

Example file:

test space at back 
 test space at front
TAB at end  
    TAB at front
some empty lines with differing TABS and spaces:





 test space at both ends 
  • 1
    To anyone looking here for a solution to remove newlines, that is a different problem. By definition a newline creates a new line of text. Therefore a line of text cannot contain a newline. The question you want to ask is how to remove a newline from the beginning or end of a string: stackoverflow.com/questions/369758, or how to remove blank lines or lines that are just whitespace: serverfault.com/questions/252921 – Tony Jun 25 '18 at 23:24
187
awk '{$1=$1;print}'

or shorter:

awk '{$1=$1};1'

Would trim leading and trailing space or tab characters1 and also squeeze sequences of tabs and spaces into a single space.

That works because when you assign something to one of the fields, awk rebuilds the whole record (as printed by print) by joining all fields ($1, ..., $NF) with OFS (space by default).

1(and possibly other blank characters depending on the locale and the awk implementation)

  • 2
    Semicolon on second example is superfluous. Could use: awk '{$1=$1}1' – Brian Nov 3 '15 at 19:18
  • 7
  • Interesting... No semicolon is supported by gawk, mawk and OS X's awk. (At least for my versions (1.2, 4.1.1, and 20070501, respectively) – Brian Nov 3 '15 at 22:21
  • Although, even though gawk supports it, it confirms the semicolon usage with: Multiple statements may be put on one line by separating them with a ";". This applies to both the statements within the action part of a pattern-action pair (the usual case), and to the pattern-action statements themselves. – Brian Nov 3 '15 at 22:22
  • 2
    echo ' hello ' | xargs – JREAM Apr 3 '18 at 10:32
40

The command can be condensed like so if you're using GNU sed:

$ sed 's/^[ \t]*//;s/[ \t]*$//' < file

Example

Here's the above command in action.

$ echo -e " \t   blahblah  \t  " | sed 's/^[ \t]*//;s/[ \t]*$//'
blahblah

You can use hexdump to confirm that the sed command is stripping the desired characters correctly.

$ echo -e " \t   blahblah  \t  " | sed 's/^[ \t]*//;s/[ \t]*$//' | hexdump -C
00000000  62 6c 61 68 62 6c 61 68  0a                       |blahblah.|
00000009

Character classes

You can also use character class names instead of literally listing the sets like this, [ \t]:

$ sed 's/^[[:blank:]]*//;s/[[:blank:]]*$//' < file

Example

$ echo -e " \t   blahblah  \t  " | sed 's/^[[:blank:]]*//;s/[[:blank:]]*$//'

Most of the GNU tools that make use of regular expressions (regex) support these classes.

 [[:alnum:]]  - [A-Za-z0-9]     Alphanumeric characters
 [[:alpha:]]  - [A-Za-z]        Alphabetic characters
 [[:blank:]]  - [ \x09]         Space or tab characters only
 [[:cntrl:]]  - [\x00-\x19\x7F] Control characters
 [[:digit:]]  - [0-9]           Numeric characters
 [[:graph:]]  - [!-~]           Printable and visible characters
 [[:lower:]]  - [a-z]           Lower-case alphabetic characters
 [[:print:]]  - [ -~]           Printable (non-Control) characters
 [[:punct:]]  - [!-/:-@[-`{-~]  Punctuation characters
 [[:space:]]  - [ \t\v\f]       All whitespace chars
 [[:upper:]]  - [A-Z]           Upper-case alphabetic characters
 [[:xdigit:]] - [0-9a-fA-F]     Hexadecimal digit characters

Using these instead of literal sets always seems like a waste of space, but if you're concerned with your code being portable, or having to deal with alternative character sets (think international), then you'll likely want to use the class names instead.

References

  • Note that [[:space:]] is not equivalent to [ \t] in the general case (unicode, etc). [[:space:]] will probably be much slower (as there are many more types of whitespaces in unicode than just ' ' and '\t'). Same thing for all the others. – Olivier Dulac Nov 21 '13 at 12:44
  • sed 's/^[ \t]*//' is not portable. Atually POSIX even requires that to remove a sequence of space, backslash or t characters, and that's what GNU sed also does when POSIXLY_CORRECT is in the environment. – Stéphane Chazelas Aug 11 '16 at 14:56
  • What if I want to trim newlines characters? '\n \n text \n \n' – Eugene Biryukov Jun 1 '18 at 8:54
  • I like the sed solution because of the lack of other side-affects as in the awk solution. The first variation does not work when I tried it in bash on OSX jsut now, but the character class version does work: sed 's/^[[:blank:]]*//;s/[[:blank:]]*$//' – Tony Jun 25 '18 at 23:13
  • @EugeneBiryukov see my comment on the original post – Tony Jun 25 '18 at 23:27
22

As suggested by Stéphane Chazelas in the accepted answer, you can now
create a script /usr/local/bin/trim:

#!/bin/bash
awk '{$1=$1};1'

and give that file executable rights:

chmod +x /usr/local/bin/trim

Now you can pass every output to trim for example:

cat file | trim

(for the comments below: i used this before: while read i; do echo "$i"; done
which also works fine, but is less performant)

  • 1
    Good luck if your file is huge and/or contains backslashes. – don_crissti Dec 31 '14 at 1:31
  • 1
    @don_crissti: could you comment a bit more?, which solution would be better fitting for huge files, and how could I modify my solution if the file contained backslashes? – rubo77 Dec 31 '14 at 10:42
  • 3
    You'll have to use while read -r line to preserve backslashes and even then.... As to huge files / speed, really, you picked the worst solution. I don't think there's anything worse out there. See the answers on Why is using a shell loop to process text bad practice ? including my comment on the last answer where I added a link to a speed benchmark. The sed answers here are perfectly fine IMO and far better than read. – don_crissti Dec 31 '14 at 12:24
  • @don_crissti ...and/or has lines starting with - and followed by combinations of 1 or more e, E or n characters, and/or contains NUL characters. Also, a non-terminated line after the last newline will be skipped. – Stéphane Chazelas May 27 '15 at 14:52
  • 1
    You can also add an alias in /etc/profile (or your ~/.bashrc or ~/.zshrc etc...) alias trim="awk '{\$1=\$1};1'" – Jeff Clayton Nov 20 '15 at 16:26
21

xargs without arguments do that.

Example:

trimmed_string=$(echo "no_trimmed_string" | xargs) 
  • 1
    This also contracts multiple spaces within a line, which was not requested in the question – roaima Sep 9 '15 at 16:04
  • 1
    @roaima - true but the accepted answer also squeezes spaces (which was not requested in the question). I think the real problem here is that xargs will fail to deliver if the input contains backslashes and single quotes. – don_crissti Sep 9 '15 at 18:28
  • @don_crissti that doesn't mean the accepted answer correctly answers the question as asked, though. But in this case here it wasn't flagged as a caveat whereas in the accepted answer it was. I've hopefully highlighted the fact in case it's of relevance to a future reader. – roaima Sep 9 '15 at 19:22
  • It also breaks on single quotes, double quotes, backslash characters. It also runs one or more echo invocations. Some echo implementations will also process options and/or backslashes... That also only works for single-line input. – Stéphane Chazelas May 21 at 17:19
17
sed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//'

If you're reading a line into a shell variable, read does that already unless instructed otherwise.

  • 1
    +1 for read. So if you pipe to while read it works: cat file | while read i; do echo $i; done – rubo77 Nov 21 '13 at 3:36
  • 1
    @rubo except that in your example the unquoted variable is also reprocessed by the shell. Use echo "$i" to see the true effect of the read – roaima Sep 9 '15 at 19:19
11

If you store lines as variables, you can use bash to do the job:

remove leading whitespace from a string:

shopt -s extglob
echo ${text##+([[:space:]])}

remove trailing whitespace from a string:

shopt -s extglob
echo ${text%%+([[:space:]])}

remove all whitespace from a string:

echo ${text//[[:space:]]}
  • Removing all white-space from a string is not same as removing both leading and trailing spaces (as in question). – catpnosis Mar 24 '18 at 16:04
  • Far the best solution - it requires only bash builtins and no external process forks. – peterh Jul 5 '18 at 13:56
  • 2
    Nice. Scripts run a LOT faster if they don't have to pull in outside programs (such as awk or sed). This works with "modern" (93u+) versions of ksh, as well. – user1683793 Jul 10 '18 at 22:54
8

To remove all leading and trailing spaces from a given line thanks to a 'piped' tool, I can identify 3 different ways which are not completely equivalent. These differences concern the spaces between words of the input line. Depending on the expected behaviour, you'll make your choice.

Examples

To explain the differences, let consider this dummy input line:

"   \t  A   \tB\tC   \t  "

tr

$ echo -e "   \t  A   \tB\tC   \t  " | tr -d "[:blank:]"
ABC

tr is really a simple command. In this case, it deletes any space or tabulation character.

awk

$ echo -e "   \t  A   \tB\tC   \t  " | awk '{$1=$1};1'
A B C

awk deletes leading and tailing spaces and squeezes to a single space every spaces between words.

sed

$ echo -e "   \t  A   \tB\tC   \t  " | sed 's/^[ \t]*//;s/[ \t]*$//'
A       B   C

In this case, sed deletes leading and tailing spaces without touching any spaces between words.

Remark:

In the case of one word per line, tr does the job.

  • None of this trims trailing/leading newlines though – highmaintenance Nov 29 '16 at 12:50
  • +1 for a list of solutions with their (sometimes unexpected) output. – Tony Jun 25 '18 at 23:15
  • @user61382 this is rather late, but see my comment on the original post. – Tony Jun 25 '18 at 23:28
  • @highmaintenance : use [:space:], instead of [:blank:], for the command tr, like: ... | tr -d [:space:], to remove newlines too. (see: man tr) – tron5 Aug 9 at 13:52
6

sed is a great tool for that:

                        # substitute ("s/")
sed 's/^[[:blank:]]*//; # parts of lines that start ("^")  with a space/tab 
     s/[[:blank:]]*$//' # or end ("$") with a space/tab
                        # with nothing (/)

You can use it for your case be either piping in the text, e.g.

<file sed -e 's/^[[...

or by acting on it 'inline' if your sed is the GNU one:

sed -i 's/...' file

but changing the source this way is "dangerous" as it may be unrecoverable when it doesn't work right (or even when it does!), so backup first (or use -i.bak which also has the benefit to be portable to some BSD seds)!

2

translate command would work

cat file | tr -d [:blank:]
  • 4
    This command is not correct as it removes all spaces from the file, not just leading/trailing whitespace. – Brian Redbeard Sep 28 '18 at 16:41
  • @BrianRedbeard You are correct. This is still a useful answer for a monolithic string, without spaces. – Anthony Rutledge May 18 at 23:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.