394

I would like to remove all leading and trailing spaces and tabs from each line in an output.

Is there a simple tool like trim I could pipe my output into?

Example file:

test space at back 
 test space at front
TAB at end  
    TAB at front
sequence of some    space in the middle
some empty lines with differing TABS and spaces:





 test space at both ends 
1
  • 11
    To anyone looking here for a solution to remove newlines, that is a different problem. By definition a newline creates a new line of text. Therefore a line of text cannot contain a newline. The question you want to ask is how to remove a newline from the beginning or end of a string: stackoverflow.com/questions/369758, or how to remove blank lines or lines that are just whitespace: serverfault.com/questions/252921
    – Tony
    Commented Jun 25, 2018 at 23:24

21 Answers 21

500
awk '{$1=$1;print}'

or shorter:

awk '{$1=$1};1'

Would trim leading and trailing space or tab characters1 and also squeeze sequences of tabs and spaces into a single space.

That works because when you assign something to any one field and then try to access the whole record ($0, the thing print prints be default), awk needs to rebuild that record by joining all fields ($1, ..., $NF) with OFS (space by default).

To also remove blank lines, change it to awk 'NF{$1=$1;print}' (where NF as a condition selects the records for which the Number of Fields is non-zero). Do not do awk '$1=$1' as sometimes suggested as that would also remove lines whose first field is any representation of 0 supported by awk (0, 00, -0e+12...)


¹ and possibly other blank characters depending on the locale and the awk implementation

18
  • 4
    Semicolon on second example is superfluous. Could use: awk '{$1=$1}1'
    – Brian
    Commented Nov 3, 2015 at 19:18
  • 23
    @Brian, no, the ; is required in the standard awk syntax Commented Nov 3, 2015 at 22:07
  • 8
    The only thing I don't like about this approach is that you lose repeating spaces within the line. For example, echo -e 'foo \t bar' | awk '{$1=$1};1' Commented Jun 23, 2017 at 1:12
  • 9
    echo ' hello ' | xargs
    – JREAM
    Commented Apr 3, 2018 at 10:32
  • 1
    @pmor, the file or stream to process must be fed as input to awk. Like with awk '{$1=$1};1' < 1.txt or awk '{$1=$1};1' 1.txt. Commented Jul 28, 2022 at 16:14
115

The command can be condensed like so if you're using GNU sed:

$ sed 's/^[ \t]*//;s/[ \t]*$//' < file

Example

Here's the above command in action.

$ echo -e " \t   blahblah  \t  " | sed 's/^[ \t]*//;s/[ \t]*$//'
blahblah

You can use hexdump to confirm that the sed command is stripping the desired characters correctly.

$ echo -e " \t   blahblah  \t  " | sed 's/^[ \t]*//;s/[ \t]*$//' | hexdump -C
00000000  62 6c 61 68 62 6c 61 68  0a                       |blahblah.|
00000009

Character classes

You can also use character class names instead of literally listing the sets like this, [ \t]:

$ sed 's/^[[:blank:]]*//;s/[[:blank:]]*$//' < file

Example

$ echo -e " \t   blahblah  \t  " | sed 's/^[[:blank:]]*//;s/[[:blank:]]*$//'

Most of the GNU tools that make use of regular expressions (regex) support these classes (here with their equivalent in the typical C locale of an ASCII-based system (and there only)).

 [[:alnum:]]  - [A-Za-z0-9]     Alphanumeric characters
 [[:alpha:]]  - [A-Za-z]        Alphabetic characters
 [[:blank:]]  - [ \t]           Space or tab characters only
 [[:cntrl:]]  - [\x00-\x1F\x7F] Control characters
 [[:digit:]]  - [0-9]           Numeric characters
 [[:graph:]]  - [!-~]           Printable and visible characters
 [[:lower:]]  - [a-z]           Lower-case alphabetic characters
 [[:print:]]  - [ -~]           Printable (non-Control) characters
 [[:punct:]]  - [!-/:-@[-`{-~]  Punctuation characters
 [[:space:]]  - [ \t\v\f\n\r]   All whitespace chars
 [[:upper:]]  - [A-Z]           Upper-case alphabetic characters
 [[:xdigit:]] - [0-9a-fA-F]     Hexadecimal digit characters

Using these instead of literal sets always seems like a waste of space, but if you're concerned with your code being portable, or having to deal with alternative character sets (think international), then you'll likely want to use the class names instead.

References

10
  • Note that [[:space:]] is not equivalent to [ \t] in the general case (unicode, etc). [[:space:]] will probably be much slower (as there are many more types of whitespaces in unicode than just ' ' and '\t'). Same thing for all the others. Commented Nov 21, 2013 at 12:44
  • 1
    sed 's/^[ \t]*//' is not portable. Atually POSIX even requires that to remove a sequence of space, backslash or t characters, and that's what GNU sed also does when POSIXLY_CORRECT is in the environment. Commented Aug 11, 2016 at 14:56
  • 2
    I like the sed solution because of the lack of other side-affects as in the awk solution. The first variation does not work when I tried it in bash on OSX jsut now, but the character class version does work: sed 's/^[[:blank:]]*//;s/[[:blank:]]*$//'
    – Tony
    Commented Jun 25, 2018 at 23:13
  • 1
    instead of [ \t]* why don't you use \s* to catch all white-spaces ?
    – Noam Manos
    Commented Feb 19, 2020 at 12:55
  • 1
    @MichaelK escaping that + works - s,^[[:space:]]\+,,. If you tell sed to do extended regex, -r it'll work w/ sed -r 's,^[[:space:]]+,,'. BTW I typically pipe this to cat -A to see the output more clearly.
    – slm
    Commented Sep 8, 2023 at 3:11
70

xargs without arguments do that.

Example:

trimmed_string=$(echo "no_trimmed_string" | xargs) 
8
  • 20
    This also contracts multiple spaces within a line, which was not requested in the question Commented Sep 9, 2015 at 16:04
  • 5
    @roaima - true but the accepted answer also squeezes spaces (which was not requested in the question). I think the real problem here is that xargs will fail to deliver if the input contains backslashes and single quotes. Commented Sep 9, 2015 at 18:28
  • 2
    @don_crissti that doesn't mean the accepted answer correctly answers the question as asked, though. But in this case here it wasn't flagged as a caveat whereas in the accepted answer it was. I've hopefully highlighted the fact in case it's of relevance to a future reader. Commented Sep 9, 2015 at 19:22
  • 3
    It also breaks on single quotes, double quotes, backslash characters. It also runs one or more echo invocations. Some echo implementations will also process options and/or backslashes... That also only works for single-line input. Commented May 21, 2019 at 17:19
  • 1
    This is a clever (unorthodox) answer! I agree w/ @StéphaneChazelas, essentially invoking the default xargs /bin/echo command.
    – John Doe
    Commented Aug 18, 2021 at 13:40
42

As suggested by Stéphane Chazelas in the accepted answer, you can now
create a script /usr/local/bin/trim:

#!/bin/bash
awk '{$1=$1};1'

and give that file executable rights:

chmod +x /usr/local/bin/trim

Now you can pass every output to trim for example:

cat file | trim

(for the comments below: i used this before: while read i; do echo "$i"; done
which also works fine, but is less performant)

13
  • 2
    @don_crissti: could you comment a bit more?, which solution would be better fitting for huge files, and how could I modify my solution if the file contained backslashes?
    – rubo77
    Commented Dec 31, 2014 at 10:42
  • 4
    You'll have to use while read -r line to preserve backslashes and even then.... As to huge files / speed, really, you picked the worst solution. I don't think there's anything worse out there. See the answers on Why is using a shell loop to process text bad practice ? including my comment on the last answer where I added a link to a speed benchmark. The sed answers here are perfectly fine IMO and far better than read. Commented Dec 31, 2014 at 12:24
  • 3
    You can also add an alias in /etc/profile (or your ~/.bashrc or ~/.zshrc etc...) alias trim="awk '{\$1=\$1};1'" Commented Nov 20, 2015 at 16:26
  • 3
    No need for bash, you can make it #! /usr/bin/awk -f {$1=$1};1. (beware of file names containing = characters though) Commented Aug 11, 2016 at 14:45
  • 2
    note that it has to be on 2 lines, one for the she-bang, one for the code ({$1=$1};1). Commented Aug 11, 2016 at 15:37
37

If you store lines as variables, you can use bash to do the job:

remove leading whitespace from a string:

shopt -s extglob
printf '%s\n' "${text##+([[:space:]])}"

remove trailing whitespace from a string:

shopt -s extglob
printf '%s\n' "${text%%+([[:space:]])}"

remove all whitespace from a string:

printf '%s\n' "${text//[[:space:]]}"
5
  • 3
    Removing all white-space from a string is not same as removing both leading and trailing spaces (as in question).
    – catpnosis
    Commented Mar 24, 2018 at 16:04
  • 5
    Far the best solution - it requires only bash builtins and no external process forks.
    – peterh
    Commented Jul 5, 2018 at 13:56
  • 2
    Nice. Scripts run a LOT faster if they don't have to pull in outside programs (such as awk or sed). This works with "modern" (93u+) versions of ksh, as well. Commented Jul 10, 2018 at 22:54
  • 1
    Upon testing, your ltrim and rtrm solutions do not work because they are too greedy. Given a string such as ` \n\n\n\v\f\t\t\r\r Anthony Rutledge. \n\r\v\\f\n\n\n\t\t\t `, your solutions will wipe away the entire string. Commented Jul 26, 2021 at 14:40
  • The first two methods can move the leading or trailing spaces but not at the same time, the third method will remove all spaces, which is not expected.
    – Gary Wang
    Commented May 9 at 3:19
35

To remove all leading and trailing spaces from a given line thanks to a 'piped' tool, I can identify 3 different ways which are not completely equivalent. These differences concern the spaces between words of the input line. Depending on the expected behaviour, you'll make your choice.

Examples

To explain the differences, let consider this dummy input line:

"   \t  A   \tB\tC   \t  "

tr

$ echo -e "   \t  A   \tB\tC   \t  " | tr -d "[:blank:]"
ABC

tr is really a simple command. In this case, it deletes any space or tabulation character.

awk

$ echo -e "   \t  A   \tB\tC   \t  " | awk '{$1=$1};1'
A B C

awk deletes leading and tailing spaces and squeezes to a single space every spaces between words.

sed

$ echo -e "   \t  A   \tB\tC   \t  " | sed 's/^[ \t]*//;s/[ \t]*$//'
A       B   C

In this case, sed deletes leading and tailing spaces without touching any spaces between words.

Remark:

In the case of one word per line, tr does the job.

5
  • 1
    None of this trims trailing/leading newlines though
    – pronebird
    Commented Nov 29, 2016 at 12:50
  • 2
    +1 for a list of solutions with their (sometimes unexpected) output.
    – Tony
    Commented Jun 25, 2018 at 23:15
  • @user61382 this is rather late, but see my comment on the original post.
    – Tony
    Commented Jun 25, 2018 at 23:28
  • 2
    @highmaintenance : use [:space:], instead of [:blank:], for the command tr, like: ... | tr -d [:space:], to remove newlines too. (see: man tr)
    – tron5
    Commented Aug 9, 2019 at 13:52
  • 1
    Nice comparison and the results clearly show that only sed do what we expected for this question!
    – Gary Wang
    Commented May 9 at 3:22
27
sed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//'

If you're reading a line into a shell variable, read does that already unless instructed otherwise.

3
  • 1
    +1 for read. So if you pipe to while read it works: cat file | while read i; do echo $i; done
    – rubo77
    Commented Nov 21, 2013 at 3:36
  • 2
    @rubo except that in your example the unquoted variable is also reprocessed by the shell. Use echo "$i" to see the true effect of the read Commented Sep 9, 2015 at 19:19
  • If IFS is set to other characters, for example ,, then spaces will not be trimmed.
    – Gary Wang
    Commented May 9 at 2:39
13

An answer you can understand in a glance:

#!/usr/bin/env python3
import sys
for line in sys.stdin: print(line.strip()) 

Bonus: replace str.strip([chars]) with arbitrary characters to trim or use .lstrip() or .rstrip() as needed.

Like rubo77's answer, save as script /usr/local/bin/trim and give permissions with chmod +x.

1
  • 4
    I'm not normally a fan of python in my scripts but this is by far one of the most legible scripts in comparison to all the other incantations in these answers.
    – Victor
    Commented Mar 22, 2022 at 19:10
9

sed is a great tool for that:

                        # substitute ("s/")
sed 's/^[[:blank:]]*//; # parts of lines that start ("^")  with a space/tab 
     s/[[:blank:]]*$//' # or end ("$") with a space/tab
                        # with nothing (/)

You can use it for your case be either piping in the text, e.g.

<file sed -e 's/^[[...

or by acting on it 'inline' if your sed is the GNU one:

sed -i 's/...' file

but changing the source this way is "dangerous" as it may be unrecoverable when it doesn't work right (or even when it does!), so backup first (or use -i.bak which also has the benefit to be portable to some BSD seds)!

7

You will be adding this to your little Bash library. I can almost bet on it! This has the benefit of not adding a newline character to the end of your output, as will happen with echo throwing off your expected output. Moreover, these solutions are reusable, do not require modifying the shell options, can be called in-line with your pipelines, and are posix compliant. This is the best answer, by far. Modify to your liking.

Output tested with od -cb, something some of the other solutions might want to do with their output.

BTW: The correct quantifier is the +, not the *, as you want the replacement to be triggered upon 1 or more whitespace characters!

ltrim (that you can pipe input into)

function ltrim ()
{
    sed -E 's/^[[:space:]]+//'
}

rtrim (that you can pipe input into)

function rtrim ()
{
    sed -E 's/[[:space:]]+$//'
}

trim (the best of both worlds and yes, you can pipe to it)

function trim ()
{
    ltrim | rtrim
}

Update: I have improved this solution to use bash native constructs. The Stream Editor (sed) is not required. You can use shell expansions to achieve what you want, and it works better than the sed solution!

Bash Reference Manual -- Shell Expansions

4

If the string one is trying to trim is short and continuous/contiguous, one can simply pass it as a parameter to any bash function:

    trim(){
        echo $@
    }

    a="     some random string   "

    echo ">>`trim $a`<<"
Output
>>some random string<<
4

Using Raku (formerly known as Perl_6):

raku -ne '.trim.put;'

Or more simply:

raku -pe '.=trim;'

As a previous answer suggests (thanks, @Jeff_Clayton!), you can create a trim alias in your bash environment:

alias trim="raku -pe '.=trim;'"

Finally, to only remove leading/trailing whitespace (e.g. unwanted indentation), you can use the appropriate trim-leading or trim-trailing command instead.

https://raku.org/

3

translate command would work

cat file | tr -d [:blank:]
5
  • 9
    This command is not correct as it removes all spaces from the file, not just leading/trailing whitespace. Commented Sep 28, 2018 at 16:41
  • 2
    @BrianRedbeard You are correct. This is still a useful answer for a monolithic string, without spaces. Commented May 18, 2019 at 23:37
  • 2
    You might call this stripper instead of some kind of trim. :-) Commented Jul 26, 2021 at 14:18
  • I completely agree w/ @AnthonyRutledge. Not everyone (perhaps no one) actually winds up here - 11 years after the OP - with the same exact problem as stated in the OP!! It's disappointing to me to see 4 downvotes to this answer as it has utility to many of us who wind up here via various "search terms" like I used: bash cut spaces from start of string. A pox on you plutonian supercilious downvoters - you're disgusting.
    – Seamus
    Commented Mar 8 at 2:40
  • @Seamus You can use shell expansions to achieve what you want, and it works better than a sed solution, or one that uses tr! gnu.org/software/bash/manual/bash.html#Shell-Expansions Commented Mar 9 at 3:39
3
trimpy () {
    python3 -c 'import sys
for line in sys.stdin: print(line.strip())'
}
trimsed () {
gsed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//'
}
trimzsh () {
   local out="$(</dev/stdin)"
   [[ "$out" =~ '^\s*(.*\S)\s*$' ]] && out="$match[1]"  || out=''
   print -nr -- "$out"
}
# example usage
echo " hi " | trimpy

Bonus: replace str.strip([chars]) with arbitrary characters to trim or use .lstrip() or .rstrip() as needed.

1
  • Did you just copy my answer for python? Attribution would be nice...
    – qwr
    Commented Mar 22, 2022 at 20:46
1

I wrote this shell function using awk

awkcliptor(){
    awk -e 'BEGIN{ RS="^$" } {gsub(/^[\n\t ]*|[\n\t ]*$/,"");print ;exit}' "$1" ; } 

BEGIN{ RS="^$" }:
in the beginning before start parsing set record
separator to none i.e. treat the whole input as
a single record

gsub(this,that):
substitute this regexp with that string

/^[\n\t ]*|[\n\t ]*$/:
of that string catch any pre newline space and tab class
or post newline space and tab class and replace them with
empty string

print;exit: then print and exit

"$1":
and pass the first argument of the function to be
process by awk

how to use:
copy above code , paste in shell, and then enter to
define the function.
then you can use awkcliptor as a command with first argument as the input file

sample usage:

echo '
 ggggg    

      ' > a_file
awkcliptor a_file

output:

ggggg

or

echo -e "\n ggggg    \n\n      "|awkcliptor 

output:

ggggg
1
  • Can you please explain the difference to just awk '{$1=$1};1' ?
    – rubo77
    Commented Jan 31, 2020 at 9:35
1

For those of us without enough space in the brain to remember obscure sed syntax, just reverse the string, cut the 1st field with a delimiter of space, and reverse it back again.

cat file | rev | cut -d' ' -f1 | rev
2
  • 1
    This only works if there is no more than one space leading each line and no more than one word in any line.
    – mttpgn
    Commented Oct 27, 2020 at 23:09
  • 1
    Another way to do it with cut: echo "$mystring" | cut -d " " -f 2-
    – Jim Fell
    Commented Apr 20, 2023 at 16:57
1

My favorite is using perl: perl -n -e'/[\s]*(.*)?[\s]*/ms && print $1'

Take for example:

MY_SPACED_STRING="\n\n   my\nmulti-line\nstring  \n\n"

echo $MY_SPACED_STRING

Would output:



   my
multi-line
string  


Then:

echo $MY_SPACED_STRING | perl -n -e'/[\s]*(.*)?[\s]*/ms && print $1'

Would output:

my
multi-line
string 
0

for bash example:

alias trim="awk '{\$1=\$1};1'"

usage:

echo -e  "    hello\t\tkitty   " | trim | hexdump  -C

result:

00000000  68 65 6c 6c 6f 20 6b 69  74 74 79 0a              |hello kitty.|
0000000c
3
  • 1
    The awk '{$1=$1};1' answer was given long ago.  The idea of making an alias out of it was suggested in a comment almost as long ago.  Yes, you are allowed to take somebody else’s comment and turn it into an answer.  But, if you do, you should give credit to the people who posted the idea before you.  And this is such a trivial extension of the accepted answer that it’s not really worth the bother. Commented Sep 4, 2020 at 4:08
  • Idea was to make alias. I doesn't seen that answer before. Commented Sep 5, 2020 at 18:13
  • and second thing from stack: "Thanks for the feedback! Votes cast by those with less than 15 reputation are recorded, but do not change the publicly displayed post score." Commented Sep 5, 2020 at 18:25
0

Remove start space and tab and end space and tab:

alias strip='python3 -c "from sys import argv; print(argv[1].strip(\" \").strip(\"\t\"))"'

Remove every space and tab

alias strip='python3 -c "from sys import argv; print(argv[1].replace(\"\t\", \"\").replace(\" \", \"\")"'

Give argument to strip. Use sys.stdin().read() to make pipeable instead of argv.

0

simple enough for my purposes was this:

_text_="    one    two       three         "

echo "$_text_" | { read __ ; echo ."$__". ; }

... giving ...

.one    two       three.

... if you want to squeeze the spaces then ...

echo .$( echo $_text_ ).

... gives ...

.one two three.
0

rust sd command sd '^\s*(.*)\s*' '$1'

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .