How to strip multiple spaces to one using sed?

Question

sed on AIX is not doing what I think it should. I'm trying to replace multiple spaces with a single space in the output of IOSTAT:

# iostat
System configuration: lcpu=4 drives=8 paths=2 vdisks=0

tty:      tin         tout    avg-cpu: % user % sys % idle % iowait
          0.2         31.8                9.7   4.9   82.9      2.5

Disks:        % tm_act     Kbps      tps    Kb_read   Kb_wrtn
hdisk9           0.2      54.2       1.1   1073456960  436765896
hdisk7           0.2      54.1       1.1   1070600212  435678280
hdisk8           0.0       0.0       0.0          0         0
hdisk6           0.0       0.0       0.0          0         0
hdisk1           0.1       6.3       0.5   63344916  112429672
hdisk0           0.1       5.0       0.2   40967838  98574444
cd0              0.0       0.0       0.0          0         0
hdiskpower1      0.2     108.3       2.3   2144057172  872444176

# iostat | grep hdisk1
hdisk1           0.1       6.3       0.5   63345700  112431123

#iostat|grep "hdisk1"|sed -e"s/[ ]*/ /g"
 h d i s k 1 0 . 1 6 . 3 0 . 5 6 3 3 4 5 8 8 0 1 1 2 4 3 2 3 5 4

sed should search & replace (s) multiple spaces (/[ ]*/) with a single space (/ /) for the entire group (/g)... but it's not only doing that... its spacing each character.

What am I doing wrong? I know its got to be something simple... AIX 5300-06

edit: I have another computer that has 10+ hard drives. I'm using this as a parameter to another program for monitoring purposes.

The problem I ran into was that "awk '{print $5}' didn't work because I'm using $1, etc in the secondary stage and gave errors with the Print command. I was looking for a grep/sed/cut version. What seems to work is:

iostat | grep "hdisk1 " | sed -e's/  */ /g' | cut -d" " -f 5

The []s were "0 or more" when I thought they meant "just one". Removing the brackets got it working. Three very good answers really quickly make it hard to choose the "answer".

Answer 1

/[ ]*/ matches zero or more spaces, so the empty string between characters matches.

If you're trying to match "one or more spaces", use one of these:

... | sed 's/  */ /g'
... | sed 's/ \{1,\}/ /g'
... | tr -s ' '

Answer 2

The use of grep is redundant, sed can do the same. The problem is in the use of * which also match 0 spaces. With GNU sed, you may use \+ instead:

iostat | sed -n '/hdisk1/s/ \+/ /gp'

Or, with standard sed:

iostat | sed -e '/hdisk/!d' -e 's/ \{2,\}/ /g'

to delete all lines that does not contain the substring hdisk, and to replace all runs of two or more spaces with single spaces, or

iostat | sed -e '/hdisk1/!d' -e 's/   */ /g'

Answer 3

Change your * operator to a +. You are matching zero or more of the previous character, which matches every character because everything that isn't a space is ... um ... zero instances of space. You need to match ONE or more. Actually it would be better to match two or more

The bracketed character class is also un-necessary for matching one character. You can just use:

s/  \+/ /g

...unless you want to match tabs or other kinds of spaces too, then the character class is a good idea.

Answer 4

You can always match the last occurrence in a sequence of anything like:

s/\(sequence\)*/\1/

And so you're on the right track, but rather than replacing the sequence with a space - replace it with its last occurrence - a single space. That way if a sequence of spaces is matched then the sequence is reduced to a single space, but if the null string is matched then the null string is replaced with itself - and no harm, no foul. So, for example:

sed 's/\( \)*/\1/g' <<\IN                                    
# iostat
System configuration: lcpu=4 drives=8 paths=2 vdisks=0

tty:      tin         tout    avg-cpu: % user % sys % idle % iowait
          0.2         31.8                9.7   4.9   82.9      2.5

Disks:        % tm_act     Kbps      tps    Kb_read   Kb_wrtn
hdisk9           0.2      54.2       1.1   1073456960  436765896
hdisk7           0.2      54.1       1.1   1070600212  435678280
hdisk8           0.0       0.0       0.0          0         0
hdisk6           0.0       0.0       0.0          0         0
hdisk1           0.1       6.3       0.5   63344916  112429672
hdisk0           0.1       5.0       0.2   40967838  98574444
cd0              0.0       0.0       0.0          0         0
hdiskpower1      0.2     108.3       2.3   2144057172  872444176

# iostat | grep hdisk1
hdisk1           0.1       6.3       0.5   63345700  112431123

IN

OUTPUT

# iostat
System configuration: lcpu=4 drives=8 paths=2 vdisks=0

tty: tin tout avg-cpu: % user % sys % idle % iowait
 0.2 31.8 9.7 4.9 82.9 2.5

Disks: % tm_act Kbps tps Kb_read Kb_wrtn
hdisk9 0.2 54.2 1.1 1073456960 436765896
hdisk7 0.2 54.1 1.1 1070600212 435678280
hdisk8 0.0 0.0 0.0 0 0
hdisk6 0.0 0.0 0.0 0 0
hdisk1 0.1 6.3 0.5 63344916 112429672
hdisk0 0.1 5.0 0.2 40967838 98574444
cd0 0.0 0.0 0.0 0 0
hdiskpower1 0.2 108.3 2.3 2144057172 872444176

# iostat | grep hdisk1
hdisk1 0.1 6.3 0.5 63345700 112431123

All that said, it is probably far better to avoid regexps completely in this situation and do instead:

tr -s \  <infile

Answer 5

Notice that you can also do what you attempt, that is

iostat | grep "hdisk1 " | sed -e's/  */ /g' | cut -d" " -f 5

by

iostat | while read disk tma kbps tps re wr; do [ "$disk" = "hdisk1" ] && echo "$re"; done

which might be especially useful if you later attempt to access other fields as well and/or calculate something - like this:

iostat | while read disk tma kbps tps re wr; do [ "$disk" = "hdisk1" ] && echo "$(( re/1024 )) Mb"; done

Answer 6

assuming the problem is

I deleted a bunch of things without using git rm. How do I delete them all from git without manually typing git rm <file path> again and again?

git status | grep "deleted: " | tr "\t" " " | sed -e "s/ *deleted: */git rm /" | bash

Note: the tr in the pipe is used because sed on the mac doesn't respect the \t character so you need to get rid of it first, or install GNU sed and use that. The line output by git (at the time of writing) starts with \tdeleted: Sadly, tz doesn't appear to come with macOS either. You'll want to install that with homebrew because it really helps out in a lot of situations.

Answer 7

You can use the following script to convert multiple spaces to a single space, a TAB or any other string:

$ ls | compress_spaces.sh       # converts multiple spaces to one
$ ls | compress_spaces.sh TAB   # converts multiple spaces to a single tab character
$ ls | compress_spaces.sh TEST  # converts multiple spaces to the phrase TEST
$ compress_spaces.sh help       # show the help for this command

compress_spaces.sh

#!/usr/bin/env bash

function show_help()
{
  ME=$(basename "$0")
  IT=$(cat <<EOF

  usage: $ME {REPLACE_WITH}
  
  NOTE: If you pass in TAB, then multiple spaces are replaced with a TAB character

  no args -> multiple spaces replaced with a single space
  TAB     -> multiple spaces replaced with a single tab character
  TEST    -> multiple spaces replaced with the phrase "TEST"

  $ME 
EOF
)
  echo "$IT"
  echo
  exit
}


if [ "$1" == "help" ]
then
  show_help
fi

# Show help if we're not getting data from stdin
if [ -t 0 ]; then
  show_help
fi

REPLACE_WITH=${1:-' '}

if [ "$REPLACE_WITH" == "tab" ]
then
  REPLACE_WITH=$'\t'
fi
if [ "$REPLACE_WITH" == "TAB" ]
then
  REPLACE_WITH=$'\t'
fi

sed "s/ \{1,\}/$REPLACE_WITH/gp"