0

I'm trying to use sed to convert TB to GB. The fields with TB are either single-digit (2T, 8T) or multiple digits (2.001T, 1.501T.) I thought I had my substitution working, but I've got an error and I can't seem to figure it out.

Sample of file:

ftwepsiprdsql02,ftwepsiprdsql02_F,2.001T,1.680T
ftwepsiprdsql02,ftwepsiprdsql02_G,801G,176.786G
ftwepsiprdsql02,ftwepsiprdsql02_H,501G,6.565G
ftwepsiprdsql02,ftwepsiprdsql02_I,1.001T,539.504G
ftwepsiprdsql02,ftwepsiprdsql02_J,501G,478.211G
ftwepsiprdsql02,ftwepsiprdsql02_X,1.501T,68.021G
rxi0738_foc_cl1,rxi0738_foc_cl1_lun248,8T,4.450T
rxi0738_foc_cl1,rxi0738_foc_cl1_lun250,8T,5.857T
rxi0738_foc_cl1,rxi0738_foc_cl1_lun252,8T,4.681T
rxi0738_foc_cl1,rxi0738_foc_cl1_lun254,8T,4.657T

I'm doing substitution for the 3rd & 4th fields separately (allocated/used capacity,) and my original approach was to change the single-digit capacity numbers to be consistent with the multi-digit (2.000T, 8.000T).

Code to substitute single-digit in the 3rd field:

for i in `awk -F , '{print $3}' $TMPRPT| grep \[0-9\]T | grep -v "\." `
do
TVAL=$(echo $i | sed 's/T/.000T/')
sed -i .tmp "s/$i/$TVAL/" "$TMPRPT"
done

In theory, this should change all the single-digit numbers to match the multi-digit ones, so then the final conversion from TB to GB is simple. The awk command performs perfectly, and gives me just the single-digit output, but once I put it into the for loop & the sed -i command, it is doing the substitution on all the fields with T designations, to include the multi-digit:

ftwepsiprdsql02,ftwepsiprdsql02_F,2.001.000T,1.680T
ftwepsiprdsql02,ftwepsiprdsql02_G,801G,176.786G
ftwepsiprdsql02,ftwepsiprdsql02_H,501G,6.565G
ftwepsiprdsql02,ftwepsiprdsql02_I,1.001.000T,539.246G
ftwepsiprdsql02,ftwepsiprdsql02_J,501G,478.211G
ftwepsiprdsql02,ftwepsiprdsql02_X,1.501.000T,68.021G
rxi0738_foc_cl1,rxi0738_foc_cl1_lun248,8.000T,4.450T
rxi0738_foc_cl1,rxi0738_foc_cl1_lun250,8.000T,5.857T
rxi0738_foc_cl1,rxi0738_foc_cl1_lun252,8.000T,4.681.000T
rxi0738_foc_cl1,rxi0738_foc_cl1_lun254,8.000T,4.657T

I know there is an easier way to accomplish this, so I'm open to alternatives, but I'd also really like to know how to make the sed -i behave the way I want.

(Please note that the platform is Isilon, which runs a scaled-down Linux kernel. Most commands are available, but not all.)

5
  • try this awk -F, -v OFS=, '{ for(i = 1; i <= NF; i++) if($i ~ /^[0-9.]+T$/) $i = $i * 1000 "G" } 1' your_file.txt
    – user313992
    Dec 30, 2019 at 18:40
  • assuming 1TB = 1000GB, not 1TB = 1024GB
    – user313992
    Dec 30, 2019 at 18:42
  • Thank you, @mosvy! That solved it perfectly! (I didn't put in the G at the end, as I'm converting all fields to only numeric.) Final command used was: awk -F, -v OFS=, '{ for(i = 1; i <= NF; i++) if($i ~ /^[0-9.]+T$/) $i = $i * 1000 } 1' $TMPRPT > $TMPRPT.tmp
    – TeresaM
    Dec 30, 2019 at 19:02
  • The Final command version will leave G on the fields that were not converted from T.
    – icarus
    Dec 30, 2019 at 19:05
  • Yes. Later in my script I convert MB to GB, then take away all the letter designations (T, G and M.) Conversion is to make everything numeric only in GB. Everything worked correctly except this one section, which you solved.
    – TeresaM
    Dec 30, 2019 at 19:27

2 Answers 2

1

If you have numfmt from GNU Coreutils:

$ numfmt -d, --field=3,4 --from=auto --round=nearest < file | 
    numfmt -d, --field=3,4 --to-unit=G --suffix=G
ftwepsiprdsql02,ftwepsiprdsql02_F,2001G,1680G
ftwepsiprdsql02,ftwepsiprdsql02_G,801G,177G
ftwepsiprdsql02,ftwepsiprdsql02_H,501G,7G
ftwepsiprdsql02,ftwepsiprdsql02_I,1001G,540G
ftwepsiprdsql02,ftwepsiprdsql02_J,501G,479G
ftwepsiprdsql02,ftwepsiprdsql02_X,1501G,69G
rxi0738_foc_cl1,rxi0738_foc_cl1_lun248,8000G,4450G
rxi0738_foc_cl1,rxi0738_foc_cl1_lun250,8000G,5857G
rxi0738_foc_cl1,rxi0738_foc_cl1_lun252,8000G,4681G
rxi0738_foc_cl1,rxi0738_foc_cl1_lun254,8000G,4657G

Alternatively, with Perl:

perl -F, -pe '
  $_ = join ",", map { $_ =~ s/^(\d*(\.\d+)?)T$/sprintf "%.0fG", 1000*$1/e; $_ } @F
' file
1
  • I'm on an Isilon cluster, which has a scaled-down Linux kernel. numfmt is not one of the commands I have available.
    – TeresaM
    Dec 30, 2019 at 19:29
0

Assuming that 1T = 1000G.

sed is Turning complete, and so has theoretically way more power than needed to do multiplication, but I am keeping this short and just doing the changes by simple text substitution without loops.

Let us break down the changes needed. First change single fields with T but without periods into ones with ".000". Need the 'g' as there are 2 fields. Note that as we have a nice comma to start the field take advantage of it.

  s/\(,[0-9]*\)T/\1.000T/g

Now a bit of cleanup in case there are fields without 3 decimal places. Add 3 zeros if there is nothing after the period, 2 zeros if there is one digit, a zero if there are two digits, and remove any excess digits after the third (i.e. truncation rather than rounding).

s/\(,[0-9]*\.\)T/\1000T/g
s/\(,[0-9]*\.[0-9]\)T/\100T/g
s/\(,[0-9]*\.[0-9][0-9]\)T/\10T/g
s/\(,[0-9]*\.[0-9][0-9][0-9]\)[0-9]*T/\1T/g

Now change the T to G, dropping the period.

s/\(,[0-9]*\)\.\([0-9][0-9][0-9]\)T/\1\2G/g

Now to be nice, drop leading zeros

s/,0*\([1-9][0-9]*G\)/,\1/g

Putting all the commands together and giving the sample data results in

ftwepsiprdsql02,ftwepsiprdsql02_F,2001G,1680G
ftwepsiprdsql02,ftwepsiprdsql02_G,801G,176.786G
ftwepsiprdsql02,ftwepsiprdsql02_H,501G,6.565G
ftwepsiprdsql02,ftwepsiprdsql02_I,1001G,539.504G
ftwepsiprdsql02,ftwepsiprdsql02_J,501G,478.211G                                                                                                        
ftwepsiprdsql02,ftwepsiprdsql02_X,1501G,68.021G
rxi0738_foc_cl1,rxi0738_foc_cl1_lun248,8000G,4450G
rxi0738_foc_cl1,rxi0738_foc_cl1_lun250,8000G,5857G
rxi0738_foc_cl1,rxi0738_foc_cl1_lun252,8000G,4681G
rxi0738_foc_cl1,rxi0738_foc_cl1_lun254,8000G,4657G
2
  • Thanks, @icarus. I didn't want to put that many commands in my script, but this certainly works.
    – TeresaM
    Dec 30, 2019 at 18:54
  • 7 changes, with 5 being data cleanup is not excessive in my view. If you prefer `sed -i 's/(,[0-9]*)T/\1.000T/g;s/(,[0-9]*)\.([0-9]*)T/\1\2G/g' because you know the data input is reasonable then that is OK.
    – icarus
    Dec 30, 2019 at 19:01

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .