I've been working with bash to get some data output to a multiline file with the data stored in this format for each row:

text1 "text2" text3 integer1,integer2

A "space" separates the first four columns of data in each row. A comma separates the two integers. text2 is enclosed in quotes.

Now I want to use bash to evaluate each line of the original data file and then replace integer1 and integer2 with new data based upon using a different formula to evaluate integer1 and integer2.

The results should then be output to a new file (with the same format as the original file) as shown below:

text1 "text2" text3 Newinteger1,Newinteger2

3 Answers 3


I hope I don't confuse you with this code, see the comments starting with a #.

I will test with the following input file:

text1 "text2" text3 1,2
this "/usr/strange path with spaces/" works 2,3

The input of the next script can be given as cat input | while .... The shell provides a nice way to avoid the extra process cat, that is the < input after the done on the last line of the script. For now it is important to remember, that you will be parsing one line before reading the second. I called the variable where the line is stored line, that could have been something else, like mybuf, firststring or whatever_you_want.
So let's start.

while read -r line; do
    # variable line is filled, just echo for debugging
    echo "The var line is filled with ${line}."
    # Special handling: you can cut off everything starting at the last space
    mystrings=${line% *}

    # You can also throw away everything up to a space.
    # Two # characters for throwing away the largest substring with "* "
    myints=${line##* }

    # Did it work? Two echoes just checking
    echo "mystring=${mystrings}"
    echo "myints=${myints}"

    # I'll try that strange thing with cutting up a variable again.
    # Now I want to cut the myint into two integers
    echo "My results are int1=${int1} and int2=${int2}"    
    # Some calculation.
    # The value of the vars can be used without the $ when used in (( ...))
    # Example new values: int1 will be given the original value of int2 + 5,
    # int2 will be doubled.
    # I did not add checks to see if int1 and int2 are integers
    (( int1 = int2 + 5 ))
    (( int2 = 2 * int2 ))
    # And show the result
    echo "${mystrings} ${int1},${int2}"
done < input

When you want to see exactly what is happening, you can start the script with set -x (turn debugging on, turn off again with set -).

  • thanks. this seems to be handling the data as needed. But I still struggle to get the data output saved to a file. I tried: done < input > output .... but got no file output. I also tried: echo "${mystrings} ${int1},${int2}" >> output ... but again collected no file output. Can you suggest how to collect the output into a file?
    – daniel
    Commented Sep 6, 2015 at 20:45
  • I must be missing something fundamental. I can type: echo "hello world" > output.txt .... and the file is updated as expected. However, when I change the echo in your script to: echo "${mystrings} ${int1},${int2}" >> output.txt .... the file receives no output .. yet I can see on the screen that the echo is working correctly.
    – daniel
    Commented Sep 6, 2015 at 21:01
  • Strange! Maybe set -o noclobber is set, but than the >> should still work. Try removing output.txt first, or write to /tmp/daniel.tmp. Or check the dir with some more commands like pwd and ls -l.
    – Walter A
    Commented Sep 6, 2015 at 21:49
  • My eyes failed me ... I had a syntax error ... Everything is working very well now. Many thanks.
    – daniel
    Commented Sep 7, 2015 at 6:32
  • I tried this myself, it reads the file line by line but it doesn't change the file itself. Then I saw this on stackoverflow inside the while loop using sed -i "${line_num}s/.*/$replacement_escaped/" "$file", just introduce a line_num and increment it inside the while. then replace the file directly with sed and line number
    – bherto39
    Commented Nov 9, 2018 at 2:41

A possible solution with awk

#!/usr/bin/awk -f
{ split($4, a, ","); printf("%s %s %s %d,%d\n", $1, $2, $3, a[1] / 2, a[2] + 500); }

The split command puts the two integers in column $4 into an array a[]. The printf command prints to stdout. The three placeholder %s are for the three text columns $1, $2 & $3. The two %d placefolders are the the two integers. Change the calculation of both integer values to your needs.

Save it to a file (eg. awktest) and set permissions to

chmod +x ./awktest
Run it with
./awktest < input.txt > output.txt

  • This almost works but fails on the second column of data. The second column of data are path/filenames enclosed in quotes because the filenames may contain spaces. So the awk split works until there are filename with spaces.
    – daniel
    Commented Sep 6, 2015 at 17:31
awk -F'[ ,]' '{$(NF-1)=$(NF-1)*6 "," $NF-10 ; NF=NF-1}1' multiline.file
  • -F'[ ,]' set , or <space> as separator
  • $NF — last field
  • $(NF-1) — field before last which composed by changed last and before last fields
  • $NF=$NF-1 — remove last field
  • 1 — prints composed line
  • I'm a little confused on how to save this to output. But I think that because the "text2" data is composed of path/filenames with spaces that it will parse the text2 column incorrectly.... but I'm not really sure, as i don't understand how to try this idea out.
    – daniel
    Commented Sep 6, 2015 at 18:00
  • Please do explain at least briefly what your code snippet does and whether it handles cases like spaces in commas in the second column (I think it doesn't, but feel free to prove me wrong).
    – peterph
    Commented Sep 6, 2015 at 18:20

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .