0

This is a more specific follow-up to a previous question (Arithmetic between 2 files generating a series of new files).

I have a tab separated model input file I would like to vary for an ensemble analysis formatted similar to this

cat input.txt

/* Preciptation in mm */
10 30 40 50 23

### Species description
*** sp_name LMA wsg a_h
abies_lasiocarpa 2 0.5 1
abies_grandis 2.5 0.4 1
larix_occidentalis 1.5 0.3 1

I have another file of multipliers randomly selected from a distribution, with one per line such as this

cat multipliers.txt

0.1
0.5
0.25

I would like to generate a series of new input files where one field (wsg) is multiplied by a single multiplier from the second file. In this example there would be 3 new files corresponding to the 3 multipliers (The actual analysis will involve 1000 multipliers). The output files would look like the following:

(wsg * 0.1) cat file1.txt

/* Preciptation in mm */
10 30 40 50 23

### Species description
*** sp_name LMA wsg a_h
abies_lasiocarpa 2 0.05 1
abies_grandis 2.5 0.04 1
larix_occidentalis 1.5 0.03 1

(wsg * 0.5) cat file2.txt

/* Preciptation in mm */
10 30 40 50 23

### Species description
*** sp_name LMA wsg a_h
abies_lasiocarpa 2 0.25 1
abies_grandis 2.5 0.2 1
larix_occidentalis 1.5 0.15 1

(wsg * 0.25) cat file3.txt

/* Preciptation in mm */
10 30 40 50 23

### Species description
*** sp_name LMA wsg a_h
abies_lasiocarpa 2 0.125 1
abies_grandis 2.5 0.1 1
larix_occidentalis 1.5 0.075 1

Based on my previous question, @EdMorton suggested the following:

$ cat tst.awk
NR==FNR {
    if ( pastHdr ) {
        ++numLines
        wsg[numLines] = $NF
        sub(/[[:space:]][^[:space:]]+$/,"")
        rest[numLines] = $0
    }
    else {
        hdr = hdr $0 ORS
        if ( $1 == "***" ) {
            pastHdr = 1
        }
    }
    next
}
{
    out = "file" FNR ".txt"
    printf "%s", hdr > out
    for (lineNr=1; lineNr<=numLines; lineNr++) {
        print rest[lineNr], wsg[lineNr] * $0 > out
    }
    close(out)
}
$ awk -f tst.awk input.txt multipliers.txt

This was an excellent solution to my previous question, but the arithmetic takes place on the last field of each row in the input. I would like to rework this to work on the nth field in each row, in this case the 3rd (wsg)

1

Having just re-read your question, I'd actually recommend you do the following which doesn't rely on you telling it which field is the wsg field but instead reads that information from the line starting with *** in your input file:

$ cat tst.awk
BEGIN { FS=OFS="\t" }
NR==FNR {
    if ( tgtFldNr ) {
        lines[++numLines] = $0
    }
    else {
        hdr = hdr $0 ORS
        if ( /^\*\*\*/ ) {      # in case this line is not tab-separated
            split($0,f," ")
            for (i in f) {
                if ( f[i] == "wsg" ) {
                    tgtFldNr = i-1
                    break
                }
            }
        }
    }
    next
}
{
    mult = $1
    out = "file" FNR ".txt"
    printf "%s", hdr > out
    for (lineNr=1; lineNr<=numLines; lineNr++) {
        $0 = lines[lineNr]
        $tgtFldNr *= mult
        print > out
    }
    close(out)
}

$ awk -f tst.awk input.txt multipliers.txt

$ head file*
==> file1.txt <==
/* Preciptation in mm */
10      30      40      50      23

### Species description
*** sp_name LMA wsg a_h
abies_lasiocarpa        2       0.05    1
abies_grandis   2.5     0.04    1
larix_occidentalis      1.5     0.03    1

==> file2.txt <==
/* Preciptation in mm */
10      30      40      50      23

### Species description
*** sp_name LMA wsg a_h
abies_lasiocarpa        2       0.25    1
abies_grandis   2.5     0.2     1
larix_occidentalis      1.5     0.15    1

==> file3.txt <==
/* Preciptation in mm */
10      30      40      50      23

### Species description
*** sp_name LMA wsg a_h
abies_lasiocarpa        2       0.125   1
abies_grandis   2.5     0.1     1
larix_occidentalis      1.5     0.075   1

Original answer for posterity:

$ cat tst.awk
NR==FNR {
    if ( pastHdr ) {
        lines[++numLines] = $0
    }
    else {
        hdr = hdr $0 ORS
        if ( $1 == "***" ) {
            pastHdr = 1
        }
    }
    next
}
{
    tgtFldNr = (n ? n : NF)
    mult = $1
    out = "file" FNR ".txt"
    printf "%s", hdr > out
    for (lineNr=1; lineNr<=numLines; lineNr++) {
        $0 = lines[lineNr]
        $tgtFldNr *= mult
        print > out
    }
    close(out)
}

$ awk -v n=3 -f tst.awk input.txt multipliers.txt

$ head file*
==> file1.txt <==
/* Preciptation in mm */
10 30 40 50 23

### Species description
*** sp_name LMA wsg a_h
abies_lasiocarpa 2 0.05 1
abies_grandis 2.5 0.04 1
larix_occidentalis 1.5 0.03 1

==> file2.txt <==
/* Preciptation in mm */
10 30 40 50 23

### Species description
*** sp_name LMA wsg a_h
abies_lasiocarpa 2 0.25 1
abies_grandis 2.5 0.2 1
larix_occidentalis 1.5 0.15 1

==> file3.txt <==
/* Preciptation in mm */
10 30 40 50 23

### Species description
*** sp_name LMA wsg a_h
abies_lasiocarpa 2 0.125 1
abies_grandis 2.5 0.1 1
larix_occidentalis 1.5 0.075 1

If you don't set -v n=<number> to set the number of the field to be multiplied then it'll default to multiplying the last field on each line just like you wanted in your previous question.

You said in the text of your question that your input is tab-separated but it doesn't look that way in the example you provided. If it truly is tab-separated then just add BEGIN { FS=OFS="\t" } at the top of the script, immediately before the NR==FNR { line.

2
  • it is tab separated, but using 8 spaces seemed to look weird on here to me. I guess I still need to work on formatting questions properly... Thanks! – sethparker Feb 16 at 16:03
  • 1
    Another approach is to create your question using ;s or some other visible character not otherwise present in your data where the tabs should be.Unfortunately shell treats tabs as a separator differently from other characters so it's not a perfect solution to asking a question. For example try printf 'a\t\tb\n' | while IFS=$'\t' read -r x y; do echo "<$x:$y>"; done vs printf 'a;;b\n' | while IFS=';' read -r x y; do echo "<$x:$y>"; done and notice that the former treats a sequence of contiguous tabs as a separator rather than each individual tab as a separator. AWK doesn't have that issue – Ed Morton Feb 16 at 16:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.