2

Hi I'm having a bit of trouble when using awk to add a column at the start of a .tsv file that i'm creating within a loop.

My code is as follows (can ignore the rest of the loop):

while IFS= read -r line
do
curl *URL.org* > ./folder/$line.tsv
sleep 60
awk '{print "$line\t" $0}' ./folder/$line.tsv > $line.tabbed
done < ./folder/<filetoread>.txt

So basically I have this loop that I'm using to create a tsv file from a curl command, and using the $line variable to create a file with this. The file read is simply a list of single word identifiers;

RUIAHT48
RUIAHT49
etc. 

I thought I could use awk to append this $line variable to create a new column in each of the newly created files.

Output before awk command;

$line  col1 col2  col3  col4    col5
$line  CLV_M   19     25    False   False
$line  CLV_P   59     63    False   False
$line  DEG_N   1      3     False   False
$line  DOC_C   10     14    False   False

Desired output;

new      col1   col2    col3    col4    col5
<value>  CLV_M  19      25      False   False
<value>  CLV_P  59      63      False   False
<value>  DEG_N  1       3       False   False
<value>  DOC_C  10      14      False   False

Where the new column is all the same value but the actual value instead of literally just "$line". Where am I going wrong here? Should I actually be using awk or another command with a simpler way?

Thanks :)

0

You need to pass the variable to awk using the -v switch:

while IFS= read -r line
do
    curl *URL.org* > "./folder/$file.tsv"
    sleep 60
    awk -v line="$line" '{ OFS = "\t"; print line,$0}' "./folder/$line.tsv" > "$line.tabbed"
done < ./folder/<filetoread>.txt

You can add \t to your output as you have done in the OP, but I would do it using the OFS option of awk.

Additionally I'm not sure what you're doing but are you aware that $file.tsv doesn't appear to be used? I think that should be $line.tsv?

  • Thanks for the answer, I've had a play around and the outcome is the new column just saying line, instead of $line. Also yeah apologies it was meant to be line.tsv, thanks for noticing. – daenwaels Aug 6 '18 at 13:20
  • Scratch that. I just removed the two sets of " " around the set awk variable "$line" and the print "line" and its worked. Thank you very much! – daenwaels Aug 6 '18 at 13:29
  • You should not remove the quotes as that will introduce security vulnerabilities to your code. I really don't think the quotes were causing an issue. – Jesse_b Aug 6 '18 at 13:50
  • 1
    Surely if I don't remove the quotes around the print "line" then it would just add the word line to the new column? Which is what I was getting before I removed them. I'll have a play around and see if it effects anything but for now that seems to have worked. – daenwaels Aug 6 '18 at 13:57
  • @maltedg: Sorry yes that is correct but you should not remove them around the variable assignment -v line="$line" should stay in quotes. – Jesse_b Aug 6 '18 at 15:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.