3

I have a tab delimited file with 10 columns and in one of the columns (with about 40 million rows), I would like to add a word before the existing entry in each row (same word in each row!) and a ; after the entry.

e.g. two rows before

1 2 3 4 5 6 7 8 text still more text in this column 10
1 2 3 4 5 6 7 8 text2 still more text in this column 10

to

1 2 3 4 5 6 7 8 test=text; still more text in this column 10
1 2 3 4 5 6 7 8 test=text2; still more text in this column 10

At the end of the day it's the basic "concatenate" function in excel, but I can't use excel for such large files and also need to move to linux anyways.

I looked into concatenate questions here on the forum but I only found topics dealing with merging two strings, eg.

foo="Hello"
foo="$foo World"
echo $foo  

but not using variables.

  • I have some issues with what you are trying to use. If something it the same in every single row of your file, it is complete meaningless to put it there, put it in the header of your file. – Bernhard Aug 3 '14 at 8:28
4

This is exactly what awk is good at:

awk -F'\t' -vOFS='\t' '{ $9 = "test=" $9 ";" }1'

-F'\t' tells it to use tab-separated fields. -vOFS='\t' tells it to use tabs in the output too. The actual body of it is the last argument: it's a little program that says for every line to change the value of $9 (the ninth field) to the concatenation of "test=", its original value, and ";". We leave all the other fields intact, The final 1 is to tell awk to print the new line out even though we did something to it.

If I give it your input (with tabs inserted):

$ cat data
1   2   3   4   5   6   7   8   text    still more text in this column 10
1   2   3   4   5   6   7   8   text2   still more text in this column 10

then I can run the above command:

$ awk -F'\t' -vOFS='\t' '{ $9="test=" $9 ";" }1' < data
1   2   3   4   5   6   7   8   test=text;  still more text in this column 10
1   2   3   4   5   6   7   8   test=text2; still more text in this column 10

and get the output you want. You can save that to a file with > redirection:

$ awk -F'\t' -vOFS='\t' '{ $9="test=" $9 ";" }1' < data > processed-data
4

With GNU sed:

sed 's/[^\t]*/text=&;/9'

that is, replace the 9th sequence of non-tab characters with text=&; (where & means the matched part).

On systems with other sed implementations, you may need to enter the tab character literally instead of \t.

  • How about ; in 9th tab? – cuonglm Aug 3 '14 at 14:37
  • @Gnouc, sorry I missed that part of the requirement. Answer updated. – Stéphane Chazelas Aug 3 '14 at 15:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.