so I have found a question on here that is the opposite of what I would like to do here: Concatenate multiple files with same header.

I am trying to merge or concatenate 2 files,within a directory within hundred of files that have a counterpart file with the same substring, into one file based on the '>' character in the header. There are thousands of headers within the each file and as opposed to the question asked in the link I attached, the contents other than the header are the same so I don't want to change anything but every line that begins with '>'.

Essentially I would like to combine file1 to file2 by the shared character '>' and to leave the contents of file 1 in order.



> MGBNCNOSCNC_142 > King_henry
> MGBNCNOSCNC_143 > Queen_jerry
> MGBNCNOSNC_144 > Jack_jill

Thanks for any help and let me know if I can be any clearer!

  • 1
    It might be clearer if you re-worked the formating - you might want to use code formating (indent by 4 spaces) in place of block quotes. Also clarify whether the lines begin with > (as indicated in your text) or with '> (as apparently shown in your sample input). Finally, it would be easier to test potential solutions if your File1 extract were larger (at least including the pattern that should be matched to Queen_jerry). – steeldriver Aug 20 '16 at 23:15
  • I've tried to make it clearer, thank you @steeldriver! – Alina Orozco Aug 20 '16 at 23:59
  • That's much clearer - thanks. So the entries from File2 should just be appended, in order, to the lines of File1 beginning with >? – steeldriver Aug 21 '16 at 0:12
  • That's right! I've tried variations of paste and awk but no success so far – Alina Orozco Aug 21 '16 at 0:19

You could read the lines of File2 into an indexed awk array, and then append them in turn to the corresponding lines of File

awk '
  NR==FNR {a[i++]=$0; next} 
  /^>/ {$0 = $0" "a[j++];}
  ' File2 File1

Alternatively, if you have GNU sed (with the R extension) you could try

sed '/^>/ R File2' File1 | sed '/^>/ {N;s/\n/ /}'

If your File1 has exactly one additional line for every line that is to be matched, another option might be to double space File2 and then paste the files together

sed 'G' File2 | paste -d ' ' File1 -

although that will result in an additional trailing space in the non-matching lines; if that's undesirable, you could insert a space at the start of each File2 line and paste them without a delimiter

sed 's/^/ /; G' File2 | paste -d '' File1 -
  • Oh my gosh it worked it worked!!!! I cannot thank you enough! – Alina Orozco Aug 21 '16 at 1:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.