2

I have a text file with strings/filenames in separate lines, eg. filename.txt. There are hundreds of filenames

ABC123_S386_R1_001
JKL345_S441_R1_001
filename9000_S587_R1_001

and another text file with the string/filenames and additional data eg. results.txt:

>ABC123_S386_R1_001 
NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN
>JKL345_S441_R1_001
NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN
>abc7890_S387_R1_001  
NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN
>filename9000_S587_R1_001
NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN

Now not all filenames in filename.txt are present in results.txt, neither are they in order. I want to insert a prefix to all the filenames from filename.txt to results.txt but not to the others.

How do I read an input file of strings, match with another file and change the matches?

Earlier I used to go about matching individual filenames with sequence.txt, getting their line number and used sed with line numbers to change either a single line or a block of lines.

My desired output would look like

>h-19/US/CA-ABC123_S386_R1_001 
NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN
>h-19/US/CA-JKL345_S441_R1_001
NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN
>abc7890_S387_R1_001  
NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN
>h-19/US/CA-filename9000_S587_R1_001
NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN

where h-19/US/CA- is the suffix I'd like to add to all the matches.

Edit: > is the first character of all the strings that need to be changed, there are no characters before > neither are any trailing white spaces after the filename.

0

3 Answers 3

1

Assuming that the relevant lines in results.txt do not contain whitespace after the filename, the following awk program will work:

awk -v prefix="h-19/US/CA-" 'NR==FNR{fnames[$1]; next} \
    /^>/{name=substr($0,2); if (name in fnames) {sub(/^>/, ">" prefix)} }1' filenames.txt results.txt
  • This will first parse filenames.txt and then results.txt.
  • While parsing filenames.txt (where FNR, the per-file line counter, is equal to NR,the global line counter) it will register all filenames (which are the only fields on the line) in an array fnames, but then skip execution immediately to the next line.
  • While parsing results.txt it will check if a line starts with >. If so, it will check if the substring following that character (stored temporarily in name) is found among the "array indices" of fnames. If that is the case, it will use sub() to substitute the leading > with >+the prefix, passed to awk as variable prefix (via the -v directive).
  • The seemingly "stray" 1 will instruct awk to print the current line, including all possible modifications (but only for results.txt since during processing of the first file we don't reach that part).

Note that awk by itself cannot modify files in-place, so you would need to work with a temporary file. If you have a sufficiently new version of GNU Awk (> 4.1.0), you can however use the inplace extension; of course, you will then need to turn the option off for the filenames.txt file:

awk -i inplace -v prefix=" ... " ' ... ' inplace=0 filenames.txt inplace=1 results.txt

This will turn in-place editing off for filenames.txt and on again for results.txt.

1

With sed you can collect the filenames in hold space and then for all lines in results.txt check for matches to filter what lines to change:

sed -e '1,/^$/{H;1h;d;}' -e 'G;/^>\(.*\).*\n\1\n/s_^>_>h-19/US/CA-_;P;d' filename.txt <((echo)) results.txt
  • You see that I pass an empty line with <((echo)) between the files, so 1,/^$/ addresses all lines of the first file (and the empty line)
  • Those lines get appended to hold space and then deleted with H;1h;d (1h avoids starting the hold space with a newline)
  • G appends the hold space to all lines of result.txt and /^>\(.*\).*\n\1\n/ matches those lines that start with > and a string which is a filename (enclosed in newlines in the hold space)
  • s_^>_>h-19/US/CA-_ does the replacement for those line
  • P;d prints just the first line without the appended junk. You could do s/\n.*// instead
1

Use perl for inplace edits to input file:

pfx='h-19/US/CA-' \
perl -pi -e '
  BEGIN { %h = map { tr/\n//dr => $ENV{pfx}} <STDIN>}
  s/^>\K(?=(.*))/$h{$1}/;
' results.txt < filename.txt

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .