Parsing files in directory

Question

I have several files in a directory: file1.txt file2.txt file3.txt

For each file in the directory, I want to print only unique columns. Column 1 will match either 3 or 4; I want to print the columns that are unique and save it as f_parsed.txt

file1.txt:

gene1   description1    gene1   gene88 
gene56  description2    gene67  gene56
gene6   description3    gene95  gene6

file1_parsed.txt:

gene1   description1    gene88  
gene56  description2    gene67  
gene6   description3    gene95  

Here is the code that I have so far:

for f in *.txt ; do while IFS= read -r line; do awk  -F "," '{if ($3 = $1) {print $1, $2, $3} else {print $1, $2, $4}}' > $f_parsed.txt;
    done

Then, for each parsed file, I want to grep the gene in column 3 of f_parsed.txt and look for it in file_B.txt and return line with match and the following line. All lines with a match saved as match1.txt (next file will become match2.txt)

file_B.fasta looks like this:

>gene88 | shahid | ahifehhuh
TAGTCTTTCAAAAGA...
>gene6 | shahid | ahifehhuh
TAGTCTTTCAAAAGA...
>gene4 | jeiai | dhdhd
GTCAGTTTTTA...
>gene67 | vdiic | behej
GTCAGTTTTTA...
>gene95 | siis | ahifehhniniuh
TAGTCTTTCAAAAGA...
...

cat f_parsed.txt | while IFS= read -r line; do grep "$3" file_B.fasta  |awk '{x=NR+1}(NR<=x){print}' > match1.txt ; done

My final output for the example file I began with should be called match1.txt and look like

>gene88 | shahid | ahifehhuh
TAGTCTTTCAAAAGA...
>gene67 | vdiic | behej
GTCAGTTTTTA...
>gene95 | siis | ahifehhniniuh
TAGTCTTTCAAAAGA...

Thanks in advance! I know the code is very rough, but I'm a beginner.

Answer 1

One approach can be as follows. We read in the fasta file first and build up an array keyed on the gene name. Value corresponding to this key is the current n next lines , newline separated.

Output is saved in match*.txt files.

awk -F '|' '
  # @the beginning of file, get its type
  FNR==1 {  inCsv = !(inFasta = FS == "|") }

  # get gene name n record next line number
  inFasta && /^>/ {
    t=$0; gene=$1
    gsub(/^.|[[:space:]]*$/, "", gene)
    nxtln=NR+1
  }
  # fill up the value for the current gene
  inFasta && NR==nxtln { a[gene] = t ORS $0 }

  # we are in CSV file
  # close previously open filehandle
  # open fresh file handle (match*.txt)
  # write to filehandle based on equality
  # of field1 and field3
  inCsv && NF>3 {
    if (FNR == 1) {
      close(outf)
      outf = "match" ++k ".txt"
    }
    print a[$($1==$3?4:3)] > outf
  }

' file_B.fasta FS=, file*.txt

$ cat match1.txt

>gene88 | shahid | ahifehhuh
TAGTCTTTCAAAAGA...
>gene67 | vdiic | behej
GTCAGTTTTTA...
>gene95 | siis | ahifehhniniuh
TAGTCTTTCAAAAGA..

Answer 2

awk '{if($1 == $3) {print $1,$2,$NF}else{if($1 == $NF){print $1,$2,$3}}}' filename

output

gene1 description1 gene88
gene56 description2 gene67
gene6 description3 gene95