2

A PDB file contains numerous paragraphs of conformations of a protein.

Each conformation starts with the keyword ATOM and ends with keyword END.

I am trying to read the file in bash such that I read every line from ATOM until END, but I don't want to read the word END.

I want to do this for each conformations (paragraphs) and store each paragraph in an array.

The file somewhat looks like this:

ATOM line 1...

ATOM line 2...

ATOM line 3...

# More lines....

END

ATOM line 1...

ATOM line 2...

ATOM line 3...

# more lines...

END

An ATOM to END is one conformation.

I want to be able to read each conformation into an array, including the ATOM but excluding the END.

I can do read text between two keywords exclusive of both the words, but I don't know how to include the starting word, but exclude the end word.

Also reading each conformation into an array such that conf[0] = first conformation , conf[1] = second conformation so on and so forth doesn't work.

Code:

#!/bin/bash

filename='coor.pdb'
echo Start
i=0
while read line; do
    conf[$i]=$(sed -n '/ATOM/,/END/{//!p}') 
    i=i+1           
done < $filename
echo $conf[0] > first_frame.data
2
#!/bin/bash

filename='coor.pdb'
echo Start
i=1
input=false
while read -r line
do
    if [ "${line%% *}" == "ATOM" ]
    then
        input=true
    elif [ "${line%% *}" == "END" ]
    then
        ((i++))                 # increase variable i by 1 == (i+1)
        rm -f "${i}_frame.data" # remove ${i}_frame.data" if exist
        input=false             # stop output lines until next ATOM
    fi
    if $input                   # if var INPUT is true add line to ${i}_frame.data file
    then
        echo "$line" >> "${i}_frame.data"
    fi
done < "$filename"

For future some sed hints:

sed '/ATOM/,/END/!d;/END/d'
sed -n '/ATOM/{:;N;s/\nEND//;T;p}'

So you can do task:

nl -s'.frame.data' -b p"^END" coor.pdb | 
sed -n '/ATOM/{s/^/echo \"/;:;s/ \{6,\}//;N;s/END//;T;s/\n  */\">/p}' |
bash
  • Oh! thank you so very much for your prompt reply @Costas. Can I please trouble you more to explain each line from the 2nd elif/then? – shay Aug 20 '15 at 15:59
  • @shay: be aware that text processing in pure bash is quite slow compared to something like awk. The shell's read builtin has to read one character at a time. – Peter Cordes Aug 20 '15 at 16:58
  • @Costas: you should quote "${line%% *}" if you're going to use it inside single [ rather than [[, so the shell doesn't try to glob-expand it, or process any other metacharacters. (I know word-splitting isn't a problem in this special-case, because you're already stripping off ` *`... unless there's a tab instead of space.) – Peter Cordes Aug 20 '15 at 17:01
  • 1
    @PeterCordes And even inside [[ … ]], you need quotes if the variable expansion appears on the right-hand side of == or != or =~ and you want the value of the variable to be interpreted as a string rather than a pattern. – Gilles 'SO- stop being evil' Aug 20 '15 at 21:45
2

Text processing in bash is slow. Pure-bash string manipulation is good for text you already have in variables, or for reading very small files. I suspect computational biology files won't usually be small, so use a tool like awk that has a small startup cost, but processes text much more quickly than bash.

Assuming you really just want to split your pdb file:

awk -v RS='\nEND\n' '{ fn="frame" NR ".pdb"; print > fn; close(fn) }' "$filename"

Get awk to use \nEND\n as the input record separator, instead of newline, then you can even use its record counter. The output record separator is still the default ORS="\n". (very nice suggestion by Costas. I tweaked it so END has to be at the start of a line, and added close to make sure it doesn't use a ton of file-descriptors on inputs with very many conformations.)

My original idea was:

awk 'BEGIN{i=0; fn="frame0.pdb"}
     !/^END/ { print > fn; }
     /^END/{ close(fn); fn="frame" ++i ".pdb"; }' \
     "$filename"

awk caches file-handles, so multiple print > fn don't result in close-reopen of the file. (close(fn) does that. It's there just for efficiency, so awk doesn't end up with boatloads of open files.)

The logic is: print every full line to the current filename. When you see an END line, move on to the next filename. If there isn't another line after the last END, then the new filename will never be written, and no vestigial last file will be created.

OTOH, if you want to do something with an array of line-blocks in memory:

# add a `!/^END/` condition to the concat block if you want to avoid a stray newline after each END
awk 'BEGIN{i=0}  
     !/^END/ { arr[i] = arr[i] $0 "\n"; }  # concat onto this array element
     /^END/ { i++; }
     END{for (k in arr) { print arr[k]; > ("frame" k ".pdb") } }' \
"$filename"

Then you have an awk array of lines to do with as you please in the END block. It has good regex functions.


Failed attempt at bash driving sed (nvm, fails because sed doesn't read one-byte-at-a-time the way shell read does):

i=0
while true; do
    outf="frame${i}.data";
    ##### DON'T USE THIS, sed READS TOO MUCH #####
    strace -o sed.tr sed '/^END/q42' > "$outf";  # strace to see that the 2nd sed invocation finds the file empty
    ret=$?;
    ((i++));
    if [[ $ret == 0 ]];then  # sed didn't see END before EOF
        [[ -s $outf ]] || rm -f "$outf";  # clean up empty last file
        break;
    elif [[ $ret != 42 ]]; then
        echo some other sed error;
        break;
    fi;
done < "$filename"
  • 1
    If you wants just split use RowSeparator awk -v RS='END\n' '{print > NR ".frame.data"}' – Costas Aug 20 '15 at 23:14
  • @Costas: well spotted that there's a solution that simple! I tweaked that a bit and put it as the first suggestion in my answer. – Peter Cordes Aug 20 '15 at 23:44
  • OP don't want to have END included… – Costas Aug 21 '15 at 8:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.