A PDB file contains numerous paragraphs of conformations of a protein.

Each conformation starts with the keyword ATOM and ends with keyword END.

I am trying to read the file in bash such that I read every line from ATOM until END, but I don't want to read the word END.

I want to do this for each conformations (paragraphs) and store each paragraph in an array.

The file somewhat looks like this:

ATOM line 1...

ATOM line 2...

ATOM line 3...

# More lines....


ATOM line 1...

ATOM line 2...

ATOM line 3...

# more lines...


An ATOM to END is one conformation.

I want to be able to read each conformation into an array, including the ATOM but excluding the END.

I can do read text between two keywords exclusive of both the words, but I don't know how to include the starting word, but exclude the end word.

Also reading each conformation into an array such that conf[0] = first conformation , conf[1] = second conformation so on and so forth doesn't work.



echo Start
while read line; do
    conf[$i]=$(sed -n '/ATOM/,/END/{//!p}') 
done < $filename
echo $conf[0] > first_frame.data

2 Answers 2


echo Start
while read -r line
    if [ "${line%% *}" == "ATOM" ]
    elif [ "${line%% *}" == "END" ]
        ((i++))                 # increase variable i by 1 == (i+1)
        rm -f "${i}_frame.data" # remove ${i}_frame.data" if exist
        input=false             # stop output lines until next ATOM
    if $input                   # if var INPUT is true add line to ${i}_frame.data file
        echo "$line" >> "${i}_frame.data"
done < "$filename"

For future some sed hints:

sed '/ATOM/,/END/!d;/END/d'
sed -n '/ATOM/{:;N;s/\nEND//;T;p}'

So you can do task:

nl -s'.frame.data' -b p"^END" coor.pdb | 
sed -n '/ATOM/{s/^/echo \"/;:;s/ \{6,\}//;N;s/END//;T;s/\n  */\">/p}' |
  • Oh! thank you so very much for your prompt reply @Costas. Can I please trouble you more to explain each line from the 2nd elif/then?
    – shay
    Aug 20, 2015 at 15:59
  • @shay: be aware that text processing in pure bash is quite slow compared to something like awk. The shell's read builtin has to read one character at a time. Aug 20, 2015 at 16:58
  • @Costas: you should quote "${line%% *}" if you're going to use it inside single [ rather than [[, so the shell doesn't try to glob-expand it, or process any other metacharacters. (I know word-splitting isn't a problem in this special-case, because you're already stripping off ` *`... unless there's a tab instead of space.) Aug 20, 2015 at 17:01
  • 1
    @PeterCordes And even inside [[ … ]], you need quotes if the variable expansion appears on the right-hand side of == or != or =~ and you want the value of the variable to be interpreted as a string rather than a pattern. Aug 20, 2015 at 21:45

Text processing in bash is slow. Pure-bash string manipulation is good for text you already have in variables, or for reading very small files. I suspect computational biology files won't usually be small, so use a tool like awk that has a small startup cost, but processes text much more quickly than bash.

Assuming you really just want to split your pdb file:

awk -v RS='\nEND\n' '{ fn="frame" NR ".pdb"; print > fn; close(fn) }' "$filename"

Get awk to use \nEND\n as the input record separator, instead of newline, then you can even use its record counter. The output record separator is still the default ORS="\n". (very nice suggestion by Costas. I tweaked it so END has to be at the start of a line, and added close to make sure it doesn't use a ton of file-descriptors on inputs with very many conformations.)

My original idea was:

awk 'BEGIN{i=0; fn="frame0.pdb"}
     !/^END/ { print > fn; }
     /^END/{ close(fn); fn="frame" ++i ".pdb"; }' \

awk caches file-handles, so multiple print > fn don't result in close-reopen of the file. (close(fn) does that. It's there just for efficiency, so awk doesn't end up with boatloads of open files.)

The logic is: print every full line to the current filename. When you see an END line, move on to the next filename. If there isn't another line after the last END, then the new filename will never be written, and no vestigial last file will be created.

OTOH, if you want to do something with an array of line-blocks in memory:

# add a `!/^END/` condition to the concat block if you want to avoid a stray newline after each END
awk 'BEGIN{i=0}  
     !/^END/ { arr[i] = arr[i] $0 "\n"; }  # concat onto this array element
     /^END/ { i++; }
     END{for (k in arr) { print arr[k]; > ("frame" k ".pdb") } }' \

Then you have an awk array of lines to do with as you please in the END block. It has good regex functions.

Failed attempt at bash driving sed (nvm, fails because sed doesn't read one-byte-at-a-time the way shell read does):

while true; do
    ##### DON'T USE THIS, sed READS TOO MUCH #####
    strace -o sed.tr sed '/^END/q42' > "$outf";  # strace to see that the 2nd sed invocation finds the file empty
    if [[ $ret == 0 ]];then  # sed didn't see END before EOF
        [[ -s $outf ]] || rm -f "$outf";  # clean up empty last file
    elif [[ $ret != 42 ]]; then
        echo some other sed error;
done < "$filename"
  • 1
    If you wants just split use RowSeparator awk -v RS='END\n' '{print > NR ".frame.data"}'
    – Costas
    Aug 20, 2015 at 23:14
  • @Costas: well spotted that there's a solution that simple! I tweaked that a bit and put it as the first suggestion in my answer. Aug 20, 2015 at 23:44
  • OP don't want to have END included…
    – Costas
    Aug 21, 2015 at 8:13

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .