About my code:

  1. get all .txt files
  2. for each file, execute the awk command. All files are processed in parallel
  3. if colum 3 does NOT contain "needle", write line to file


-bash: !~: event not found


ls *.txt | parallel "awk -F '\t' -v OFS='\t' '$1 { if($3 !~ /needle/){print;} ;}' {} > {}.output"


  • What am I doing wrong?

Thank you

2 Answers 2


This looks like a quoting issue. Perhaps the easiest way to debug this is to pass the option --dry-run to parallel:

$ parallel --dry-run "awk -F '\t' -v OFS='\t' '$1 { if($3 !~ /needle/){print;} ;}' {} > {}.output" ::: in
awk -F '\t' -v OFS='\t' ' { if( ~ /needle/){print;} ;}' in > in.output

There you can see that your variables $1 and $3 have disappeared, and that !~ has been expanded to ~ (in my case). All those have special meaning for the bash shell, and should be protected if they are meant for awk instead. Try the following instead (I also altered the awk script to fit your description):

parallel awk -F '\\t' -vOFS='\t' $'\'($3 !~ /needle/) { print }\'' \>{}.output {} ::: *.txt

The reason why those expansions happened is that the tokens were inside double quotes (they are within single quotes within double quotes, but the shell only concerns itself with the outer quotes).

Note that I also avoid parsing the output of ls.


When quoting gets too tough consider using bash functions:

myfunc() {
  awk -F '\t' -v OFS='\t' '$1 { if($3 !~ /needle/){print;} ;}' "$1" > "$1".output
export -f myfunc

parallel myfunc {} ::: *.txt

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .