I need to create a generic script to remove special characters [|!@#$%^&*\] from a particular column in a pipe delimited file.

For example, I have pipe (|) delimited files and I have a column 'Review Comments' common to all the files. I need to remove all the special character given above from the Review comments column.

I am able to delete special character from a file using sed command but I am not able to create a generic script.

  • Is "Review Comments" always in the same position, ie. the same column number? – JigglyNaga Sep 14 '16 at 9:33
  • No, Review Comments will not be in the same position. It will be in different position for different files – Chani Sep 14 '16 at 12:11
  • 1
    beware: in a pipe-delimited file, none of your field values can contain the pipe character, unless the field is quoted and you are using a proper CSV parser. – glenn jackman Sep 14 '16 at 14:25

This awk script searches for a field containing only "Review Comments" in first line, then removes all special characters from that field in all the remaining lines.

FNR==1 { 
    for ( N=1 ; N<=NF ; N++ ) { 
        if ( $N ~ /^Review Comments$/ )
FNR>1 {
{ print }
| improve this answer | |
  • I have intentionally omitted "|" from the list of characters to remove, but including it will make no difference, because it's the field-separator/delimiter. – JigglyNaga Sep 15 '16 at 9:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.