3

I have a text files that are laid out in a 3x66 matrix mat.txt, like this:

0 -1 0.000532 -0.00026 0.000465 etc...
0 0.000294 1 -0.000102 -0.1146 etc...
0 -0.000134 0.0000967 1 -0.9972 etc...

The values can be thought of as 3D coordinate pairs, where the first value of each line represent an (x,y,z) coordinate, the second values of each line represent another (x,y,z) coordinate, and so on. Except for the zeros at the beginning, the numbers change by file so I need to target based on location in the text file rather than by a string.

I need to remove certain coordinates from the file, and the coordinates depend on which file it is. I thought maybe using awk to separate and delete entire columns, but I am not sure how I can dynamically read which columns to delete. I have separate text files which contain the columns that need to go.

For example:

cat delete.txt
2 5 18 27 59

Could I use awk to isolate and delete the 2nd, 5th, 18th, etc columns?

for i in $(cat delete.txt)
do
awk '{print $i}' | rm $i << mat.txt
done
  • What is your OS? this would likely be quite simple with a tool such as csvcut from the csvkit suite – steeldriver Jul 26 at 13:35
1

if infile is:

0 -1 0.000532 -0.00026 0.000465 etc...
0 0.000294 1 -0.000102 -0.1146 etc...
0 -0.000134 0.0000967 1 -0.9972 etc...

delete is the columns number you are going to delete them from your infile like:

2 4 6

with awk, you could do something like:

awk 'NR==FNR { split($0, to_delete); next }
             # split 'delete' file into an array called to_delete on default FS (white-space)
             { for (col in to_delete) $to_delete[col]=""; print }' delete infile
             # delete the columns from 'infile' that match with $column getting from array

which will gives you output with columns 2, 4 and 6 deleted from the file.

0  0.000532  0.000465
0  1  -0.1146
0  0.0000967  -0.9972
  • This works! Any idea where I can work something in there that deletes the extra whitespace left behind? – jrob Jul 26 at 14:21
  • 2
    @jrob to do that you'd change print to $0=$0; $1=$1; print. The $0=$0 tells awk to re-split $0 into new fields then the $1=$1 tells awk to recombine those fields replacing every FS (chains of white space) with OFS (a single blank char) and stripping leading/trailing white space. A wee bit arcane I admit... – Ed Morton Jul 26 at 15:43
0

It sounds like this is what you're looking for:

awk '
NR==FNR { split($0,del); next }
{
    out = sep = ""
    for (i=1; i<=NF; i++) {
        if ( !(i in del) ) {
            out = out sep $i
            sep = OFS
        }
    }
    print out
}
' delete.txt mat.txt
0

Assuming delete.txt is one line only, we may obtain the desired columns using the following code:

$ perl -psale '$. == 1 and 
   @indices2P = grep { my $c=$_+1; $d !~ /\b$c\b/ } 0 .. $#F;
   $_ = "@F[@indices2P]";
' -- -d="$(< delete.txt)" mat.txt

Results:

0 0.000532 0.000465
0 1 -0.1146
0 0.0000967 -0.9972

Explanation:

Store the columns to be deleted in the scalar variable $d and on the very first line of reading the file mat.txt compute the column indices that need to be printed.

Then apply only those indices when accessing the array @F for printing.

0
$ < delete.txt \
       tr -s ' \t' '\n\n' | sort -nru |
       sed -e 's|.*|s/\\s*\\S+//&|' |
       sed -Ef - mat.txt

Results:

0 0.000532 0.000465
0 1 -0.1146
0 0.0000967 -0.9972

Explanation:

Using GNU sed with extended regexep mode turned On we first generate a sed code that when applied to the mat.txt file would get us the output that we like.

Assumptions:

o The file delete.txt comprises only positive integers and max value < 512

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.