0

I have a file with different words which correspond with my patterns

WORDS.TXT

842
897
907
967
995
1065
1095

And other file containing the lines that I want to remove when match with the word.txt

816.42743   9246.4688
832.41711   8867.3076
842.51001   23988.459
859.42639   5776.8726
870.53418   5992.9668
875.41742   6930.3584 
877.43384   35639.199 
885.51709   60137.957
891.41595   5480.983

Therefore, the new fille should remove line 3 bacause it matches with a word of the pattern, 842.

The final file should be:

816.42743   9246.4688
832.41711   8867.3076
859.42639   5776.8726
870.53418   5992.9668
875.41742   6930.3584 
877.43384   35639.199 
885.51709   60137.957
891.41595   5480.983

I also need the pattern of the words (word.txt) recognizes only the numbers before the point. Thanks, I hope to be clear

0
3

Using a way recommended into GNU awk manual to parse the files:

awk -F '.' 'pass == 1 {seen[$0]}
            pass == 2 && !($1 in seen)' pass=1 words.txt pass=2 file

At first pass, we save every line ($0) of words.txt as a key of the associative array seen.

At second pass, we print only lines form file where the first field ($1) does not exist into seen.

-F '.' sets the field separator (FS) to the literal dot.

Output:

816.42743   9246.4688
832.41711   8867.3076
859.42639   5776.8726
870.53418   5992.9668
875.41742   6930.3584 
877.43384   35639.199 
885.51709   60137.957
891.41595   5480.983
8
  • That's a suggested way to read the same file twice, not to read 2 different files. – Ed Morton Nov 13 '20 at 13:16
  • Indeed. I don't find any reason for not expanding to 2 different files. – thanasisp Nov 13 '20 at 13:25
  • It's fine, it's just not the recommended way ... to parse two files. It's not necessary as there are other ways to do the same thing with existing constructs in all awks (e.g. NR==FNR or FILENAME==ARGV[1], etc.) and GNU awk additionally has ARGIND, it's just when you have to process the same file twice that an incremental variable between appearances of that file becomes more useful. – Ed Morton Nov 13 '20 at 15:33
  • ok, I rephrased that. Of course I use FNR==NR or anything like FNR==1 for many files to identify the change of input file, because they are convenient. Probably for the limits of the input space, e.g. an empty file, using a variabe or ARGV is safier. It's good to have documented many ways of doing thngs. – thanasisp Nov 13 '20 at 16:04
  • I do too, but be aware that Stéphane Chazela recently noted correctly that the FNR==NR approach fails if the first file turns out to be empty. – AdminBee Nov 13 '20 at 16:21
2
awk -F'.' 'NR==FNR{a[$1]; next} !($1 in a)' words.txt file2
1

You can use grep, but you need another tool to enforce the "before the point" requirement

grep -v -w -f <(sed 's/^/^/' words.txt) file2
  • grep can take a file containing patterns: -f
  • but we want to transform the file of numbers to regular expressions, so the process substitution with sed
  • you want to match whole words: -w
  • and you want to invert the matching: -v

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.