Unique lines with unique words

Question

Help me to print unique lines in Linux. Word examples like

A B
B C
C D
A E
F G
H I
I J

the output should be

F G
H I

I.e., the line should be unique at first and the words in the line should also appear only once as in above example “F G” is a unique line and neither “F” or “G” never exists in any other line similarly “H I”.

Answer 1

You could get the list of unique words. And then, matching lines would be lines whose all words are unique:

tr -cs 'A-Z' '[\n*]' < words.txt |
  sort |
  uniq -u |
  perl -lne '
    if ($ARGV eq "-") {
      $u{$_}=1;
      next
    }
    for $w (/[A-Z]+/g) {
      next LINE unless $u{$w}
    }
    print' - words.txt

Change A-Z to the list of characters that constitute a word.

Answer 2

You could do it like this with awk:

unique.awk

FNR == NR {
  for(i=1; i<=NF; i++)
    if(++w[$i] > 1)
      not_unique[$i] = 1
  next
}

{
  for(i=1; i<=NF; i++)
    if(not_unique[$i])
      next
}

1

Run it like this:

awk -f unique.awk infile infile

Output:

F G

As a one-liner:

awk 'FNR == NR { for(i=1; i<=NF; i++) if(++w[$i] > 1) not_unique[$i] = 1; next } { for(i=1; i<=NF; i++) if(not_unique[$i]) next } 1' infile infile

Explanation

The file needs to be parsed twice, first to find all non-unique words, then to print those lines which contain only unique words. This is reflected in the program structure, the first block creates a hash containing words that are not unique, the second checks each line and skips it if it has non-unique words. The trailing 1 at the end is only reached when unique lines occur and it invokes awk's default action ({ print $0 }).