print lines where each word contains two same characters in Linux

Question

I have input like this

LTCYMM SVNNDA DTVEV QLOPGO CUPUR
MMWVJM LIVLI WBSVD UQCMW HBMDA HVVFY BWYSS
NOGWOS JIKKDI GCIQAD MXJNWE SMVFCB GIZVPA GZOHZR WJBMZS
KKPQBP BKDKRU ZTPDPL ZRLUEL HRZZKO KXSKCU YZQTBT RISNKS
VYQQC BFAWI NSZDV HKPGI KVJOC COPPS
JGU YLN MXW ACR BZA HOP
TMCVPT HBNGIH IQYGCI DTQPON WXANKG GMIYZS
CWVT BUBA NSGR MUPO LDNS

i am trying to print lines where each word contains at leats two same characters, useing grep command The longest line contains 8 words ، I think I can solve it like that ،but i feel that is the wrong way ,

grep '^.*\([A-Z]\)[^ ]*\1[^ ]* [^ ]*\([A-Z]\)[^ ]*\2[^ ]*   [^ ]*\([A-Z]\)[^ ]*\3[^ ]* [^ ]*\([A-Z]\)[^ ]*\4[^ ]* [^ ]*\([A-Z]\)[^ ]*\5[^ ]* [^ ]*\([A-Z]\)[^ ]*\6[^ ]* [^ ]*\([A-Z]\)[^ ]*\7[^ ]* [^ ]*\([A-Z]\)[^ ]*\8[^ ]*$/| .... for 7 words | for 6 ...

expected output

 LTCYMM SVNNDA DTVEV QLOPGO CUPUR
 KKPQBP BKDKRU ZTPDPL ZRLUEL HRZZKO KXSKCU YZQTBT RISNKS

Answer 1

With perl:

$ perl -ne 'print unless grep {!/(.).*\1/} /\S+/g' file
LTCYMM SVNNDA DTVEV QLOPGO CUPUR
KKPQBP BKDKRU ZTPDPL ZRLUEL HRZZKO KXSKCU YZQTBT RISNKS

Or with grep implementations with support for perl-like regexps:

$ grep -Pve '(?<!\S)(?!\S*(\S)\S*\1)\S' file
LTCYMM SVNNDA DTVEV QLOPGO CUPUR
KKPQBP BKDKRU ZTPDPL ZRLUEL HRZZKO KXSKCU YZQTBT RISNKS

That prints the lines that do not (with -v) contain a \S (non-whitespace character) that is not preceded by another non-whitespace ((?<!\S)) (or IOW that is the start of a white-space delimited word) and is not the start of a sequence of non-whitespaces one of which is repeated ((?!\S*(\S)\S*\1)). So in essence similar to (though less legible than) the perl approach above.

Note that they also print blank lines (as they don't contain words that don't have repeated characters). If you don't want them, you can exclude them which should be trivial (such as by adding a -e '^\s*$' in the grep one).

Answer 2

Using any awk in any shell on every Unix box:

awk '{
    for ( fldNr=1; fldNr<=NF; fldNr++ ) {
        numChars = length($fldNr)
        numUnq = 0
        split("",seen)       # you could use delete(seen) here in most awks
        for ( charNr=1; charNr<=numChars; charNr++ ) {
            if ( !seen[substr($fldNr,charNr,1)]++ ) {
                numUnq++
            }
        }
        if ( numUnq == numChars ) {
            next
        }
    }
    print 
}' file
LTCYMM SVNNDA DTVEV QLOPGO CUPUR
KKPQBP BKDKRU ZTPDPL ZRLUEL HRZZKO KXSKCU YZQTBT RISNKS

Answer 3

Using perl along with the all method from the List::Util module we can detect the desired lines (all words with atleast one duplicated charadter)

perl -MList::Util=all  -lane '
  print if all { /(.).*\1/ } @F;
' file

---

Using the GnU sed we can select the desired lines when we ensure all desired fields stretch from the beginning of line to end.

$ sed -En '/^\s*(\S*(\S)\S*\2\S*(\s+|$))+$/p' file

---

Another way with sed is to progressively check for a duplicated char amongst the nonwhitespace characters and don't print the pattern space as soon as no duplicates are found in a nonwhitespace run of characters.

sed -Ee 'h
  :loop
    s/^\s+|\s+$//g
    s/\S+/&\n/
    /(\S).*\1.*\n/!d
    s/^[^\n]*\n//
  /./bloop
  g
' file

---

We utilize awk and then loop over every word and every char in a word. Split word over the char and check if it breaks into more than 2 parts => dup detected in this word. Like wise at the end of current line if count of dups detected equals the number of fields => line fit to print.

awk '
{
  for (p=i=1+(w=0); i<=NF; i++) {
    while (p <= length($i)) {
      c = substr($i,p++,1)
      if (split($i,a,c) > 2) {
        w += p = 1
        break
      }
    }
  }
}
w==NF
' file

Answer 4

Here’s another solution in pure Bash — no perl, no grep, no awk.

#!/bin/bash
set -euo pipefail

containssametwice() {
  local -Ai chars=()
  local -i i
  for ((i = 0; i < ${#1}; ++i)); do
    ((++chars["${1:i:1}"] < 2)) || return 0
  done
  return 1
}

while IFS= read -r line; do
  read -ra words <<< "$line"
  for word in "${words[@]}"; do
    containssametwice "$word" || continue 2
  done
  printf '%s\n' "$line"
done