7

I have a scrambledA.txt with data in a list like so:

efrrsu
aehmmr
aeeglnr
alnors
acflno
aaahmy
aceimru
1469en
aelprsy
cehrry

and a corresponding wordlistC.txt that contains many many key-value-pairs:

...-...
...-...
1469en-ne1469
aaahmy-yamaha
aceimru-maurice
acflno-falcon
aeeglnr-general
aehmmr-hammer
aelprsy-players
alnors-larson
cehrry-cherry
efrrsu-surfer
...-...
...-...

I'm trying to match my scrambledA.txt with the correct words in the wordlistC.txt. First I tried

grep -f scrambledA.txt wordlistC.txt | cut -d'-' -f2

but it doesn't give me the output in the order of scrambledA.txt which I need. I then figured I have to do something with a for loop but couldn't get it to work - what am I doing wrong?

for line in $(cat "scrambledA.txt")
do
    #grep -f "$line" wordlistC >> scrambledBB.txt
    #echo $line | grep -f wordlistC.txt >> scrambledBBB.txt
done

Intended output is

surfer
hammer
general
larson
falcon
yamaha
maurice
ne1469
players
cherry
0

5 Answers 5

13

Strictly speaking, you are getting the words in the order they occur in your input file. It's just that your "input file" to grep is the wordlist file, not your list of scrambled words.

What you are currently feeding to grep with its -f option is the list of scrambled words, which is not actually the patterns that you want to match with. The patterns that you want to match with are the keys in your wordlist file (the strings before the first - on each line). You would want to match those against the words in your scrambled file to pick out the corresponding unscrambled strings.

We can do this fairly easily with awk by just reading the wordlist into an associative array and then looking up the words from the scrambled file in that array for each line read:

$ awk -F - '!wordlist_processed { dict[$1]=$2; next }
            { print $0 in dict ? dict[$0] : "UNKNOWN WORD" }
           ' wordlistC.txt wordlist_processed=1 scrambledA.txt
surfer
hammer
general
larson
falcon
yamaha
maurice
ne1469
players
cherry

This will additionally print out the string UNKNOWN WORD for any scrambled word that does not occur as a key in the wordlist.

A totally different approach is to convert your wordlist into a sed script that simply substitutes the words in the input:

$ sed 's,\([^-]*\)-\(.*\),s/^\1$/\2/;t,' wordlistC.txt | sed -f /dev/stdin scrambledA.txt
surfer
hammer
general
larson
falcon
yamaha
maurice
ne1469
players
cherry

The first sed command generates a script that the second sed applies to the scrambled words. The script would look like this, given the example in the question (with the ...-entries removed):

s/^1469en$/ne1469/;t
s/^aaahmy$/yamaha/;t
s/^aceimru$/maurice/;t
s/^acflno$/falcon/;t
s/^aeeglnr$/general/;t
s/^aehmmr$/hammer/;t
s/^aelprsy$/players/;t
s/^alnors$/larson/;t
s/^cehrry$/cherry/;t
s/^efrrsu$/surfer/;t

(Note that we use the words from the wordlist as-is, which means that the keys would need to be proper regular expressions and that the unscrambled words would need to be valid for inclusion on the right-hand side in a sed substitution command.)

As you can see, each substitution matches a particular lone scrambled word on a line and replaces it with the unscrambled word. The naked t command after each substitution would branch to the end of the script if a substitution was made. It's to avoid making more than one substitution on any line of input.

In both approaches above, we "apply the wordlist" to the scrambled words in order to unscramble them. Not the other way around, which is what you did in your question.

0
9

Provided that your wordlistC.txt can be read entirely into memory you can descramble your scrambledA.txt using awk:

awk -F'-' '
    # Save lookup key/value
    FNR==NR { w[$1]=$2 }

    # Lookup or error
    FNR<NR { if($1 in w) { print w[$1] } else { print "Unknown:", $1 } }
' wordlistC.txt scrambledA.txt

Output

surfer
hammer
general
larson
falcon
yamaha
maurice
ne1469
players
cherry
1
  • This. The question asks about grep, but grep is not the right tool for the needed task. Awk is perfect for this kind of tasks. Apr 16 at 9:41
3

A common technique when you need to preserve the order of a list is to add numbers to it before processing. Arrange to preserve the numbers and use them to sort after processing, then you can cut the numbers out.

nl scrambledA.txt | … | sort -k1n | cut -f2

You can use join to merge lines from two files according to the value of a field. Note that both inputs must be sorted according to the common field. When joining, retain the numbering from the scrambled file, and the descrambled word from the word list: you don't need the scrambled word anymore.

nl scrambledA.txt | sort -k2 |
join -t $'\t' -1 2 -2 1 -o 1.1,2.2 - <(<wordlistC.txt tr - \\t | sort) |
sort -t $'\t' -k1n | cut -f2
1
  • @dhm I had tested something almost like it, but made a typo when reformatting it to post it here. Fixed. Apr 16 at 8:56
1

You can control the order of search by putting your command in a loop:

$ while IFS= read -r term; do grep -F "${term}" wordlistC.txt || echo "not found: ${term}"; done < scrambledA.tx | cut -d'-' -f2
t
surfer
hammer
general
larson
falcon
yamaha
maurice
ne1469
players
cherry
1
  • Calling grep like this multiple times in a loop would take orders of magnitude longer to run and be less robust and less portable compared to calling awk once.
    – Ed Morton
    Apr 16 at 11:00
1

Using Raku (formerly known as Perl_6)

#taking scrambledA.txt inline:

~$ raku -ne 'BEGIN my @scrambled = <efrrsu aehmmr aeeglnr alnors acflno aaahmy aceimru 1469en aelprsy cehrry>; 
               state %hash; 
               %hash.append: $_.split("-"); 
             END put %hash{$_} // next for @scrambled;'  wordlistC.txt

OR:

#taking scrambledA.txt using a filepath:

~$ raku -ne 'BEGIN my @scrambled = "/path/to/scrambledA.txt".IO.lines; 
               state %hash; 
               %hash.append: $_.split("-"); 
             END put %hash{$_} // next for @scrambled;'  wordlistC.txt

Above are answers coded in Raku, a member of the Perl-family of programming languages. Raku is called with the -ne non-autoprinting (awk-like) linewise command line flags.

Like Perl, Raku has hashes (i.e. associative arrays / dictionaries) built-in. The strategy used above is to take the scrambled words in as an @-sigiled array, to maintain the desired output order (taking scrambled words either of two ways, above). Then the associative wordlist is split to create a %hash, with the scrambled word as the key and the plaintext word as the value.

Finally in the END block, we iterate through the @scrambled array using for: each element (key) is looked-up in the hash via %hash{$_}, and the corresponding value is output (if found).

Sample Input (scrambledA.txt):

efrrsu
aehmmr
aeeglnr
alnors
acflno
aaahmy
aceimru
1469en
aelprsy
cehrry

Sample Input (wordlistC.txt):

1469en-ne1469
aaahmy-yamaha
aceimru-maurice
acflno-falcon
aeeglnr-general
aehmmr-hammer
aelprsy-players
alnors-larson
cehrry-cherry
efrrsu-surfer

Sample Output:

surfer
hammer
general
larson
falcon
yamaha
maurice
ne1469
players
cherry

Scrambled words that cannot be looked-up in the wordlistC.txt file are skipped using // next ("defined-or" next). Instead of using next to only return matches, you can leave blank lines for non-matches by changing the RHS of the // "defined-or" operator to "" (empty string). Otherwise--if you want more verbose feedback--add a string of your choice, like so:

END put %hash{$_} // "Unknown: $_" for @scrambled;`

https://docs.raku.org/language/hashmap#Hashes_and_maps
https://docs.raku.org/
https://raku.org

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .