2

I have access to busybox only, v 1.27.

I'm trying to use a huge TXT file I extracted from a PDF document as a TXT dictionary, to be console friendly. The word I'm trying to get is the very first word of the line. What's next on the line, is the definition.

Word I'm grepping for:

patético

The one liner I'm using is this:

cat ./rae.txt | grep '^patético'

cat is being used because I have a quicker access to delete the word I'm looking for and that's more efficient. It saves a couple of keystrokes by just deleting the last part of the line, the term.

Most of the time I get the definition of the word I'm searching, for example anhelo.

$ grep -e '^anhelo' ./rae.txt
anhelo. (Del lat. anhelus). 1. m. Deseo vehemente.     
anheloso, sa. (Del lat. anhelosus). 1. adj. Que tiene o siente anhelo. Anhelosos de con-   cluir. 2. adj. Propio de lo que muestra anhelo. Mirada, búsqueda anhelosa. 3. adj. Dicho de la   respiración: Frecuente y fatigosa. 4. adj. desus. Que respira de este modo.     

As you can see, each line has the definition, anhelo and anheloso.

First I thought the issue was the dot . and/or comma , right after the word, I assumed it was being expanded or interpreted as a special character and not as plain text. But is not the case.

My questions are:

Why I can't grep ^patético? Why this does not work?

grep -e '^patético' ./rae.txt

The text I'm running grep on is:

patería. 1. f. coloq. Chile y Perú. Muestra ocasional y fingida de amistad.     

paternal. (De paterno). 1. adj. Propio del afecto, cariño o solicitud de padre.     

paternalismo. 1. m. Tendencia a aplicar las formas de autoridad y protección propias del   padre en la familia tradicional a relaciones sociales de otro tipo; políticas, laborales, etc. U. m.   en sent. peyor.     

paternalista. 1. adj. Dicho de una persona: Que adopta el paternalismo como forma de   conducta. U. t. c. s. 2. adj. Que responde o parece responder a dicha actitud.     

paternalmente. 1. adv. m. De modo propio o digno de un padre.     

paternidad. (Del lat. paternitas, -atis). 1. f. Cualidad de padre. 2. f. Tratamiento que en   algunas órdenes dan los religiosos inferiores a los padres condecorados de su orden, y que   los seculares dan por reverencia a todos los religiosos en general, considerándolos como   padres espirituales.     

paterno, na. (Del lat. paternus). 1. adj. Perteneciente o relativo al padre. 2. adj. Propio   del padre. 3. adj. Derivado de él.  V. casa paterna     

paternóster. (Del lat. Pater noster, Padre nuestro, palabras con que principia la oración   dominical). 1. m. Oración del padrenuestro. 2. m. Padrenuestro que se dice en la misa, y es   una de las partes de ella. 3. m. Cada uno de los aditamentos de alambre que se adaptan al   chambel para aumentarle su capacidad de pesca. 4. m. Chambel ya preparado con estos   artilugios. 5. m. coloq. Nudo gordo y muy apretado.     

patero, ra. (De pato1). 1. adj. Chile y Perú. Adulador, lisonjeador. U. t. c. s. 2. m.   Cazador de patos salvajes.     

pateta. (De pata1). 1. m. coloq. diablo (|| príncipe de los ángeles rebelados). Ya se lo   llevó pateta. No lo hiciera pateta. 2. m. coloq. Persona que tiene un vicio en la conformación   de los pies o de las piernas. 3. m. pl. u. c. sing. Méx. diablo (|| príncipe de los ángeles rebela-   dos). EL patetas.     

patéticamente. 1. adv. m. De modo patético.     

patético, ca. (Del lat. patheticus, y este del gr. o, que impresiona, sensible). 1. adj. Que   es capaz de mover y agitar el ánimo infundiéndole afectos vehementes, y con particularidad   dolor, tristeza o melancolía.     

patetismo. 1. m. Cualidad de patético.     

patí. (De or. guar.). 1. m. Arg. y Ur. Pez grande de río, de color gris azulado con man-   chas oscuras. 2. m. C. Rica. Especie de empanada rellena de carne o papas.     

-patía. (Del lat. -pathia, y este del gr. a, de la raíz -, sufrir, experimentar). 1. elem. com-   pos. Significa 'sentimiento', 'afección' o 'dolencia'. Homeopatía, telepatía.     

patiabierto, ta. (De pata1 y abierto). 1. adj. coloq. Que tiene las piernas torcidas e irreg-   ulares, y separadas una de otra.     

By the way, is not just patético is every word with a tilde; in this word is é. My system doesn't have UTF-8 support, nor a language package installed. I assumed this should not be a problem, because letters are just a stream of bytes and also that I don't have issues if I utilize less to search for words not showing up with grep.

EDIT0

locale

LANG=C
LC_CTYPE="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_COLLATE="C"
LC_MONETARY="C"
LC_MESSAGES="C"
LC_PAPER="C"
LC_NAME="C"
LC_ADDRESS="C"
LC_TELEPHONE="C"
LC_MEASUREMENT="C"
LC_IDENTIFICATION="C"
LC_ALL=

file -i rae.txt

rae.txt: text/plain; charset=utf-8

EDIT1

Console codepage

C
POSIX

ANSI_X3.110-1983
ANSI_X3.4-1968
ISO-8859-1
ISO-8859-15
ISO-8859-2

EDIT 2 After processing rae.txt with iconv to be ISO8859-1

./rae-iso88591.txt: text/plain; charset=iso-8859-1
23
  • 3
    with ASCII text, letters are just a stream of bytes (actually, just a single byte each, for ascii). With unicode text, things are much more complicated, and you need a unicode locale installed and the system needs to be configured to use it.
    – cas
    Nov 13, 2021 at 7:31
  • 2
    @abacox Grep does not use wildcard characters. It uses regular expressions ^pat*tico means: p at the beginning, followed by a, followed by 0 or more t characters, followed by tico.
    – mattdm
    Nov 13, 2021 at 19:05
  • 1
    I expect ^pat.\+tico would probably work. But... that's got its own problems!
    – mattdm
    Nov 13, 2021 at 19:07
  • 1
    @mattdm, \+ is undefined in standard basic regex. It only works in GNU. If you want an equivalent for the ERE .+, use ..* (or in general a\{1,\} for a+)
    – ilkkachu
    Nov 13, 2021 at 19:24
  • 1
    for us, spanish speakers, the acute accent is called "tilde"; the glyph over the n is technically called "virgulilla", but this terminology is not quite strictly enforced.
    – jarnosz
    Nov 13, 2021 at 20:59

4 Answers 4

2

Well, not sure if this is your issue, but one possible reason for not finding the correct line is the two encodings for the letter é. There's the Unicode character U+00E9 (Latin small letter E with acute), which is the whole glyph in itself; and there's the combination of e and U+0301 (Combining acute accent). The UTF-8 representations of those would be c3 a9 and 65 cc 81 in hex.

Copying the text from your question, the letter comes off as U+00E9.

So:

$ grep ^patético file.txt    # no output
$ grep ^patético file.txt
patético, ca. (Del lat. patheticus, ...

And of course depending on how the system is set up, there might be issues with entering the character in the first place, but you could work around that with something like:

$ grep $'^pat\xc3\xa9tico' file.txt
patético, ca. (Del lat. patheticus, ...    

Converting to ISO 8859-1 turns it into the single byte e9, so in the C locale, a single . will be enough to match it. Though at least on my system, with an UTF-8 locale, the lone e9 byte would be an invalid character and . would not match it.

Though of course it's still an invalid character for my UTF-8 terminal, so the output looks like this:

$ iconv -f utf8 -t latin1 file.txt  | LC_ALL=C grep $'^pat.tico'
pat�tico, ca. (Del lat. patheticus, ...
6
  • Would you say that is better to work with the TXT archive according to the settings on the system? Like, if UTF-8 is supported then have the file encoded in such way, same case if just C is supported, to have the file in ISO8859-1, and so on?
    – abacox
    Nov 13, 2021 at 21:48
  • I was taking a closer look at the hex grepping you've done there. What a awesome way to get around the issue without re-encoding the content.
    – abacox
    Nov 13, 2021 at 23:30
  • 2
    +1. You could grep for both characters at once with grep '^pat\(é\|é\)tico' (or as an extended regex with grep -E '^pat(é|é)tico'. or GNU grep's perl-compatible regex with a non-capturing group: grep -P '^pat(?:é|é)tico'). You can still use $'...' instead of '...' to use the hex codes inside the parentheses if you want.
    – cas
    Nov 14, 2021 at 11:07
  • and (i should have included this earlier because it's obvious, duh) you can use bracket expressions like pat[eéé]tico in your regex too.
    – cas
    Nov 14, 2021 at 18:18
  • 2
    That's what i get for posting so late at night, I suppose. So...bracket expressions match any of the individual characters (so multi-byte chars will work in a unicode locale), but without a working unicode locale (i.e. support for multi-byte chars), they match any of the individual bytes making up a character (because without unicode support, é is not interpreted as the character é, it is interpreted as two bytes C3 and A9). A capture group with alternation, however, isn't limited to just matching individual bytes, it matches the exact byte sequences: C3A9 (é) or 65CC81 (é).
    – cas
    Nov 15, 2021 at 5:10
2

As @cas suggested, piping the utf-8 input through iconv may do the trick. Try

iconv -f utf-8 -t CP1252 rae.txt | grep -e '^patético'

Just make sure that the console codepage is Win1252 or Latin1 to make it work.

Tested with busybox-w32 v1.33 on Win10, and v1.29 on TinyCore10.

If you are running on a real flavor of *nix, you should check and/or fix the codepage of your console, just as @Inian hinted at, to make it work.

Addendum

Please notice that you may avoid guessing the CharCodes of the letters in clean 8bit codepages simply by searching

cat rae.txt | grep -e '^pat.tico'

or by two (or as many as necessary) dots on UTF-8.

cat rae.txt | grep -e '^pat..tico'

Your mileage may vary.

3
  • Thank you. Done it, no luck. Now I'm trying to get the expansion trick to work ^pat?tico. For that I'm aiming to get a 7bit encoding. So far UTF-7, ISO8859-1, CP1252 didn't make the trick. :/
    – abacox
    Nov 13, 2021 at 19:07
  • Oh!, I did not mentioned it, but I added the console codepages. It seems that at least ISO8859-1 should work. Please see my edit.
    – abacox
    Nov 13, 2021 at 19:14
  • @abacox you're confusing wildcards (i.e. shell globs) with regex. In regex, ? doesn't mean "any character". It means the the previous character (or capture group) is optional (or, "zero-or-one". compare with * which means "zero-or-more"). For "any character", use . - so ^pat.tico
    – cas
    Nov 14, 2021 at 10:54
0

YES! THANK YOU GUYS. :)

What finally worked out for me was converting the file from UTF-8 to ISO8859-1. And using regular expressions instead of wild cards. I finally got the definition I'm looking for.

grep -e '^pat.tico' ./rae-iso88591.txt

Yields

patético, ca. (Del lat. patheticus, y este del gr. o, que impresiona, sensible). 1. adj. Que   es capaz de mover y agitar el ánimo infundiéndole afectos vehementes, y con particularidad   dolor, tristeza o melancolía.     

2
  • 1
    i prefer converting utf8 to cp1252 because latin1 (8859-1) covers a narrower range, and I lose some characters in the process.
    – jarnosz
    Nov 13, 2021 at 19:32
  • Thank you. I tried with CP1252 and it does work as well. I'll take your word for it and will stick with CP1252.
    – abacox
    Nov 13, 2021 at 19:33
-2

Some time if it contains extra spaces in beginning so we used below wild card

grep "[[:space:]]*\?patético," file

output

patético, ca. (Del lat. patheticus, y este del gr. o, que impresiona, sensible). 1. adj. Que   es capaz de mover y agitar el ánimo infundiéndole afectos vehementes, y con particularidad   dolor, tristeza o melancolía.     
1
  • 2
    Can you explain why this would work and ^patético does not? You don't actually say that ^patético does not work on your system.
    – Kusalananda
    Nov 13, 2021 at 9:50

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .