Piping ls to grep. Why does grep disregard the output from the ls command?

Question

I am trying to use this command ls -p | egrep "\<[A-Z]+\>" to print all uppercase file and dir names with dir names having / appended. I do not understand why my command works even though I said to grep only file/dir names with all upper case letters. I get the correct output but don't understand how a dir like XXX/ is also listed by grep.

Answer 1

I presume that you question is "why does DIRNAME/ match the extended regular expression \<[A-Z]+\>, even though it has a character in it that is not an uppercase letter (/)?".

Your regular expression matches anything that contains a word that is all uppercased:

$ printf '%s\n' "this is not matched" "this IS matched" | egrep "\<[A-Z]+\>"
this IS matched

The \< and \> matches a zero-width "word boundary", i.e. the space between a "word character" and a character that is not of that type (or the start/end of the line). A word character is any character in the class [[:alpha:]_] (letters and underscore).

Your expression matches something like DIRNAME/ since DIRNAME matches the expression (there's a word boundary before the D and after the E).

---

To filter out specific names from a directory listing, don't use grep or other line-based text manipulation tools. Filenames can contain newlines, so line-based tools would have a hard time doing the right thing unless you impose restrictions on filenames.

Instead, to get all uppercase names in a directory in bash:

$ ls -p -d *
DIR/     FILE     TEST123  dir/     file     test123

$ ls -p -d !(*[[:lower:]]*)
DIR/     FILE     TEST123

This requires shopt -s extglob to enable extended globbing patterns. The extended globbing pattern !(*[[:lower:]]*) matches anything that does not contain any lowercase letters.

Note that the pattern does not care about the / that ls -p adds to directory names. This is because the pattern matches filenames, and there is no filename containing the character /. Also, the pattern is expanded before ls is invoked.

If you additionally want to weed out names containing digits, use

$ ls -p -d !(*[[:lower:][:digit:]]*)
DIR/  FILE

(this excludes any name that contains at least one lowercase letter or digit) or,

$ ls -p -d !(*[[:lower:]]*|*[[:digit:]]*)
DIR/  FILE

(this excludes any name that contains at least one lowercase letter, and also excludes any name that contains at least one digit).

Answer 2

The syntax \> means match at end of word. What you want is probably

ls | egrep "^[A-Z.]+$"

where ^ matches at the beginning of the line and and $ matches at the end of the line.

Answer 3

$  ls -p | egrep "[A-Z]+"
ABC
$  ls -p ???             
ABC  fif  out

For me it was the word-boundary matching that changed the behaviour.

ls -p | egrep "\<[A-Z]+\>" gives no output in my case.

Compare ls -f |cat and ls -p |cat. When you start tweaking your regex you will find out: one shouldn't bother with these things: too complicated, and in the end it is not advised to parse ls systematically: ls output is only a "report", not a list of files to work with i.e. to do operations on. (Like a screenshot of a "explorer" window in MS Windows, almost.)

$  ls -p |cat 
ABC
XYZ/

$  ls -f |cat 
XYZ
ABC

Even the ordering is different; that slash / should not be used to filter directories.

It takes something like find . -type d (lists only directories, in all subdirs also) and then you have a good starting point for any filesystem search.

$  find . -type d 
.
./XYZ
./d3
./d3/d2
./d3/d1
./d2
./d1

Your specifation is right between a simple ls (maybe with a grep trick) and some clean solution with find. I don't give one - all depends on your temperament and your future plans regarding listing of "raw" filesystem info.

This illustrates the difference:

 $  find . -name "d1"
./d3/d1
./d1

$  ls -R |grep d1
d1
./d1:
d1
./d3/d1:

With ls-grep you get "additional" lines and characters. In a desperate situation, in the shell interactively: why not? (you know that file "d1" is somewhere)

But anything scripted should rely on find and it's many possibilities. It turns your filesystem into a database: query and report: two steps. ls is only the quick all-in-one tool on the command line.

$ ls -p -d !(*[[:lower:]]*|*[[:digit:]]*)

Is this really serious? What about some strategical advice, instead of this? See comments etc.

O maybe you want to be around for every one liner any user ever could come up with. So baroque. Me I do roots style.

---