Group files and directories based on first letter of their name and move them to directories of that letter

Question

I have a directory with the following content:

$ mkdir dir && cd "$_"
~/dir $ mkdir a1 a2 a3 a4 b1 c1 c2 1a 2a 2b 2c 2d 3a _1 _2
~/dir $ touch a_1 a_2 a_3 a_4 b_1 c_1 c_2 1_a 2_a 2_b 2_c 2_d 3_a __1 __2
~/dir $ ls
__1  1_a  __2  2_a  2_b  2_c  2_d  3_a  a_1  a_2  a_3  a_4  b_1  c_1  c_2
_1   1a   _2   2a   2b   2c   2d   3a   a1   a2   a3   a4   b1   c1   c2

Now I want to group all these files and directories based on their first letter and move them to directories of the same letter. So the output would be:

~/dir $ ls
_  1  2  3  a  b  c

And using exa, the tree would look something like this:

~/dir $ exa --tree
.
├── 1
│  ├── 1_a
│  └── 1a
├── 2
│  ├── 2_a
│  ├── 2_b
│  ├── 2_c
│  ├── 2_d
│  ├── 2a
│  ├── 2b
│  ├── 2c
│  └── 2d
├── 3
│  ├── 3_a
│  └── 3a
├── _
│  ├── _1
│  ├── _2
│  ├── __1
│  └── __2
├── a
│  ├── a1
│  ├── a2
│  ├── a3
│  ├── a4
│  ├── a_1
│  ├── a_2
│  ├── a_3
│  └── a_4
├── b
│  ├── b1
│  └── b_1
└── c
   ├── c1
   ├── c2
   ├── c_1
   └── c_2

I know I can move using wildcards:

~/dir $ mkdir a && mv a* a

Which throws an error:

mkdir: cannot create directory ‘a’: File exists

But it gets the job done. And I can do something like this to avoid the error:

~/dir $ mkdir temp && mv a* temp && mv temp a

And then I could use that in a for loop for every letter that I know. But the problem is that I don't know what those first letters could possibly be, we have quite a lot of letters. Is there a way I can achieve this without the need to know those letters?

Answer 1

Just iterate over all the things:

#!/bin/bash
for f in *; do 
    firstChar="${f:0:1}"; 
    mkdir -p -- "$firstChar"; 
    mv -- "$f" "$firstChar"; 
done

The -p tells mkdir not to complain if the directory already exists and ${f:0:1} is Ksh/Bash/Zsh syntax for "get the first character".

And here it is in action:

$ mkdir a1 a2 a3 a4 b1 c1 c2 1a 2a 2b 2c 2d 3a _1 _2
$ touch a_1 a_2 a_3 a_4 b_1 c_1 c_2 1_a 2_a 2_b 2_c 2_d 3_a __1 __2
$ ls -F
__1  _1/  1_a  1a/  __2  _2/  2_a  2a/  2_b  2b/  2_c  2c/  2_d  2d/  3_a  3a/  a_1  a1/  a_2  a2/  a_3  a3/  a_4  a4/  b_1  b1/  c_1  c1/  c_2  c2/

$ for f in *; do 
    firstChar="${f:0:1}"; 
    mkdir -p "$firstChar"; 
    mv -- "$f" "$firstChar"; 
done

$ tree
.
├── _
│   ├── __1
│   ├── _1
│   ├── __2
│   └── _2
├── 1
│   ├── 1_a
│   └── 1a
├── 2
│   ├── 2_a
│   ├── 2a
│   ├── 2_b
│   ├── 2b
│   ├── 2_c
│   ├── 2c
│   ├── 2_d
│   └── 2d
├── 3
│   ├── 3_a
│   └── 3a
├── a
│   ├── a_1
│   ├── a1
│   ├── a_2
│   ├── a2
│   ├── a_3
│   ├── a3
│   ├── a_4
│   └── a4
├── b
│   ├── b_1
│   └── b1
└── c
    ├── c_1
    ├── c1
    ├── c_2
    └── c2

22 directories, 15 files

Note that if you can have existing file or directory names that are only one character, that will cause errors. For instance, if you create a file called a and try the above approach, you will get:

mkdir: cannot create directory ‘a’: File exists
mv: 'a' and 'a' are the same file

If that is a problem, you could do something convoluted like renaming to a temp file, creating the directory, and moving the file into it:

for f in *; do 
    firstChar="${f:0:1}";
    ## if the first character exists as a dir, move the file there
    if [[ -d "$firstChar" ]]; then
       mv -- "$f" "$firstChar"; 
    ## if it exists but is not a dir, move to a temp location,
    ## re-create it as a dir, and move back from temp
    elif [[ -e "$firstChar" ]]; then
        tmp=$(mktemp ./tmp.XXXXX)
        mv -- "$firstChar" "$tmp"
        mkdir -p "$firstChar"
        mv -- "$tmp" "$firstChar"/"$f"
    else    
        mkdir -p "$firstChar"; 
        mv -- "$f" "$firstChar"; 
    fi
done

Answer 2

POSIXly:

find . -path './*' -prune -exec sh -c '
    j="${1%${1#???}}"; mkdir "$j" && mv -- "$1" "$j/"
' _ {} \; 2>/dev/null

Answer 3

Another approach that hasn't been mentioned is to find all the unique first letters of files and directories, move everything to a temporary directory, create the first letter directory, and then move back everything.

letters=($(ls -1 | sort -n | cut -c 1 | uniq))
temp=$(mktemp -dp .)
for letter in "${letters[@]}"; do
    mv "$letter"* "$temp";
    mkdir "$letter";
    mv "$temp/"* "$letter";
done
rm -r "$temp"

With this approach, one-letter directories and files are also taken care of. In action:

$ mkdir a1 a2 a3 a4 b1 c1 c2 1a 2a 2b 2c 2d 3a _1 _2
$ touch a_1 a_2 a_3 a_4 b_1 c_1 c_2 1_a 2_a 2_b 2_c 2_d 3_a __1 __2
$ mkdir a b c
$ touch 1 2 3 _
$ ls -F
_    1    __2  2_a  2b/  2_d  3_a  a_1  a2/  a_4  b_1  c_1  c2/
__1  1_a  _2/  2a/  2_c  2d/  3a/  a1/  a_3  a4/  b1/  c1/
_1/  1a/  2    2_b  2c/  3    a/   a_2  a3/  b/   c/   c_2

And after that script, the result becomes:

$ ls -F
_/  1/  2/  3/  a/  b/  c/
$ exa --tree
.
├── 1
│  ├── 1
│  ├── 1_a
│  └── 1a
├── 2
│  ├── 2
│  ├── 2_a
│  ├── 2_b
│  ├── 2_c
│  ├── 2_d
│  ├── 2a
│  ├── 2b
│  ├── 2c
│  └── 2d
├── 3
│  ├── 3
│  ├── 3_a
│  └── 3a
├── _
│  ├── _
│  ├── _1
│  ├── _2
│  ├── __1
│  └── __2
├── a
│  ├── a
│  ├── a1
│  ├── a2
│  ├── a3
│  ├── a4
│  ├── a_1
│  ├── a_2
│  ├── a_3
│  └── a_4
├── b
│  ├── b
│  ├── b1
│  └── b_1
└── c
   ├── c
   ├── c1
   ├── c2
   ├── c_1
   └── c_2

With this approach, you create only the necessary directories. So the mkdir commands runs the least amount of times possible.

Edit: As @terdon mentioned, parsing the output of ls will cause the command to fail on files with newline in their names. To fix that, we can use this:

temp=$(mktemp -dp .)
for f in *; do
    letter="${f:0:1}";
    mv -- "$letter"* "$temp";
    mkdir "$letter";
    mv "$temp/"* "$letter";
done
rm -r "$temp"

Answer 4

First, the solution for your move pattern is simple, just add a ?. That only matches a followed by at least another character.

mv a?* a

We don't have that many letters, but to find all actually present letters, you can use this command. sed removes everything after the first letter, sort uniq removes the duplicates.

ls | sed -r 's/^(.).*$/\1/' | sort -u

Answer 5

With zsh:

autoload -Uz zmv # best in ~/.zshrc
() { mkdir ${(uM)@#?}; } ??* && zmv '(?)?*' '$1/$f'

We have:

• () { body; } args: an anonymous function, here passed the expansion of ??* (the file names made of at least 2 characters) as arguments. In the body:
• ${@#?} would be the args stripped of their first character, but with the M (for matched) parameter expansion flag, that's reversed and only the first character is retained instead. u (for unique) removes duplicates.
• then zmv '(?)?*' '$1/$f' moves the files with at least 2 characters into their respective folder.