Bash: Zipping/grouping files by common substring

Question

I have about 100 files.

They are named like this.

3000_ABCD_XXXXXXX.csv
3000_ABCD_YYYYYYY.csv
3000_ABCD_XYXYZYZ.csv

3000_EFGH_XXXXXXX.csv
3000_EFGH_YYYYYYY.csv
3000_EFGH_XYXYZYZ.csv

3000_IJKL_XXXXXXX.csv
3000_IJKL_YYYYYYY.csv
3000_IJKL_XYXYZYZ.csv

Currently I'm zipping each file individually but I want to group them based on their common substring like for example ABCD.zip will store

3000_ABCD_XXXXXXX.csv
3000_ABCD_YYYYYYY.csv
3000_ABCD_XYXYZYZ.csv

EFGH.zip will store

3000_EFGH_XXXXXXX.csv
3000_EFGH_YYYYYYY.csv
3000_EFGH_XYXYZYZ.csv

etc.

I'm very new to Unix/Bash scripting. Could anyone point me in the right direction?

Edit: ABCD,EFGH,IJKL are not known in advance. Their position and width within the file name is guaranteed, though.

Answer 1

With zsh:

setopt extendedglob
typeset -A a
for f (./*) {
  [[ $f = (#b)*_(*)_* ]] &&
    a[$match]+=$f$'\0'
}
for z (${(k)a}) {
  echo zip ./$z.zip ${(ps:\0:)a[$z]}
}

(remove the echo to actually do it when satisfied).

Using perl (from zsh/bash or any other non-csh-like shell):

perl -e 'for (@ARGV) {push @{$a{$1}}, $_ if (/_(.*)_/s)}
  system "echo", "zip", "./$_.zip", @{$a{$_}} for (keys %a)' ./*_*_*

(again, remove the "echo", to actually do it).

Answer 2

You could do something like:

IFS='
'
set -f
for group in $(set +f; printf '%s\n' 3000_*.csv | sed 's/3000_\([^_]*\).*/\1/' | LC_ALL=C uniq)
do
  set +f
  zip "$group.zip" "3000_$group"*.csv
done

Should work in bash or a POSIX shell, provided the filenames don't contain newlines.

Answer 3

You can try the below script.

##The find command below finds all the csv files in the current directory. 

find ~/home/file-directory-location/*.csv -type f > filenames.txt

##We know the second substring after _ will contain the index. 
##I am sorting the file based on that second substring and getting the 
##indices into a new file for zipping.
##The uniq will specify how many zip files we are creating.  

LC_ALL=C sort -t_ -k2,2 filenames.txt | cut -d '_' -f 2 | LC_ALL=C uniq > indexes

##Now, for the created indices just zip the CSV files based on the index name. 
while read index; 
do
        tar cvzf "$index".tgz /home/file-directory-location/3000_"$index"*
done <indexes