Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have about 100 files.

They are named like this.

3000_ABCD_XXXXXXX.csv
3000_ABCD_YYYYYYY.csv
3000_ABCD_XYXYZYZ.csv

3000_EFGH_XXXXXXX.csv
3000_EFGH_YYYYYYY.csv
3000_EFGH_XYXYZYZ.csv

3000_IJKL_XXXXXXX.csv
3000_IJKL_YYYYYYY.csv
3000_IJKL_XYXYZYZ.csv

Currently I'm zipping each file individually but I want to group them based on their common substring like for example ABCD.zip will store

3000_ABCD_XXXXXXX.csv
3000_ABCD_YYYYYYY.csv
3000_ABCD_XYXYZYZ.csv

EFGH.zip will store

3000_EFGH_XXXXXXX.csv
3000_EFGH_YYYYYYY.csv
3000_EFGH_XYXYZYZ.csv

etc.

I'm very new to Unix/Bash scripting. Could anyone point me in the right direction?

Edit: ABCD,EFGH,IJKL are not known in advance. Their position and width within the file name is guaranteed, though.

share|improve this question
    
What do you mean by "pick them up using indexes"? –  Joseph R. Jun 10 at 17:59
    
Like I know where the 'ABCD' will be in the file name i.e it will be from index 7 to 11 for eg... but i dont knw that it is called 'ABCD' –  user72070 Jun 10 at 18:24

3 Answers 3

up vote 4 down vote accepted

With zsh:

setopt extendedglob
typeset -A a
for f (./*) {
  [[ $f = (#b)*_(*)_* ]] &&
    a[$match]+=$f$'\0'
}
for z (${(k)a}) {
  echo zip ./$z.zip ${(ps:\0:)a[$z]}
}

(remove the echo to actually do it when satisfied).

Using perl (from zsh/bash or any other non-csh-like shell):

perl -e 'for (@ARGV) {push @{$a{$1}}, $_ if (/_(.*)_/s)}
  system "echo", "zip", "./$_.zip", @{$a{$_}} for (keys %a)' ./*_*_*

(again, remove the "echo", to actually do it).

share|improve this answer

You could do something like:

IFS='
'
set -f
for group in $(set +f; printf '%s\n' 3000_*.csv | sed 's/3000_\([^_]*\).*/\1/' | LC_ALL=C uniq)
do
  set +f
  zip "$group.zip" "3000_$group"*.csv
done

Should work in bash or a POSIX shell, provided the filenames don't contain newlines.

share|improve this answer
    
That assumes filenames don't contain space, tab newline, globbing (*, ?, [) characters. That will run the same zip commands several times. –  Stéphane Chazelas Jun 10 at 19:05
    
@Stéphane, fixed. Forgot the uniq at first. –  Graeme Jun 10 at 19:23
    
You also need to fix the locale to C (LC_ALL=C). Try for instance with a list of files like 3000_a_a.csv, 3000_a@b_a.csv, and 3000_a_c.csv in a typical en_US.UTF-8 locale. That's because in most non-C locales, _ is ignored (at first) for sorting. –  Stéphane Chazelas Jun 10 at 19:37

You can try the below script.

##The find command below finds all the csv files in the current directory. 

find ~/home/file-directory-location/*.csv -type f > filenames.txt

##We know the second substring after _ will contain the index. 
##I am sorting the file based on that second substring and getting the 
##indices into a new file for zipping.
##The uniq will specify how many zip files we are creating.  

LC_ALL=C sort -t_ -k2,2 filenames.txt | cut -d '_' -f 2 | LC_ALL=C uniq > indexes

##Now, for the created indices just zip the CSV files based on the index name. 
while read index; 
do
        tar cvzf "$index".tgz /home/file-directory-location/3000_"$index"*
done <indexes
share|improve this answer
    
If any of the cvs files in ~/home/file-directory-location are of type directory, then you will include all the regular files in there. I'm not sure what's the point of your find command there. You've got some missing quotes in there. You need LC_ALL=C for sort and uniq. –  Stéphane Chazelas Jun 10 at 19:08
    
I just want to get the list of files that need to be zipped before I find out the indices. So, I use the above find command to get the list of csv files. –  Ramesh Jun 10 at 19:09
    
@StéphaneChazelas, I added the quotes for the variable. Please let me know if this is fine. –  Ramesh Jun 10 at 19:18

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.