How to split files with common prefix in bash

Question

I have a directory with many files with common prefix, like gsc*_other.foo, where * is a character that goes from 0 to 9 or a/b (where a stands for 10 and b for 11). My goal is to create text files with the same gsc* per each one (that is, the first text file has all the gsc0_other.foo, the second all the gsc1_other.foo, and so on). I just know that I need something like split, but splitting over a common prefix, instead of a number of lines or size of the files. I already made a similar script, to list files up to a maximum number of files:

find ./J0902-405/*.evt -maxdepth 1 -type f -printf '%f\n' | 
  split -l498 -d - files_xselect.list

And I looked for many other way to change this script in what I need (like more split, xargs, sed), but don't have a way to go along with that.

Answer 1

There are many ways to achieve this, I'd just loop over the different values you'd like to split, i.e.

for i in {0..9} a b; do
    ls -1 gsc"${i}"* > filelist"${i}".txt
done

This will effectively run

ls -1 gsc0* > filelist0.txt
ls -1 gsc1* > filelist1.txt
ls -1 gsc2* > filelist2.txt
...

Note that if no file exist, the error message will be printed on stderr, i.e. on the terminal, not redirected to the filelist, the filelist will be created but remains empty.

Answer 2

Assuming all your files are of the pattern gsc*_*, I mean where the string between the gsc and the _ is whatyou want to use as the index, this should work:

find ./J0902-405/*.evt -maxdepth 1 -type f -printf '%f\n' | 
   awk -F'_' '{print $0 > $1"_list.txt"}'

That will create files called gsc0_list.txt, gsc1_list.txt etc.

The trick is to give awk an underscore (_) as its field separator (-F'_') so that the first field ($1) will be gsc0, or gsc1 or gsc11 etc. Then, you append (print >>) the current line (the file name) to a file called "whatever value $1 has"_list.txt.

---

A test:

$ tree
├── foo
│   └── bar
│       ├── gsc0_12630.foo
│       ├── gsc10_14894.foo
│       ├── gsc11_23911.foo
│       ├── gsc1_18215.foo
│       ├── gsc2_14017.foo
│       ├── gsc3_22263.foo
│       ├── gsc4_16461.foo
│       ├── gsc5_29327.foo
│       ├── gsc6_14337.foo
│       ├── gsc7_27295.foo
│       ├── gsc8_7591.foo
│       └── gsc9_31840.foo
├── gsc0_26853.foo
├── gsc10_30741.foo
├── gsc11_27136.foo
├── gsc1_25097.foo
├── gsc2_1446.foo
├── gsc3_7110.foo
├── gsc4_7399.foo
├── gsc5_14557.foo
├── gsc6_21869.foo
├── gsc7_13413.foo
├── gsc8_2952.foo
└── gsc9_20981.foo

$ find . -type f -printf '%f\n' | awk -F'_' '{print $0 > $1"_list.txt"}'
$ ls *txt
gsc0_list.txt   gsc11_list.txt  gsc2_list.txt  gsc4_list.txt  
gsc6_list.txt   gsc8_list.txt   gsc10_list.txt gsc1_list.txt   
gsc3_list.txt  gsc5_list.txt  gsc7_list.txt  gsc9_list.txt
$ cat gsc6_list.txt
gsc6_21869.foo
gsc6_14337.foo

Note that this does not give you the path of the file and you can easily have duplicate names but you are explicitly removing the path with your find -printf command so I assume that's what you want to do.