Move files into separate directories based on grouping defined in a CSV file

Question

RELATED: Move all files with matching prefixes to folder based on a csv list

I have a two-column CSV file (comma-separated values), with headers:

"id","group"
"F1256","old"
"E51651","new"
"X56369","new"
"G5481369","old"
"54564564T","old"
"544-5F5","new"
"1298FFF","old"
"JKL-wew_w","new"

And I have these files in a single directory:

2014-12-15_T921_F1256.png
E51651_hf_2018-9-19.jpg
hf_oldX56369_15-10-2014.xml
2018-07_xx54564564T_hfdata.bmp
G5481369oldbackup_2018-01-01.txt

I want to use grep (or any other similar tool) on these files, and match their filenames against the id column of my CSV file. Upon a match (i.e. if id is exactly found in the filename), the file has to be moved to the appropriate group sub-directory.

Therefore, two folders, old and new must be created in the current directory and all those files must be moved according to the described condition.

RESULT

old
├──2014-12-15_T921_F1256.png
├──2018-07_xx54564564T_hfdata.bmp
├──G5481369oldbackup_2018-01-01.txt

new
├──E51651_hf_2018-9-19.jpg
├──hf_oldX56369_15-10-2014.xml

How can I do this?

Answer 1

An awk solution can be:

awk -F, 'NR>1 { group[$2]= group[$2]? group[$2] "* *" $1: $1 ;next }
    END { 
        for (x in group) printf( "echo mv *%s* -t %s\n" , group[x], x )
    }' infile.csv| sh

Remove echo if you were happy with the result.

.
├── infile.csv
├── new
│   ├── E51651_hf_2018-9-19.jpg
│   └── hf_oldX56369_15-10-2014.xml
└── old
    ├── 2014-12-15_T921_F1256.png
    ├── 2018-07_xx54564564T_hfdata.bmp
    └── G5481369oldbackup_2018-01-01.txt

this will move all files belonging to related directory group at once.
about the awk explanation, please look at my recent answer.

Answer 2

You could do this by first sed n xargs

 sed -e '
       s/","/* /;s/^"/*/;s/"$//;1d
 '   |  xargs -l sh -c 'mv $1 "$2"'  _

Note : all the caveats apply that come with using an xargs pipeline, e. g. quotes, whitespace, etc.

Answer 3

In Python:

import csv, os, glob
filenames = []
filedir = 'files'
with open('filelist.csv', 'rb') as f:
    reader = csv.reader(f)
    filelist = list(reader)
filelist.pop(0)

for k, filename in enumerate(glob.glob(filedir + '/*')):
    filenames.append(os.path.basename(filename))

for (id, directory) in filelist:
    matches = [e for e in filenames if id in e]
    for (filename) in matches:
        if not os.path.exists(directory):
            os.makedirs(directory)
        os.rename(filedir + '/' + filename, directory + '/' + filename)

Answer 4

Try this:

    #!/bin/bash

input_file="$1"
base_dir="$2"
delim=","

while read -r line
do
    id=${line%$delim*}
    group=${line#*$delim}
    mv *"${id}"* "$base_dir//$group"
done < "$input_file"

This example process input file which does not contains the definition (first) line, and id, group are without quotation marks