Currently my mbsync
is erroring out with a bunch of duplicated UIDs
. This means I have to find all files that have duplicates of the sub-string "U=[0-9]+:"
and change the most recent file to remove that sub-string.
For example, a simple case on one file would be:
$> fd ".*U=17:.*" --exec ls -la {} \;
-rw------- 1 djm staff 95903 Mar 2 06:57 cur/1583291317.13980_115.DJM2,U=17:2,ST
-rw------- 1 djm staff 13654 Sep 30 2015 cur/1580615936.64042_2698.DJM2,U=17:2,S
$> mv ./cur/1583291317.13980_115.DJM2,U=17:2,ST ./cur/1583291317.13980_115.DJM2
The problem is, I have about 1000 of these to do. So I'm trying to
- find all files that have that matching sub-string,
- sort them by the matching sub-string,
- show only the duplicate matching files
- Of those duplicates change the name of the latest file to remove that sub-string.
I'm having trouble finding all the duplicates while keeping the file-path intact to use later when changing the file name.
This is what I have so far:
fd ".*U=[0-9]+:.*" | sort -t , -k2.3n | xargs -I{} printf "%s\t${%s//.*,}\n" "{}" "{}"
Note, I'm using the rust utility fd
similar to find
.
I find all the files matching the pattern, I then numerically sort the output split on ,
starting at the third character.
This is where I'm having trouble filtering it down to duplicates as just piping into uniq -d
gives me nothing because of the file-paths. So I'm thinking, I could printf
the sub-string spaced away from the file path and then pipe that into uniq -d
.
I'm not quite sure how to move forward. An answer would inform me how to filter my output down to duplicated file-paths based on the specified sub-string.
-u
to thesort
invocation to pick the first of dupes but that's not guaranteed to be the latest file, it'll be the first among equals as reported byfd
find ./cur/ -type f -newerXY '2020-01-10' | egrep "U=[0-9]*:" | sed 's/,/ /' | awk '{print $2, $1}' | sort -n
After that command, I would use the output and run it through a for a loop.for i in $(ls);do name=$(echo $i|sed 's/,.*//g'); mv $i $name;done