0

I have data recovered from a backup and the data recovered has multiple versions of the same folders/files. The folders/files have been named with _000 _001 _002 etc on the files and folders. All modified time stamps are the same. So the folders/fileslook like this

[PATH  ~/Folder/9] (VERSION 1) EMPTY - and can be deleted  
[PATH  ~/Folder/9_000] (VERSION 2) EMPTY - and can be deleted  
[PATH  ~/Folder/9_001] (VERSION 3) Data is in this last one  

FILEA.TXT (VERSION 1) remove/delete this one  
FILEA_000.TXT (VERSION 2) remove/delete this one  
FILEA_001.TXT (VERSION 3) remove/delete this one  
FILEA_002.TXT (VERSION 4) I need to keep this one and then rename  

FILEB.TXT (VERSION 1) remove/delete this one  
FILEB_000.TXT (VERSION 2) I need to keep this one and then rename  

Some of these folders go 6+ deep. The pattern I have found is that the last file or folder version for each folder/file is the good one and the folders that are unwanted are empty. The bad thing is not all files/folders have the same number of versions. So just because FILEA.TXT has 4 versions (FILEA.TXT to FILEA_002.TXT) FILEB.TXT only has 2 versions (FILEB.TXT & FILEB_000.TXT). So I can't search all files with _002 and move or delete. I was easily able to delete all the unwanted empty folders with bash (find . -type d -empty -delete) which takes care of part of my problem on the folders except for renaming back.

I need to remove all the files that are unwanted and THEN remove the version numbers on folders/files so my software can correctly import the recovered data. Here is a example of a path and the files and folders in the path.

[PATH  ~/Folder/9] EMPTY - and can be deleted  
[PATH  ~/Folder/9_000] EMPTY - and can be deleted  
[PATH  ~/Folder/9_001] Data is in this last one  

[PATH  ~/Folder/9_001/62BF7CA1] EMPTY - and can be deleted  
[PATH  ~/Folder/9_001/62BF7CA1_000] EMPTY - and can be deleted  
[PATH  ~/Folder/9_001/62BF7CA1_001] Data is in the last one  

[ FOLDERS/FILES IN "~/Folder/9_001/62BF7CA1_001" ]  

Archive - unwanted deleted it  
Archive_000 - unwanted deleted it  
Archive_001 - unwanted deleted it  
Archive_002 - unwanted deleted it  
Archive_003 - keep  
Documents - unwanted deleted it  
Documents_000 - keep
FolderX - unwanted deleted it  
FolderX_000 - unwanted deleted it  
FolderX_001 - unwanted deleted it  
FolderX_002 - keep
62BF7CA1.PDF - unwanted deleted it  
62BF7CA1_000.PDF - unwanted deleted it  
62BF7CA1_001.PDF - unwanted deleted it  
62BF7CA1_002.PDF - keep  
62BF7CA1.TXT- keep
62BF7CA1.DOC - unwanted deleted it  
62BF7CA1_000.DOC - unwanted deleted it  
62BF7CA1_001.DOC - unwanted deleted it  
62BF7CA1_002.DOC - keep  
62BF7CA1.QIF - unwanted deleted it  
62BF7CA1_000.QIF - unwanted deleted it  
62BF7CA1_001.QIF - unwanted deleted it  
62BF7CA1_002.QIF - keep  

The files and folders in this path to keep would be

Archive_003  
Documents_000  
FolderX_002  
62BF7CA1_002.PDF  
62BF7CA1.TXT  
62BF7CA1_002.DOC  
62BF7CA1_002.QIF  

... and then I need to drop any _000 _001 _002 etc if it exist on each file folder

Archive  
Documents  
FolderX  
62BF7CA1.PDF  
62BF7CA1.TXT  
62BF7CA1.DOC  
62BF7CA1.QIF  

Probably easy with things like sed and pipes but I have no idea.

  • 1
    Why FILEB_000.TXT is deleted that there is no FILEB_001.TXT while Documents_000 is not deleted. What is the logic then? and are you on linux machine or windows? If you are on Windows you should ask on superuser.com – αғsнιη Feb 2 '18 at 6:13
  • Thanks for catching that it was a type-o. The data is from Windows but I am on a linux machine to manipulate the data into the form I need – wade812 Feb 2 '18 at 14:19
  • Your case is difficult to achieve with an easy solution, please take a look here for start Select greatest numbered filename – αғsнιη Feb 2 '18 at 15:00
  • So, are all the excess directories empty?  Or might you have to deal with Archive_001/FileA_001.txt, Archive_001/FileA_002.txt, Archive_002/FileA_001.txt and Archive_002/FileA_002.txt? – G-Man Feb 3 '18 at 4:23
  • Your question is not clear kindly post with input and excepted output what i have undestood and done below is code from my end @praveen_2 j1]# for i in Archive Documents EF PDF TXT DOC QIF ; do sed -n "/$i/p" l.txt | tail -1; done Archive_003 Documents_000 EF_002 62BF7CA1_002.PDF 62BF7CA1.TXT 62BF7CA1_002.DOC 62BF7CA1_002.QIF You have new mail in – Praveen Kumar BS Feb 3 '18 at 10:45
1

Iterate across all files. Find the ones that also match with the _nnn component. Identify the last one and delete the others.

If you need this to apply across a tree of folders one option is to put this into a script and call the script from a find -type d -exec construct.

for file in *.*
do
    [[ -d "$file" || $file =~ _[[:digit:]]{3}\. ]] && continue
    echo -n "Considering $file: " >&2

    extn="${file/*.}"
    versions=("$file")
    keep="$file"

    # Look at matching files
    for version in "${file%.$extn}"_???."$extn"
    do
        [[ -f "$version" ]] || continue

        # Save every one. Identify the current last
        versions+=("$version")
        keep="$version"
        echo -n "$version " >&2
    done
    echo "==> keep $keep" >&2

    # Delete them all except the last
    for version in "${versions[@]}"
    do
        [[ "$version" != "$keep" ]] && echo rm -f -- "$version"
    done
    [[ "$keep" != "$file" ]] && echo mv -f -- "$keep" "$file"
done

Remove the echo statements preceding the rm -f -- "$version" and the mv -f -- "$keep" "$file" when you're comfortable it's deleting the files you want deleted and keeping the ones you want kept.

  • Thank you for the response Roaima. I ran the for loop and it produced the output I am looking for. Appreciate it. Sorry I have to ask but how would I put this into a script and call the script from a find -type d -exec construct? like this? #!/bin/bash find -type -d -exec myscript {} \; – wade812 Feb 3 '18 at 19:15
  • (1) *.* expands to a list of filenames that contain ..  The question mentions Archive, Documents, and EF.  Yes, the first two are probably directories, but I don’t know what EF is, and *.* will ignore it.  (Of course, by definition, filenames with no . have no extension, so you would need slightly different code to handle them.)  (2) You could use globstar to search a directory tree.  … (Cont’d) – G-Man Feb 3 '18 at 19:56
  • (Cont’d) …  (3) This will mv 62bf7ca1_002.doc 62bf7ca1.doc without first doing rm 62bf7ca1.doc.  This will cause problems if 62bf7ca1.doc is read-only.  (I would suggest initializing versions=("$file"), so the unnumbered version gets deleted if there are other versions.  Doing mv -f is another option.) (4) You should probably use -- with rm and mv to protect against filenames beginning with -. – G-Man Feb 3 '18 at 19:56
  • -- G-Man -- I updated the question to make it clear what are folders and what are files. Yes 'EF' was a folder but I changed it to 'FolderX' to make it more clear. Again don't have to worry about Folder finding and renaming as I have already taken care of that. Just concerned with files. Thank you for following up with me. Appreciate it! Still need to understand how to call 'roaima' code from a contruct. :-) – wade812 Feb 3 '18 at 20:46
  • @G-Man all the examples dictated files with an extension. Those are what I have matched. I think I've fixed the remainder of your suggestions (at least one slipped through my rewrite of prototype code). Thanks – roaima Feb 3 '18 at 23:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.