Extracting duplicity backup data

Question

I've made some backups of my previous systems using Deja Dup (duplicity). In my current distribution it's not available, so I've simply extracted the data. Now in order to make that data meaningful again I have to cat the files like that:

cat ? ?? > file

Question marks are dependable from number of files of course.

The structure of extracted data is like this:

some/long/path/to/file/[original_filename_converted_to_folder.extension]/

so the filename is a folder and within that folder there are numbered files (like 1, 2..20..300..etc). Different files (which now are folders) have different number of those numbered files inside them.

So I thought to get all directories where . is present. Then if there are no subdirectories in that directory cat all files to one, give that file folder name (because it's the actual file name), transfer it one directory upwards (..) and then delete that directory (the one used to create a file with filename)

So I started writing script, but I have some issues with it:

directories = "$(find . -type d)"
for i in "$directories"
do
  fdir = "$i"
  rdir = "$(dirname "$i")";
  fname = "$(basename "$i")";
  if [[ "$(basename "$i")" == *"."* ]] then
    cd "$i"
    FILECOUNT="$(find . -type f -maxdepth 1 -printf x | wc -c)"
    DIRCOUNT="$(find . -type d -maxdepth 1 -printf x | wc -c)"
    DIRCOUNT=$(( DIRCOUNT - 1 ))
    if DIRCOUNT == 0 then
      if FILECOUNT != 0 then
        # >> this is the problematic part
        DIGITS = "$(grep -o "[s|S]" <<<"$x" | wc -l)"
          cat "$QUESTIONMARKS" > "$fname"
          mv "$fname" "$rdir" +"/"+ "$fname"+".restored"
          rm *
          cd ..
          rmdir fname
          mv "$fname"+".restored" "$fname"
        # << end of problems
      fi
    fi
  fi
done

So the problem is - how do I generate question marks depending on file number in directory? if there is a 1 digit file number in directory then command should look like this:

cat ? > file

if there is 2 digit file number in directory, then:

cat ? ?? > file

if 3, then:

cat ? ?? ??? > file

You get the idea..

After this, I wrote a list of commands which should proceed, although - do I simply add them like this or do I have to wrap them to () ?

Is it ok to cd in such script? Will I not get into some issues?

Any other possible solutions to achieving this?

Thanks.

Answer 1

If I understand correctly, you have directories that contains files called 1, 2, 3, etc. (up to a variable limit) and you want to replace each directory by a file which has the same name as the directory, and is the concatenation of the files in numerical order.

In this answer, I'm going to use zsh, because it's a lot easier to use than a combination of bash and other tools. You can translate the method to sh and find (bash and GNU find will be easier than plain sh and POSIX find).

First, to locate the directories, the presence of a dot doesn't seem like a very useful heuristics: you'll miss files whose name doesn't contain a dot and you'll hit directories whose name does contain a dot. Looking for directories that contain numerically-named files seems more reliable (though it isn't perfect, nothing is). Warning: untested code:

#!/bin/zsh
for dir in **/1(:h); do
  files=($dir/*(Nn:t))
  for ((i=1; i<=$#files; i++)); do
    # if the ith file isn't called i, skip this directory
    if [[ $files[i] != $i ]]; then files=; break; fi
  done
  if [[ -z $files ]]; then continue; fi
  # The directory contains only numbered files, and they're in sequence.
  # Concatenate the files, move the result into place and remove the directory.
  tmp=$(TMPDIR=$dir:h mktemp -d)
  mv -- $dir $tmp &&
  for x in $tmp/$dir:t/$^files; do cat -- $x; done >$tmp/file &&
  mv -- $tmp/file $dir &&
  rm -r $tmp
done

Explanation of zsh features:

• :t and :h are history expansion modifiers to take the directory part and the basename of a file respectively.
• *(Nn) uses glob qualifiers: n to sort file names in numerical order instead of lexicographic order, and N to get an empty array rather than an error if the directory is empty.
• The ^ parameter expansion modifier in $dir/$^files adds the prefix $dir/ to each element of the array files (as opposed to adding it only to the first element).