I've made some backups of my previous systems using Deja Dup (duplicity). In my current distribution it's not available, so I've simply extracted the data. Now in order to make that data meaningful again I have to cat the files like that:

cat ? ?? > file

Question marks are dependable from number of files of course.

The structure of extracted data is like this:


so the filename is a folder and within that folder there are numbered files (like 1, 2..20..300..etc). Different files (which now are folders) have different number of those numbered files inside them.

So I thought to get all directories where . is present. Then if there are no subdirectories in that directory cat all files to one, give that file folder name (because it's the actual file name), transfer it one directory upwards (..) and then delete that directory (the one used to create a file with filename)

So I started writing script, but I have some issues with it:

directories = "$(find . -type d)"
for i in "$directories"
  fdir = "$i"
  rdir = "$(dirname "$i")";
  fname = "$(basename "$i")";
  if [[ "$(basename "$i")" == *"."* ]] then
    cd "$i"
    FILECOUNT="$(find . -type f -maxdepth 1 -printf x | wc -c)"
    DIRCOUNT="$(find . -type d -maxdepth 1 -printf x | wc -c)"
    DIRCOUNT=$(( DIRCOUNT - 1 ))
    if DIRCOUNT == 0 then
      if FILECOUNT != 0 then
        # >> this is the problematic part
        DIGITS = "$(grep -o "[s|S]" <<<"$x" | wc -l)"
          cat "$QUESTIONMARKS" > "$fname"
          mv "$fname" "$rdir" +"/"+ "$fname"+".restored"
          rm *
          cd ..
          rmdir fname
          mv "$fname"+".restored" "$fname"
        # << end of problems

So the problem is - how do I generate question marks depending on file number in directory? if there is a 1 digit file number in directory then command should look like this:

cat ? > file

if there is 2 digit file number in directory, then:

cat ? ?? > file

if 3, then:

cat ? ?? ??? > file

You get the idea..

After this, I wrote a list of commands which should proceed, although - do I simply add them like this or do I have to wrap them to () ?

Is it ok to cd in such script? Will I not get into some issues?

Any other possible solutions to achieving this?



If I understand correctly, you have directories that contains files called 1, 2, 3, etc. (up to a variable limit) and you want to replace each directory by a file which has the same name as the directory, and is the concatenation of the files in numerical order.

In this answer, I'm going to use zsh, because it's a lot easier to use than a combination of bash and other tools. You can translate the method to sh and find (bash and GNU find will be easier than plain sh and POSIX find).

First, to locate the directories, the presence of a dot doesn't seem like a very useful heuristics: you'll miss files whose name doesn't contain a dot and you'll hit directories whose name does contain a dot. Looking for directories that contain numerically-named files seems more reliable (though it isn't perfect, nothing is). Warning: untested code:

for dir in **/1(:h); do
  for ((i=1; i<=$#files; i++)); do
    # if the ith file isn't called i, skip this directory
    if [[ $files[i] != $i ]]; then files=; break; fi
  if [[ -z $files ]]; then continue; fi
  # The directory contains only numbered files, and they're in sequence.
  # Concatenate the files, move the result into place and remove the directory.
  tmp=$(TMPDIR=$dir:h mktemp -d)
  mv -- $dir $tmp &&
  for x in $tmp/$dir:t/$^files; do cat -- $x; done >$tmp/file &&
  mv -- $tmp/file $dir &&
  rm -r $tmp

Explanation of zsh features:

  • :t and :h are history expansion modifiers to take the directory part and the basename of a file respectively.
  • *(Nn) uses glob qualifiers: n to sort file names in numerical order instead of lexicographic order, and N to get an empty array rather than an error if the directory is empty.
  • The ^ parameter expansion modifier in $dir/$^files adds the prefix $dir/ to each element of the array files (as opposed to adding it only to the first element).
  • Thanks for the answer. Although I can't run this script. zsh is complaining about the opening clause on line 1: ./zshscript.sh: line 1: syntax error near unexpected token (' ./zshscript.sh: line 1: for dir in **/1(:h); do' – Ežiukas Stebintis Pasaulį Apr 26 '15 at 12:44
  • @EžiukasStebintisPasaulį Are you sure you ran the script with zsh? I just tested (not extensively, just with a sample input), and it is syntactically correct and seems to do what it's supposed to do. – Gilles Apr 26 '15 at 12:57
  • I've changed tmp=$(TMPDIR=$dir:h mktemp -d) mv -- $dir $tmp && cat -- $dir/$^files >$tmp/file && mv -- $tmp/file $dir && rm -r $tmp to fname=$dir:t cat -- $dir/$^files >$dir:h/$fname".restored" && rm -r $dir && mv -- $dir:h/$fname".restored" $dir:h/$fname because I two problems with your code: 1. filename was not the original one (now I get one), 2. After running script instead of restored files there was temporary directory with root rights and inside there was a restored file with name 'file' and original directory from which contents the file was created. – Ežiukas Stebintis Pasaulį Apr 27 '15 at 20:27
  • But still thank you - this helped a lot. And I still have one problem, though - because the paths inside are quite long, when there is too much files, then I get argument list too long: cat any ideas how can I fix that? Most likely that is why I was trying to use question marks as arguments - at least such recommendation I found when there is a lot of files to cat. Any ideas? – Ežiukas Stebintis Pasaulį Apr 27 '15 at 20:30
  • @EžiukasStebintisPasaulį It's hard to follow what you did in a comment without the line breaks. Please edit my answer with your fix. Regarding “argument list too long”, wildcards wouldn't help (it's the length after wildcard expansion that matters). You can replace cat -- $dir/$^files >$tmp/file by (cd -- $dir && cat $files) >$tmp/file to make the problem unlikely (shorter command line), or by for x in $dir/$^files; do cat -- $x; done >$tmp/file to be sure. – Gilles Apr 27 '15 at 20:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.