3

I have a backup from a disk that contains data files and analysis. The dir structure and names are not really consistent. To save space, I would like to zip all the subdirs that only contain data (*.txt) and delete the original afterwards. There are several threads on zipping subdirs, but not on the conditions I have.

To be more precise: I do not want to zip all *.txt files, but all directories that exclusively contain txt-files into a directory.zip and delete the original directory.

|improve this question
  • Hi Anthon I was trying things like this, but the solution only works on one level. So, I don't know how to make it recursively. And then, similar to the solution given below, I'd like to zip the entire folder, not the containing text files alone. Note that like in the discussion linked above, the original data is from a Windows machine and folder and file names may contain spaces. – mikuszefski Dec 8 '13 at 13:58
  • not quite what you are looking for but : find -type f -name "*.txt" -print0|xargs -0r gzip will compress all the txt files. – hildred Dec 8 '13 at 14:07
  • Hi hildred, the point is really zipping the folder. The idea is also to reduce the number of files, as a measurement folder contains several hundreds up to thousands of text files. – mikuszefski Dec 8 '13 at 14:21
  • If you are so keen on backing up your stuff and saving space, then if a directory contains text files and other files why not just backup the text files from it? – barlop Mar 14 '18 at 22:49
  • @barlop interesting late comment, but I do not see how making only a partial backup would help. Lets assume you have measured a lot of ASCII data and merged the info into JPGs. Zipping JPGs doesn't make sense, zipping the ASCII measurements does. In detail my problem was different, but try to think of a rather complex project like a PhD thesis in physics with large amounts of data files, analysis, images, etc. – mikuszefski Mar 15 '18 at 9:07
4

This script will archive then optionally remove all folders containing "*.txt" files and nothing else.

folders=$(find . -type d -exec sh -c 'cd "$1";[ "$(ls *.txt 2>/dev/null)" ] \
          && [ -z "$(ls -ad * | grep -v '\.txt$')" ] && echo "$1"' sh {} \;)

echo "$folders" | zip -r@ archive && echo "$folders" | while read folder; do
  echo "will remove $folder"
  # Uncomment next line for the folder to be removed
  # rm -rf "$folder"
done

Edit: here is a solution that creates individual zip files:

find . -depth -type d -exec sh -c '
cd "$1" || exit
[ "$(ls ./*.txt 2>/dev/null)" ] &&
[ -z "$(ls -ad ./* | grep -v '\.txt$')" ] &&
(
  b=$(basename "$1")
  cd ..
  zip -r "$b.zip" "$b" && rm -rf "$b"
)' sh {} \;
|improve this answer
  • If I understand this correctly, this zips each text file. I want to zip the entire folder---with the condition that it only contains text files---and then delete the original folder. – mikuszefski Dec 8 '13 at 13:53
  • Do you mean if the folder contains both .txt and non .txt files, it shouldn't be archived at all ? – jlliagre Dec 8 '13 at 14:00
  • Yes, somewhat like this. Only the folders that are definitively data folders, i.e. exclusively containing *.txt files. One might even think about adding an additional (if bigger than 5 MB or so). I guess it's not done in a single line. – mikuszefski Dec 8 '13 at 14:04
  • So a folder containing many .txt files but also either a subfolder or a file named foo.jpg should be ignored ? – jlliagre Dec 8 '13 at 14:12
  • oh, no. I see my lack of precision here. The folders to be zipped do not contain any more subfolders. So I have a complex file tree and if one branch ends in a folder that contains only *.txt, that is to be zipped. – mikuszefski Dec 8 '13 at 14:24
1

Here's a simpler approach: Use find to get a list of directories; in each directory, check if the list of .txt files is the same as the list of all files. If it is, go ahead and zip it.

This is made much easier by the fact that the zipped folders may not contain subfolders. I'm assuming that none of your files or folders begin with a dot; if that's not true, please let me know and I'll add the necessary checks.

for DIR in `find . -type d -print`
do
    TXT=`echo "$DIR"/*.txt`
    ALL=`echo "$DIR"/*`
    if [ "$TXT" == "$ALL" ]
    then
        echo only txt: $DIR
        # zip "$DIR"
    fi
done
|improve this answer
  • 2
    Note that due to for DIR in $(find …) only works if none of the directory names contain whitespace or \[*?. – Gilles Dec 8 '13 at 22:05
1

The following snippet traverses all subdirectories of the current directory. If it finds a directory containing only subdirectories or files with the .txt extension, it zips the directory and removes it.

find . -type d -exec sh -c '
  if [ -z "$(find "$0" ! -type d ! -name "*.txt" | head -n 1)" ]; then
    zip -r "$0.zip" "$0" && rm -r "$0"
  else
    exit 1
  fi
' {} \; -prune
|improve this answer
  • Ouch. I'm afraid there are a couple of design and syntax errors in your script. find cannot return an empty string with -type d given the fact the current directory will always match. You have the .txt test reversed (should be ! -name "*.txt"). The zip file will contain the whole hierarchy while the OP might prefer only a single directory. You both missed to pass {} and missed to end the -exec action. – jlliagre Dec 9 '13 at 11:05
0

The following script recursively finds directories and performs a zip on them if they contain only .*txt files. It works like this: all subdirectories are checked with a pattern for non-txt files and a pattern for txt-files. If there are any txt files matched and there are no other types of files matched, the directory is zipped and destroyed.

#!/bin/bash

shopt -s dotglob nullglob extglob
for dir in $(find "$1" -type d); do
    non_txt=("$dir"/!(*.txt)); txt=("$dir"/+(*.txt))
    if ((${#txt[@]} && ! ${#non_txt[@]})); then
        zip -r "$dir.zip" "$dir" && rm -r "$dir"
    fi
done

Another variant that considers directory names with \[*? characters in them:

#!/bin/bash

find "$1" -depth -type d -exec sh -c '
    shopt -s dotglob nullglob extglob
    non_txt=("$1"/!(*.txt)); txt=("$1"/+(*.txt))
    if ((${#txt[@]} && ! ${#non_txt[@]})); then
        zip -r "$1.zip" "$1" && rm -r "$1"
    fi
' sh {} \;
|improve this answer
  • Note that for dir in $(find …) only works if none of the directory names contain whitespace or \[*?. – Gilles Dec 8 '13 at 22:04
  • @Gilles How would I modify it to handle those cases? Could you make an edit maybe? – user13742 Dec 8 '13 at 22:06
  • The robust way is to use the -exec primary, as in jlliagre's answer. – Gilles Dec 8 '13 at 22:09
  • @Gilles I presented another variant using -exec, though I can't really tell the two results apart, considering directories with these characters, except whitespace which is not handled good by for i in $(find …). – user13742 Dec 8 '13 at 23:05
  • For GNU find (and a non-antique bash), you can use find ... -print0 | while read -d $'\0' i; do stuff with "$i"; done -- should be bulletproof, and I think it looks neater than cramming it all into the -exec. – evilsoup Dec 9 '13 at 0:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.