0

My previous question, Cleaning out mail folders with cron task didn't result in a good enough answer to help me. I have narrowed down the task as follows:

I wish to move mail out of each user's .spam/cur and .spam/new folders into the spam-teaching folder (by nightly cron job). There they will be processed and deleted.

mv ~/mail/*/*/.spam/{cur,new}/* ~/mail/.sa-learn

The above line generates an error if the folder doesn't exist. e.g.,

+-- mail
  +-- sitename.com
    +-- username1
    | +-- .spam
    |   +-- cur
    |   +-- new
    +-- username2   <-- no ".spam" folder.
    +-- username3
    | +-- .spam
    |   +-- cur
    |   +-- new

Since it would be useful to create the folders for the users if they don't exist I am considering using touch to create them if they don't exist.

Q1: Is the following approach robust enough?

for dir in ~/mail/*/*/; do touch "$dir/.spam"; done
for dir in ~/mail/*/*/.spam/; do touch "$dir/cur"; done
for dir in ~/mail/*/*/.spam/; do touch "$dir/new"; done

Q2: Will the move command now work without error even if the folders are empty?

mv ~/mail/*/*/.spam/{cur,new}/* ~/mail/.sa-learn

Q3: Is there a smarter way to do this?


Update 4

#!/bin/bash
# SpamAssassin Learn script.
# With help from Kusalananda's answer 
# to https://unix.stackexchange.com/questions/336412/creating-spam-folders-in-each-mail-users-account
#
# Any mail the user drops into their spam folder will be moved to a temp folder,
# fed to SpamAssassin's sa-learn and then deleted.
# The script also creates the .spam folders for each account if they don't already exist.
# Run daily as a cron task.

myDomain=sitename.com

# Create .spam/, .spam/cur/ and .spam/new/ folders for each user.
#mkdir -p "$HOME"/mail/"$myDomain"/*/.spam/{cur,new}
for userdir in "$HOME"/mail/"$myDomain"/*; do test -d "$userdir" && mkdir -p "$userdir"/.spam/cur; done
for userdir in "$HOME"/mail/"$myDomain"/*; do test -d "$userdir" && mkdir -p "$userdir"/.spam/new; done

# Create a temp folder.
mkdir -p "$HOME"/mail/.sa-learn/ 

# Find all the .spam emails and move them to the temp folder.
# IMAP users should see their spam folder empty.
find "$HOME"/mail/"$myDomain" -type f -path "*/.spam/cur/*" -print0 | xargs -0 -I XX mv "XX" "$HOME"/mail/.sa-learn/
find "$HOME"/mail/"$myDomain" -type f -path "*/.spam/new/*" -print0 | xargs -0 -I XX mv "XX" "$HOME"/mail/.sa-learn/

# Feed the emails into the SpamAssassin spam learner.
sa-learn -p ~/.spamassassin/user_prefs --spam "$HOME"/mail/.sa-learn

# Remove the temporary folder and its contents.
rm -rf "$HOME"/mail/.sa-learn

This works. I seemed to be having trouble with the {cur,new} syntax.

User now drops spam into .spam folder. Cron job moves them to .sa-learn folder, feeds them to sa-learn and then deletes the folder.

  • Instead of the first mkdir to create the .spam folders, you will have to loop through the contents of "$HOME"/mail/mydomain.com/ and create the folder for each of the users individually. Maybe something like for userdir in "$HOME"/mail/mydomain.com/*; do test -d "$userdir" && mkdir -p "$userdir"/.spam/{cur,new}; done – Kusalananda Jan 12 '17 at 7:44
  • @Kusalananda: Your for loop worked, thank you. Any ideas how to debug the third line? – Transistor Jan 12 '17 at 20:42
  • You don't have a development setup? I'd advise you to set up a simple Linux machine and test this there. Testing a shell script through running it in cron is definitely not ideal, especially not if the system you're doing it on is a production system and the data you're running your tests over is live! – Kusalananda Jan 12 '17 at 20:54
  • Thanks for the "prod". I pulled out the family Ubuntu laptop and got it going. See Update 4 and thanks for your help. – Transistor Jan 12 '17 at 23:50
3

You can not create a directory with touch, use mkdir for this:

mkdir -p "$HOME"/mail/.sa-learn/{cur,new,tmp}

The -p flag tells mkdir to create all intermediate folders if they do not exist, and makes it not an error to try to create a folder that already exist.

That is assuming a shell that does brace expansion. If it doesn't, break it up inte three separate mkdir invocations.

When moving mail (untested):

find "$HOME"/mail -type f -path "*/.spam/*" -print0 |
xargs -0 -I XX mv "XX" "$HOME"/mail/.sa-learn/new/

This will find all mail (new or old) in any .spam folder under $HOME/mail and move them into the .sa-learn/new folder.

I'm using "$HOME" rather than ~ in scripts because I think it looks nicer, it's self-documenting, and it behaves as any other variable (~ doesn't).

  • "$HOME"/mail/.sa-learn should be -pruned. Why xargs -I? it's suboptimal (one exec per file), rather use xargs -0 mv -t dir (one exec for all files). Or better: find ... -exec mv -t dir {} +. xargs is not needed after all. – xhienne Jan 10 '17 at 22:28
  • @xhienne I'm not using mv -t because I'm on a system without GNU coreutils, and I don't know what Unix user Transistor is on. I don't prune .sa-learn because I haven't set up a test for this, and I tend to get it wrong, and pruning .sa-learn is bound to not save much time anyway. I'd expect this to run quite fast unless there are tens of thousands of spam messages in .sa-learn, which I don't think there will be as it's cleared out regularly. – Kusalananda Jan 10 '17 at 22:35
  • Thanks, guys. I'm on a reseller hosting package without shell access so I'll be testing this by triggering a cron job. Host says all systems "run on CentOS Linux x86, which is identical to Red Hat Enterprise Linux." Probably max of 200 spam items to process per day. I've done plenty of php/sql stuff over the last decade but not enough shell stuff to become familiar with the syntax. This is a big help. – Transistor Jan 10 '17 at 22:50
  • @Transistor what xhienne is talking about is using the -prune flag with find to stop it from entering the .sa-learn folder (see the find manual). It will do this to look for .spam folders while walking through everything under the mail folder so "pruning .sa-learn" means stopping it from entering .sa-learn (which is reasonable since there's no .spam folder in there). If the target system uses GNU coreutils mv (i.e. one can use mv -t) his suggestions will be giving a speedup. – Kusalananda Jan 10 '17 at 23:04
  • Good point, I hadn't pay attention to the GNU aspect of the -t option. With CentOS, it will work. For having worked on mail servers, it's always beneficial to spare I/Os, Between smtpd, popd, imapd, spamassasin, amavisd, webmail, etc. mail servers are often working continuously with a high load. There may be a few spams in one user maildir but there may be thousands of users. As for the -prune option, it's not to spare CPU or I/Os, it's to avoid alarming error messages during the move: "X and X are the same file". But you are right there is no spam folder, so forget this idea. – xhienne Jan 10 '17 at 23:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.