Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have some files like this:

2012-12-17_083213_1.log  
2012-12-17_083213_1.log_stats  
2012-12-17_083213_1.logaccount_ptr  
2012-12-17_083213_1.loginitial_ptr  
2012-12-17_083213_1.logptr  
2012-12-17_093049_2.log  
2012-12-17_093049_2.log_stats  
2012-12-17_093049_2.logaccount_ptr  
2012-12-17_093049_2.loginitial_ptr  
2012-12-17_093049_2.logptr

In here every 1G of data, a new file is created with the name in Year-Month-Day_HourMinuteSecond_number.log format (as you can see above).

I want to

  1. Copy all *.log files to a new folder named with the format Year-Month-Day, so for these it would be named: 2012-12-17
  2. tgz that folder
  3. Send it to a server IP.

I don't know how exactly. I want it as a bash script so that I can run it every day for the newly added files.

share|improve this question
1  
What have you tried? –  jordanm Dec 17 '12 at 19:29
    
i have tried this: # !/bin/bash localbakdir=/opt/me/fw1/log mkdir date +%Y-%m-%d –  Jimmy A Dec 17 '12 at 19:31
    
@jordanm i have tried this: # !/bin/bash localbakdir=/opt/me/fw1/log mkdir date +%Y-%m-%d –  Jimmy A Dec 17 '12 at 19:41

1 Answer 1

up vote -1 down vote accepted

I will copy you a script to do it, but next time do some research first and try to write it by yourself. That is the only way to learn. And, please, don't use it without understanding it.

#!/bin/bash
tosend=""
for i in *.log; do
    dir=$(echo $i | awk '{split($1,a,"_"); print a[1]}');
    if [ ! -d $dir ]; then
        mkdir $dir;
    fi;
    cp $i $dir;
    tosend="$tosend\n$dir"
done
for i in $(echo -e $tosend | uniq);do
    echo Compressing $i
    tar -zcvf $i.tar.gz $i
    scp $i.tar.gz user@server
done

The key part is using awk to split the file name, and then keeping a list of the folders you have modified. It is not the most elegant but it works.

Note that if you copy the files, next time you run it they will be copied again. The best thing would be to move them.

Btw, I added the uniq part instead of adding the name to the "array" within the if because it's possible that the folders already exist when you run the script.

However, if you only wish to send the new files and you run it at midnight, this is probably the best approach:

#!/bin/bash
tosend=""
for i in *.log; do
    dir=$(echo $i | awk '{split($1,a,"_"); print a[1]}');
    if [ ! -d $dir ]; then
        mkdir $dir;
        tosend="$tosend\n$dir"
    fi;
    mv $i $dir;
done
for i in $tosend;do
    echo Compressing $i
    tar -zcvf $i.tar.gz $i
    scp $i.tar.gz user@server
done

This would only send the new folders (even though some of the old ones may have new files). And with the mv the main folder will be kept clean.

For more sophisticated/bulletproof approaches, I think rsync is the best shot.

share|improve this answer
1  
mywiki.wooledge.org/ParsingLs –  jordanm Dec 17 '12 at 20:18
    
@balkian thank you so much, ofcourse i want to understand but i am new in this and i want to learn, here I dont understand how you are checking the file name and make a directory with the same name? –  Jimmy A Dec 17 '12 at 20:19
    
Jimmy, I suggest you check this tutorial or something similar to get familiar with bash. –  balkian Dec 17 '12 at 20:37
    
@balkian thank you i am reading this. but my question is that in here all *.log files are put in the new folder, what if i want all the files dated today to be put there? i mean in here 2012-12-17_083213 is my folder and then inside it i have the .log and .log_stats and etc with this name inside it? –  Jimmy A Dec 17 '12 at 22:36

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.