2

I have this directory structure at /var/sync:

# tree -d /var/sync/
/var/sync/
├── sync_bi
│   ├── replicator => Replicator.php
│   sync.php 
├── sync_pfizer
│   ├── replicator => Replicator.php
│   sync.php
│ replicator.sh
│ sync.sh

As you can see on each sync_* directory there is script called sync.php and on each replicator directory there is a script called Replicator.php. I need to run those scripts each night. Replicator.php should run at 00:00 AM and sync.php should run at 12:30 AM. I know the right path here is obviously a cron job. I have made two scripts replicator.sh and sync.sh sharing almost the same code the main difference is the script name. See code below:

replicator.sh
#! /bin/bash
cd $1
/usr/bin/php Replicator.php

sync.sh
#! /bin/bash
cd $1
/usr/bin/php sync.php

Then I have added each to cronjob as follow:

0 0 * * * /var/sync/replicator.sh /var/sync/sync_bi/replicator/
0 30 * * * /var/sync/sync.sh /var/sync/sync_bi/
0 2 * * * /var/sync/replicator.sh /var/sync/sync_pfizer/replicator/
30 2 * * * /var/sync/sync.sh /var/sync/sync_pfizer/

My idea behing all this explanation is to optimize my environment by changing script to just one that allow me to perform all the actions in just one cronjob call and keep less files to maintain. So I did a little research and I have found a helpful topics:

  • By using find some/dir -type f -execdir somecommand {} \; is a proper way to achieve the same thing as I am doing with cd
  • And by moving into each directory under root path - in my case /var/sync

     for D in ./*; do
        if [ -d "$D" ]; then
            cd "$D"
            run_something
            cd ..
        fi
    done
    

This is a pseudo code of what I am talking about (in case is not clear enough):

read tree structure at /var/sync
foreach (directory in root+path)
    if (directory_name is replicator)
        cd replicator/
        /usr/bin/php replicator.php
    else if (directory_name is sync_*)
        cd sync_*/
        /usr/bin/php sync.php
endforeach

I am not so bash expert so I am asking for some help to translate that pseudo code in something functional that allow me to achieve what I am looking for, any advice and/or help?

UPDATE

Perhaps I am wrong but I am not following at all what you guys are trying to do so here is the order of the events:

# Run Replicator at mid day and mid night
# this two can run at same time since they are pushing data to different DBs
/var/sync/sync_bi/replicator/Replicator.php
/var/sync/sync_pfizer/replicator/Replicator.php

# Run sync at mid day and mid night
# this two should run one first and the second after 30 min since they are 
# pushing data to the same DBs and second depends on first one
/var/sync/sync_bi/sync.php
/var/sync/sync_pfizer/sync.php

Having that info (and sorry for not clear this in first) how your solutions should works? As you have them right now or you need to change anything in order to cover this?

4
  • Do you also want to run /var/sync/sync.sh and /var/sync/replicator.sh or only the ones in the subdirectories? Will all sync_* directories contain a replicator subdirectory or could some of them not have one? Can the tree be more complex or will it always be a maximum of 2 deep? Can you have something like /var/sync/foo/bar/baz/foo_sync/ for example?
    – terdon
    Nov 4, 2015 at 14:43
  • Also, where in the pseudocode do you check whether it's 0:00 AM or 12:30 AM?
    – choroba
    Nov 4, 2015 at 14:44
  • @terdon I do not want to run /var/sync/sync.sh either /var/sync/replicator.sh that will never happen. Yes, syc_* will always have a replicator directory inside. That's how the three looks like with no more deep rather than 2, what could happen here is the appearance of another sync_* dir with same structure as the rest of them and no for that last one of your questions
    – ReynierPM
    Nov 4, 2015 at 14:52
  • @choroba I think this is part of cronjob, the idea is to wait until parents tasks ends I don't have any idea in how to achieve this
    – ReynierPM
    Nov 4, 2015 at 14:53

3 Answers 3

3

This translates your pseudocode to bash:

#!/bin/bash
cd /var/sync
for dir in sync*/ ; do
    ( # Subshell to make "cd" not global.
        cd "$dir"
        [[ -f sync.php ]] && /usr/bin/php sync.php
        if [[ -d replicator ]] ; then
            cd replicator
            [[ -f replicator.php ]] && /usr/bin/php replicator.php
        fi

    )
done

It runs both the sync.php and replicator.php at the same time, though, but you originally wanted to call them at different times. Maybe something like

#!/bin/bash

run=$1

cd /var/sync
for dir in sync*/ ; do
    ( # Subshell to make "cd" not global.
        cd "$dir"
        [[ -f "$run" ]] && /usr/bin/php "${run##*/}"
    )
done

called from cron like

0 30 * * * /path/to/script.sh sync.php
0 0 * * * /path/to/script.sh replicator/Replicator.php

${run##*/} removes everything before the last /, i.e. returns just the basename.

Update

If you want to run the scripts as follows:

0:00 sync_a/sync.php
0:30 sync_a/replicator/Replicator.php
1:00 sync_b/sync.php
1:30 sync_b/replicator/Replicator.php
2:00 sync_c/sync.php
...

etc., you can just regenerate the crontab every time you change the directory structure:

#!/bin/bash
add=$1
php=/usr/bin/php
hour=0
minute=0

update_time () {
    (( minute += add ))
    if (( minute >= 60 )) ; then
        (( hour += minute / 60 ))
        (( minute %= 60 ))
    fi
    if (( hour > 11 )) ; then
        echo "Can't fit into a day!"
        exit 1
    fi
}
cd /var/sync

for dir in sync*/ ; do
    if [[ -f "$dir"/sync.php ]] ; then
        echo $minute $hour 0 0 0 "$php" /path/to/run.sh "$dir" sync.php
        update_time

        if [[ -f "$dir"/replicator/Replicator.php ]] ; then
            echo $minute $hour 0 0 0 "$php" path/to/run.sh "$dir" Replicator.php
        fi
        update_time
    fi
done

Call as generate_crontab.sh 30 or similar. Note that some of the dirs could get skipped after a change, as the files are sorted alphabetically.

run.sh should just

dir=$1
script=$2
if [[ -f "$dir$script" ]] ; then
    cd "$dir"
    /usr/bin/php $script
fi
7
  • Excellent but I want to add more thing, the first script to be run on each subdirectory should be Replicator.php and 30m after this I should run sync.php so in my case the order should be: 00:00 AM => run /sync_bi/replicator/Replicator.php 00:30 AM => run /sync_bi/sync.php 01:00 AM run /sync_pfizer/replicator/Replicator.php 01:30AM run /sync_pfizer/sync.php
    – ReynierPM
    Nov 4, 2015 at 14:56
  • In fact I do like to run all in just one script so I made just one cronjob call starting at 00:00 AM and then running next script each 30min (this mins could be a configurable parameter on the script since that time can change)
    – ReynierPM
    Nov 4, 2015 at 14:59
  • Having a script running for hours makes the system unreliable.
    – choroba
    Nov 4, 2015 at 15:04
  • @ReynierPM: Check the update.
    – choroba
    Nov 4, 2015 at 15:22
  • Many thanks, sorry while you was editing your answer I was editing my question so pls check the update and tell me if your recently update does that since that's exactly what I am trying to do
    – ReynierPM
    Nov 4, 2015 at 15:23
2

The simplest approach would be two find commands:

find /var/sync/ -name 'sync.php' -execdir php {} \;
find /var/sync/ -name 'Replicator.php' -execdir php {} \;

That will look for files called either sync.php or Replicator.php, then cd into their parent directory and execute them with php. You could add the commands directly to your crontab:

0 30 * * * find /var/sync/ -name 'sync.php' -execdir php {} \;
0 0 * * * find /var/sync/ -name 'Replicator.php' -execdir php {} \;

If you need the Replicator scripts to be run with a 30 minute pause between them, you could do something like:

find /var/sync/ -name 'Replicator.php' -execdir php {} \; -exec sh -c "sleep 30m" \;

That will first run the script, then wait for 30 minuntes before moving to the next one.

Finally, if you need to make sure that i) you never have >1 sync.php running at the same time and ii) each sync script is run after the corresponding replicator script has finished, you can use this as the command you give to cron:

find /var/sync/ -name 'Replicator.php' -execdir php {} \; ; find /var/sync/ -name 'sync.php' -execdir php {} \;

The above will run each of the Replicator.php scripts first (one after the other, not in parallel) and, once they've finished, will run each of the sync.php scripts.

12
  • I am still having a doubt here see this gist where I try to run each script at mid day and mid night, right? Now if you look at /var/sync there are two Replicator.php and two sync.php scripts. How do I know which one is running or your solution will put both with same name running at same time? In that case for Replicator.php could work since uses different DB but for sync don't since use the same and scripts should be run one first then the next and so
    – ReynierPM
    Nov 4, 2015 at 15:14
  • @ReynierPM no, the scripts will be run sequentially, one after the other. Use the updated version (with \; instead of +) just to be sure, but the original should also have run them sequentially. I don't understand what you mean by knowing which one is running. Do you want some kind of log produced?
    – terdon
    Nov 4, 2015 at 15:22
  • Many thanks, sorry while you was editing your answer I was editing my question so pls check the update and tell me if your recently update does that since that's exactly what I am trying to do
    – ReynierPM
    Nov 4, 2015 at 15:23
  • @ReynierPM if you really need to wait 30 whole minutes, see update. Otherwise, if you just need to run the second script after the first has finished, you can use the original version. Are you sure that 30 minutes are necessary?
    – terdon
    Nov 4, 2015 at 15:27
  • I don't need to wait the whole 30 min this is just a approximated period I just need to wait until the previous script ends its execution, so which one to take on this case?
    – ReynierPM
    Nov 4, 2015 at 15:34
1
{   cat >    sync.sh  
    ln -s    sync.sh replicator.sh 
    chmod +x sync.sh
} <<"" 
#!/bin/sh
cmd=${0%.*}
cmd=${cmd##*/}
find /var/sync -name "$cmd.php" -exec /usr/bin/php {} \;

So when you call a shell script the shell sets the special shell parameter $0 to the name of the script it reads, as a C program would set argv[0]. You're halfway there already in that you're running two scripts each named for its wrap target. But you don't need two scripts if you have two names - so you can just symlink the same script to the second name and refer to $0 when you call it.

For cron you need only

i=-30
for  s in sync replicator
do   echo "0 $((i+=30)) * * * /var/sync/$s.sh" | crontab
done
0

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .