1

I tried posting this on Stackexchange but I think I may have more chance of a correct answer here as its very linux specific. I have an sh script which updates data in a csv, then runs a perl script ($match) from within the sh script that matches data between two csv files and populates the file $matches with the matching results. I'm only going to show a snippet from the end of the script because only this bit is relevant to the question:

#!/bin/sh
export PATH="/opt/bin:/opt/sbin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/syno/sbin:/usr/syno/bin:/usr/local/sbin:/usr/local/bin"

match=/home/perl_experiments/newitems/match.pl       
matches=/home/perl_experiments/newitems/matches.txt


/usr/bin/perl $match > $matches

if [[ -s $matches ]] ; then
    date >> $log
    echo "matches has data." >> $log
    $sendmail
else
    date >> $log                  
    echo "matches is empty." >> $log
    exit                              
fi

EDIT: Here is the $match script

#!/usr/bin/perl                                                                                                                                                                              

my @csv2 = ();                                                                                                                                                                               
open CSV2, "<csv2" or die;                                                                                                                                                                   
@csv2=<CSV2>;                                                                                                                                                                                
close CSV2;                                                                                                                                                                                  

my %csv2hash = ();                                                                                                                                                                           
for (@csv2) {                                                                                                                                                                                
  chomp;                                                                                                                                                                                     
  my ($title) = $_ =~ /^.+?,\s*([^,]+?),/; #/ match the title                                                                                                                                
  $csv2hash{$_} = $title;                                                                                                                                                                    
}                                                                                                                                                                                            

open CSV1, "<csv1" or die;                                                                                                                                                                   
while (<CSV1>) {                                                                                                                                                                             
  chomp;                                                                                                                                                                                     
  my ($title) = $_ =~ /^.+?,\s*([^,]+?),/; #/ match the title                                                                                                                                
    my %words;                                                                                                                                                                               
    $words{$_}++ for split /\s+/, $title;    #/ get words                                                                                                                                    
    ## Collect unique words                                                                                                                                                                  
    my @titlewords = keys(%words);                                                                                                                                                           
  my @new;                          #add exception words which shouldn't be matched                                                                                                          
  foreach my $t (@titlewords){                                                                                                                                                               
        push(@new, $t) if $t !~ /^(and|the|to|uk)$/i;                                                                                                                                        
  }                                                                                                                                                                                          
  @titlewords = @new;                                                                                                                                                                        
  my $desired = 5;                                                                                                                                                                           
  my $matched = 0;                                                                                                                                                                           
  foreach my $csv2 (keys %csv2hash) {                                                                                                                                                        
    my $count = 0;                                                                                                                                                                           
    my $value = $csv2hash{$csv2};                                                                                                                                                            
    foreach my $word (@titlewords) {                                                                                                                                                         
            my @matches   = ( $value=~/\b$word\b/ig );                                                                                                                                       
            my $numIncsv2 = scalar(@matches);                                                                                                                                                
            @matches      = ( $title=~/\b$word\b/ig );                                                                                                                                       
            my $numIncsv1 = scalar(@matches);                                                                                                                                                
            ++$count if $value =~ /\b$word\b/i;                                                                                                                                              
            if ($count >= $desired || ($numIncsv1 >= $desired && $numIncsv2 >= $desired)) {                                                                                                  
                $count = $desired+1;                                                                                                                                                         
                last;                                                                                                                                                                        
            }                                                                                                                                                                                
    }                                                                                                                                                                                        
    if ($count >= $desired) {                                                                                                                                                                
      print "$csv2\n";                                                                                                                                                                       
      ++$matched;                                                                                                                                                                            
    }                                                                                                                                                                                        
  }                                                                                                                                                                                          
  print "$_\n\n" if $matched;                                                                                                                                                                
}                                                                                                                                                                                            
close CSV1;                                                                                                                                                                                  
~           

Now, i've deliberately left some data in the 2 csv's that matches to test the script. When I run the script manually it works as expected and I get the message "matches has data" in the $log file. However, when its run from crontab (as root, just like when I run it manualy) it doesn't produces any data in $matches- and the $log files entry states "matches is empty."

Here is my crontab entry, which definitely runs the script, it just doesn't produce the expected output:

*/10    09-21   *       *       1,2,3,4,5       root    /home/perl_experiments/newitems/newitems.sh

So my question is, why is this happening, and what can I amend to ensure crontab executions are the same as my manual executions? Is it something to do with crontab having issues with running a perl script inside an sh script?

As I say when run manually it does exactly what I expect it to, but when run my crontab, no matches are produced. Suggestions welcome.

  • you're using [[ as a test operator, but that's bash-specific, and your she-bang line explicitly says /bin/sh – Jeff Schaller Jun 6 '16 at 12:48
  • Really? In that case why does it work when I manually execute the script? – nmh Jun 6 '16 at 13:40
  • do you have bash as your interactive shell? – Jeff Schaller Jun 6 '16 at 13:42
  • not sure, when I type "which bash" nothing comes up – nmh Jun 6 '16 at 13:46
  • Ahh I see evidence of a Synology NAS; if you enter /bin/sh and then type [[ -d / ]] && echo yes, does it work or give an error? Not sure if their /bin/sh supports [[ – Jeff Schaller Jun 6 '16 at 13:56
3

Your perl script has

open CSV2, "<csv2" or die;
...
open CSV1, "<csv1" or die;

Where are those files located? cron's current directory is the home directory of the user. If the files are in the "newitems" directory, you have to cd there first.

Make sure you're not making any other assumptions about the environment in your programs.

I find this is a handy command to enable in your crontab one time:

#* * * * * { date; pwd; echo "env:"; env; echo "set:"; set; } > ~/cron.env

An excellent point by @Otheus:

if ! /usr/bin/perl "$match" > "$matches"; then
    status=$?
    echo "$match script returned unsuccessfully"
    exit $status
fi
  • ok, let me give that a go, i'll be back like Arnie – nmh Jun 6 '16 at 13:41
  • 1
    @nmh i was going to raise same issue, with note that you never check output status of perl run in the shell script. You should test $? or wrap yhe execution in if clause. – Otheus Jun 6 '16 at 15:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.