I tried posting this on Stackexchange but I think I may have more chance of a correct answer here as its very linux specific. I have an sh script which updates data in a csv, then runs a perl script ($match) from within the sh script that matches data between two csv files and populates the file $matches with the matching results. I'm only going to show a snippet from the end of the script because only this bit is relevant to the question:

export PATH="/opt/bin:/opt/sbin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/syno/sbin:/usr/syno/bin:/usr/local/sbin:/usr/local/bin"


/usr/bin/perl $match > $matches

if [[ -s $matches ]] ; then
    date >> $log
    echo "matches has data." >> $log
    date >> $log                  
    echo "matches is empty." >> $log

EDIT: Here is the $match script


my @csv2 = ();                                                                                                                                                                               
open CSV2, "<csv2" or die;                                                                                                                                                                   
close CSV2;                                                                                                                                                                                  

my %csv2hash = ();                                                                                                                                                                           
for (@csv2) {                                                                                                                                                                                
  my ($title) = $_ =~ /^.+?,\s*([^,]+?),/; #/ match the title                                                                                                                                
  $csv2hash{$_} = $title;                                                                                                                                                                    

open CSV1, "<csv1" or die;                                                                                                                                                                   
while (<CSV1>) {                                                                                                                                                                             
  my ($title) = $_ =~ /^.+?,\s*([^,]+?),/; #/ match the title                                                                                                                                
    my %words;                                                                                                                                                                               
    $words{$_}++ for split /\s+/, $title;    #/ get words                                                                                                                                    
    ## Collect unique words                                                                                                                                                                  
    my @titlewords = keys(%words);                                                                                                                                                           
  my @new;                          #add exception words which shouldn't be matched                                                                                                          
  foreach my $t (@titlewords){                                                                                                                                                               
        push(@new, $t) if $t !~ /^(and|the|to|uk)$/i;                                                                                                                                        
  @titlewords = @new;                                                                                                                                                                        
  my $desired = 5;                                                                                                                                                                           
  my $matched = 0;                                                                                                                                                                           
  foreach my $csv2 (keys %csv2hash) {                                                                                                                                                        
    my $count = 0;                                                                                                                                                                           
    my $value = $csv2hash{$csv2};                                                                                                                                                            
    foreach my $word (@titlewords) {                                                                                                                                                         
            my @matches   = ( $value=~/\b$word\b/ig );                                                                                                                                       
            my $numIncsv2 = scalar(@matches);                                                                                                                                                
            @matches      = ( $title=~/\b$word\b/ig );                                                                                                                                       
            my $numIncsv1 = scalar(@matches);                                                                                                                                                
            ++$count if $value =~ /\b$word\b/i;                                                                                                                                              
            if ($count >= $desired || ($numIncsv1 >= $desired && $numIncsv2 >= $desired)) {                                                                                                  
                $count = $desired+1;                                                                                                                                                         
    if ($count >= $desired) {                                                                                                                                                                
      print "$csv2\n";                                                                                                                                                                       
  print "$_\n\n" if $matched;                                                                                                                                                                
close CSV1;                                                                                                                                                                                  

Now, i've deliberately left some data in the 2 csv's that matches to test the script. When I run the script manually it works as expected and I get the message "matches has data" in the $log file. However, when its run from crontab (as root, just like when I run it manualy) it doesn't produces any data in $matches- and the $log files entry states "matches is empty."

Here is my crontab entry, which definitely runs the script, it just doesn't produce the expected output:

*/10    09-21   *       *       1,2,3,4,5       root    /home/perl_experiments/newitems/newitems.sh

So my question is, why is this happening, and what can I amend to ensure crontab executions are the same as my manual executions? Is it something to do with crontab having issues with running a perl script inside an sh script?

As I say when run manually it does exactly what I expect it to, but when run my crontab, no matches are produced. Suggestions welcome.

  • you're using [[ as a test operator, but that's bash-specific, and your she-bang line explicitly says /bin/sh
    – Jeff Schaller
    Jun 6, 2016 at 12:48
  • Really? In that case why does it work when I manually execute the script?
    – nmh
    Jun 6, 2016 at 13:40
  • do you have bash as your interactive shell?
    – Jeff Schaller
    Jun 6, 2016 at 13:42
  • not sure, when I type "which bash" nothing comes up
    – nmh
    Jun 6, 2016 at 13:46
  • Ahh I see evidence of a Synology NAS; if you enter /bin/sh and then type [[ -d / ]] && echo yes, does it work or give an error? Not sure if their /bin/sh supports [[
    – Jeff Schaller
    Jun 6, 2016 at 13:56

1 Answer 1


Your perl script has

open CSV2, "<csv2" or die;
open CSV1, "<csv1" or die;

Where are those files located? cron's current directory is the home directory of the user. If the files are in the "newitems" directory, you have to cd there first.

Make sure you're not making any other assumptions about the environment in your programs.

I find this is a handy command to enable in your crontab one time:

#* * * * * { date; pwd; echo "env:"; env; echo "set:"; set; } > ~/cron.env

An excellent point by @Otheus:

if ! /usr/bin/perl "$match" > "$matches"; then
    echo "$match script returned unsuccessfully"
    exit $status
  • ok, let me give that a go, i'll be back like Arnie
    – nmh
    Jun 6, 2016 at 13:41
  • 1
    @nmh i was going to raise same issue, with note that you never check output status of perl run in the shell script. You should test $? or wrap yhe execution in if clause.
    – Otheus
    Jun 6, 2016 at 15:06

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .