0

In various scripts I frequently need to generate computationally complex file, BUT reuse the file if it already exists.

How can I create a reusable function that does just that:

  • checks for condition (eg. if pregenerated file exists and is more recent than source files)
  • if condition is met, just output the file
  • otherwise, run a sequence of commands provided to generate it, and also save results to a file to be reused in future runs.

Ideally the code using that function would look like:

if_needs_update_then_generate_into <condition> <generated_filename> \
(
  sequence
  of
  commands
  generating
  the
  file
)

I.e. I don't like making a separate function for generation as it affects readability;

I tried the following, but under CentOS it always create a separate process for "long sequence of commands", even when pregenerated file already exists and is actually reused. That process never ends and keeps using RAM:

function if_needs_update_then_generate_into()
{
  condition="$1"
  pregenerated="$2"
  
  if condition; then
    cat |
    tee "$pregenerated"
  else
    cat "$pregenerated"
  fi
}

What I liked about it that usage looks just like I need:

if_needs_update_then_generate_into \
  condition \
  pregenerated \
< <(
  sequence
  of
  commands
  generating
  the
  file
)
3
  • 1
    Have you considered using make? Jul 16, 2022 at 20:36
  • Thanks! make looks like way to go for future, but looks overkill for now. Also, not sure make will allow complex data-dependent logic for condition. Jul 16, 2022 at 20:38
  • A simple Makefile seems to fit the bill nicely, and would likely be less complex than anything you do in a shell script, at least if the only condition you will have to care about is a test for an outdated or non-existing target file. It's unclear in the question what types of other conditions you want to incorporate and how you envisage passing these to the function.
    – Kusalananda
    Jul 16, 2022 at 21:07

0

You must log in to answer this question.

Browse other questions tagged .