Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

At the moment I'm developing a larger Bash script (it's an Open Source project of mine) and it's starting to become a mess. I've split up logic into functions, use local variables where I can and have only declared a handful global variables. Still, it is becoming pretty hard to maintain.

I thought about splitting the script up into multiple scripts and source them in my main script (similar to imports in other languages).

But I wonder if this is a feasible approach. First, sourcing multiple scripts could severly slow the execution time of the script, and second, it makes distribution harder.

So, is this a good approach, and do other (Open Source) projects do it the same way?

share|improve this question
3  
I went looking for a really long shell script and at 3500 lines and 125KB, I'd not want to try to maintain it. The shell is really friendly when hooking up programs, but when it tries to do computation, it gets pretty ugly. I know the code you have mostly works and the costs of porting it are high, but you might want to consider something else in the future. –  msw May 7 '13 at 15:47
1  
I have run into the same problem. I have a moderately large bash project sourceforge.net/projects/duplexpr . Currently, each script is self contained, but I am thinking of moving all the common functions to a separate file(s) and including them where needed to eliminate having to update them in multiple places. In general, I think it would be better to just call each successive script rather than source it. It makes running the parts independently possible. Then, you have to pass variables as arguments or in parameter files (or you can just export them if you don't want standalone.) –  Joe May 11 '13 at 22:11
add comment

6 Answers

up vote 10 down vote accepted

Yes, it is a common practice. For example, in the early days of Unix the shell code responsible for guiding the system through its boot phases to multiuser operation was a single file, /etc/rc. Today the boot process is controlled by many shell scripts, broken up by function, with common functions and variables sourced as needed from central locations. Linux distributions, Macs, BSDs, all have adopted this approach to varying degree.

share|improve this answer
1  
though in that case, it's more for reusability. Different scripts use a common library of shell functions. –  Stephane Chazelas May 7 '13 at 19:47
add comment

Is shell the right tool for the job at that point? As a developer who has bumped into issues of outgrowing code I can tell you that a rewrite should not be considered but instead considering separating out pieces into something better suited for the scale you are looking to grow your application - perhaps python or ruby or even perl?

Shell is a utility language - it's a scripting language - and so it will be hard to grow it to those sizes.

share|improve this answer
    
This is exactly what I was about to post. –  Zack May 7 '13 at 20:35
    
especially given that even the older OSes like Solaris are shipping with perl, and python, etc, now. one reason older systems used shell scripts is that the shell was guaranteed to always be available, but bigger Unices, like HP-UX and Solaris, and AIX, couldn't be guaranteed to include other tools. –  Tim Kennedy May 20 '13 at 16:28
add comment

A script could be broken up as you describe - just about anything can be done. I would say that the 'good approach' would be to compartmentalize your large script, figure out where parts of it could be run as a separate process, communicating through IPC mechanisms.

Beyond that, for a shell script, I would package it as a single file. As you say, it makes distribution harder: you either have to know where the 'library' scripts are located - there is no nice standard for shell scripts - or rely on the user to set their path correctly.

You could distribute an installation program that handles all this for you, extracting the files, putting them in the proper place, telling the user to add something like export PROGRAMDIR=$HOME/lib/PROGRAM to the ~/.bashrc file. Then the main program could fail if $PROGRAMDIR is not set or doesn't contain the files you expect.

I wouldn't worry as much about the overhead of loading the other scripts. The overhead is really just opening a file; the processing of the text is the same, especially if they are function definitions.

share|improve this answer
add comment

If it makes your maintenance easier, you can have both. Split it up into logical parts so you can maintain it easily, then write (e.g.,) a Makefile to put it all back together for distribution You could write some quick scripts to copy the functions from the include file to the output file in place of the source line or just do something trivial like this (you'll have to re-tabify this, as make requires tabs):

all: myscript

myscript: includes/* body/*
    cat $^ > "$@" || (rm -f "$@"; exit 1)

You then have a "source" version (used for editing) and a "binary" version (used for trivial installation).

share|improve this answer
1  
Aha! Someone else that uses the simple cat approach :) –  Clayton Stanley May 9 '13 at 8:01
add comment

Common practice or not, I don't think sourcing anything other than a set of exports is a good idea. I find that executing code by sourcing it is just confusing, and it limits re-use, because environmental and other variable settings make the sourced code highly dependent on the sourcing code.

You are better off to break up your application into smaller, self-contained scripts, then execute them as a series of commands. This will ease debugging, since you can execute each self-contained script in an interactive shell, examining files, logs, etc between each command invocation. Your single, large application script turns into a simpler controlling script that just runs a series of commands after you've finished debugging.

A group of smaller, self-contained scripts does run into the harder-to-install problem.

share|improve this answer
add comment

An alternative to sourcing the scripts is simply calling them with arguments. If you've already broken up most of your functionality into shell functions, you're probably quite close to being able to do this already. The following snippet of Bash allows any function declared in a script to be used as a subcommand:

if [[ ${1:-} ]] && declare -F | cut -d' ' -f3 | fgrep -qx -- "${1:-}"
then "$@"
else main "$@" # Try the main function if args don't match a declaration.
fi

The reason not to source is to avoid environment and option pollution.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.