Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a multi-line string in the variable $PAT. $PAT must be searched for within the file $FILE. If $PAT is in $FILE, it needs to print the file with $PAT removed. If $PAT is not found, then print nothing. It is unknown if $PAT contains any special characters, and it must matched literally. For example, if $PAT is //\/\\|* then that exact same 8-character string should be searched for in $FILE.

The real world use for this is for installing and removing text within already existing files/scripts. If you want to append $PAT in $FILE, you want to know if it has already been appended previously. If $PAT is already in $FILE, then the output without $PAT allows you to easily uninstall it.

The systems I'm needing such a script for (Android devices) only have BusyBox on them. No Perl or other scripting languages.

share|improve this question
What BusyBox utilities are available? Only what comes with Android (which aren't actually BusyBox), or more? Android ships a very limited set of utilities, but more and more with each release: what minimal version are you targetting? – Gilles Dec 6 '12 at 22:53
Also, how big are the files? Will they fit comfortably in RAM (so no more than ~100MB)? – Gilles Dec 6 '12 at 22:56
These answers to this question so far have been so imaginatively brilliant. I am so impressed, I'm beyond words. Excellent thinking everyone. – Sepero Dec 7 '12 at 17:30
Yes, the files would generally be small Gilles. Your answer was great. The Android devices will be rooted androids, so default busybox. ;) – Sepero Dec 7 '12 at 17:32
up vote 2 down vote accepted

I assume that you're rewriting a text file that fits in memory (it looks like you're rewriting a configuration file).

The following script only uses shell builtin features and cat. It should work on Android's shell, at least since Gingerbread and definitely since Ice Cream Sandwich. It prints the file contents minus the first occurrence of $PAT if there is one; if $PAT does not occur, it prints nothing.

contents=$(cat "$FILE")
case $contents in
    echo "${contents%%$PAT*}${contents#*$PAT}";;

This snippet assumes that the file does not contain any null byte, ends in a single newline, and does not start with a dash. Also, if the pattern ends with a newline, it won't be found at the end of the file. The following more complex snippet copes with arbitrary text files:

contents=$(cat "$FILE"; echo a)
case $contents in
    echo -n "$dashes"
    echo -n "${contents#$dashes}";;

(Note that your proposed behavior makes it impossible to distinguish a file that contained exactly the pattern and an empty file.)

It's actually easier to implement your append/remove script directly than to use the proposed intermediate function.

contents=$(cat "$FILE"; echo a)
case $contents in
  *"$PAT"*) contents="${contents%%$PAT*}${contents#*$PAT}";;
  *) contents="$contents$PAT"
{ echo -n "$dashes"; echo -n "${contents#$dashes}"; } >"$FILE.new"
mv -- "$FILE.new" "$FILE"
share|improve this answer
You're so right about impossible distinguishment. I made a real error. Gosh your answer is great! Thanks so much. – Sepero Dec 7 '12 at 17:40
Wouldn't it be safer to catch newlines using this? case "$contents" in – Sepero Dec 7 '12 at 17:43
@Sepero You mean case "$contents" in instead of case $contents in? This is one of the few cases where you can leave out the double quotes. There can only be a single word, so the shell doesn't do word splitting and globbing even without double quotes. You can also leave out the double quotes in assignments: foo=$bar is equivalent to foo="$bar". However, export foo=$bar is not safe: if bar is hello world, this runs export foo=hello world, i.e. set foo to hello and export both foo and world). If you don't want to memorize such exceptions, use double quotes all the time. – Gilles Dec 7 '12 at 17:50

If you want to match $PAT as complete lines, I have a solution. By complete lines, I mean that, in the case of a match, you can split $FILE in three sub-files (f1, f2 & f3) where:

  • cat f1 f2 f3 is $FILE,
  • f2 is $PAT.

Note that f1 and/or f3 can be empty.

First, create the f2 file:

cat << EOF > f2

Then, diff $FILE and f2, saving the result:

diff $FILE f2 > diff_res

If $res is zero, then f1 and f3 are empty, and $FILE is equal to $PAT. I will suppose that you want an empty file in this case.

If diff_res contains a line starting by ">", f2 contains at least one line not in $FILE. To test that:

grep -q '^> ' diff_res
test $? -eq 0 && echo "PAT not found"

If diff_res does not contain lines starting by ">", all lines of f2 are in $FILE, but maybe not contiguously. If it is contiguously, diff_res will contain either:

  • A single line not starting with "<" (if f1 or f3 are empty),
  • Two lines not starting with "<", the 1st always starting with "1d" or "1,".

To test this, we have:

nb=$(grep -v "^< " diff_res | wc -l)
if test $nb -gt 2; then
elif test $nb -eq 1; then
  pat_found=$(sed -n -e '1{/^1d/p;/^1,/p}' diff_res | wc -l)

Then, if pat_found is 1, the file without $PAT is the diff result with only the lines starting by "<" without those 2 char:

grep '^< ' diff_res | cut -c 3-

The complete and reorganized script would look like:

# Output the desired result on stdin.

f2=/tmp/f2              # Use of PID or mktmp would be better'
diff_res=/tmp/diff_res  # Use of PID or mktmp would be better'

cat << EOF > $f2

diff $FILE $f2 > $diff_res
if test $? -ne 0; then
  grep -q '^> ' $diff_res
  if test $? -ne 0; then
    nb=$(grep -v "^< " $diff_res | wc -l)
    if test $nb -eq 1; then
      grep '^< ' $diff_res | cut -c 3-
    elif test $nb -eq 2; then
      pat_found=$(sed -n -e '1{/^1d/p;/^1,/p}' $diff_res | wc -l)
      test $pat_found -eq 1 && grep '^< ' $diff_res | cut -c 3-

rm -f $f2 $diff_res
share|improve this answer
I had to pick Gilles answer for simplicity, but this was just brilliant! Excellent stuff. – Sepero Dec 7 '12 at 17:35

Read the file character by character. If the character matches the first character of the variable, compare the next one, and so on. If the whole variable is not matched, return back. You can even implement a more advanced algorithm to make it work faster, but as your language happens to be the shell, it would be terribly slow anyway.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.