This is a bare-bones awk version to count changes and affected lines:
#! /bin/bash
Awk='
BEGIN { fmtEnd = "Made %d substitutions on %d lines.\n"; }
{
n = gsub (/exit/, "return");
if (n) { Lines++; Count += n; }
print;
}
END { printf (fmtEnd, Count, Lines) > "/dev/stderr"; }
'
awk "${Awk}" doFifo > doFifo.fix
Output (stderr) is merely this, which could be rearranged to make it easier to recover the count:
Made 8 substitutions on 6 lines.
GNU/awk does have the -i inplace extension, but I am ultra-conservative about update-in-situ. My clients complain a lot, and assert their data is always 100% correct, so I keep audit trails and every data version.
Below is an awk variant which notes every changed line. This is still not production level: I would want it to accept the pattern and replacement as args, deal with multiple files in one run, name the output files based on the inputs, and summarise by file and overall total. Maybe allow an array of pattern -> replacement too.
#! /bin/bash
AwkFull='
BEGIN {
reFix = "exit"; txFix = "return";
fmtEnd = "Made %d substitutions on %d lines.\n";
fmtSub = "\n.... %d Changes on file %s line %d:\n";
fmtSub = fmtSub "Was: %s\nNow: %s\n";
}
{
New = $0;
n = gsub (reFix, txFix, New);
if (n == 0) { print $0; next; }
Lines++; Count += n;
printf (fmtSub, n, FILENAME, FNR, $0, New) > "/dev/stderr";
print New;
}
END { printf (fmtEnd, Count, Lines) > "/dev/stderr"; }
'
awk "${AwkFull}" doFifo > doFifo.fix
This shows every changed line like:
.... 2 Changes on file doFifo line 64:
Was: (exit) printf 1>&7 '%(%T)T Received exit command\n' -1
Now: (return) printf 1>&7 '%(%T)T Received return command\n' -1
Edit: Making the parameters into command arguments.
The first version above embedded the pattern and replacement text inside the gsub command itself. The second version made it easier to change these by (a) giving them names, and (b) declaring them at the head of the code.
The next stage in generalising the code would be to pass these from the shell. This is easy in awk. First, remove the line that defines reFix and txFix (convention is re
for regular expression and tx
for text, but call variables anything you like provided you are consistent).
To get shell strings into awk variables, there is a -v
option. So your awk command becomes:
awk -v reFix="exit" -v txFix="return" "${AwkFull}" doFifo > doFifo.fix
and the final step to use shell variables is to use any form of shell substitution, like:
awk -v reFix="${1}" -v txFix="${myNew}" "${AwkFull}" doFifo > doFifo.fix
There are two (maybe more) downsides:
(1) awk knows that /exit/
is a pattern. In some cases, you may need to clarify the syntax: for example, a simple line-match like /exit/
will need to be rewritten as $0 ~ reFix
. But awk knows the first arg to gsub()
is a pattern, so that syntax works unchanged. (See https://www.gnu.org/software/gawk/manual/gawk.html#Strong-Regexp-Constants for more.)
(2) Patterns in variables do not get syntax-checked when the awk program is first read, only when they are used. So user-entered patterns may easily break in the middle of a run, and with obscure error messages.
awk
would be a much better fit. It has a gsub() function which returns the count of substitutions it made. You can add that to a global count, and send it to stderr in an END block. You could also print each line that had any substitutions made for context, and with a file name and line number.sed
? and not a JSON parser likejq
?echo "aaa" | grep -o a | wc -l
will return3
.jq
ever before :)