0

When i try to use line number from variable in sed it works individually but when same is done using while loop it not works.

eg.

cat file
den 
run
ten
run
fan

linenumber=2
replacetext=34
sed -i ""$linenumber"s/run/run\n"$replacetext"/" file

Output:
cat file
den
run
34
ten
run
fan

Above works and sed command is generated after variable expansion.

But when i use same in while loop it does work only for first instance and not all as i wanted to replace all instances with incrementing line number using pure sed solution. Awk or perl solution is not wanted.

eg.

replacetext=1
while read in; do replacetext=$((replacetext++)) && sed ""$in"s/run/run\n"$replacetext"/" file; done <<< "$(cat file | grep 'run' -n | cut -f1 -d ':')"

Expected Output: Match text will be replaced with itself followed by newline and a incrementing number.

den 
run
2
ten
run
3
fan

Obtained output:

den
run
2
ten
run
fan

Please only sed based solution required. And i wanted to know what unusual behaviour was occuring in while loop that it was not working.Because the loop variable is given the value of line number and then that line number is used in sed to replace macthed text on that line with itself folowed by newline and incremented number.

Sed seems to work when same variable line number is provided but not when used in while loop.

8
  • @Quasímodo is close. (I thing i minor bug in there comment). – ctrl-alt-delor Jun 27 '20 at 11:51
  • Dollar expansion happens in double quotes, so no need to close and re-open. – ctrl-alt-delor Jun 27 '20 at 11:52
  • If you have GNU sed, you might consider doing something like seq 1 "$(wc -l <file)" | sed -i '/run/R/dev/stdin' file instead. The seq just generates a suitably large number of indices and the sed reads as many of them as it needs. – steeldriver Jun 27 '20 at 12:16
  • Quasimodo your solution did not worked at all. – Daichi Ash Jun 27 '20 at 12:21
  • steel driver your solution worked. Thanks – Daichi Ash Jun 27 '20 at 12:25
2

It fails because the stuff after <<< gets a list of the correct line numbers in the original file.

Then you edit the file multiple times, inserting extra lines.

So every sed except the first is being told the wrong line numbers for the s/// command.

One solution would be to reverse the list of line numbers, so edits did not affect earlier numbering.

This is a sed solution, in one pipeline, somewhat quirky.

The first sed gets the line numbers where the Pattern is found, using the = operator. The nl prefixes a sequence number to each line: like 3 10

The second sed outputs sed expressions: it converts each such line to your required sed edit, like: 10 s/$/\n3/.

The third sed collects and applies those edits from stdin, and does an in-situ edit of the file. In this case, it edits itself.

This is the script itself:

#! /bin/bash

Pattern="[Ss]ed"

RegEx='[^0-9]*\([0-9]*\)[^0-9]*\([0-9]*\).*'
DoSed='\2 s/\$/\\n\1/'

sed -n -e "/${Pattern}/=" badSed |
    nl -ba |
    sed -e "s:${RegEx}:${DoSed}:" |
    sed -i -f - badSed
 

This is what it does to itself:

#! /bin/bash

Pattern="[Ss]ed"

RegEx='[^0-9]*\([0-9]*\)[^0-9]*\([0-9]*\).*'
DoSed='\2 s/\$/\\n\1/'
1

sed -n -e "/${Pattern}/=" badSed |
2
    nl -ba |
    sed -e "s:${RegEx}:${DoSed}:" |
3
    sed -i -f - badSed
4
3
  • It worked reversing the line numbers with tac – Daichi Ash Jun 28 '20 at 13:02
  • That's a trick I learned 40 years ago when fixing compiler errors, which show line numbers. If you start editing at the bottom of the error list, the source line numbers match the error messages all the way up. But you get to check a lot of knock-on errors that way. – Paul_Pedant Jun 28 '20 at 13:22
  • It is still a shed-load easier in the proper tool: awk '{ print; if (/run/) print ++n } One process, no shell loops, ten seconds to write. – Paul_Pedant Jun 28 '20 at 13:29
0

The line numbers are being messed up where the substitution needs to happen.

So we devise another strategy, we allow the while loop to first generate a valid set of sed commands, which are then applied back to the input file.

replacetext=1
while read in; do ((replacetext++)); echo "$in s/\$/\\n$replacetext/"; done <<<"$(grep -n run file | cut -d: -f1)" | sed -f - file > tmp && mv tmp file 
3
  • If you don't use sed -i then you will see duplicated output. Because the sed is invoked as many times as the number of lines comprising the string "run". – Rakesh Sharma Jun 27 '20 at 19:34
  • Cut off the last part (the sed itself) and just inspect the sed commands that are coming down the pipe. – Paul_Pedant Jun 28 '20 at 13:16
  • In this case, you are only running one sed, so the -i option will work properly here. The problem with it previously was that the file was altered between each iteration. – Paul_Pedant Jun 28 '20 at 13:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.