3

I know the line numbers in advance, and keep them in another file:

cat linenos
2
15
42
44
... etc

as you see the lines are non-adjacent so I cannt use one range for sed. The goal is to modify target file lines by, say, prepending them with a marker like MARKER

Straight forward way is to call sed multiple times to modify each line:

for l in $(cat linenos)
do 
  sed -i "${l}s/^/MARKER/" target_file
done

which apparently will call sed multiple times.

CAUTION: *Not only is this approach inefficient it can also make things go wrong if the modification is other than inserting a marker like this. Any line deletion or insertion sed command like d a r, will make initial line number in linenos invalid for the next sed runs in the loop.

What would you suggest to improve/optimize that?

Sample linenos file

cat linenos
2
5

Sample target_file

cat target_file
line one
line two
line three
line four
line five
line six

Expected result of modified target_file

cat target_file
line one
MARKERline two
line three
line four
MARKERline five
line six

Possible approach i came up with is dynamically create sed scenario

SEDCMD=$(for l in $(cat linenos); do echo -n "${l}s/^/MARK/;" ; done)

sed -i -e "$SEDCMD" targetfile

@steeldriver's below approach shares the idea, but is more elegant and concise

  • I have realized that the 1st straight forward approach is faulty. Made an update on that. – Tagwint Feb 28 at 16:31
  • Unless MARKER contains a newline, how are the line numbers affected? – chepner Feb 28 at 22:07
  • @chepner you are right, I was influenced by my final goal which goes beyond of the scope of the example I used in my question. Corrected my note accordingly. Thank you – Tagwint Feb 28 at 23:32
  • Maybe you should select one answer, it is expected that you do. – Isaac Mar 1 at 19:20
2

If fileN contains the numbers of lines to be modified, and target_file is the text file that contain the lines to be modified. The minimum solution will require to read each file once.

Sorted

If the file that contains the line numbers contains one number (bigger than 1) per line, is sorted and there are no repetitions, we can use:

awk 'BEGIN{ getline lineN <"fileN"} {
     if(NR==lineN){$0="MARKER " $0;getline lineN <"fileN"}
     }1' target_file

Which will keep only one line in memory (of each file) and walk both files from start to end. However once awk has processed a line, line 15 for example, it won't go back to line 12, for example. So, the file lineN has to be sorted (not repeated, and greater than 1) for this to work.

Unsorted

Of course, the naive solution is that the line numbers file could be sorted sort -nu fileN.

But, if the list of line numbers may be unsorted (and repeated), we may use either sed , ed (the precursor of sed), or awk (later):

Convert each line in lineN to a sed editing command like s/^/MARKER /. Either shell printf or sed could do that:

printf '%ss/^/MARKER /\n' $(<fileN) | sed -f - target_file
sed 's#$#s/^/MARKER /#' fileN       | sed -f - target_file

{ printf '%ss/^/MARKER /\n' $(<fileN); printf '%s\n' ,p Q; } | ed -Gs target_file
{ sed 's#$#s/^/MARKER /#' fileN ; echo "w"      ; } | ed target_file

Note that in the last case the editing is done directly and at the original file. The last command w writes the modifications to file. If what is needed is to print the result then use the third option, which will print all lines.

awk

In awk, capture the whole fileN in memory and process target_file

awk '{ if(NR==FNR){
                     a[$1]=1
                  }else{
                     if(a[FNR]==1){ printf("%s","MARKER ")};
                     print 
                  }
     }' fileN target_file

Or, with a variable to control when the list of files with line numbers has ended:

awk '{ if (dofile==1) {   if(a[FNR]==1){ printf("%s","MARKER ")};
                          print
                      }else{
                          a[$1]=1
                      }
     }' fileN fileK   dofile=1   target_file

Note that the last version allows several files with line numbers, like fileN and fileK in the example.

Also note that the awk versions do not process repeated line numbers. All repeated line numbers are processed just once.

|improve this answer|||||
  • special thanks for $(<file) idiom – Tagwint Mar 1 at 0:18
10

You could use sed itself (or another text-processing utility of your choice) to turn the line numbers into sed expressions, then pass those to sed using the -f switch

Ex.

sed 's:$:s/^/MARKER/:' linenos | sed -f- -i target_file

This at least only calls sed twice.

|improve this answer|||||
  • That's great! Do you think sorting linenos content will improve the sed efficiency in case it is unsorted initially? – Tagwint Feb 27 at 17:55
  • @Tagwint that's an interesting question - I don't know – steeldriver Feb 27 at 17:58
  • 1
    @Tagwint - no. For each line in your target file, sed will process the entire lineos-based script (that is, check each editing command and execute it, if applicable). – don_crissti Feb 27 at 18:54
  • @don_crissti thanks, that makes sense indeed. – Tagwint Feb 27 at 19:25
  • @steeldriver can you explain the first command, or at least give a name which can I can google. – Prvt_Yadav Mar 1 at 15:03
6

With perl (where GNU sed got -i from):

perl -pi -e '
  BEGIN{$l{0+$_}=1 while <STDIN>}
  $_ = "MARKER$_" if $l{$.}' target_file < linenos

We feed the list of line numbers on perl's stdin. That is read in the BEGIN block.

For each line of input, we convert that line to a number with 0+$_. That makes the newline character go away and also canonicalises the numbers (all of 1e0, 1, 01 become 1).

The %l hash table is populated with value 1 for each line number as the key.

The target_file is processed in the main -p loop where MARKERS is prepended to lines where the current line number ($.) is found in %l with a non-zero value.

|improve this answer|||||
4
$ awk 'NR==FNR{a[$1]="MARKER"; next} {print a[FNR] $0}' linenos target_file
line one
MARKERline two
line three
line four
MARKERline five
line six

or to save a bit of memory:

$ awk 'NR==FNR{a[$1]; next} {print (FNR in a ? "MARKER" : "") $0}' linenos target_file
line one
MARKERline two
line three
line four
MARKERline five
line six

If you want "inplace" editing (same as perl and GNU sed with -i) use GNU awk and change awk '...' to awk -i inplace '...' and add a print; before the next statement so your linenos file doesn't get emptied. IMHO it's simpler to just do this with any awk (or any other UNIX tool):

awk 'script' linenos target_file > tmp && mv tmp target_file
|improve this answer|||||
  • If linenos contains repeated line numbers, they will be processed only once. – Isaac Feb 29 at 22:53
1

Another way, using ed instead of sed to modify the target_file in place:

(while IFS= read n; do echo "${n}s/^/MARKER/"; done < linenos; echo w) |  ed -s target_file
|improve this answer|||||

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.