4

This question and its answers show the content of an existing file ("processedFile") can be inserted into another existing file ("receivingFile") before one of its patterns inside, such that:

sed -i -e '/pattern/r processedFile' receivingFile

At times, such content might result from a previous command-unit, and is not yet written.

At that point, is it possible to insert the outcome of the command-unit into receivingFile without creating/writing processedFile?

Example:

Say, we preprocessed someFile as follows (reading its content starting from the 4th line to the end of file):

awk 'NR>3' someFile

Now, without writing this to a new file, we want to directly insert it into receivingFile before a pattern such that:

awk 'NR>3' someFile | <insertion commands>

PS: Preferably by using awk or sed.


Based on the request from @RomanPerekhrest,

someFile:

# random output
# random
11
0 0 0 23.259058 2.1756592 -1.2097659 -0.074128056 0.34343502 -0.017659865
0 0 0.05 51.091269 4.8520157 -0.0011590485 0.00096592555 0.0059719715 -0.050598505

Preprocessing:

awk 'NR>3' someFile :

0 0 0 23.259058 2.1756592 -1.2097659 -0.074128056 0.34343502 -0.017659865
0 0 0.05 51.091269 4.8520157 -0.0011590485 0.00096592555 0.0059719715 -0.050598505

receivingFile :

stackExchange is the best.
pattern
Unix stackExchange is the best of the bests.
  • from what I see, I’d say use cat someFile | awk 'NR>3' | <insertion commands> – Stan Strum Feb 12 '18 at 18:10
4

I can't tell if it's portable accross all *NIX, depending on implementation. It's working on my Linux Debian.

TL;DR: use /dev/stdin as included file, which is sed's input and will get the output of the command before the pipe, like

awk 'NR>3' someFile | sed -i -e '/pattern/r /dev/stdin' receivingFile

Example: destination file contents called receivingFile:

stackExchange is the best.
pattern
Unix stackExchange is the best of the bests.

Input sample called someFile:

# random output
# random
11
0 0 0 23.259058 2.1756592 -1.2097659 -0.074128056 0.34343502 -0.017659865
0 0 0.05 51.091269 4.8520157 -0.0011590485 0.00096592555 0.0059719715 -0.050598505

To insert after the pattern:

awk 'NR>3' someFile | sed -e '/pattern/r /dev/stdin' receivingFile

To insert before the pattern, reusing the answer linked by OP and dump to output:

awk 'NR>3' someFile | sed -n -e '/pattern/r /dev/stdin' -e 1x -e '2,${x;p}' -e '${x;p}' receivingFile

To do what was asked and also replace the file:

awk 'NR>3' someFile | sed -i -n -e '/pattern/r /dev/stdin' -e 1x -e '2,${x;p}' -e '${x;p}' receivingFile

New content of receivingFile:

stackExchange is the best.
0 0 0 23.259058 2.1756592 -1.2097659 -0.074128056 0.34343502 -0.017659865
0 0 0.05 51.091269 4.8520157 -0.0011590485 0.00096592555 0.0059719715 -0.050598505
pattern
Unix stackExchange is the best of the bests.

UPDATE: as noted by @don_crissti, this doesn't work if pattern is the last line. Here's an other command, adapted from his answer to the same linked question! Some other command is needed to accomodate with the temporary addition (and suppression by sed) of the extra blank line. These two would work:

awk 'NR>3' someFile | sed -e '/pattern/{r/dev/stdin' -e 'N;:l;$!n;$!bl};${/^$/!{s/\n$//};//d}' receivingFile <(printf '\n') > receivingFile.new
mv receivingFile.new receivingFile

or just (not needing bash and leaving sed -i to handle the final file):

awk 'NR>3' someFile | { echo '' >> receivingFile; sed -i -e '/pattern/{r/dev/stdin' -e 'N;:l;$!n;$!bl};${/^$/!{s/\n$//};//d}' receivingFile ; }

Look at the linked answer for the line-splitted sed code.

  • Related to the question of portability: unix.stackexchange.com/questions/338667/… – Kusalananda Feb 12 '18 at 23:22
  • If you read all the answers there (instead of just copying jfg956's solution which partially works) you'll learn that the stuff you used in your answer here only works if pattern is not on the last line. If it's on the last line the stuff from /dev/stdin will be inserted after pattern (not before it, as required here). – don_crissti Feb 13 '18 at 11:23
3

Just use ex, the POSIX-specified scriptable form of vi (which is also POSIX-specified).

printf '%s\n' '/pattern/-r !awk "NR>3" somefile' x | ex receivingFile

Or more generally:

printf '%s\n' '/pattern/-r !somecommand' x | ex somefile
  • ed is better than ex for this. /bin/ed is an ~ 50KB binary. /usr/bin/ex is a symlink or hardlink to vi - if that vi is vim, then it'll be at least a 1MB binary (for vim.tiny on debian. or more depending on what's compiled into vim - e.g. 2.5MB for vim.basic, 3.1MB for vim.gtk3). Even nvi is about 470KB. – cas Feb 13 '18 at 0:50
  • @cas, "ed is better than ex for this" — that really depends on context. If the context is an embedded system or one where performance is absolutely critical, that may be true. It's not an absolute truth. You could also say that handling it in assembly is "better" because if it's well written you can get an even smaller binary—which we would both agree is nonsense. ex is far more flexible than ed and also has the advantage that it's already the editor of choice for most sysadmins (who use vi)—they just don't know it exists yet. – Wildcard Feb 13 '18 at 0:55
  • What do you mean by "far more flexible"? ex is a clone of ed (actually, of em which was a clone of ed for use on new-fangled video terminals), - is there any scriptable editing operation that ex can do that ed can not? BTW, it is an absolute truth that executing a 50KB binary is always better than executing a 1MB binary that does the exact same thing. – cas Feb 13 '18 at 1:25
  • @cas, you can compare the specs for ed and the specs for ex and find for yourself the differences in functionality. (Registers come to mind.) But the crucial distinction is that ex is already built in to the editor of choice used by most sysadmins. ed is not. Knowledge of ex is directly and immediately usable in everyday editing tasks, without leaving your vi session. (NB: If you reply to this pugnaciously do not expect another response.) – Wildcard Feb 13 '18 at 4:11
  • The "specs" for ed and ex are trivially different. The edit operations in ex are the same as those in ed. Your alleged advantage that "ex is already built in to the editor of choice used by most sysadmins" is no advantage at all because ex commands are identical to ed. The actual difference is the difference between running a 50KB binary and a 500K or 1M or 2.5MB symlink to vi/vim. BTW, it is you that has been replying pugnaciously - you seem to have taken offence at my initial comment and responded in an obnoxiously defensive manner as if I had made some personal criticism. – cas Feb 13 '18 at 5:07
1

With sed (GNU sed) 4.4

sed -i '/pattern/{h;s/.*/awk "NR>3" somefile/e;p;x}' receivingFile

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.