I have a command that uses an awk and a sed.

awk '{$1=""; print $0}' file.txt | sed "1s/.*/D,,3/" #awk command removes first column from file.txt and prints the rest, sed inserts "D,,3" on the first line

I am getting the desired result for the command above. However, I am trying to avoid using pipes, is there a way to combine the two commands above and get the same result without using a pipe.

first : awk '{$1=""; print $0}' file.txt
second: sed "1s/.*/D,,3/"

row 3:
name      john doe
state     Florida
age       32
  • 3
    Why do you need sed at all? there are many ways to modify the first line only in awk e.g. awk 'NR==1 {print "D,,3"; next} {$1=""; print $0}' – steeldriver Feb 22 '17 at 1:14
  • I used sed to insert D,,3 on the first line. I tried you command and it worked the same. What does NR==1 and next do? – venka99 Feb 22 '17 at 1:20
  • What's wrong with pipes? – DarkHeart Feb 22 '17 at 1:25
  • I was told that each command in a long pipe adds runtime compare to a single command without a pipe. – venka99 Feb 22 '17 at 1:29
  • A single command that does two things to a stream may well be slower than two separate piped commands, because those will run concurrently. – dhag Feb 22 '17 at 2:22

Awk and sed are both Turing-complete, so whatever one can do, the other can also do. (In terms of text transformation, that is; awk has more OS interaction facilities.) However, each has their own strengths and weaknesses. Awk can easily do most of what sed can do but a few things such as regex replacements with groups are more difficult. Sed can theoretically do anything awk can do but with considerable difficulty (e.g. sed doesn't have integer arithmetic so you'd have to code it up with text transformations). So if you want to put everything in a single command, awk is usually your best bet.

awk 'NR==1 {print "D,,3"; next} {$1=""; print $0}' file.txt

Here, sed can also easily do what you're doing with awk: normalize whitespace and cut everything up to the first space.

sed "s/[ \t][ \t]*/ /g; s/^ *[^ ][^ ]*/ /; s/^ $//; 1s/.*/D,,3/" file.txt

Combining the two commands isn't particularly useful here. I don't anticipate a performance gain. There is a performance penalty for every pipe, since the data has to flow from one command to the next. However, there can also be performance gains. If you have multiple CPUs then the two commands can execute in parallel. If a more specialized command is used for certain things (e.g. using special-purpose tools like grep and head instead of a general-purpose tool like awk) then that more specialized command is usually faster. Whether the gains compensate the overhead of the pipe depends on the data, on the tools, on how many cores you have, etc.

Unless this script is a performance bottleneck, think of clarity first. In this case, I would avoid the pure sed version — it may be slightly faster, but it's a lot less readable. (It could perhaps be simplified under certain assumptions about your data, e.g. if you know that there will always be at least two fields and that the field separator is always a single space or you don't care to preserve the amount of whitespace.) I find it clearer to do everything in awk, but it it's a near draw between that and your awk+sed version.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.