3

Fairly new to Bash, about a grueling week. I love it so far and really like long chaining pipes. What I'm noticing though is that I have to break the pipe if I need to use the STDOUT as a variable.

Here's an example:

echo -e 'Going to the movies can be fun.
When a dog found a cat a trouble began.
Do not text while driving please.' > example

Suppose I want to replace every "A" with "THE" in the second line. Here's how I'd currently do it.

cat example | head -2 | tail -1 | sed 's/ a / the /g'

Here's how I'd like to do it

# None of these work
cat example | head -2 | tail -1 | ${xargs//a /the } 
cat example | head -2 | tail -1 | ${cat//a /the }
cat example | head -2 | tail -1 | ${$0//a /the }
cat example | head -2 | tail -1 | ${\1//a /the }

Right now I'd have to create a variable then use bash string manipulation.

middle=$(cat example | head -2 | tail -1)
echo ${middle//a /the }

I do realize for this example you can use many tools

cat example | awk 'gsub(/ a/," the");'

What's I'm really trying to figure out is if there is any way to use the pipe STDOUT as a variable.

  • Maybe tee could help you, if you want to save the output of the current pipe to a file while continuing the pipe afterwards. You could then read the file into a variable. Not sure if that helps you though. – confetti Aug 12 '18 at 7:38
  • 1
    I'm not entirely sure I understand what you mean by "pipe as a variable". Could you please clarify that bit? To change a on the second line of the text in example, I would do sed -i '2s/a/the/' example, without involving variables. – Kusalananda Aug 12 '18 at 7:42
  • Sorry I am struggling to understand this question. Are you asking how do i replace a with the on the second line, and then continue the pipeline? What do you mean by “use the pipe stdout as a variable”? – ctrl-alt-delor Aug 12 '18 at 10:49
  • 1
    The fact that we can not find a solution, to do it your way, may indicate that your way is not the Unix way. Or that we need a more concrete example. You say that you added the head and tail to make the pipe longer, for the example. This shows that your example is an abstraction. This is often a good thing, but I wonder if this time, as the problem is abstract, the examples should be concrete. Can you add a real life example. – ctrl-alt-delor Aug 12 '18 at 11:18
2

How do I replace a with the on the second line, and then continue the pipeline?

First fix the sed to match all as, event as start and end.

sed -r -e 's/(^| )a($| )/\1the\2/g'

Then make it match only on line 2

sed -r -e '2 s/(^| )a($| )/\1the\2/g'

Now you can do

echo -e 'Going to the movies can be fun.
When a dog found a cat a trouble began.
Do not text while driving please.' | sed -r -e '2 s/(^| )a($| )/\1the\2/g' | less

Another solution

First drop the cat

cat is just opening a file and passing it to its output, that is then piped to the next command. We can get the shell to open a file and direct it to the input of the next (now first) command, by using input-redirection (<). We can write this as command args <input-file or <input-file command args. It does not matter where within the command we put the <input-file. I put the redirection first, so that the pipe-line can be read left to right.

< example head -2 | tail -1 | sed 's/ a / the /g'

Alternative (that does exactly the same thing):

head -2 <example | tail -1 | sed 's/ a / the /g'

these do the same as the following, but with one less process.

cat example | head -2 | tail -1 | sed 's/ a / the /g'

Next encapsulate: Here we put brackets around 3 commands, we can then pipe the output of all 3 to another command (here I use less).

{  
  < example head -1
  < example head -2 | tail -1 | sed 's/ a / the /g'
  < example tail -1
} | less
  • Honestly, I'm not sure what 66% of your second solution is doing. I've accepted the fact that my thinking just isn't the Unix way, as you put it. I'm going to select yours as the answer, thanks for killing my cat. -_o – Proximo Aug 12 '18 at 16:43
  • I have added some explanation, hope it is enough. – ctrl-alt-delor Aug 13 '18 at 9:36
1

Command substitution is the way to go (as you do with $middle) but IMHO just piping through sed in these situations is easier.

Having said that: I know that you are just learning, but combining head, tailand sed is hardly ever necessary. Your example could be expressed in pure sed as

sed -n '2s/ a / the /gp' example
  • -n suppresses output by default
  • 2 matches the second line
  • s/ a / the /g replaces all ("g"lobal) occurences of a.
  • p prints the result
  • That's for the comments. I added head and tail just to make the pipe longer. I like SED and AWK but if pipe stdout was a usable variable like say $0 I really think the language would be more dynamic. – Proximo Aug 12 '18 at 7:55
  • 1
    @proximo ${} only work on variables so there is no way to use this to process the output of a command without assignment. It would make a nice addition though (even if the use is limited probably, as anything a bit more complex requires sed etc anyway). – nohillside Aug 12 '18 at 9:17
  • 4
    @Proximo The thing with stream processing in Unix is that you very seldom want the output of pipeline in a variable at all. Variables commonly hold short strings like single words or a line of text, while the result of a pipeline may well be several gigabytes of data. – Kusalananda Aug 12 '18 at 11:07
  • An array[@] at $0 would be nice. I am starting to see the error in my thinking. I will just accept things for what they are. – Proximo Aug 12 '18 at 16:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.