redirecting output of a command as if it was file input

Question

I find myself always having to arrow up and going backwards through the command line to change one part of an earlier command that is pipped to a later grep or head or whatever command.

For a crude example: searching dmesg for a given string and only wanting the last 5 occurrences.

I would do the above such that:

dmesg | grep -i USB | tail -n 5

but then if I want to change the search term or what I am searching for or the source, I am waiting for my cursor to backtrace to the relevant part of the line. I'd like to move that to the end of the line such that the above example could be represented as (I know this is incorrect):

tail -n 5 < dmesg | grep -i USB (and I could then search for sda)

as I said, this is a crude example and is indicative of the sort of thing I'd LIKE to do, but not actually what I want to do, ie. this isn't about dmesg or the use of grep and tail, but how to interact between these programs placing the "variable" at the end.

This could be further exampled on getting the seek time on a DNS enquiry such that:

dig google.com | grep msec (let's move google.com to the end so that I can then test another domain) via
grep msec < dig google.com

again, poor examples. I am talking more about when I have input/outputs flowing between multiple programs

I do admit I only have a basic understanding of piping and redirection which could be summarised as:

• piping (|) using the stdout of one program to feed into the stdin of another
• redirect (>) taking the stdout of one program and feeding it into a file (aside: could it be redirected elsewhere other than the only obvious ones I can think of
  > /dev/null or
  > /dev/sda or
  > {said file}
  whereas piping is program output as program input going left to right.
• redirect (<) (which I call "file as input") taking the contents of a file and using those as if they'd been typed at the command line

Answer 1

I'm often annoyed by the same thing, and there are a few ways of dealing with it:

1. Just accept it, sometimes it's not worth spending any time to work around it.

2. as PSkocik mentioned, you can use e.g. tail -n 5 <(dmesg | grep USB)

3. Become more adept with your shell's editing capabilities - e.g. in bash (and many other programs that use readline) you can use CtrlA to get to beginning of line, ESCf and ESCb to move forward and back a "word", and Ctrl-XCtrl-E to edit the current line in $EDITOR (e.g. vim)

There are many more editing commands available and readline is fully documented in .info files. On a Debian system, install the readline-doc package. Other distros may include the documentation in the readline package itself or may separate it as Debian does.

I also recommend installing and using pinfo for a more lynx web-browser-like experience (IMO the GNU info browser is ghastly and almost unusable). If it's not already in your distro, you can find it at http://pinfo.alioth.debian.org/

4. readline also has a vi mode for editing (the default is an emacs-like mode), which some people prefer.

5. In simple cases, you can use quick substitution: e.g. if the last command you entered was:

dmesg | grep -i USB | tail -n 5

then typing ^USB^sda^Enter would result in this being executed:

dmesg | grep -i sda | tail -n 5

For more details on this, see man bash and search for HISTORY EXPANSION, especially the section Event Designators.

and yes, doing this repeatedly also becomes annoying.

6. For more complex cases, the best solution is to write a shell script or function that does what you want and run that instead of your long complicated pipeline of commands.

e.g. write a shell script called dmesg-grep that looks something like this:

#! /bin/bash

# regexp to search for is arg 1.  needs to be an extended regexp
# because we're using grep -E aka egrep.
re="$1"

# number of lines to output is optional arg 2 (default 5)
lines=${2:5}

dmesg | grep -iE "$re" | tail -n "$lines"

Then you can just run dmesg-grep usb or dmesg-grep sda.

If you do this a lot, make a bin subdirectory of your home directory and add ~/bin to your default PATH (e.g. in ~/.bash_profile or ~/.bashrc) and save your scripts in there.

Answer 2

In shell that support <() (bash, ksh, zsh), you can transform

$ cmd1 | cmd2 

into

$ cmd2 < <(cmd1)

These two should be equivalent in terms of piping. (However, I think there are some minor differences in terms of how the two forms handle things like process group and signals).

For better command line editing, I recommend you explore your shell's facilities for that. E.g., bash has set -o vi which allows you to use vi-like shortcuts or switch into vim directly to edit your command more efficiently. The default set -o emacs uses emacs shortcuts (search man bash for readline to learn more about them (/readline)).

Answer 3

You may think this is a sideways answer, but often the solution is to simply use variables in your command line. They are not just for shell scripts. Eg you are developing a complex "one-liner" by trial-and-error and have:

 $ some complex command using value 23 at this point

then you want to vary just one value in the middle of the command line. A single edit and then the line becomes easy to recall without it needing any further editing:

 $ a=23
 $ some complex command using value "$a" at this point
 $ a=37
 $ some complex command using value "$a" at this point
 $ a=42
 $ some complex command using value "$a" at this point

It's even better when you need $a in 2 or more places in the command, such as for a directory or filename.