1

Sometimes I have output from a command line script that I would like to further process/filter with cli tools. I can't re-run the command because it takes a long time or will not produce the same output again.

Currently I paste the output into a new file in the text editor, save it and then use cat on cli to pipe it into tr, sed and other tools. This is cumbersome.

Is there a quicker way for such text processing tasks?

7
  • 2
    "I paste the output into a new file in the text editor, save it" - you could also the command for accessing the clipboard in your OS (e.g, xclip on Linux, pbpaste on macOS, etc.) to paste the the copied output directly into a file (pbpaste > some-file).
    – muru
    Jul 22, 2022 at 7:28
  • I don't know of any shell that would keep the output of the last command by default, but I think it should be possible to set up an strace background process tracking the output of your shell's child processes and make the last output available somehow.
    – Philippos
    Jul 22, 2022 at 7:41
  • as the command was already executed in the past and OP can't re-run the command, setting up strace would be useless; could help only with future executions of the command for which tee or piping to file is much better for text parsing purposes than strace
    – magor
    Jul 22, 2022 at 8:11
  • Making a shell that automatically saves the output of commands would be an interesting exercise, BUT a simplistic implementation would make the programs detect their stdout connected to a pipe instead of a terminal, which would affect the behaviour of many programs, and the output would be less than useful for e.g. ncurses programs that move the cursor around to redraw parts of the screen. Plus storing the output of someone's week-long Emacs session might also start being a bit unwieldy...
    – ilkkachu
    Jul 22, 2022 at 11:23
  • @magor The question is not about the command executed in the past, but about the same situation in the future. Starting a background process now will make the output available in all future, if the same situation happens again.
    – Philippos
    Jul 25, 2022 at 5:21

4 Answers 4

1

You can do exactly the same thing as you describe, but without using a file, nor an editor. Just do not specify a filename as argument to the cat command. This means cat will read from stdin. Then, with the filter pipeline waiting, you just paste the text to the terminal, and press CTRL-D to close stdin for cat.

The drawback is that the filtered output is mingled with the input text. This does not hurt though if the filter stores the output into some file.

0

Your use case is not specific enough, but if using Bash is not a problem, then you can benefit from PROMPT COMMAND.

PROMPT_COMMAND='touch ~/.last_command_output; LAST_COMMAND_OUTPUT="$(cat ~/.last_command_output)"; exec >/dev/tty; exec > >(tee ~/.last_command_output)'

From now on you can work with $LAST_COMMAND_OUTPUT variable which should contain output from your last command.

What happens under the hood:

  • PROMPT_COMMAND is a variable that defines command executed before BASH displays prompt (last command finished working in most of cases)
  • touch ~/.last_command_output; - creates file ~/.last_command_output so it will be used for storing purposes later on
  • LAST_COMMAND_OUTPUT="$(cat ~/.last_command_output)" stores contents of the file mentioned before in environment variable named LAST_COMMAND_OUTPUT so using it is a matter of simple echo later
  • exec >/dev/tty; - exec with no arguments is used to redirect output of current shell to /dev/tty (current terminal device) this is necessary to drop old redirection from previous command in terminal (see next bullet point)
  • exec > >(tee ~/.last_command_output) - runs tee (it stores contents of stdin inside of file and also outputs it to stdout) using process substitution so exec can redirect output to this file descriptor that is like piping it to tee.

This seems a good place to start with. Then you probably could add some other features to it, like storing output from N last commands or making sure, that multiple sessions will not overwrite file used for storing output from command.

Also there are drawbacks. Redirecting output that way causes some commands notice that output is not stdout and you might experience different behaviour comparing to running in terminal without PROMPT_COMMAND. There might be only something like no color in terminal, but ls will output it's listing in one line instead of each entry line by line. Most noticeable are problems with terminal programs that require input from user like vi. Warning that input is not a terminal and messed output in $LAST_COMMAND_OUTPUT variable.

Also, to disable this simply unset PROMPT_COMMAND.

0

Wrap the call to your existing command in a function like:

doit() {
    local ret=0

    exec 3>&1                           # save fd 1 (stdout) in fd 3 to restore later
    exec > >(tee output.txt)            # redirect stdout of this script to go to tee

    your_command "${@:--}"              # run whatever command you want

    ret="$?"                            # save that command's exit status

    exec 1>&3; 3>&-                     # restore stdout and close fd 3

    return "$ret"
}

then call doit instead of your_command. That will run your_command and save it's output in the file output.txt for editing later. Obviously name the function and output file something appropriate.

0

As suggested by @muru I use a clipboard-to-shell tool now: xsel.

When I find that cli output needs to be mangled further, I select and copy it with ctrl+shift+c to the clipboard. Then I use xsel -b (= xsel --clipboard) to paste it as standard input to other tools:

$ xsel -b | grep foo | sed s/bar/baz/

This is solution that works everywhere without needing setup.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .