I know that to capture a pipeline's contents at an intermediate stage of processing, we use tee as ls /bin /usr/bin | sort | uniq | tee abc.txt | grep out
, but what if i don't want to redirect the contents after uniq to abc.txt but to screen(through stdout, ofcourse) so that as an end result , i'll have on screen, the intermediate contents after uniq as well as the contents after grep.
7 Answers
sometimes /dev/tty can be used for that...
ls /bin /usr/bin | sort | uniq | tee /dev/tty | grep out | wc
-
3
-
1
ls /bin /usr/bin | sort | uniq | tee /dev/tty | grep out | wc > out
only contains the output fromwc
.– user7543Feb 12, 2021 at 20:14 -
1@user7543 tee -o out will give you tee output you can dan do wc >> out to append output to the file (depending on nonglobering settings that is). I would use file descriptor for both tee and grep and cat it.– netheroApr 16, 2021 at 14:58
-
ls /bin /usr/bin | sort | uniq | tee /dev/fd/2 | grep out | wc
On a linux system you can use the the /dev/fd/[num]
links like named pipes in many cases. This will duplicate stdout to stderr, which, typically, is your terminal screen, but doesn't need to be.
-
Only works with bash, not dash /bin/sh on Ubuntu. I use this command for docker build:
ID=$(docker build "$dockercontext" 2>&1 | tee /dev/fd/3 - | tail -1)
Oct 15, 2015 at 13:15 -
@DanielAlder - it works fine with
dash
. but you need to direct it somewhere. where does&3
go? thefd/2
above assumes the command is run from a terminal and&2
goes to thetty
. for example:dash -c '{ echo hey; read v; echo "${v#?}" >/dev/fd/2; } <>/dev/fd/1 |:'
– mikeservOct 16, 2015 at 5:24 -
You are right. Didn't realize that /dev/fd/* are now also supported in dash. fd/3 is redirected to stdout in my case. I use this command line now, the other variant was overkill:
ID=$(docker build "$1" 2>&1 | tee /dev/fd/2 - | tail -1)
Oct 16, 2015 at 7:51 -
@DanielAlder - those dont really have anything to do with the shell. theyre just files. or... links to file-descriptors anyway. but the shell doesnt put them there or control their behavior at all - those are handled by the kernel. the shell just does
open()
anddup2()
with file-descriptors and it treats those exactly the same as it does any other file.– mikeservOct 16, 2015 at 7:59 -
You don't know the history, and I didn't know the current situation. Not many years ago, there was no /dev/fd/ in the system. It was compiled directly in bash like /dev/tcp/ too which can be used to open tcp connections. Actually, bash still supports the emulation in case it can't find /dev/fd. Obviously I missed some developments of /proc/self, udev and the kernel so I didn't know that this is now system default. Oct 16, 2015 at 8:30
This command worked for me.
ls /bin /usr/bin | sort | uniq | tee /dev/pts/0 | grep out
You could check what is your terminal using the command tty
and replace the tee to redirect the output to that terminal.
References
-
-
@JoeSewell, you could see which is the running terminal using
tty
. I have updated with that information.– RameshJan 12, 2015 at 17:49 -
1Nice update, but why not use
/dev/tty
instead? That assumes, of course, that the command even has access to the terminal. If it's run from, for example, insidegvim
, there may be no terminal involved at all. Doing this withingmake
could become even more complex with multiple jobs, since not all processes can get to "the terminal." Jan 12, 2015 at 17:53 -
How to do it (example):
exec 3>&1; ( ls |( tee >&3 ) >/dev/null ); exec 3>&-
Which will show the ls result and send it to nirvanah.
To grok the key part, 3>&1
, you may read I/O Redirection and esp. this example.
In short: >somefile
is short for 1>somefile
, which in turn means Assign the file handle of somefile to file descriptor 1 (and drop the former value of that descriptor, for the scope of this process).
So, 3>&1
means: Assign the file descriptor 1 (which may but need not be tty) to the (until now unused) file descriptor 3. We're effectively using &3
as a temporary variable.
-
The exact command provided doesn't work for me:
$ 3>&1 ls |( tee >&3 ) >/dev/null
givesbash: 3: Bad file descriptor
Apr 27, 2021 at 9:00 -
Thanks @ColinPitrat, i seem to have missed something, but using exec it works for me. Apr 27, 2021 at 11:55
-
Cool, this makes it a great answer because it's now more portable and conceptually simpler than all the others. Apr 28, 2021 at 6:15
mkfifo myfifo
cat myfifo& ls /bin /usr/bin | sort | uniq | tee myfifo | grep out
mkfifo
creates a FIFO (first in, first out) special file, a.k.a. a named pipe.
Start an asynchronous cat
to read from the fifo, and then run your pipeline,
tee
ing the intermediate result to the fifo.
This will produce a [1]+ Done cat myfifo
message at the end.
You can suppress that with this magic trick:
(cat myfifo&); ls /bin /usr/bin | sort | uniq | tee myfifo | grep out
For a long term, robust solution, you might want to create a permanent fifo
(e.g., $HOME/myfifo
) rather than creating a new one every time.
But that will fail if you may be running multiple instances of this simultaneously.
Alternatively,
- Generate a unique name (e.g., with
mktemp
). - Create the fifo in a directory that's guaranteed to be writable (e.g.,
/tmp
). - Remove the fifo at the end of the command.
In the case this is being run from a terminal you could
- start a terminal multiplexer such as tmux or screen
- split your terminal - in tmux Ctrl B"
- in the first window
touch abc.txt
thentail -f abc.txt
- switch to the second window and run your command. You'll see the file created by tee update.
http://tldp.org/LDP/abs/html/io-redirection.html
cat stuff | 3>&1 tee 3 | xsel -i
Edit: unfortunately, the above ↑ is incorrect. An earlier answer of
cat stuff | tee >(cat) | xsel -i
is good though.
-
1(1) It seems to me that this does not do what the question asks for. (2) TLDP is not a good source of information; please do not use it. (3) I don’t see much correlation between the page you linked to and your answer. Why are you giving TLDP credit (/ blame)? (4) The question is about a pipeline involving
ls
,sort
,uniq
andgrep
. It’s good manners to answer the question that was asked — and yet your answer shows a pipeline involvingcat
(a UUOC, BTW) andxsel
. … (Cont’d)– ScottMar 16, 2020 at 3:15 -
1(Cont’d) … (5) Please don’t clutter our site with answers that you know to be wrong, unless you have a good reason to do so. Instead of adding a second answer (and, inexplicably, calling it an “earlier answer”), just delete the wrong answer and replace it with the one that’s less bad.– ScottMar 16, 2020 at 3:15
stdout
is a pipe togrep out
. It's not your terminal anymore. Do you have any sort of guarantee that this will always be run from a terminal?gvim
may leave you without a terminal at all. Having this line in agmake
makefile may cause problems if the-j
option is used, and the subshell in use doesn't have access to the terminal.cron
jobs wouldn't have a terminal, either. A shell script, on the other hand, might have a terminal, unless it's run without one.