3

I'm copying dd output through netcat with the following command

$dd if=/dev/zero bs=1024K count=1 | nc <IP_ADDR> <PORT> -q 0
1+0 enregistrements lus
1+0 enregistrements écrits
1048576 bytes (1,0 MB, 1,0 MiB) copied, 0,0590934 s, 17,7 MB/s

However when I try to parse the output nothing happens

$ dd if=/dev/zero bs=1024K count=1 | nc <IP_ADDR> <PORT> -q 0 | grep copied
1+0 enregistrements lus
1+0 enregistrements écrits
1048576 bytes (1,0 MB, 1,0 MiB) copied, 0,058937 s, 17,8 MB/s

It should print only the last line, why the output is not sent to grep? I tried few redirections but I'm not able to have it redirected as I want.

I would like to have the data sent through netcat but having the output messages (both stderr and stdin) sent to stdout or a file to parse it afterwards.

4
  • You can use -o to output to a file and then parse the file Commented Oct 2, 2017 at 14:49
  • Seems that dd hasn't a -o parameter
    – Arkaik
    Commented Oct 2, 2017 at 15:34
  • nc has though.. Commented Oct 3, 2017 at 8:23
  • My bad though..
    – Arkaik
    Commented Oct 6, 2017 at 15:59

1 Answer 1

3

In

dd if=/dev/zero bs=1024K count=1 | nc <IP_ADDR> <PORT> -q 0 | grep copied

there's no way that dd status output could go to grep. grep is reading the output of nc, not dd. If dd wrote that output on its stdout, it would go to nc, not grep.

Thankfully dd does not write that status message to its stdout (otherwise it would be sent to <IP_ADDR> which we don't want), but it writes it on a separated stream: stderr (as it's a diagnostic message, not part of its normal output).

To have dd's stderr connected to a pipe that goes to grep (and nc's stdout+stderr unchanged), you could do:

{ {
  dd if=/dev/zero bs=1M count=1 2>&3 3>&- |
    nc -q 0  <IP_ADDR> <PORT> 3>&-
} 3>&1 >&4 4>&- | grep copied 4>&-; } 4>&1

Assuming the shell's stdin/stdout/stderr go to I, O, E (all would be the tty device open in read+write mode if run from a terminal), in the above we would have:

cmd \ fd | stdin stdout stderr  3       4
---------+------------------------------------
      dd | I     pipe1  pipe2   closed  closed
      nc | pipe1 O      E       closed  closed
    grep | pipe2 O      E       closed  closed

Or to have the stderr of dd and the stdout+stderr of nc go to grep (but the stdout of dd still go to nc):

{ 
  dd if=/dev/zero bs=1M count=1 |
    nc -q 0  <IP_ADDR> <PORT>
} 2>&1 | grep copied

Our table of fd assignment per command becomes:

cmd \ fd | stdin stdout stderr
---------+--------------------
      dd | I     pipe1  pipe2
      nc | pipe1 pipe2  pipe2
    grep | pipe2 O      E

Yet another approach:

{ 
  dd if=/dev/zero bs=1M count=1 2>&1 >&3 3>&- |
    grep copied >&2 3>&-
} 3>&1 | nc -q 0  <IP_ADDR> <PORT>

cmd \ fd | stdin stdout stderr  3
---------+-----------------------
      dd | I     pipe1  pipe2
      nc | pipe1 O      E
    grep | pipe2 E      E

But note that that output won't be very relevant. That 1MiB of data will probably fit in the pipe buffer, nc's internal read buffer and the socket send buffer, so you won't be really timing the network throughput. It's likely dd will return before the first data packet is sent over the network (shortly after the TCP connection has been enabled and nc starts reading its stdin). Look at iperf instead for that.

Without iperf, you could get a better measurement of sending throughput if you did something like:

{
  dd bs=1M count=50 2> /dev/null # buffers filled and the TCP connection 
                                 # established and into a steady state
  dd bs=1M count=100 2>&1 >&3 3>&- | grep copied >&2 3>&-
} < /dev/zero 3>&1 | nc -q 0  <IP_ADDR> <PORT>
5
  • Wooow I'm impressed ^^ I would never have found this alone. Just to confirm, the information is not relevant only with the second solution right? I only tried the first one which seems to work perfectly. Thanks a lot
    – Arkaik
    Commented Oct 2, 2017 at 15:31
  • @Arkaik, no. What I'm saying is that it's not a valid approach to measure network throughput. It's independant of how dd's stderr is redirected. Commented Oct 2, 2017 at 15:39
  • Alright, I though it was not so bad. I'll dig around iperf instead. Thx
    – Arkaik
    Commented Oct 2, 2017 at 15:42
  • @Arkaik, it would be more relevant if you sent several hundred megabytes and start measuring after the first few megabytes have been sent for the TCP congestion algorithm to have attained its cruise speed. Commented Oct 2, 2017 at 15:49
  • I finnaly kept using dd and netcat but I'm measuring time before and after with date. I also send 1Go through network so measured time should be relevant.
    – Arkaik
    Commented Oct 3, 2017 at 7:52

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .