2

I'm copying dd output through netcat with the following command

$dd if=/dev/zero bs=1024K count=1 | nc <IP_ADDR> <PORT> -q 0
1+0 enregistrements lus
1+0 enregistrements écrits
1048576 bytes (1,0 MB, 1,0 MiB) copied, 0,0590934 s, 17,7 MB/s

However when I try to parse the output nothing happens

$ dd if=/dev/zero bs=1024K count=1 | nc <IP_ADDR> <PORT> -q 0 | grep copied
1+0 enregistrements lus
1+0 enregistrements écrits
1048576 bytes (1,0 MB, 1,0 MiB) copied, 0,058937 s, 17,8 MB/s

It should print only the last line, why the output is not sent to grep? I tried few redirections but I'm not able to have it redirected as I want.

I would like to have the data sent through netcat but having the output messages (both stderr and stdin) sent to stdout or a file to parse it afterwards.

  • You can use -o to output to a file and then parse the file – Raman Sailopal Oct 2 '17 at 14:49
  • Seems that dd hasn't a -o parameter – Arkaik Oct 2 '17 at 15:34
  • nc has though.. – Raman Sailopal Oct 3 '17 at 8:23
  • My bad though.. – Arkaik Oct 6 '17 at 15:59
3

In

dd if=/dev/zero bs=1024K count=1 | nc <IP_ADDR> <PORT> -q 0 | grep copied

there's no way that dd status output could go to grep. grep is reading the output of nc, not dd. If dd wrote that output on its stdout, it would go to nc, not grep.

Thankfully dd does not write that status message to its stdout (otherwise it would be sent to <IP_ADDR> which we don't want), but it writes it on a separated stream: stderr (as it's a diagnostic message, not part of its normal output).

To have dd's stderr connected to a pipe that goes to grep (and nc's stdout+stderr unchanged), you could do:

{ {
  dd if=/dev/zero bs=1M count=1 2>&3 3>&- |
    nc -q 0  <IP_ADDR> <PORT> 3>&-
} 3>&1 >&4 4>&- | grep copied 4>&-; } 4>&1

Assuming the shell's stdin/stdout/stderr go to I, O, E (all would be the tty device open in read+write mode if run from a terminal), in the above we would have:

cmd \ fd | stdin stdout stderr  3       4
---------+------------------------------------
      dd | I     pipe1  pipe2   closed  closed
      nc | pipe1 O      E       closed  closed
    grep | pipe2 O      E       closed  closed

Or to have the stderr of dd and the stdout+stderr of nc go to grep (but the stdout of dd still go to nc):

{ 
  dd if=/dev/zero bs=1M count=1 |
    nc -q 0  <IP_ADDR> <PORT>
} 2>&1 | grep copied

Our table of fd assignment per command becomes:

cmd \ fd | stdin stdout stderr
---------+--------------------
      dd | I     pipe1  pipe2
      nc | pipe1 pipe2  pipe2
    grep | pipe2 O      E

Yet another approach:

{ 
  dd if=/dev/zero bs=1M count=1 2>&1 >&3 3>&- |
    grep copied >&2 3>&-
} 3>&1 | nc -q 0  <IP_ADDR> <PORT>

cmd \ fd | stdin stdout stderr  3
---------+-----------------------
      dd | I     pipe1  pipe2
      nc | pipe1 O      E
    grep | pipe2 E      E

But note that that output won't be very relevant. That 1MiB of data will probably fit in the pipe buffer, nc's internal read buffer and the socket send buffer, so you won't be really timing the network throughput. It's likely dd will return before the first data packet is sent over the network (shortly after the TCP connection has been enabled and nc starts reading its stdin). Look at iperf instead for that.

Without iperf, you could get a better measurement of sending throughput if you did something like:

{
  dd bs=1M count=50 2> /dev/null # buffers filled and the TCP connection 
                                 # established and into a steady state
  dd bs=1M count=100 2>&1 >&3 3>&- | grep copied >&2 3>&-
} < /dev/zero 3>&1 | nc -q 0  <IP_ADDR> <PORT>
  • Wooow I'm impressed ^^ I would never have found this alone. Just to confirm, the information is not relevant only with the second solution right? I only tried the first one which seems to work perfectly. Thanks a lot – Arkaik Oct 2 '17 at 15:31
  • @Arkaik, no. What I'm saying is that it's not a valid approach to measure network throughput. It's independant of how dd's stderr is redirected. – Stéphane Chazelas Oct 2 '17 at 15:39
  • Alright, I though it was not so bad. I'll dig around iperf instead. Thx – Arkaik Oct 2 '17 at 15:42
  • @Arkaik, it would be more relevant if you sent several hundred megabytes and start measuring after the first few megabytes have been sent for the TCP congestion algorithm to have attained its cruise speed. – Stéphane Chazelas Oct 2 '17 at 15:49
  • I finnaly kept using dd and netcat but I'm measuring time before and after with date. I also send 1Go through network so measured time should be relevant. – Arkaik Oct 3 '17 at 7:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.