2

Inspired by this question: sed: N command does not read single line I wanted to understand why there appears to be an inconsistency between the following 2 commands:

#1. echo

This command produces no output

$ echo -en 'abc\n' | sed -n 'N;p'
$

A hex dump of the output from echo:

od - octal dump

$ echo -en 'abc\n' | od -tx1
0000000 61 62 63 0a
0000004

hexdump

$ echo -en 'abc\n' | hexdump -C
00000000  61 62 63 0a                                       |abc.|
00000004

#2. cat

If I put the string 'abc\n' in a file such as this:

1: abc
2:

That's a line with the string abc on it followed by a linefeed (\n aka. 0x0A). Then pipe it to sed as before, I get this:

$ cat abc.txt | sed -n 'N;p'
abc

$

That's output that includes the string abc followed by 2 linefeeds.

od - octal dump

$ cat abc.txt | od -tx1
0000000 61 62 63 0a 0a
0000005

hexdump

$ cat abc.txt | hexdump -C
00000000  61 62 63 0a 0a                                    |abc..|
00000005

I'm a little perplexed as to what's going on?

UPDATE

OK so my issue was with how I was creating the file abc.txt as @choroba pointed out. I was creating the file in vim and not realizing that it was adding 2 linefeeds. When I enabled visibility of special characters it became a little more obvious:

:set listchars=eol:$,tab:>-,trail:~,extends:>,precedes:<,
:set list

Now in vim file abc.txt looked like this:

abc$
$

If I created the file as @choroba suggested then the file abc.txt showed up as expected:

$ cat abc.txt | od -tx1
0000000 61 62 63 0a
0000004

And it behaved identical to the original echo example:

$ cat abc.txt | sed -n 'N;p'
$ 

Original issue

My original issue with why sed -n 'N;p' wasn't displaying anything was answered thanks to @enzotib's answer. The bit from the POSIX standard is what I wasn't picking up on:

If no next line of input is available, the N command verb shall branch to the end of the script and quit without starting a new cycle or copying the pattern space to standard output

4
  • You might find that hd produces easier-to-read hex dumps than od -x. Or even od -tx1, although personally I prefer to see the characters where possible, as with hd. At any rate, both of those will solve the endian issue.
    – rici
    Aug 16 '13 at 15:42
  • @rici - thanks I always forget the alternatives, I've used od for 15+ years so it's hard to unlearn it 8-). I'll modify my Q to use the -tx1 switches as you've suggested though so it's easier to read.
    – slm
    Aug 16 '13 at 15:43
  • hd is also known as hexdump.
    – slm
    Aug 16 '13 at 15:52
  • On debian, it's in bsdmaintools, along with hexdump (and various other bsd tools like column and cal). hd is the same as hexdump -C.
    – rici
    Aug 16 '13 at 15:54
2

The POSIX standard states:

[2addr] N Append the next line of input, less its terminating <newline>, to the pattern space, using an embedded <newline> to separate the appended material from the original material. Note that the current line number changes. If no next line of input is available, the N command verb shall branch to the end of the script and quit without starting a new cycle or copying the pattern space to standard output.

so the behavior is very different if there is or not a next line.

Your input, as you can see from the output of od -x, differ just in a newline.

0
3

How did you create the file? It contains two newlines, not one. Try again with

echo abc > abc.txt
cat abc.txt | sed -n 'N;p'
1
  • You'd be correct, I've updated my Q with your feedback. Thanks for setting me straight 8-).
    – slm
    Aug 16 '13 at 14:06

This site is temporarily in read only mode and not accepting new answers.

Not the answer you're looking for? Browse other questions tagged .