Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

Inspired by this question: sed: N command does not read single line I wanted to understand why there appears to be an inconsistency between the following 2 commands:

#1. echo

This command produces no output

$ echo -en 'abc\n' | sed -n 'N;p'
$

A hex dump of the output from echo:

od - octal dump

$ echo -en 'abc\n' | od -tx1
0000000 61 62 63 0a
0000004

hexdump

$ echo -en 'abc\n' | hexdump -C
00000000  61 62 63 0a                                       |abc.|
00000004

#2. cat

If I put the string 'abc\n' in a file such as this:

1: abc
2:

That's a line with the string abc on it followed by a linefeed (\n aka. 0x0A). Then pipe it to sed as before, I get this:

$ cat abc.txt | sed -n 'N;p'
abc

$

That's output that includes the string abc followed by 2 linefeeds.

od - octal dump

$ cat abc.txt | od -tx1
0000000 61 62 63 0a 0a
0000005

hexdump

$ cat abc.txt | hexdump -C
00000000  61 62 63 0a 0a                                    |abc..|
00000005

I'm a little perplexed as to what's going on?

UPDATE

OK so my issue was with how I was creating the file abc.txt as @choroba pointed out. I was creating the file in vim and not realizing that it was adding 2 linefeeds. When I enabled visibility of special characters it became a little more obvious:

:set listchars=eol:$,tab:>-,trail:~,extends:>,precedes:<,
:set list

Now in vim file abc.txt looked like this:

abc$
$

If I created the file as @choroba suggested then the file abc.txt showed up as expected:

$ cat abc.txt | od -tx1
0000000 61 62 63 0a
0000004

And it behaved identical to the original echo example:

$ cat abc.txt | sed -n 'N;p'
$ 

Original issue

My original issue with why sed -n 'N;p' wasn't displaying anything was answered thanks to @enzotib's answer. The bit from the POSIX standard is what I wasn't picking up on:

If no next line of input is available, the N command verb shall branch to the end of the script and quit without starting a new cycle or copying the pattern space to standard output

share|improve this question
    
You might find that hd produces easier-to-read hex dumps than od -x. Or even od -tx1, although personally I prefer to see the characters where possible, as with hd. At any rate, both of those will solve the endian issue. –  rici Aug 16 '13 at 15:42
    
@rici - thanks I always forget the alternatives, I've used od for 15+ years so it's hard to unlearn it 8-). I'll modify my Q to use the -tx1 switches as you've suggested though so it's easier to read. –  slm Aug 16 '13 at 15:43
    
hd is also known as hexdump. –  slm Aug 16 '13 at 15:52
    
On debian, it's in bsdmaintools, along with hexdump (and various other bsd tools like column and cal). hd is the same as hexdump -C. –  rici Aug 16 '13 at 15:54

2 Answers 2

up vote 2 down vote accepted

The POSIX standard states:

[2addr] N Append the next line of input, less its terminating <newline>, to the pattern space, using an embedded <newline> to separate the appended material from the original material. Note that the current line number changes. If no next line of input is available, the N command verb shall branch to the end of the script and quit without starting a new cycle or copying the pattern space to standard output.

so the behavior is very different if there is or not a next line.

Your input, as you can see from the output of od -x, differ just in a newline.

share|improve this answer
    
Thank you, this was the bit I was missing. –  slm Aug 16 '13 at 14:06

How did you create the file? It contains two newlines, not one. Try again with

echo abc > abc.txt
cat abc.txt | sed -n 'N;p'
share|improve this answer
    
You'd be correct, I've updated my Q with your feedback. Thanks for setting me straight 8-). –  slm Aug 16 '13 at 14:06

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.