2

Inspired by this question: sed: N command does not read single line I wanted to understand why there appears to be an inconsistency between the following 2 commands:

#1. echo

This command produces no output

$ echo -en 'abc\n' | sed -n 'N;p'
$

A hex dump of the output from echo:

od - octal dump

$ echo -en 'abc\n' | od -tx1
0000000 61 62 63 0a
0000004

hexdump

$ echo -en 'abc\n' | hexdump -C
00000000  61 62 63 0a                                       |abc.|
00000004

#2. cat

If I put the string 'abc\n' in a file such as this:

1: abc
2:

That's a line with the string abc on it followed by a linefeed (\n aka. 0x0A). Then pipe it to sed as before, I get this:

$ cat abc.txt | sed -n 'N;p'
abc

$

That's output that includes the string abc followed by 2 linefeeds.

od - octal dump

$ cat abc.txt | od -tx1
0000000 61 62 63 0a 0a
0000005

hexdump

$ cat abc.txt | hexdump -C
00000000  61 62 63 0a 0a                                    |abc..|
00000005

I'm a little perplexed as to what's going on?

UPDATE

OK so my issue was with how I was creating the file abc.txt as @choroba pointed out. I was creating the file in vim and not realizing that it was adding 2 linefeeds. When I enabled visibility of special characters it became a little more obvious:

:set listchars=eol:$,tab:>-,trail:~,extends:>,precedes:<,
:set list

Now in vim file abc.txt looked like this:

abc$
$

If I created the file as @choroba suggested then the file abc.txt showed up as expected:

$ cat abc.txt | od -tx1
0000000 61 62 63 0a
0000004

And it behaved identical to the original echo example:

$ cat abc.txt | sed -n 'N;p'
$ 

Original issue

My original issue with why sed -n 'N;p' wasn't displaying anything was answered thanks to @enzotib's answer. The bit from the POSIX standard is what I wasn't picking up on:

If no next line of input is available, the N command verb shall branch to the end of the script and quit without starting a new cycle or copying the pattern space to standard output

4
  • You might find that hd produces easier-to-read hex dumps than od -x. Or even od -tx1, although personally I prefer to see the characters where possible, as with hd. At any rate, both of those will solve the endian issue.
    – rici
    Aug 16, 2013 at 15:42
  • @rici - thanks I always forget the alternatives, I've used od for 15+ years so it's hard to unlearn it 8-). I'll modify my Q to use the -tx1 switches as you've suggested though so it's easier to read.
    – slm
    Aug 16, 2013 at 15:43
  • hd is also known as hexdump.
    – slm
    Aug 16, 2013 at 15:52
  • On debian, it's in bsdmaintools, along with hexdump (and various other bsd tools like column and cal). hd is the same as hexdump -C.
    – rici
    Aug 16, 2013 at 15:54

2 Answers 2

3

How did you create the file? It contains two newlines, not one. Try again with

echo abc > abc.txt
cat abc.txt | sed -n 'N;p'
1
  • You'd be correct, I've updated my Q with your feedback. Thanks for setting me straight 8-).
    – slm
    Aug 16, 2013 at 14:06
2

The POSIX standard states:

[2addr] N Append the next line of input, less its terminating <newline>, to the pattern space, using an embedded <newline> to separate the appended material from the original material. Note that the current line number changes. If no next line of input is available, the N command verb shall branch to the end of the script and quit without starting a new cycle or copying the pattern space to standard output.

so the behavior is very different if there is or not a next line.

Your input, as you can see from the output of od -x, differ just in a newline.

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.