23

I've been using sed for quite some time but here is a quirk I came around with, which I am not able to resolve.

Let me explain my problem with the actual case.


Scene#1

printf "ls" | xclip -selection clipboard
echo "ls" | xclip -selection clipboard

In the first command, I pipe printf output to xclip so that it gets copied to the clipboard. Now, printf, unlike echo does not insert a new line at the end by default. So, if I paste this content into terminal, the ls command that is copied does not automatically run.

In the second, there is a new line at the end, so pasting the clipboard content also results in the running of the command in the clipboard.

This is undesirable for me. So, I wanted to remove the newline using sed, but it failed, as explained in the scene below.

Scene#2

echo "ls" | sed -r 's/\n//g' | xclip -selection clipboard

The content in the clipboard still contains new-line. When I paste it into terminal, the command automatically runs.

I also tried removing carriage return character \r. But nada. It seems I am missing something very crucial/basic here.

5 Answers 5

32

sed delimits on \newlines - they are always removed on input and reinserted on output. There is never a \newline character in a sed pattern space which did not occur as a result of an edit you have made. Note: with the exception of GNU sed's -z mode...

Just use tr:

echo ls | tr -d \\n | xclip -selection clipboard

Or, better yet, forget sed altogether:

printf ls | xclip -selection clipboard
2
  • This will remove all newlines, often not desirable in the generate-text | xclip case. This answer only removes the last newline.
    – Tom Hale
    Commented Sep 27, 2019 at 6:31
  • Really? What about sed -e ':lbl N; s/\n//; t lbl' ?
    – kavadias
    Commented Feb 1, 2022 at 13:00
11

You can replace newlines in sed by passing it the -z option.

sed -z 's/\n/ /g'

Sed man page:

-z, --null-data separate lines by NUL characters

1
  • This will remove all newlines, which is generally not be desired in the | xclip case.
    – Tom Hale
    Commented Sep 27, 2019 at 6:26
9

Many text processing tools, including sed, operate on the content of the line, excluding the newline character. The first thing sed does when processing a line is to strip off the newline at the end, then it executes the commands in the script, and it adds a final newline when printing out. So you won't be able to remove the newline with sed.

To remove all the newlines, you can use tr instead:

echo "ls" | tr -d '\n' | xclip

0
3

To remove only the last newline, pipe through:

sed -z '$ s/\n$//'

sed won't add a \0 to then end of the stream if the delimiter is set to NUL via -z, whereas to create a POSIX text file (defined to end in a \n), it will always output a final \n without -z.

Eg:

$ { echo foo; echo bar; } | sed -z '$ s/\n$//'; echo tender
foo
bartender

And to prove no NUL added:

$ { echo foo; echo bar; } | sed -z '$ s/\n$//' | xxd
00000000: 666f 6f0a 6261 72                        foo.bar

To remove multiple trailing newlines, pipe through:

sed -Ez '$ s/\n+$//'
2

If you are just putting commands in the clipboard

echo -n "ls " | xclip -selection clipboard

If you additionally need to make more complex transformations,

echo "ls  " | perl -pe 's/\n//' | xclip -selection clipboard
5
  • Hmm. Smooth solution. I guess it is perhaps time to move on to perl or python for regex.
    – shivams
    Commented Jan 25, 2015 at 0:49
  • 3
    @shivams - perl and python are both much slower than sed can be when used correctly. Learning to use it correctly is the key, though.
    – mikeserv
    Commented Jan 25, 2015 at 1:26
  • @mikeserv - I agree; special for oneliner sed is wonderful. If the transformations are going to become more complex, then I prefer perl/python/gawk.
    – JJoao
    Commented Jan 25, 2015 at 12:35
  • 1
    @JJoa: Your preference is your own, but sed can do far more than handle just your one-liners, and it can likely do so faster than any of those other tools you mention.
    – mikeserv
    Commented Jan 25, 2015 at 14:54
  • 1
    @mikeserv - once again, I agree with you. Some year ago I had a translation system EN-PT written in sed (20000 s/.../.../), generated by sed! And it worked! Later it became Perl, to have arrays, hashes, functions, packages and CPAN.
    – JJoao
    Commented Jan 25, 2015 at 21:10

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .