9

I'm looking for something that behaves like Perl's chomp. I'm looking for a command that simply prints its input, minus the last character if it's a newline:

$ printf "one\ntwo\n" | COMMAND_IM_LOOKING_FOR ; echo " done"
one
two done
$ printf "one\ntwo" | COMMAND_IM_LOOKING_FOR ; echo " done"
one
two done

(Command substitution in Bash and Zsh deletes all trailing new lines, but I'm looking for something that deletes one trailing new line at most.)

8

This should work:

printf "one\ntwo\n" | awk 'NR>1{print PREV} {PREV=$0} END{printf("%s",$0)}' ; echo " done"

The script always prints previous line instead of current, and the last line is treated differently.

What it does in more detail:

  1. NR>1{print PREV} Print previous line (except the first time).
  2. {PREV=$0} Stores current line in PREV variable.
  3. END{printf("%s",$0)} Finally, print last line withtout line break.

Also note this would remove at most one empty line at the end (no support for removing "one\ntwo\n\n\n").

14

You can use perl without chomp:

$ printf "one\ntwo\n" | perl -0 -pe 's/\n\Z//'; echo " done"
one
two done

$ printf "one\ntwo" | perl -0 -pe 's/\n\Z//'; echo " done"
one
two done

But why not use chomp itself:

$ printf "one\ntwo\n" | perl -pe 'chomp if eof'; echo " done"
4

If you want an exact equivalent to chomp, the first method that comes to my mind is the awk solution that LatinSuD already posted. I'll add some other methods that don't implement chomp but implement some common tasks that chomp is often used for.

When you stuff some text into a variable, all newlines at the end are stripped. So all these commands produce the same single-line output:

echo "$(printf 'one\ntwo') done"
echo "$(printf 'one\ntwo\n') done"
echo "$(printf 'one\ntwo\n\n') done"
echo "$(printf 'one\ntwo\n\n\n\n\n\n\n\n\n\n') done"

If you want to append some text to the last line of a file or of a command's output, sed can be convenient. With GNU sed and most other modern implementations, this works even if the input doesn't end in a newline¹; however, this won't add a newline if there wasn't one already.

sed '$ s/$/ done/'

¹ However this doesn't work with all sed implementations: sed is a text processing tool, and a file that isn't empty and doesn't end with a newline character is not a text file.

  • This is not exactly equivalent to chomp, as chomp only deletes at most one trailing newline. – Flimm Jul 8 '14 at 11:48
  • @Flimm Yes, the most obvious exact equivalent to chomp would be the awk solution that LatinSuD already posted. But in many cases chomp is just a tool to do a job, and I provide ways to do some common tasks. Let me update my answer to clarify this. – Gilles 'SO- stop being evil' Jul 8 '14 at 12:01
1

Another perl approach. This one reads the entire input into memory so it might not be a good idea for large amounts of data (use cuonglm's or the awk approach for that):

$ printf "one\ntwo\n" | perl -0777pe 's/\n$//'; echo " done"
one
two done
0

I snagged this from a github repo somewhere, but can't find where

delete-trailing-blank-lines-sed

#!/bin/bash
#
# Delete all trailing blank lines.
# From http://sed.sourceforge.net/sed1line.txt
#
# Version: 1.3.0
# Created: 2011-01-02
# Updated: 2015-01-25
# Contact: Joel Parker Henderson (joel@joelparkerhenderson.com)
# License: GPL
##
set -euf
sed -e :a -e '/^\n*$/{$d;N;ba' -e '}'
0

abstract

Print lines without newline, add a newline only if there is another line to print.

$ printf 'one\ntwo\n' | 

     awk '{ printf( "%s%s" , NR>1?"\n":"" , $0 ) }';   echo " done"

one
two done

Other solutions

If we were working with a file, we can just truncate one character from it (if it ends on a newline):

removeTrailNewline(){ [[ $(tail -c 1 "$1") ]] || truncate -s-1 "$1"; }

That is a fast solution as it needs to read only one character from the file and then remove it directly (truncate) without reading the whole file.

However, while working with data from stdin (an stream) the data must be read, all of it. And, it is "consumed" as soon as it is read. No backtrack (as with truncate). To find the end of an stream we need to read to the end of the stream. At that point, there is no way to move back on the input stream, the data has been already "consumed". This means that data must be stored in some form of buffer until we match the end of the stream and then do something with the data in the buffer.

The most obvious of solutions is to convert the stream into a file and process that file. But the question asks for some kind of filter of the stream. Not about the use of additional files.

variable

The naive solution would be to capture the whole input into a variable:

FilterOne(){ filecontents=$(cat; echo "x");        # capture the whole input
             filecontents=${filecontents%x};       # Remove the "x" added above.
             nl=$'\n';                             # use a variable for newline.
             printf '%s' "${filecontents%"$nl"}";  # Remove newline (if it exists).
       }

printf 'one\ntwo'     | FilterOne ; echo 1done
printf 'one\ntwo\n'   | FilterOne ; echo 2done
printf 'one\ntwo\n\n' | FilterOne ; echo 3done

memory

It is possible to load a whole file in memory with sed. In sed it is impossible to avoid the trailing newline on the last line. GNU sed might avoid printing a trailing newline, but only if the source file is already missing it. So, no, simple sed can't help.

Except on GNU awk with the -z option:

sed -z 's/\(.*\)\n$/\1/'

With awk (any awk), slurp the whole stream, and printf it without the trailing newline.

awk '    { content = content $0 RS } 
     END { gsub( "\n$", "", content ); printf( "%s", content ) }
    '

Loading a whole file into memory might not be a good idea, it may consume a lot of memory.

Two lines in memory

In awk, we can process two lines per loop by storing the previous line in a variable and printing the present one:

awk 'NR>1{print previous} {previous=$0} END {printf("%s",$0)}'

Direct processing

But we could we do better.

If we print the present line without a newline and print a newline only when a next line exists, we process one line at a time and the last line will not have a trailing newline:

awk 'NR==1{ printf("%s",$0);next }; { printf( "\n%s", $0 ) }'

Or, written in some other way:

awk 'NR>1{ print "" }; { printf( "%s", $0 ) }'

Or:

awk '{ printf( "%s%s" , NR>1?"\n":"" , $0 ) }'

So:

$ printf 'one\ntwo\n' | awk '{ printf( "%s%s" , NR>1?"\n":"" , $0 ) }'; echo " done"
one
two done

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.