8

I'm looking for a way to only execute replacement when the last character is a newline, using sed.

For instance:

lettersAtEndOfLine

is replaced, but this is not:

lettersWithCharacterAfter&

Since sed does not work well with newlines, it is not as simple as

$ sed -E "s/[a-zA-Z]*\n/replace/" file.txt

How can this be accomplished?

15

With standard sed, you will never see a newline in the text read from a file. This is because sed reads line by line, and there is therefore no newline at the end of the text of the current line in sed's pattern space. In other words, sed reads newline-delimited data, and the delimiters are not part of what a sed script sees.

Regular expressions can be anchored at the end of the line using $ (or at the beginning, using ^). Anchoring an expression at the start/end of a line forces it to match exactly there, and not just anywhere on the line.

If you want to replace anything matching the pattern [A-Za-z]* at the end of the line with something, then anchoring the pattern like this:

[A-Za-z]*$

...will force it to match at the end of the line and nowhere else.

However, since [A-Za-z]*$ also matches nothing (for example, the empty string present at the end of every line), you need to force the matching of something, e.g. by specifying

[A-Za-z][A-Za-z]*$

So, your sed command line will thus be

$ sed 's/[A-Za-z][A-Za-z]*$/replace/' file.txt

I did not use the -E switch here because it's not needed. With it, you could have written

$ sed -E 's/[A-Za-z]+$/replace/' file.txt

It's a matter of taste.

  • Though I knew how to do that, you will get a +1 just for using the technical term for it. :) So this is called anchoring - nice to know. For up to now, I always had to paraphrase it... Another note about the + : you CAN use it even without using extended regex, just remember to write it like \+. So sed -e 's/[A-Za-z]\+$/replace/' file.txt will work perfectly even without GNU sed installed. And not to be forgotten: Do not use -E, as GNU sed does not support it. – syntaxerror Jun 17 '15 at 22:11
  • 1
    @syntaxerror - I think you can remove the last sentence or at least unbold it as gnu sed definitely supports -E. – don_crissti Nov 12 '15 at 1:10
  • @don_crissti Well, I thought you've been on this network long enough to know that there is no way to unbold parts of a comment (unless you rewrite it entirely). So let me correct to: GNU sed may "silently" support -E, but it is not documented in the manpage (nor in the Texinfo manual (checked both)). Hence I assumed it is not supported (which was a wrong assumption, after all). Anyways, you're right, because at least GNU sed won't complain if you use this option. – syntaxerror Nov 15 '15 at 13:27
  • @don_crissti Glad you did! So at least it has been confirmed that sed will take a particular option that has not been properly documented yet. This always comes in handy; if no one is aware about the lack of documentation, nobody will ever fix it. – syntaxerror Nov 15 '15 at 16:22
  • @syntaxerror, see unix.stackexchange.com/a/310454/135943. Of course, if you have to work with old systems such as RHEL 5, then you'll be using a GNU sed version that doesn't support -E. – Wildcard Aug 10 '18 at 22:20
1
sed "s/[a-zA-Z]*$/replace/" input.txt > result.txt

Or, the long complex unnecessary way:

I've found out, this can be done, still using sed, with the help of tr. You can assign another character to represent the end of the line. Another temporary character has to be used, in this case "`". Let's use "~" to represent the end of the line:

tr '\n' '`' <input.txt >output.txt
sed -i "s/`/~`/" output.txt
tr '`' '\n' <output.txt >result.txt

And then to perform the actual search and replace, use "~" rather than "\n":

sed -i -E "s/[a-zA-Z]*~/replace/" result.txt

And then clean up the extra character on the other lines:

sed -i "s/~//" result.txt

Obviously, this can all be piped together resulting in something like:

tr '\n' '`' <input.txt | sed -e "s/`/~`/" | tr '`' '\n' | sed -E -e "s/[a-zA-Z]*~/replace/" | sed "s/~//" > result.txt
  • 3
    Not sure I understand... Why don't you just anchor to end of line with $ ? e.g s/[a-zA-Z]*$/replace/ – don_crissti Jun 1 '15 at 23:53
  • 1
    2 points: 1) You'd better use \+ instead of * since the latter allows zero letters at end of string; 2) You can use a character class [[:alpha:]]. So: sed 's/[[:alpha:]]\+$/replace/' file – glenn jackman Jun 2 '15 at 0:25
  • @glennjackman What's the backslash for before the plus? Wouldn't that match the addition character? – Matthew D. Scholefield Jun 2 '15 at 0:39
  • 1
    GNU sed without the -r option uses this regular expression syntax. – glenn jackman Jun 2 '15 at 1:28
0

From the (broken) code snippet you posted, you seem to want to replace the newline as well. In that case, regex anchoring by itself can't help you. The following is a solution:

sed '/[[:alpha:]]\+$/{N;s/[[:alpha:]]\+\n/replace/}' your_file

Broken down:

  • /[a-zA-Z]\+$/{} means apply whatever comes inside the curlies to lines that match the regex.
  • The regex is the one that uses anchoring as seen in your own answer, modified to take glenn jackman's comments into account.
  • Inside the curlies, N means "append the next line to the active buffer" (what sed calls the 'pattern space')
  • Finally the s/// statement is your required substitution. It now works because the pattern space contains two successive lines and the newline is therefore a part of it.
0

To find the end of line, just use the $-sign:

Without end of line anchor:

sed -n '/pattern/p' file 

Without end of line anchor:

sed -n '/pattern$/p' file

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.