0

I have some text that I'd like tidied up. Basically, to add a newline before each comment so there's an empty line between each two lines.
Here's a sample of the contents to be modified for reference:

# quickly backup or copy a file with bash
cp filename{,.bak}
# Rapidly invoke an editor to write a long, complex, or tricky command
ctrl-x e
# Copy ssh keys to user@host to enable password-less ssh logins.
$ssh-copy-id user@host
# Empty a file
> file.txt
# Execute a command without saving it in the history
<space>command
# Capture video of a linux desktop
ffmpeg -f x11grab -s wxga -r 25 -i :0.0 -sameq /tmp/out.mpg
# Salvage a borked terminal
reset
# start a tunnel from some machine's port 80 to your local post 2001
ssh -N -L2001:localhost:80 somemachine
# Execute a command at a given time
echo "ls -l" | at midnight
# Query Wikipedia via console over DNS
dig +short txt <keyword>.wp.dg.cx
# currently mounted filesystems in nice layout
mount | column -t
# Update twitter via curl
curl -u user:pass -d status="Tweeting from the shell" http://twitter.com/statuse
s/update.xml
# Place the argument of the most recent command on the shell
'ALT+.' or '<ESC> .'
# output your microphone to a remote computer's speaker
dd if=/dev/dsp | ssh -c arcfour -C username@host dd of=/dev/dsp    

I'd like it to look like this:

# quickly backup or copy a file with bash
cp filename{,.bak}

# Rapidly invoke an editor to write a long, complex, or tricky command
ctrl-x e

# Copy ssh keys to user@host to enable password-less ssh logins.
$ssh-copy-id user@host

# Empty a file
> file.txt

# Execute a command without saving it in the history
<space>command

# Capture video of a linux desktop
ffmpeg -f x11grab -s wxga -r 25 -i :0.0 -sameq /tmp/out.mpg

# Salvage a borked terminal
reset

I tried using OSX sed to get the formatting correct, but to no avail:

sed 's/^\#/\n&/g' <filename.txt>

I'm not opposed to using awk if that's a better way to tackle the problem.

  • don -> Looks like you're right. I believe Mac's version of sed treats text formatting differently and wrongly assumed (I know, I know...) it would work similarly. With the Mac, it was inserting a literal "n" character to every line at the beginning. – lmind Oct 17 '17 at 14:00
  • yes, OSX sed doesn't support \n in the RHS; use a literal newline and next time mention your sed version. – don_crissti Oct 17 '17 at 14:01
2
awk '{if (/^#/) {if (!n++ && NR>1) print ""} else n=0; print}'

would insert a an empty line before the first line in a sequence of lines that start with #. So on an input like:

xx
#1
#2
yy

gives:

xx

#1
#2
yy

n counts the number of comment lines in groups of comment lines. We print an empty line (print "") if we're on the first line in each group of comment lines (and exclude the first line of input with NR>1).

  • 1
    Thanks Stéphane! Looks like my sed works on Linux, but not on Mac. Any chance you could break down that awk command for the laymen/noob (e.g. me)? – lmind Oct 17 '17 at 14:04
2

The correct sed is

sed '2,$s/^#/\n#/' filename

This will add a newline only before the lines that start with a comment.

The 2,$ tells sed to start evaluating from the second line to the last line of the file. In the substitute command s, we're looking for lines that start with the comment symbol (^#), and we replace this occurrence within the line with a newline and the comment symbol we found (\n#).

You don't need the g option at the end of the command because you don't want to replace every # found in a line.

This will only work with GNU sed, because \n in the replacement string is undefined in the standard. But you can do

sed '1n;/^#/{x;p;x;}' filename

instead. The 1n prints out the first line and continues with the next one. This way we avoid the "addressing" of lines 2 until end. So now we can address all lines starting with # and for those everything inside the {} is performed: x exchanges the contents of pattern space and hold space. The hold space is empty, so p prints an empty line, the second x changes buffers back, so hold space is empty again and the original line get printed by default.

  • Assumes GNU sed or compatible. POSIXly, you need a backslash followed by a newline character instead of backslash followed by n. Also note that it would insert a blank line before the first line of the file if it was a comment, or between each line of multi-line comments. – Stéphane Chazelas Oct 17 '17 at 13:57
  • cesarv -> thanks! Can you please break this down for the uninformed? I realize now that Macs are POSIX and Linux GNU, hence the difference and why my sed doesn't work. – lmind Oct 17 '17 at 14:07
  • @lmind, POSIX is a standard followed by both GNU and BSD sed. The behaviour for \n is unspecified by POSIX (is an extension of GNU sed over the standard), you need \<newline> portably (will work with both GNU and BSD sed (and other POSIX-compliant sed implementations)). Linux is not an operating system, it's the kernel found in some GNU and non-GNU systems (some of which are not even POSIX). – Stéphane Chazelas Oct 17 '17 at 14:10
  • And the sed command? What does the "2,s4$" do? I believe I understand that carat hash means look for the hash symbol at the beginning of each line, then / to replace, then \n for newline, but why must we specify another hash symbol instead of an ampersand? – lmind Oct 17 '17 at 15:26
  • The 2,$ part tells sed to run the s command from line 2 to last line ($) in the file, so the first line is untouched. – cesarv Oct 17 '17 at 15:28
0

It is also possible to add a newline only before the first line of every block of comments using sed branches:

sed -n '1{p;d}; /^#/{s/^/\n/;:x;p;n;/^#/bx}; /^#/!p' filename

Here:

  • 1{p;d} avoids adding a new line before the first line.

  • /^#/{s/^/\n/;:x;p;n;/^#/bx} is to add a new line before every block of comments. Here is a breakdown of what it does:

    • /^#/ matches lines that start with #. For these lines, sed will execute the block of commands within curly brackets.

    • s/^/\n/ adds a newline before the line that matches.

    • :x;p;n;/^#/bx defines a label x, prints the pattern space (p), replaces the pattern space by the next line of input (n), and returns to x if the new pattern space is a comment (/^#/bx).

  • /^#/!p prints normally lines that are not comments.

EDIT: You can also make the command indempotent as follows:

sed -n '1{p;d}; /^\s*$/bx; /^#/{s/^/\n/;:x;p;n;/^#/bx}; /^#/!p' filename
  • Thanks for breaking this down, Rasta. It really helps to see how it all breaks down. – lmind Oct 17 '17 at 15:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.