4

I'm having a lot of difficulty figuring out how to phrase this so Google-fu is failing.

I have a text file with a table of data. I'd like to insert newlines to visually separate subgroups.

For example, if I start with:

jan   ford
jan   trillian
mar   trillian
sep   marvin

And the first field is my subgroup field then the output should be:

jan   ford
jan   trillian

mar   trillian

sep   marvin

I can do something like ^(a-z){3}\t(.*)\n\1\t(.*)$ to identify two lines where the month is the same but I don't know how to match when they're different.

Ideally I'd love this to be a regex I can throw into BBedit but I'm open to other solutions.

5

It looks like bbedit is some kind of paid OSX editor. I'm afraid I;ve never used it and can't install it so I can't help you there. Based on the regex you show, it has its own regular expression syntax so it's unlikely you'll find a solution on a general *nix site using it. However, here are a couple of other options. In both, the idea is to save the first field and print a blank line if it is different than the one seen on the previous line:

$ awk '{if($1!=last && NR>1){print ""}last=$1;}1;' file
jan ford
jan trillian

mar trillian

sep marvin

awk is a scripting language that is designed to deal with field-based data. It will automatically split each line into fields which can then be referred to as $1,$2 ... $N. So, the script above will save the first field in the variable last, and for each line but the first (that's what the NR>1 means), it will print an empty line if last is not the same as the currently saved value. The 1; is awk shorthand for "print every line".

Alternatively, you could also do this in perl:

$ perl -lape '$F[0] ne $last && $.>1 && print ""; $last=$F[0]' file
jan ford
jan trillian

mar trillian

sep marvin

Here, we're using perl command line switches to do most of the work. The -a makes perl act like awk and split each input line into the array @F. Therefore, $F[0] is the first field. The -l makes perl add a newline to each print call, so print "" just prints an empty line. The -p makes it print each input line after applying the script given by -e. The script itself is exactly the same as the awk one above.

  • BBEdit has a free cousin called TextWrangler if you want to look at it. – Dave Noonan Oct 30 '15 at 13:10
  • I asked here largely because Google brought me here and I was okay with *nix style solutions. The regex should be standard but I think it wasn't displaying properly. Looks okay now. Awk, which I don't use often so didn't think about, works nicely, Thanks, and I very much appreciate the explanation with the answer. – Dave Noonan Oct 30 '15 at 13:16
4
awk 'NR > 1 && $1 != last {print ""}; {print; last = $1}'
2

Since you can match the dups, then you'll want to negate your action. You could do like:

sed -e'/^\n/!{$n;G;N;s/^\(\(...\).*\)\n\2/\1\2/;}' -eP\;D <in >out

In other words start with a double spaced file, and prune out the second space between your groups.

Running that against your example input prints:

jan   ford
jan   trillian

mar   trillian

sep   marvin

Which actually leads me to a question for you: I think I get how Trillian might have two birthdays with the parallel universes and all, but could you explain why you think Marvin might have one at all? I guess September would fit, though - it's always gloomy.


Here's another one, courtesy @don_crissti (if slightly modified):

sed -etD -e'$q;N;/^\(...\).*\n\1/!s/\n/&&/;:D' -eP\;D

I like his a lot better. He initially asked me: why edit out the newlines and not just edit them in? I didn't have any answer, really, and was mostly ambivalent because I didn't consider it to make much of a difference.

The thing is, though, the lookahead technique used here requires a knife-edge buffer - it could report false positives if the buffer grows any larger than the most current two input lines at a time. Balancing a single line of lookahead gets to be more difficult when you're also actually inserting newlines into the very same stream you're matching and requires that you can easily distinguish between a line of input you've inserted or a line you need to read. That balance requires at least one extra test per cycle - and I get by with the /^\n/! test above.

But a very basic part of the sed machinery is the s///ubstitution statement's tested return, which does conditional branching in the event of success. Because his substitutions always generate an extra line the sed line cycle - which is what clears a test's return - does not roll over when pattern space is Deleted and test still works at the top of the script. In this way don can very simply test over every insertion and reliably evaluate the result of the previous iteration at the top of the next.

So go upvote one of his answers, cause I guess he isn't going to put it here.


The Get command appends sed's hold space to pattern space following an inserted \newline delimiter. The hold space is never used here, though, and so all that is Got is the \newline.

The Next command appends the Next input line to pattern space following an inserted \newline delimiter. Doing Get then Next for each input line gets two \newlines per input line.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.