5

My data:

Question Nr.  311
Main proteins are in the lorem ipsun
A Lorem RNA test
B Cells
C Metoclom
D Cells
E Muscles

Question Nr.  312
Main proteins are in the lorem ipsun
A Lorem
B Cells
C Metoclom
D Cells
E Muscles

...

Wanted format:

\item 
Main proteins are in the lorem ipsun

A Lorem RNA test

B Cells

C Metoclom

D Cells

E Muscles

\item
Main proteins are in the lorem ipsun

A Lorem

B Cells

C Metoclom

D Cells

E Muscles

\item ...

Where I am planning to present the options each on new line.

My attempt:

sed s/Question Nr.*/\item/g

Which should replace all lines having Question Nr[anything on the line] - problem is in the detection what comes after, since there can be many options, but the end of options is \n\n i.e. the newline.

Semistage problem here:

\item 
Main proteins are in the lorem ipsun
A Lorem RNA test
B Cells
C Metoclom
D Cells  
E Muscles

\item 
Main proteins are in the lorem ipsun
A Lorem
B Cells
C Metoclom
D Cells  
E Muscles

Other challenges

  • Have capitalized words like HIV and RNA in the options; some solutions below insert empty line after HI and RN

How can you get my wanted output by sed/perl?

5

Another way with tr+sed:

tr -s \\n <infile | sed '$!G;s/Question Nr.*/\\item/'

tr squeezes all newlines and then sed appends hold space content (empty newline) to each line except the last one, replacing Question Nr.* with \item. With this method you won't be able to edit the file in-place. I chose tr here as it's faster then sed's regex (even if it's not as clean as a sed-only solution)

  • 1
    Very nice. Add an empty line check at the beginning of the sed command like so (untested): /^$/d; Or if you like, use /^[[:space:]]*$/d; to catch whitespace only lines. This gives sed '/^[[:space:]]*$/d;/Question Nr.*/!G;s//\\item/' as probably the cleanest solution. – Wildcard Nov 21 '15 at 0:35
6

With sed:

sed 's/^Question Nr\..*/\\item/; s/^\([A-Z] .*\)/\n\1/' file
  • The first s/// replaces Question Nr. with \item similar to the sed command in your question.
  • The second one replaces line that start with a capital letter from A to Z, but only one followed by a space. This whole line is replaced with itself \1 repending a newline \n.

The output:

\item
Main proteins are in the lorem ipsun

A Lorem

B Cells

C Metoclom

D Cells

E Muscles

\item
Main proteins are in the lorem ipsun

A Lorem

B Cells

C Metoclom

D Cells

E Muscles
| improve this answer | |
5

If it doesn't need to be sed, Perl's "paragraph mode" is perfect for this. From man perlrun:

   -0[octal/hexadecimal]
        specifies the input record separator ($/) as an octal or
        hexadecimal number.  [...]

        The special value 00 will cause Perl to slurp files in paragraph
        mode.  [...]

So, using -00 tells perl to define "lines" as paragraphs, it uses \n\n as the end of line character. With that in mind, you could do something like:

$ perl -00pe 's/Question.*/\\item/; s/[A-Z] /\n$&/g;' file
\item
Main proteins are in the lorem ipsun

A Lorem

B Cells

C Metoclom

D Cells

E Muscles

\item
Main proteins are in the lorem ipsun

A Lorem

B Cells

C Metoclom

D Cells

E Muscles

The first substitution operator replaces any lines matching the string Question with \item and the second adds a newline before each capital letter followed by a space.

| improve this answer | |
  • Again only 1 second diference =) – chaos Nov 20 '15 at 22:48
  • @chaos indeed :) But yours was cleverer so I copied it. It is much simpler to replace Question with \item than to do it in two steps as I was doing. Thanks! – terdon Nov 20 '15 at 22:49
  • There are some problems when the data contains words like "HIV" in the body. I am trying to second match to start at the beginning of the line by s/^[A-Z] / unsuccessfully. Why s/^ does not work here as expected? – Léo Léopold Hertz 준영 Nov 21 '15 at 10:11
  • Unsuccessful attempt also s/^\([A-Z] \)/ – Léo Léopold Hertz 준영 Nov 21 '15 at 11:08
  • 1
    @Masi because the "line" is now a paragraph so ^ matches the beginning of the paragraph. You'll have to use s/\n[A-Z] // instead. – terdon Nov 21 '15 at 11:21
5
sed -e'/./!d;$!G;/^Q/c\' -e'\\item' <in >out

That will delete every blank line in input, Get a blank line out of hold space and append it to all non-blank lines which are ! not the $ last, and change any pattern space ^beginning with the character Q to the one-line fixed-string \item on output.

When run on your example input, the output is:

\item
Main proteins are in the lorem ipsun

A Lorem

B Cells

C Metoclom

D Cells

E Muscles

\item
Main proteins are in the lorem ipsun

A Lorem

B Cells

C Metoclom

D Cells

E Muscles

(with no trailing blank line at the tail of output)

Portably the sed -expression statement should not end in a backslash like that, and so it might be written:

sed -e'/./!d;$!G;/^Q/c\'"$(printf '\n\\\item')" <in >out
| improve this answer | |
4

Now with awk:

awk '$1 ~ /[ABCDEM]/ {print $0"\n"} $1 ~ /Question/ {print "\\item"}' inputfile

If the line starts with A, B, C, D, E or M (for Main), it prints that line and an extra \n. If the line starts with "Question", it simply prints \item.

| improve this answer | |
  • Pretty sure "Main proteins" was an example. I don't think this check is sufficient for the OP's actual input. – Wildcard Nov 21 '15 at 0:32
  • 1
    I thought that also, but it is quite easy to adapt. – Kira Nov 21 '15 at 1:49
  • i agree. i consider that if you can demonstrate how a command works and how it might be modified to taste, then you've written a pretty good answer. – mikeserv Nov 21 '15 at 2:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.