sed: how to insert a newline into a regular expression?

Question

I'm trying to format a text file. Right now it generally looks like this:

s ApartCD Compact DiscCD-ROM Compact Disc-Read-Only MemoryCD-RW Compact Disc-RewritableCDFS Compact Disc File SystemCERT Computer Emergency Response TeamCFS Central File System, Common File System, or Command File SystemACRONYM SPELLED OUTCGA Computer Graphics and ApplicationsCIDR Classless Inter-Domain RoutingCIFS Common Internet File SystemCMOS Complementary Metal-Oxide SemiconductorCNR Communications and Networking RiserCOMx Communication port (x=port number)CPU Central Processing UnitCRT Cathode-Ray TubeDaaS Data as a ServiceDAC Discretionary Access ControlDB-25 Serial Communications D-Shell Connector, 25 pinsDB-9 Serial Communications D-Shell Connector, 9 pinsDBaaS Database as a Service DC Direct CurrentDDoS Distributed Denial of ServiceDDR Double Data RateDDR RAM Double Data Rate Random Access MemoryDFS Distributed File SystemDHCP Dynamic Host Configuration ProtocolDIMM Dual Inline Memory ModuleDIN Deutsche Industrie NormDLT Digital Linear TapeDLP Digital Light P

This is just a sample I copied. To start with, I'd like to insert a newline between every instance of a lowercase letter immediately followed by an uppercase letter. I've figured out from reading the GNU sed manual that I can use the bracket ranges [:lower:] and [:upper:] for this, (I think [[:lower:]]+[[:upper:]] is the right pattern) but I have yet to nail down the right command or general syntax to make this work. I've found several commands for appending newlines, but none for inserting them into a regex.

This is my first time using sed. I have no idea if this is the best tool for the job or not. Any help is greatly appreciated, and if it's not too much trouble, please explain your command so I can perhaps learn something and not have to bother anybody in the future. Thank you.

Answer 1

Try this using GNU sed:

$ sed 's/\([a-z]\+\)\([A-Z]\)/\1\n\2/g' text.txt 

For your input, it yields:

s Apart
CD Compact Disc
CD-ROM Compact Disc-Read-Only Memory
CD-RW Compact Disc-Rewritable
CDFS Compact Disc File System
CERT Computer Emergency Response Team
CFS Central File System, Common File System, or Command File System
ACRONYM SPELLED OUTCGA Computer Graphics and Applications
CIDR Classless Inter-Domain Routing
CIFS Common Internet File System
CMOS Complementary Metal-Oxide Semiconductor
CNR Communications and Networking Riser
COMx Communication port (x=port number)CPU Central Processing Unit
CRT Cathode-Ray Tube
Daa
S Data as a Service
DAC Discretionary Access Control
DB-25 Serial Communications D-Shell Connector, 25 pins
DB-9 Serial Communications D-Shell Connector, 9 pins
DBaa
S Database as a Service DC Direct Current
DDo
S Distributed Denial of Service
DDR Double Data Rate
DDR RAM Double Data Rate Random Access Memory
DFS Distributed File System
DHCP Dynamic Host Configuration Protocol
DIMM Dual Inline Memory Module
DIN Deutsche Industrie Norm
DLT Digital Linear Tape
DLP Digital Light P

Some explanation:

I will assume you know what "capture group" is, since you understand the pattern.

The string 's/\([a-z]\+\)\([A-Z]\)/\1\n\2/g' is splited into 4 parts by character /.

The first part s means to substitute.

The second part is the pattern. The pattern contains two capture groups: [a-z]\+ and [A-Z]. Take the word "DDoS" in you input for an example. the first capture groups will capture "o" and the second capture group will capture "S". The whole pattern matches "oS".

The third part \1\n\2 will replace the string matched by the pattern.\1 refers to the first capture group [a-z]\+ and in the "DDoS" example, it's "o". \2 refers to the second capture group [A-Z] and in the "DDoS" example, it's "S". \n means line feed character. So combine then together you will get "o\nS", which will replace the matched string "oS".

The fourth part g means global. If omitted, sed will find the first match, do the substitution and quit. With g, sed will search all the matched strings and do the substitution.

You may refer to the sed manual for detailed description.

Answer 2

GNU sed is able to insert newlines with its s/// command:

sed -e 's/\([[:lower:]]\)\([[:upper:]]\)/\1\n\2/g' file

Non-GNU sed (as found on macOS, BSD, and other non-Linux systems) does not know what \n means in the replacement text of the s/// command and would just insert an n character.

To substitute in a newline you would have to escape a literal newline, as in

sed -e 's/\([[:lower:]]\)\([[:upper:]]\)/\1\
\2/g' file

or use a work-around, for example by inserting some other place-holder character that does not already exist in the data, and then use y/// to replace these with newlines (the y/// command understands \n):

sed -e 's/\([[:lower:]]\)\([[:upper:]]\)/\1@\2/g' -e 'y/@/\n/' file

or

sed -e 's/\([[:lower:]]\)\([[:upper:]]\)/\1@\2/g' file | tr '@' '\n'

Note that [[:lower:]] matches a single character whereas [[:lower:]]+ (or [[:lower:]]\{1,\} when written as a basic regular expression) matches at least one character. In this case, we don't need the + (or \{1,\}) since it's enough to match a single lower case character.