0

I'm trying to use sed for updating a commented config file. Rather than tacking everything at the EOF, I'm trying to keep things sectioned off. My config looks something like this.

# SECTION ONE
data...
data...
data...
<insert line here>

# SECTION TWO
data...
data...

I'm trying to insert lines at the end of section one, but I'm having a hard time writing a search pattern since it won't allow "\n" and you can't have multiple "^$" in a pattern. I'd like something like the following:

sed -i "/^\n\n# SECTION TWO.*/i data..." somefile.conf

    or

sed -i "/^$^$# SECTION TWO.*/i data..." somefile.conf

I'm open to other suggestions as well, but I'd like to keep it to a single line if possible. This is part of a much larger script. I know this is pretty easy with Python, Perl, etc., but I'm trying to keep this to a "shell" solution.

5
  • 1
    Is the configuration file in some form of structured format like XML, YAML, or JSON? Does the file contain multiple other sections (possibly before the section of interest)? Is it the SECTION ONE text that identifies the section that you want to add the text to or just the fact that it's the first section? – Kusalananda May 4 at 8:28
  • 1
    This looks like a "XYProblem" (that occurs so often that one named it) : You are not asking help on what you need to achieve (ie, add a line before 2 blank lines) but instead ask help on one (of many) ways you tried to achieve it. Here, you specificly asked to use sed, but I don't see any reason to do so, as it makes the answer much more difficult, as sed is not the best tool for that job, or at least not the most readable, imo. Unless sed was really mandatory, @Kusalananda correctly gave the answer to help you in what you need to achieve, and not on what you actually asked for. – Olivier Dulac May 4 at 10:35
  • @OlivierDulac: Sed is one of the best tools for that job and the answer is not difficult, if know the basics of sed. For people with a basic sed knowledge but who are unfamiliar with awk (or ruby, python, you name it), a cryptic sed command is far better, than a cryptic awk command. Look how verbose the awk-solution is! Sed instead: Around 20 bytes, mostly using things, Nilpo already knows, much easier to support. – user unknown May 4 at 16:09
  • @userunknown A short solution is not necessarily a good solution. And a cryptic solution is worse than a more verbose one. You can write crypto solutions privately, obviously, but if you are writing for others, you'd better make sure that they understand it and that they can modify it to fit their needs. – Kusalananda May 4 at 16:22
  • The AWK-solution is more verbose without being less cryptic, so this remark is missing the point. The point was mainly, that a canonical way to solve a problem which starts by learning the programming language XY first, seems a much better candidate to be named XYProblem. – user unknown May 4 at 18:07
2

Using awk in place of sed:

newdata='This is the new data'

newdata=$newdata awk -F '\n' -v OFS='\n' -v RS= -v ORS='\n\n' \
    '$1 == "# SECTION ONE" { $(NF+1) = ENVIRON["newdata"] }; 1' file

or

newdata='This is the new data'

newdata=$newdata awk '
    BEGIN { FS = OFS = "\n"; RS = ""; ORS = "\n\n" }
    $1 == "# SECTION ONE" { $(NF+1) = ENVIRON["newdata"] }; 1' file

This puts awk in "paragraph reading mode" by using an empty value for RS, the input record separator. This means that awk will read a paragraph at a time. A "paragraph" is any collection of lines delimited by at least one empty line.

We then set the input field separator, FS, to a newline character with -F '\n' so that each line in a paragraph becomes its own field.

We also set the output-related corresponding variables ORS and OFS in such a way that records (paragraphs) are outputted with a trailing empty line, and so that fields (lines within a paragraph) are outputted with a terminating newline character.

The actual code detects when the first line of a paragraph is exactly the string # SECTION ONE. If that is the case, a new field is added to the end of the current record with the new data. The new data is taken from the newdata environment variable.

All paragraphs, whether modified or not, are then outputted unconditionally.

Note that if the actual configuration file (which we never get to see in the question) is written in XML, YAML, JSON, or some other structured document format, then this answer is invalid, as those document formats require proper format-aware tools for reading and writing (since they are not line-oriented and since the data needs encoding/decoding).

0
1

Using GNU sed we can do as follows:

sed -e '/# SECTION ONE/,/^$/s/^$/__NEW-DATA__\n/' file

Range operator in awk can be used to get the desired output

awk '
  /# SECTION ONE/,!NF{
    if (!NF) print "__NEW-DATA__"
  }1
' file

This method uses POSIXly sed constructs

sed -e '
  /# SECTION ONE/!b
  :a
    n;/^$/!ba
  G;s/^/__NEW-DATA__/
' file

In this method we place the new data before the section name, provided it's not the first. Assuming file size is at least 3 lines.

sed -e '
  1,2N
  $q;N
  s/^\n\n# SECTION TWO/__NEW-DATA__\n&/;t
  P;D
' file
0

After reading Kusalandras comment and the question again, I understand it like him or her.

GNU-sed has a switch to turn on POSIX style, so I hope this time, the command is agnostic to the sed-flavour.

Original file:

cat FILE
# SECTION ONE
data...
data...
data...

# SECTION TWO
data...
data...

Command:

sed --posix "s/^[ \t]*$/...new data .../; /# SECTION TWO/i\\ " FILE

# SECTION ONE
data...
data...
data...
...new data ...
 
# SECTION TWO
data...
data...

Note, that the insertion leaves a blank in the file, which can be remmoved with a follow up sed command or it can be handled by usind s/^[ \t]*$ as a regular expression, which matches empty lines and such of purely white space of blanks and/or tabs.

Alternative:

sed --posix "s/^$/...new data .../; /# SECTION TWO/i\ " -e "s/^ $//" FILE 

The second form uses a second sed command (-e - arguably, it's the third, after using two commands, separated by ; in the first quotes) to remove the space, if it isn't tolerable, and hence is dogmatic in finding empty lines.

1
  • 1
    @Kusalananda: Thanks for the hint. After considering your comment I agree to your understanding of the question, which makes more sense. I corrected my answer accordingly. I had misinterpreted the question as <add lines here> with plural "lines", and thought it should improve the layout with big margins between sections. – user unknown May 4 at 18:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.