4
  • Goal: Delete all but the last comment line for each comment block. If the file ends with a comment block, delete it completely. Each comment line begins with a #.

  • Command that I tried

    sed -z -e 's/#.*\n#/#/g' "${InputP}"
    
  • Input file

    # Life/Living
    # Life/Passion
    - [Mindfulness.md](file:///home/nikhil/Documents/Git/Life/Passion/PassionSrc/Sports/Yoga/Mindfulness/Mindfulness.md)
    # Life/PersonalManagement
    # Life/Social
    # Linux/AmazingNotes
    # Linux/Backintime
    # Linux/DotFiles
    # Linux/GitScripts
    - [Peaceful.m3u](file:///home/nikhil/Documents/Git/../Mobile/Documents/PortableNotes/PortableNotesSrc/SocialActivity/Music/SongsPlaylist/Data/Peaceful.m3u)
    - [AuxiliaryFiles.sh](file:///home/nikhil/Documents/Git/Linux/GitScripts/GitScriptsSrc/GitInit/GitNew/Src/AuxiliaryFiles.sh)
    # PythonWs/NumericalProgramming
    # PythonWs/Python
    # PythonWs/ScientificComputing
    
  • Expected Output

    # Life/Passion
    - [Mindfulness.md](file:///home/nikhil/Documents/Git/Life/Passion/PassionSrc/Sports/Yoga/Mindfulness/Mindfulness.md)
    # Linux/GitScripts
    - [Peaceful.m3u](file:///home/nikhil/Documents/Git/../Mobile/Documents/PortableNotes/PortableNotesSrc/SocialActivity/Music/SongsPlaylist/Data/Peaceful.m3u)
    - [AuxiliaryFiles.sh](file:///home/nikhil/Documents/Git/Linux/GitScripts/GitScriptsSrc/GitInit/GitNew/Src/AuxiliaryFiles.sh)
    
  • But I get this Output
    # PythonWs/ScientificComputing
    

Does anyone know how to solve the problem?

  • 1
    Porcupine, I have reworded your question in such a way that other users may more easily find it. If you disagree with the edit, feel free to roll it back. – Quasímodo Nov 26 '20 at 11:09
  • 1
    I think this edit helps the understanding of the question. – AdminBee Nov 26 '20 at 11:20
2

Using GNU sed with slurp mode -z and utilizing extended regexes -E we can do as shown:

$ sed -Ez '
    s/(^|\n)(#[^\n]*\n)+$/\1/
    s/(^|\n)(#[^\n]*\n)+/\1\2/g
' file
  • Remove a trailing comment block.
  • Remove all comment blocks but keep the last line in each.

The GNU sed model is as follows:

  • Sed reads a file line by line unless -z is in effect, when it reads the whole file. The record separator by default is a newline \n unless -z is in use then it is \0 the NULL ascii.
  • After reading in a record, the trailing record separator is clipped and the resulting string is stored in the pattern space register. The pattern space is where all the sed commands operate.
  • Now let's say there are 5 sed commands in our sed script. Then the first one is applied on the pattern space, this modifies the pattern space and on this modified pattern space the next sed command is applied ... and so forth sequentially till the last. Then the pattern space is printed to stdout unless the -n is in effect. After this the next record is read in and the same sequence of sed commands are applied to the pattern space.

Please note that the above is a very simplified narrative, valid when no flow control commands are used in the sed script.

Yes you are right, in the slurp mode the $ signifies the end of file as also the end of pattern space since there is just one pattern space.

When you have this construct (regex)+ then the brackets hold the last regex match because of the greedy nature of regexes.

Alternatively, it can also be done as

$ sed -e '
    /^#/{h;d;} 
    H;z;x;s/^\n//
' file 
  • Thanks! Your solution is elegant! Could you please explain a bit more? When you have 2 sed commands, does the second executes only after the first is run for the whole file, or after each line? Also does $ signifies the end of file due to -z? How do you achieve this: Remove all comment blocks but keep the last line in each? Shouldn't this keep the first line in each? – Porcupine Nov 27 '20 at 12:55
  • @Porcupine I have added a brief summary of the steps and answers to the queries. – guest_7 Nov 27 '20 at 15:02
9

The problem is that .* is greedy and so sed -z -e 's/#.*\n#/#/g' will match from the very first line containing # up to the last line starting with #. This only happens because of the -z flag, that slurps the whole file in the pattern space at once (assuming no null bytes in the text file).

The Sed script to solve your problem is

sed -n '/^#/N;/\n#/D;p' file
  • /^#/N If the line begins with #, append the next line to the pattern space.
  • /\n#/D If the pattern space contains a newline followed by #, delete all up to the newline and start a new cycle.
  • p Print the pattern space if this command is reached.

Useful links

7

You obviously want to remove all comment lines that are followed by other comment lines from your input. The sed call fails because regular expressions are by default "greedy" (i.e. consume as much as possible), which cannot be easily changed.

So I will add an awk-based solution to the stated goal:

awk '/^#/{buf=$0;next} {if (buf) {print buf; buf=""}}1' "${InputP}"

or, slightly more compact:

awk '/^#/{buf=$0;next} buf{print buf; buf=""}1' "${InputP}"
  • This will print all lines that are not comment lines unchanged (the 1 outside the rule blocks means "print the current line, including all modifications made so far" - which is none in this case).
  • If a comment line is encountered (the line matches the patten /^#/), the content will be stored in a buffer buf, but not yet printed. The next command skips execution to the next line so the remaining code only applies to non-comment lines.
  • If a non-comment line is encountered, the buffer content is printed first (if any) and the buffer emptied (to prevent multiple printout) before the actual line content is printed.
  • nice approach and it gives the output as it was ask, but the OP asked for sed – BANJOSA Nov 24 '20 at 17:15
  • 3
    @BANJOSA Yes and no. In the end, the OP stated a goal, and called the sed call the "command that I tried", so they seem to be open to other approaches. – AdminBee Nov 24 '20 at 17:19
  • 3
    @BANJOSA as a general rule here, we answer the question not only for the person asking but for future visitors as well. Giving answers using alternative tools is fine since i) the OP may be OK with another tool and ii) the next person to have the same issue might not have the same preferences as the OP. – terdon Nov 24 '20 at 18:59
1

This should work:

perl -ne 'print $x,$_ unless /^#/; $x = /^#/ ? $_ : ""' < infile

I get the expected output you posted.

Edit: explanation

  • take $x to be $left_over_line_to_be_printed, if you will :-)
  • the first statement prints any leftover line, then the current line, if the current line is not a header line.
  • the second statement sets "leftover" to the last header line seen, or to an empty string. (That is, for a NON-header line, the "left over" will be the empty string, since that has already been printed by the first statement. For a header line, it will be the line. If multiple header lines come together, it will end up being the last one).
  • Thanks! Maybe it might help others if you could explain a bit more :) – Porcupine Nov 27 '20 at 12:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.