Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have a sed expression to remove the first 2 characters from all the lines in a file.

sed 's/^..//' raw.txt > converted.txt

However, the problem is some lines are large and get appended to the next line. For example,if I consider the input as in the below lines,

□ I have box and space in front
□ I am a large line which will
get appended to next line.
□ I am another line.

As per my sed syntax, I am getting the output as,

I have box and space in front
I am a large line which will
t appended to next line. 
I am another line. 

However, the output am expecting is,

I have box and space in front. 
I am a large line which will get appended to next line. 
I am another line. 

How should I modify the sed expression to achieve this?

share|improve this question
    
perhaps s/^[^A-Za-z]// will work –  user55518 Feb 19 at 16:49
    
yeah, I am getting all the characters. But I am still trying to figure out to get the output as I had mentioned in my expected output. –  Ramesh Feb 19 at 16:53
    
Do you actually have a or does that represent a space? –  terdon Feb 19 at 17:06
    
it's also possible that is a multichar utf8 character. –  user55518 Feb 19 at 17:22
    
Yeah, I get the box from a web page. –  Ramesh Feb 19 at 17:25

3 Answers 3

up vote 4 down vote accepted
awk '
    {
        if (/^□ /) {
            if (prev_line) 
                print prev_line
            prev_line = substr($0, 3)
        } else {
            prev_line = prev_line " " $0
        }
    }    
    END {print prev_line}
' raw.txt
I have box and space in front
I am a large line which will get appended to next line.
I am another line.
share|improve this answer
    
It works perfectly fine. Thanks for your answer. –  Ramesh Feb 19 at 19:18

Might be easier with awk:

awk '{
       if (/^□ /) printf "%s", (NR>1?"\n":"") substr($0,3)
       else printf " %s", $0
     }
     END {if (NR>0) print ""}'
share|improve this answer
    
This one works as expected. Thanks for the answer. –  Ramesh Feb 19 at 19:18

If your file is not huge and you can load it directly into memory, this should work:

$ perl -000pe 's/\n([^□])/ $1/g; s/^□ //mg;' raw.txt 
I have box and space in front
I am a large line which will et appended to next line.
I am another line.

The -000 activates paragraph mode and (assuming there are no consecutive newlines in your file) will load the whole file into the special variable $_. You then do the necessary replacements and the -p causes $_ to be printed automatically.

share|improve this answer
    
This one seems a pretty elegant solution. Thanks! –  Ramesh Feb 19 at 19:17
    
I would anchor the 2nd substitution: s/^□ //mg –  glenn jackman Feb 19 at 21:48
    
@glennjackman good point, I've done so. And thanks for catching the bug too. –  terdon Feb 20 at 16:56

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.