Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I need join 2 lines into one in stdout - replace new line AND 4x space with one space but new line without space don't touch

tcpdump -vvv ... -l | xyz

this is output:

2014-06-06 AAA
    BBB
2014-06-06 CCC
    DDD

but I need:

2014-06-06 AAA BBB
2014-06-06 CCC DDD
share|improve this question
    
This feels like a Mexican stand-off. I'm upvoting everybody - these are all awesome. –  mikeserv Jun 6 at 22:06

5 Answers 5

up vote 4 down vote accepted

I thought of another way to do this that doesn't rely on paired lines but instead only a <space> character following a \newline character:

sed ':n;N;s/\n  */ /;tn;P;D' 

That uses a sliding window of newline data and the test function to operate. The flow is like this:

  1. define the :next label
  2. append Next line in input to pattern space
  3. s/elect a \newline character followed by 1 or more <space> characters and replace them with a single /<space>/
  4. test if last s/// was successful and, if so, branch to :next label, else...
  5. Print up to the first \newline character in pattern space and...
  6. Delete up to and including first \newline character in pattern space and restart cycle with remaining pattern space or with next line

So, basically, all occurrences of \n [ ]* in input get squeezed into [ ]:

sed ':n;N;s/\n  */ /;tn;P;D' <<\DATA
2014-06-06 AAA
    BBB
2014-06-06 CCC
    DDD
2014-06-06 EEE
2014-06-06 FFF
    GGG
DATA

OUTPUT:

2014-06-06 AAA BBB
2014-06-06 CCC DDD
2014-06-06 EEE
2014-06-06 FFF GGG

OLD:

sed -n 'h;n;H;x;s/\n */ /p' <<\DATA
2014-06-06 AAA
    BBB
2014-06-06 CCC
    DDD
DATA

OUTPUT:

2014-06-06 AAA BBB
2014-06-06 CCC DDD

I use 5 sed commands.

  1. overwrite hold space with pattern space...
  2. overwrite pattern space with the next line in input
  3. append pattern space to Hold space following an automatically inserted \newline character
  4. exchange the contents of hold and pattern spaces
  5. s/elect the first \newline character and any or *all following spaces then /replace the selection with a single space and /print the result.

Though, now that I think about it, a simplified version of both mine and @Falsenames answers - which really is better than this one - would just be:

sed 'N;s/\n */ /' <<\DATA
2014-06-06 AAA
    BBB
2014-06-06 CCC
    DDD
DATA    

OUTPUT:

2014-06-06 AAA BBB
2014-06-06 CCC DDD
share|improve this answer

Editing this to give more error correction.

The 'N' command in sed appends the next line. Using this in conjunction with a substitution to swap out the newline and a bunch of the spaces with nothing.

$ cat foo | sed 's/^/    /'
2014-06-06 AAA
    BBB
2014-06-06 CCC
    DDD
2014-06-06 EEE
2014-06-06 FFF
    GGG

My new answer:

$ sed ':a;$!{N;s/\n   //;ba;}' foo
2014-06-06 AAA BBB
2014-06-06 CCC DDD
2014-06-06 EEE
2014-06-06 FFF GGG

That seems to be far superior to my old answer:

$ sed '/^[^ ]/N;s/\n   //' foo
2014-06-06 AAA BBB
2014-06-06 CCC DDD
2014-06-06 EEE
2014-06-06 FFF
    GGG

Or the pretty slick one @mikeserv listed.

$ sed -n 'h;n;H;x;s/\n */ /p' foo
2014-06-06 AAA BBB
2014-06-06 CCC DDD
2014-06-06 EEE 2014-06-06 FFF
    GGG 
share|improve this answer
    
I wish I could upvote this twice. I love screwing with this stuff - experimenting just like you have. I don't know what drove me to do all of that crazy stuff in the first place - I think I was counting on $(date) ...\n for every line and I was going to strip the second $(date) and the \newline. Anyway, I think mine works even in the last line case - which I didn't handle because they were always supposed to be paired - to handle odd cases without recursion you could do this: sed '/^[0-9]/!{H;$!d};x;s/\n */ /g' though I use the method you demo quite often as well. –  mikeserv Jun 7 at 2:08
1  
I always run into issues with h/H. Just starting to learn some of the more advanced control stuff with sed. I use the basics often enough at work that it's nearly embarrassing to be unfamiliar with the control ranges. That last modification of yours here works great. –  Falsenames Jun 7 at 3:25
    
I think the H one breaks depending on whether the last line is an odd or even case - cleaning that stuff up is usually the annoying bit. P,D make it much easier, but you have to think about them differently. info sed has some examples - usually they H the first 5 or 10 lines or so then x and just N;P;D till eof - always keeping 10 or so lines in pattern space. If I can offer you any advice though - debug with l - especially if you're having trouble with hold/pattern space. My problems there usually have to do with not clearing it after use. –  mikeserv Jun 7 at 3:39
awk -v ORS= '/^ {4}/{print substr($0,4); next}
     {print s $0; s="\n"}
     END{print s}'

That one should work for inputs like:

2014-06-06 AAA
    BBB
2014-06-06 CCC
    DDD
    EEE
2014-06-06 EEE
2014-06-06 FFF

And produce:

2014-06-06 AAA BBB
2014-06-06 CCC DDD EEE
2014-06-06 EEE
2014-06-06 FFF

With perl:

perl -0777 -pe 's/\n {4}/ /g'

But that means slurping the while input in memory first.

share|improve this answer
echo "2014-06-06 AAA
      BBB
2014-06-06 CCC
    DDD
" | paste - - | tr -s '[:blank:]' '[ *]'
2014-06-06 AAA BBB
2014-06-06 CCC DDD
share|improve this answer

Another awk command,

$ awk '{var=$0; getline; print var,$1}' file
2014-06-06 AAA BBB
2014-06-06 CCC DDD
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.