1

I have a text file that has lines like:

LIN This is the value which I need
      from this line which has a very long line
SHR This1;This2;
     This3;
GYC This is an extra.

And the output is supposed to be as:

LIN This is the value which I need from this line whi a very..
SHR This1;This2;This3;
GYC This is an extra.

I was trying this in sed first before I could do it in Python. So I managed to come up with..

sed 's/     //' filename.txt 

The above snippet does only one thing, it removes the empty 4+1 spaces (always fixed), however now i am stuck not sure knowing how to go ahead with because I have to move the lines to the LINE&SHRT tags so i am able to extract information. Also, when the tag line ( with LINE/SHRT/EKY/EKC/USER) line is continuing it gets indicated by a single extra space. And if supposed to be new line then it continues to next line.

Question, can sed be used to move the line to previous line? How can i differentiate the empty space to indicate if line is carry on or terminates

  • I don't know what you mean by "LINE/SHRT/EKY/EKC/USER" and "LINE&SHRT". These things do not occur in the data that you have posted. – Kusalananda Sep 1 '17 at 13:55
  • Anhaa sorry. I mean the tags: EKY/EKC/ are just like LIN/SHR. Apologies for that – Noel Alex Makumuli Sep 1 '17 at 14:13
2

Keep it simple:

sed 'H;1h;$!d;g;s/\n  */ /g'

This short script will join all lines that begin with at least one space with the previous line.

How it works: H appends each line to the hold space. To avoid a leading newline the first line is copied by 1h. If this was not the last line, delete it, otherwise move the hold space to the pattern space with g. Now the whole file is in the pattern space and now the s command replaces all newlines with spaces by one space.

With GNU sed you can make it even simpler:

sed -z 's/\n  */ /g'
  • For sure this works like a charm.. There is the ^M when the new moved to previous.. I can try to google around on how to remove the ^M. If you can assist will be cool too.. – Noel Alex Makumuli Sep 1 '17 at 17:40
  • Shoved the file to dos2unix then called sed -z 's/\n */ /g' now works perfectly. – Noel Alex Makumuli Sep 1 '17 at 19:37
  • Sorry, wasn't online. But you were right: There were DOS-style line endings. If your question is solved, you may consider making it answered. – Philippos Sep 2 '17 at 17:17
1

Another way with AWK:

awk '{$1=$1;printf("%s ",$0)};NR%2==0{print ""}' FILE.txt

OUT:

LIN This is the value which I need from this line which has a very long line
SHR This1;This2; This3;
GYC This is an extra.
  • $1=$1 : delete space from beginning line
  • printf("%s ",$0) delete \n (newline) from end of line
  • NR%2==0{print ""} print \n (newline) for even lines (e.g: 2,4,6,...)
1
/^[A-Z]/        { if (line) { print line }; line =      $0 }
/^ /            { sub(/     /, "")        ; line = line $0 }
END             { if (line) { print line }                 }

This awk script will produce

LIN This is the value which I need from this line which has a very long line
SHR This1;This2;This3;
GYC This is an extra.

given the data provided in the question in file.in.

  1. The first block will execute for every line that starts with a uppercase ASCII character. It will output the contents of line if there is anything in it, and then save the current input line in line. This takes care of outputting the constructed line for the previous input "block" and starts to assemble the next output line.

  2. The second block will execute for each line that has at least one space at the start and will remove the first five spaces before adding it to the end of line. This builds up the output line from continuation lines in the input.

  3. The END block will output the saved line if there is anything in it. This takes care of outputting the assembled line from the final "block" of input.

You run this with

$ awk -f script.awk file.in

This script handles the cases where there might be multiple indented (continued) lines.

  • If there's more than one "continued" lines, you won't concatenate them all. – glenn jackman Sep 1 '17 at 16:13
  • @glennjackman Fixed. – Kusalananda Sep 1 '17 at 16:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.