0

I am trying to format data in a file that needs to be imported to a system. This system does not take more than 65 characters of text in a single line. So, I am trying to format the text in Description column to split as multiple lines as shown in sample below. It would be really helpful for the user if it is a single line Linux command, can you please check if it is possible?

Sample input in file -

 
Depth   Description
(ft)    -
12370.0 LS: Mdst, blsh gry, sft, occ mod firm, crpxln, prly, arg, SLTST: blk, firm-mod hd, amor, gt, mod calc, CLST: lt gry-m gry, sft, amor, wxy
12400.0 LS: Mdst, blsh gry, mod firm, crpxln, chky, arg, SLTST: blk-dk gry, firm-mod hd, amor, gt, mod calc, CLST: lt gry-m gry, occ rdsh gry, mod firm, amor, wxy, tr CHK
12430.0 LS: Mdst, blsh gry, mod firm, crpxln, chky, arg, SLTST: blk-dk gry, firm-mod hd, amor, gt, mod calc, SH: blk-dk gry, mod firm, blky-plty, occ brit, wxy
12460.0 SH: blk-dk gry, mod firm, blky-plty, occ brit, ea, SLTST: blk-dk gry, firm-mod hd, amor, gt, mod calc, SST: gry-dk gry, wl consol, v f, ang, p srt, cotd, slily calc cmt, no fluor

Depth and Description and the line below it are considered headers. Depth column contains all numbers, Description column contains text that should be formatted as multiple lines for the system to accept the data.

Desired output with Linux command - 

 
Depth   Description
(ft)    -
12370.0 LS: Mdst, blsh gry, sft, occ mod firm, 
        crpxln, prly, arg, SLTST: blk, firm-mod hd, 
        amor, gt, mod calc, CLST: lt gry-m gry, sft, 
        amor, wxy
12400.0 LS: Mdst, blsh gry, mod firm, crpxln, chky, 
        arg, SLTST: blk-dk gry, firm-mod hd, amor, 
        gt, mod calc, CLST: lt gry-m gry, occ rdsh gry, 
        mod firm, amor, wxy, tr CHK
12430.0 LS: Mdst, blsh gry, mod firm, crpxln, chky, 
        arg, SLTST: blk-dk gry, firm-mod hd, amor, 
        gt, mod calc, SH: blk-dk gry, mod firm, 
        blky-plty, occ brit, wxy
12460.0 SH: blk-dk gry, mod firm, blky-plty, occ brit, 
        ea, SLTST: blk-dk gry, firm-mod hd, amor, gt, 
        mod calc, SST: gry-dk gry, wl consol, v f, ang, 
        p srt, cotd, slily calc cmt, no fluor

Note - User will not be able to install any Linux package on their machine. (example - par)

|improve this question
  • but your line in the output 12460.0 SH: blk-dk gry, mod firm, blky-plty, occ brit, contains 55 chars – RomanPerekhrest Sep 5 '17 at 16:34
  • Sorry, updated the question to say 65 characters. Also, I formatted the file just to get an idea but did not count the number of characters on each line. – NewCoder Sep 5 '17 at 16:48
  • You can get pretty close to your desired output with just the fmt command (from coreutils) e.g. fmt -tw65 file – steeldriver Sep 5 '17 at 17:41
  • steeldriver - Thanks for the suggestion. I tried it out but it does not work as expected. When I use this command and import the formatted data, all data is going under Depth instead of placing the text under Description column. – NewCoder Sep 6 '17 at 20:24
  • Thanks Kusalananda & RomanPerekhrest. All your solutions work, it gets the format as requested in the question. – NewCoder Sep 7 '17 at 14:00
1

With GNU sed:

$ sed -r '3,$s/(.{55} )/\1\n\t/g' file.in
Depth   Description
(ft)    -
12370.0 LS: Mdst, blsh gry, sft, occ mod firm, crpxln, prly,
        arg, SLTST: blk, firm-mod hd, amor, gt, mod calc, CLST:
        lt gry-m gry, sft, amor, wxy
12400.0 LS: Mdst, blsh gry, mod firm, crpxln, chky, arg,
        SLTST: blk-dk gry, firm-mod hd, amor, gt, mod calc, CLST:
        lt gry-m gry, occ rdsh gry, mod firm, amor, wxy, tr CHK
12430.0 LS: Mdst, blsh gry, mod firm, crpxln, chky, arg,
        SLTST: blk-dk gry, firm-mod hd, amor, gt, mod calc, SH:
        blk-dk gry, mod firm, blky-plty, occ brit, wxy
12460.0 SH: blk-dk gry, mod firm, blky-plty, occ brit, ea,
        SLTST: blk-dk gry, firm-mod hd, amor, gt, mod calc, SST:
        gry-dk gry, wl consol, v f, ang, p srt, cotd, slily calc
        cmt, no fluor

This leaves spaces at the end of lines that may be stripped b running it through sed 's/ $//'.

The expression 3,$s/(.{55} )/\1\n\t/g will replace any run of 55 characters followed by a space by those same 55 characters (and the space) followed by a newline and an indentation by one tab. This repeats for all matches of the pattern on the line.

|improve this answer
  • Thanks for your help! I am almost there. The formatted output looks as required with the solution proposed but looks like something is missing. If I edit the file using notepad - Like split the lines <65 characters and import it, it takes it as required with depth on first column and description on second column. But, when I used the script output file and import it, the split lines from description are being imported to the first column instead of second column. – NewCoder Sep 5 '17 at 18:24
  • @NewCoder Does the importer expect the continued lines to be indented by tabs? Does the columns need a tab between them maybe? Is there a tab in the input data? – Kusalananda♦ Sep 5 '17 at 18:27
  • That's correct, I think it is looking for lines with tabs. When I edited the file, I used tab instead of spaces and the importer took it as required. Yes, there is a tab between two header columns. – NewCoder Sep 5 '17 at 18:39
  • @NewCoder That's a significant fact to leave out... Hmm... need to think. – Kusalananda♦ Sep 5 '17 at 18:41
  • @NewCoder I've fixed this sed solution. I will have to redo the awk solution too... – Kusalananda♦ Sep 5 '17 at 18:43
3

With standard awk:

{ printf("%s\t", $1) }
{
    linelen = 8;  # assume 8 spaces for the first field

    for (i = 2; i <= NF; ++i) {
        wordlen = length($i);

        if (wordlen + linelen > 65) {
            # break output line since adding next word
            # would make it too long
            printf("\n\t");
            linelen = 8; # count tab as 8 spaces
        }

        if (i < NF) {
            printf("%s ", $i);
            linelen += wordlen + 1; # +1 for the space
        } else {
            printf("%s\n", $i);
        }
    }
}

Testing:

$ awk -f script.awk file.in
Depth   Description
(ft)    -
12370.0 LS: Mdst, blsh gry, sft, occ mod firm, crpxln, prly, arg,
        SLTST: blk, firm-mod hd, amor, gt, mod calc, CLST: lt
        gry-m gry, sft, amor, wxy
12400.0 LS: Mdst, blsh gry, mod firm, crpxln, chky, arg, SLTST:
        blk-dk gry, firm-mod hd, amor, gt, mod calc, CLST: lt
        gry-m gry, occ rdsh gry, mod firm, amor, wxy, tr CHK
12430.0 LS: Mdst, blsh gry, mod firm, crpxln, chky, arg, SLTST:
        blk-dk gry, firm-mod hd, amor, gt, mod calc, SH: blk-dk
        gry, mod firm, blky-plty, occ brit, wxy
12460.0 SH: blk-dk gry, mod firm, blky-plty, occ brit, ea, SLTST:
        blk-dk gry, firm-mod hd, amor, gt, mod calc, SST: gry-dk
        gry, wl consol, v f, ang, p srt, cotd, slily calc cmt, no
        fluor

The only downside is that this leave trailing spaces on most output lines. You may strip these by passing the output though sed 's/ $//'.

|improve this answer
0

awk solution:

awk 'NR<=2{ print }NR>2{ c=0; r=$1; 
            for(i=2;i<=NF;i++) { 
               if ((length(r) + length($i)) > (!c? 65:61)) { c++; printf "%s\n\t",r; r=$i } 
               else { r=r FS $i } if(i==NF) printf "%s",r } 
            print "" 
     }' file

  • NR<=2{ print } - print the first 2 lines as is

  • r variable, containing each processed/formatted line

The output:

Depth   Description
(ft)    -
12370.0 LS: Mdst, blsh gry, sft, occ mod firm, crpxln, prly, arg,
    SLTST: blk, firm-mod hd, amor, gt, mod calc, CLST: lt gry-m
    gry, sft, amor, wxy
12400.0 LS: Mdst, blsh gry, mod firm, crpxln, chky, arg, SLTST:
    blk-dk gry, firm-mod hd, amor, gt, mod calc, CLST: lt gry-m
    gry, occ rdsh gry, mod firm, amor, wxy, tr CHK
12430.0 LS: Mdst, blsh gry, mod firm, crpxln, chky, arg, SLTST:
    blk-dk gry, firm-mod hd, amor, gt, mod calc, SH: blk-dk gry,
    mod firm, blky-plty, occ brit, wxy
12460.0 SH: blk-dk gry, mod firm, blky-plty, occ brit, ea, SLTST:
    blk-dk gry, firm-mod hd, amor, gt, mod calc, SST: gry-dk gry,
    wl consol, v f, ang, p srt, cotd, slily calc cmt, no fluor
|improve this answer
  • RomanPerekhrest - I get the output as shown above but when I import it - It is considering first column as '12370.0 LS: Mdst, blsh gry, sft, occ mod firm, crpxln, prly, arg,' under Depth which is incorrect since it has to take only 12370.0 and remaining two lines as second column under description. Can you please let me know if anything needs to be changed in the script to make it right? – NewCoder Sep 6 '17 at 20:14
  • @NewCoder, when you say I import it - no one see how are doing this. So how can solve your additional issue if I don't see your actual logic? – RomanPerekhrest Sep 6 '17 at 20:57
  • RomanPerekhrest - Sorry, I understand. I was just thinking about the formatting part which might be missing a small thing. For some reason, the entire first line is going to the first column but the 2nd and third line are going to the second column. Just wondering if it is possible to send any text after the number (ex - 12370.0) to the next line which may fix the issue? So, in the above case the first line after headers is 12370.0 and the next 3 lines show description. – NewCoder Sep 6 '17 at 21:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.