8

Is it possible to sort between two strings in a large file ?

e.g. Current file is as:

    0cf  Front Brake
    0d0  Rear Brake
    0ce  Handle Bars
HUT 03  VR Controls
    009  Vest
    001  Belt
    002  Body Suit
    020  Stereo Enable
    003  Flexor
    007  Hand Tracker
    004  Glove
    006  Head Mounted Display
    008  Oculometer
    00a  Animatronic Device
    000  Unidentified
    021  Display Enable
    005  Head Tracker
HUT 04  Sport Controls
    000  Unidentified
    002  Golf Club
    001  Baseball Bat

And the desired output is as:

    0ce  Handle Bars
    0cf  Front Brake
    0d0  Rear Brake
HUT 03  VR Controls
    000  Unidentified
    001  Belt
    002  Body Suit
    003  Flexor
    004  Glove
    005  Head Tracker
    006  Head Mounted Display
    007  Hand Tracker
    008  Oculometer
    009  Vest
    00a  Animatronic Device
    020  Stereo Enable
    021  Display Enable
HUT 04  Sport Controls
    000  Unidentified
    001  Baseball Bat
    002  Golf Club

Here, Section HUT 03 VR Controls and HUT 04 Sports Controls is sorted out.

In given file, Section headers starts with non-space characters while section content always begins with space or tab. Since this file have 100+ sections then it will not be feasible to hard-code section name in script/command

  • Are the sections on fixed line numbers, or defined by patterns? – Sparhawk May 16 at 7:33
  • Section headers starts as a first character of line, while it's contents begins with space/tab. Sections are not on fixed numbers. – SHW May 16 at 7:36
  • Are you wanting to sort one section only (as per title of question and text), or every section? – Kusalananda May 16 at 8:06
  • @Kusalananda I agree the question is ambiguous on this point; the example output however does show all sections (or parts thereof) being sorted. – Stephen Kitt May 16 at 8:17
  • I wouldn't say that "HUT" uses hex characters. – jlliagre May 16 at 9:00
7

In Python:

#!/usr/bin/python3

with open("file.txt", "r") as ins:
    lines = []
    for line in ins:
        if line.startswith((" ", "\t")):
            lines.append(line)
        else:
            lines.sort()
            print(*lines, end = "", sep = "")
            print(line, end = "")
            lines = []
    lines.sort()
    print(*lines, end = "", sep = "")

This sorts all the sections (separately), not only those between two specific lines.

  • Superb! This is masterstroke. – SHW May 16 at 8:01
6

For the fun, here is a way to sort a single section using ex:

ex file <<%
/HUT
+1,/HUT/-1!sort
w file.sorted
q
%
6
$ awk 'BEGIN { OFS="\t"; s=0 } /^[^[:blank:]]/ { print ++s "\b", $0; next } { print s, $0 }' file | sort -n | cut -f 2-
    0ce  Handle Bars
    0cf  Front Brake
    0d0  Rear Brake
HUT 03  VR Controls
    000  Unidentified
    001  Belt
    002  Body Suit
    003  Flexor
    004  Glove
    005  Head Tracker
    006  Head Mounted Display
    007  Hand Tracker
    008  Oculometer
    009  Vest
    00a  Animatronic Device
    020  Stereo Enable
    021  Display Enable
HUT 04  Sport Controls
    000  Unidentified
    001  Baseball Bat
    002  Golf Club

This uses awk to add a number (and a tab separator) in front of each line corresponding to the section that this line is in. For section headers, we add a number followed by a backspace character (only because backspace sorts before tabs). Then we simply sort the resulting data on these numbers before removing them and the added tab separators.

Section headers are detected by looking for non-blank characters at the start of the line.

  • 1
    Nice! I especially like the backspace trick. – Stephen Kitt May 16 at 8:28
  • 1
    With this approach you could also use the section number (after the HUT field) as the prefix, to sort the sections too. – Stephen Kitt May 16 at 9:20
3

You could get awk and sort to cooperate to get the job done.

awk '
    /^[[:blank:]]/{print | "sort"; next}
    {close("sort"); print}; 
    END{close("sort")}
' file
  • Pipe each content line into sort
  • Call close on sort when a section marker is encountered; this causes sort to flush its output to standard output and exit
  • Print the section marker
  • A new instance of sort takes over for the content lines following the section marker
  • Call close on sort at the end to take care of trailing content
0

For such tasks I often find it tedious to write a script. If it only needs to be done once and maybe for a few files, it can be done quite nicely using a macro if you open the file in vim and type:

  • GoFAKE SECTION<ESC>: add a fake section in the end, and make sure this is at the start of the line (you may have cindent or autoindent enabled). This is required for sorting the last section as well.
  • gg: back go to the start of the file, then the file starts with a section go down one line with j
  • qq: start recording a macro to register q
  • v: start selection
  • /^\S\+<Enter>: search for the start of the next section
  • k: up one line
  • :!sort<Enter: sort the section
  • nj: go to the first element of the next section
  • q: stop recording the macro
  • @q: repeat the macro
  • 100@@: repeat the macro a few times (until no sections left)
  • dd: delete the last line of the file (the FAKE SECTION)

You may want to :set lazyredraw to speed up macro execution.

New contributor
MarcDefiant is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.