1

So I have a text file that's 20,000 "columns" long and 2 rows. Data looks something like this:

  FP1 amp     FP1 lat     FP2 amp       FP2 lat    FP3 amp       FP3 lat      AF1 amp         AF1 lat 
  4.1         231         2.2           232        1.3            233         4.4             234

Every 120 columns or so, the header values repeat themselves with different values in the 2nd row. How can I separate these "columns" every 120 columns, start a new line and continue doing this until the end of the file?

  • 1
    If the file only has 2 rows, how can something happen every 120 lines? Did you mean every 120 columns? – Barmar Aug 25 '16 at 19:43
  • 2
    Use a for() loop that iterates over the columns. When col % 120 == 0, print a newline. – Barmar Aug 25 '16 at 19:47
  • What separates the columns? Is it consistently a set number of spaces? If so, how many? – bashBedlam Aug 25 '16 at 23:30
1

Suppose we transpose the data with BSD rs:

$ rs -T
a b c d e f g h
1 2 3 4 5 6 7 8
[Ctrl-D][Enter]
a  1
b  2
c  3
d  4
e  5
f  6
g  7
h  8

Now we have it as two long columns, susceptible for awk processing. We can sweep through the data and accumulate a pair of strings from $1 and $2. Whenever NR modulo the desired row size is zero, we output the accumulated strings.

$ awk '{ hdrs = (hdrs ? hdrs OFS $1 : $1);
         vals = (vals ? vals OFS $2 : $2); }
       !(NR % 3) { print hdrs; print vals;
                   hdrs = vals = "" } 
       END { print hdrs; print vals; }'
a  1
b  2
c  3
d  4
e  5
f  6
g  7
h  8
[Ctrl-D][Enter]
a b c
1 2 3
d e f
4 5 6
g h
7 8

If a different output shape is required such as the following (individually wrapped headers and values, in a congruent pattern), it's not difficult to massage that out of the Awk code:

a b c
d e f
g h
1 2 3
4 5 6
7 8

Okay now the actual data is messy because it has field headings delimited with multiple spaces, and they contain spaces themselves.

Assuming that the fields contain only single internal spaces, and are always separated with mutiple spaces, what we can do is preprocess the data to replace the internal spaces with a non-whitespace character (that doesn't already occur in the data), such as a tilde (~). For instance using Sed:

$ sed -e 's/\([^ ]\) \([^ ]\)/\1~\2/g'
foo bar      xyzzy quux      alpha beta     gamma     delta
[Ctrl-D][Enter]
foo~bar      xyzzy~quux      alpha~beta     gamma     delta

Filtering back is easy with

tr '~' ' '

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.