1

I am new to UNIX and am trying to learn basic level text processing in UNIX so maybe this question may seem very basic but I would really appreciate any guidance.

I have a text file as below

A 1
B 2
C 9
D 1
A 5
B 3
C 4
A 6
C 7
A 5
C 1

for which I was able to piece together a command with some help as below

cat | awk 'BEGIN{OFS=FS=" "}!(($1,$2)in c){c[$1,$2]=1;r[$1]=r[$1] OFS $2}
END{for(i in r){print i r[i]}}'

inside a shell script to get it to the format below:

A 1 5 6 5
B 2 3
C 9 4 7 1
D 1

My shell script will also take an argument k (=7 say)

In this case, my script should sample 1 element from each box (A,B,C,D) until it has a total of 7 elements sampled. This will be like follows: First pick 1 from A, then 2 from B, 9 from C, 1 from D, 5 from A, 3 from B, 4 from C and display

A 1 5
B 2 3
C 9 4
D 1

If instead I had k = 9 as my input, then my output should be

A 1 5 6
B 2 3
C 9 4 7
D 1
  • You will always have 4 A,B,C&D Boxes only? And all Boxes granted will exist in your txt file or you can have for example only A&C Boxes? – αғsнιη Mar 10 '18 at 23:34
1

In a high-level language you'd use an array of arrays, but bash doesn't have those. Problems that suggest multilevel data structures, as this one does, tend to be very tedious to solve in the shell.

But since your goal is to learn Unix text processing, and not Python, let's solve it in the shell.

In this solution, we read through the file once to get the row headers, and then again as many times as it takes to collect the requisite number of elements. We keep two arrays: outrow is an array of output lines with each line being appended to as we go; cursor is an array of integers that stores our place on each line.

Note that this script will loop forever if there aren't enough elements to satisfy the request. Solving that is left as an exercise for the reader.

#!/bin/bash
k=$1
input=input.txt
declare -a outrow
declare -a cursor
K=0
n=0
while read line
do
    outrow[$n]=${line%% *}
    cursor[$n]=1
    (( n++ ))
done < $input

while [[ $K -lt $k ]]
do
    n=0
    while read line
    do
        declare -a col=( $line )
        if [[ ${#col[@]} -gt ${cursor[$n]} ]]
        then
            outrow[$n]+=" ${col[ ${cursor[$n]} ]}"
            (( cursor[$n]++ ))
            (( K++ ))
            [[ $K -lt $k ]] || break
        fi
        (( n++ ))
    done < $input
done

for row in "${outrow[@]}"
do
    echo "$row"
done
  • Hi OMG, thanks for your input. However I should have clarified this earlier. I actually dont have a txt file with this pattern but get this pattern in my script using awk. file=cat | awk 'BEGIN{OFS=FS=" "}!(($1,$2)in c){c[$1,$2]=1;r[$1]=r[$1] OFS $2} END{for(i in r){print i r[i]}}' I am unable to pipe the output from this awk command which gives the format I mentioned stupidly as in input text file. Could you please help. Thanks! – Sabrish Gopalakrishnan Mar 10 '18 at 15:26
  • @SabrishGopalakrishnan Redirect the output of your command to a file with > input.txt, then run the script above. – nohillside Mar 10 '18 at 16:36
1

Note: by changing the num variable, you can regulate the amount of elements.

gawk -v num=5 '
BEGIN {
    PROCINFO["sorted_in"] = "@ind_str_asc"
}
{
    ### 
    # Traverse throught input.txt from first to last line
    # and store all elements in the two-dimensional array - table
    # along the way, maintain the array of counters for each letter
    ###

    # The array of counters for each unique element from the first column.
    # In our case the indexes of array are capital letters (A, B, C, D)
    # and values are the amount of each letter occurrences.
    cnt_arr[$1]++

    # Two dimension array - table
    # it looks like chess board - rows named by letters (A, B, C, D)
    # and columns named by numbers (1, 2, 3, 4, 5... etc).
    # Its cells contains numbers from the second column.
    # For example, if letter A occurrences 5 times in the input.txt
    # then, the table will have the A row with 5 columns 
    table[$1][cnt_arr[$1]] = $2
}
# At this point, all lines from input.txt are processed
# and stored in the table
END {
    # Do needed number of iterations - specified by the num variable
    for(i = 0; i < num; i++) {

        # On each iteration run the inner loop,
        # which iterating through all rows in the table
        for(row_name in table) {

            # Check each cell - if it is non-empty
            # add its value to the result_arr[row_name], separated by OFS.
            # OFS - output field separator, the space by default
            if(table[row_name][i]) {
                result_arr[row_name] = result_arr[row_name] OFS table[row_name][i]
                # and count the number of succesful occurences
                cnt++
            }

            # If count of non-empty cells equals to the num variable
            # or equals to the NR (number of records|lines)
            # print the result_arr and exit
            if(cnt == num || cnt >= NR) {
                for(i in result_arr) {
                    print i result_arr[i]
                }
                exit
            }
        }
    }
}' input.txt

Information about PROCINFO["sorted_in"] = "@ind_str_asc" line is here.


Input

A 1
B 2
C 9
D 1
A 5
B 3
C 9
A 6
C 7
A 5
C 1

Output

A 1 5
B 2
C 9
D 1
  • 1
    Great trick with the @ind_str_asc but I'm afraid this won't work if you don't have at least one occurrence of each letter in the 1st 7 lines. If, for instance, the first case of C is after the 7th line, you'll have no output for C. – terdon Mar 10 '18 at 16:59
  • @terdon From my understanding, OP doesn't needed the algorithm described by him. In really he want just pick the first seven elements (lines) and put them in rows, grouped by the first column (capital letters). So, if the C letter doesn't occur in the first 7 lines, then it shouldn't appear in the resulting output. – MiniMax Mar 10 '18 at 17:18
  • 1
    The question isn't very clear, but I don't think you're right: "my script should sample 1 element from each box(A,B,C,D) until have a total of 7 elements sampled." – terdon Mar 10 '18 at 17:49
  • 1
    @MiniMax Thanks for your help. However the code does not sample one element from each box (A,B,C,...) but takes the first n occurences in the first txt file – Sabrish Gopalakrishnan Mar 10 '18 at 18:48
  • @SabrishGopalakrishnan I made a new solution. Check the update. – MiniMax Mar 10 '18 at 20:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.