2

I have a file that contains data something like:

a 1
b 2
c,d,e 3,4,5
f 6
g,h 7,8 

... and I need the output like:

a 1
b 2
c 3
d 4
e 5
f 6
g 7
h 8

I can do this using python, but I want to try this using shell scripting. I was thinking of first isolating the rows that contain the delimiter ',' and then work ahead. I have used this so far to isolate the the rows:

perl -F, -ane 'print if $#F >=1' filename

... but I am stuck in the next step.

4
  • 1
    Why don't you do it with Python?
    – Kusalananda
    Jun 12 '18 at 7:08
  • Why do you say you want to try shell scripting and then post a perl program?
    – doneal24
    Jun 12 '18 at 16:22
  • I want to try shell scripting....however, the perl solution was the one available at that time. Also, I have done it with python, but want to get into shell scripting.
    – Anurag
    Jun 13 '18 at 7:14
  • You have so many working answers. Why don't you accept the one that helped you to most to close this question?
    – Socowi
    May 11 '19 at 9:08
3

With perl

$ perl -lane '@v=split/,/,$F[1]; $i=0;
              print "$_ $v[$i++]" for split/,/,$F[0]' ip.txt
a 1
b 2
c 3
d 4
e 5
f 6
g 7
h 8

Split either the first/second column, initialize index counter and then print the pairs by iterating over splitting the other column

-a option will auto-split the input line on whitespaces and have the results in @F array

2
  • this will take more time because of splitting in a loop? isn't it?
    – Anurag
    Jun 13 '18 at 7:31
  • splitting is done once.. loop happens on the result of split, which is an array with one or more elements.. if the solution is slow, you'll have to benchmark and improve as needed..
    – Sundeep
    Jun 13 '18 at 11:28
2

Awk solution (assuming that number of "keys" (contained in the 1st field $1) would always correspond to number of "values" (contained in the 2nd field $2)):

awk '$1 ~ /,/{
         len = split($1, keys, ",");
         split($2, vals, ",");
         for (i = 1; i <= len; i++) print keys[i], vals[i];
         next
    }1' file

The output:

a 1
b 2
c 3
d 4
e 5
f 6
g 7
h 8
3
  • 1
    or just awk '{split($1,keys,","); split($2,vals,","); for (x in keys) print keys[x], vals[x]}' infile Jun 12 '18 at 7:11
  • @αғsнιη, that is not optimal approach, cause it tries to call split twice even in cases when it's not needed Jun 12 '18 at 7:13
  • Yes the keys correspond to the values. This looks great. thank you
    – Anurag
    Jun 13 '18 at 7:16
2

One way to do it using the sed editor is:

sed -e '
   s/,/\n/
   s/\(\n.*[[:blank:]]\)\([^,]*\),/ \2\1/
   P;D
' input.file

Working:

  • Clip the leading comma-separated element from the 2nd field.
  • Then append this element in the 1st field's leading comma-separated element.
  • Print the leading element of the 1st field and delete it after this.
  • Repeat this procedure with what remains in the pattern space till it is empty.

Another method using Perl is:

perl -lane '
   my($kref, $vref, %h) = map { [split /,/] } @F[0,1];
   @h{@$kref} = @$vref;
   print "$_  $h{$_}" for @$kref;
' input.file

Another way is shown here:

perl -lpe 'print "$1 $3" while s/^([^,]*),(.*\h)([^,]*),/$2/' input.file

Working:

  • Look at the regex this way: (Perl reads in a line at a time from the file) then:
    • ^([^,]*) shall pick the first-field's leading comma-separated element of the current line. This is stored in $1 variable.
    • (.*\h) shall preserve, for the next iteration of the while loop, the intermediate contents beginning from the second comma-separated element of the first field to the beginning of the second comma-separated element of the second field. This is stored in the $2 variable.
    • ([^,]*) shall pick the leading comma-separated element from the second field of the current line.This gets stored in the $3 variable.
    • Now "$1 $3" is printed out to STDOUT and the line is shrunk to $2. The while loop now performs the operation all over again on this edited line, which is $2 of the previous line,..... this repeats till the s/// succeeds. The failure comes when we run out of commas. At which point , what remains in the line, "c 5" is printed to STDOUT by the default behavior of perl in the -p mode.
  • Pluck out the leading comma-separated elements from the first and second fields.
  • Print those elements and also shrink the current record by removing.
  • Loop over the current record while it's having 2 commas.
  • Last pair is auto-printed due to the -p option of Perl.

perl -lane '
   my($kref, $vref) = map { [split /,/] } @F;
   print shift @$kref, " ", shift @$vref while @$kref && @$vref;
' input.file

Working:

  • keys are stored in an array @$kref, corresponding values in @$vref. Note no hashes involved here.
  • print the top of the arrays simultaneously and then remove the top ...rinse,repeat while both the arrays are non-empty.

Output:

a 1
b 2
c 3
d 4
e 5
f 6
g 7
h 8
2
  • I do not understand the perl -lpe 'print "$1 $3" while s/^([^,]*),(.*\h)([^,]*),/$2/' input.file . So this loop will take into account one par at a time?
    – Anurag
    Jun 13 '18 at 7:39
  • @Anurag pls. see the explanation in the reply above. Jun 13 '18 at 8:42
1

Using awk:

awk '{gsub(","," "); for(i=0;i<NF/2;i++) print $(i+1),$(i+1+NF/2)}' file     

After removing the comma from each line, the script loops through half of the parameters of each line to print the first fields together with the field from the second half of the line.

1
  • So, if i understand correctly, the gsub replaces the comma with a whitespace and now every delimiter is a space. And then the loop does its work. Nice...Thank you
    – Anurag
    Jun 13 '18 at 7:27
1

"using shell scripting" -- this is bash:

while read -r key value; do
    IFS=, read -ra keys <<<"$key"
    IFS=, read -ra vals <<<"$value"
    for ((i=0; i < ${#keys[@]}; i++)); do
        echo "${keys[i]} ${vals[i]}"
    done
done <<END
a 1
b 2
c,d,e 3,4,5
f 6
g,h 7,8 
END
0
0

With gnu sed

sed -E ':A;s/([^,]*),([^ ]*) ([^,]*),(.*)/\1 \3\n\2 \4/;tA' infile
0

Sorry for the obfuscation...

perl -pe '1 while s/(.*),(.*\h)(.*),/$1 $3\n$2/' infile

Each substitution extracts last pair:

a,b,c 1,2,3  →  a,b 1,2  →  a 1
                c 3         b 2
                            c 3
0

There are a lot of solutions here programmed in perl, awk, sed, bash, ... but no script-solution.

# /bin/bash
f="yourFile"
paste -d' ' <(cut -d' ' -f1 "$f" | tr , '\n') <(cut -d' ' -f2 "$f" | tr , '\n')

or written differently

# /bin/bash
f() { cut -d' ' -f"$1" yourFile | tr , '\n'; }
paste -d' ' <(f 1) <(f 2)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.