Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

Let's say I have a file that looks something like this:

23: a, b, c, d
24: b, d, f
25: c, g

and I want to get output like this:

23.a
23.b
23.c
23.d
24.b
24.d
24.f
25.c
25.g

Of course it's not too hard to just bang something out, but I was wondering if there was a slick one-liner using something like awk.

share|improve this question

7 Answers 7

up vote 19 down vote accepted

Maybe something like:

sed 's/: /./;s/\(\([^.]*\.\)[^,]*\), /\1\
\2/;P;D'

That's two lines (\<LF> can be replaced with \n with some sed implementations).

The D command is one way to implement while loops in sed. It removes the first line of the pattern space and as long as there's something remaining in the pattern space starts all over again with what's left. So the above can be read as:

do {
  - change ": " to "." so we start with "23.a, b, c"
  - change "23.x, y, z" to "23.x\n23.y, z"
  - print the first line ("23.x"): P
  - remove it
} while (pattern space is not empty)

We don't need the first s command to be part of the loop, but to avoid that, we'd need to use a more verbose type of loop like using labels (:) and branching commands (b, t).

share|improve this answer
3  
Nice, but we like explanations of that the code does ... –  Bananguin Mar 24 '13 at 20:57
    
This wins for sheer cleverness. –  Daniel McLaury Mar 24 '13 at 21:09

Never mind, I just remembered the awk split function, which makes this pretty straightforward.

awk -F ":" '{
  split($2, ps, ",");
  for (i in ps) {
    gsub(" ", "",ps[i]);
    print $1 "." ps[i];
  }
}'

(the gsub is stripping extraneous whitespace.)

Thanks for the other answers, though.

share|improve this answer
    
I guess the same as @Stephane Chazelas, but more blunt: awk -F ':' '{gsub(/[^a-z]/,",",$2);gsub(/,+/,"\n"$1".",$2);gsub(/^\n/,"",$2);print $2}' –  XzKto Mar 25 '13 at 7:48
1  
Usually I prefer a more complex FS in such cases: awk -F '[:,]' '{for(i=2;i<=NF;i++)printf"%s%s\n",$1,$i}'. –  manatwork Mar 25 '13 at 10:49
1  
Note that not all awk implementations guarantee that your i in ps expression will result in looping in order through the array. For example mawk does, but gawk not. –  manatwork Mar 25 '13 at 10:52
    
That's bizarre... what possible advantage is there to looping through in a different order? –  Daniel McLaury Mar 26 '13 at 3:44
    
awk's arrays are associative arrays and associative arrays usually work like that (for example HashMap in Java, hash in Perl, dict in Python, Hash in Ruby before 1.9.2, array in Tcl). That is because the internal representation of the data. Programmers has a related question, Is an assocative array ordered?, –  manatwork Mar 26 '13 at 9:19

Here's a Perl one:

 perl -nle '/(.+?):\s*(.+)/; print "$1.$_" for split(/[,\s]+/,$2);' foo.txt

EXPLANATION:

  • perl -nle : this tells Perl to parse the input file one line at a time (-n), execute the script given as an argument to -e and add a new line (\n) to every printed string (-l).

  • /(.+?):\s*(.+)/ : Match the first characters until the first colon that is followed by 0 or more spaces (:\s*), then the rest of the line. The parentheses are Perl syntax for capturing patterns, the two matches are saved as $1 and $2.

  • split(/[,\s]*/,$2); : this will split $2 (the second matched pattern from the match operation above) at , and/or spaces, creating an anonymous array.

  • print "$1.$_" for split() : iterate through the anonymous array created by the split above, saving each array member as $_ and print it along with $1 (the first pattern captured in the first step) and a dot ..

share|improve this answer
    
I recommend print "$1.$_\n" for ..." instead of map { print "$1.$_\n" } .... –  Christoffer Hammarström Mar 25 '13 at 12:21
    
Also, with -l you don't need "\n". But might be even better to use -E and say. –  Christoffer Hammarström Mar 25 '13 at 12:23
    
@ChristofferHammarström, interesting recommendation. What is the reason? –  manatwork Mar 25 '13 at 12:24
    
map builds and returns a list of values. Here it is being used as a for or foreach. –  Christoffer Hammarström Mar 25 '13 at 12:41
    
@ChristofferHammarström don't forget that say is new (perl >=5.10 I think) and may not always be available. I used map cause this is a one liner and I wanted it shorter. I realised it is not street legal in a CompSci department but it really makes no difference in this context. –  terdon Mar 25 '13 at 12:41

Here's a Ruby one:

ruby -ane '$F.drop(1).each{|f| puts $F.first.gsub(":",".")+f.chomp(",")}' <file.txt

Explanation

  • ruby -ane : this tells Ruby to auto split the lines, one line at time, and execute the argument as a script.

  • In an auto split file $F is an array of the split result.

  • drop(1) skips the first field (the row number) and .each loops over the following fields.

  • gsub replaces the : and chomp removes a trailing separator from the string.

share|improve this answer

An awk one-liner that I think is a bit more elegant than the other awk solution:

awk -F'[:, ]+' '{for(i=2;i<=NF;i++)printf $1"."$i"\n"}' file.in

It takes advantage of the fact that the awk field separator is a regex.

share|improve this answer

Perl:

perl -nE '($first,$rest)=split ": "; say "$first.$_" for split ", ", $rest'

Splits the line into the first number and the rest, then prints "$first.$_" for each of the letters.

share|improve this answer

How about a simple bourne shell script (mostly):

tr -d ':,' file.txt | while read p r; do for i in $r; do echo "$p.$i"; done; done

The "tr" command just cleans the colons (:) and commas (,) out - this answer relies on there being whitespace in the data (which the sample data has - otherwise you need to use sed to convert : and , into whitespace instead of tr).

The output of "tr" is piped into the outer loop "while read...; do ...; done" which reads lines and breaks them into two, at the first occurence of whitespace (or rather the contents of "$IFS" - the shell input field separator, which defaults to whitespace), leaving the prefix in "$p" and the rest of the line in "$r".

The inner loop "for i in ...; do ...; done" then breaks the contents of "$r" up at whitespace ("$IFS") and puts each item into "$i" before executing the echo command.

EDIT: see comments - you don't need "tr" at all ... the colons and commas can be cleaned by including them in the IFS variable like so:

OIFS="$IFS"; IFS=":,       "; while read p r; do 
 for i in $r; do echo "$p.$i"; done; done <file.txt; IFS="$OIFS"

all done within the shell - no calls to external programs ... (unless echo is not builtin). Note the IFS= above has a space and a tab char. Also note that the $r in the second for loop does not have quotes around it - this is deliberate so the shell will split it on whitespace.

share|improve this answer
    
You could tr ':,' ' ' | tr -s ' '... –  vonbrand Mar 30 '13 at 14:27
    
yes - would probably be cheaper than sed, but you don't need the second tr - the shell's read command will treat sequences of whitespace as a single separator ... and this just made me think - we don't need "tr" at all! Here is an entirely Bourne shell script solution: OIFS="$IFS"; IFS=":, "; while read p r; do for i in $r; do echo "$p.$i"; done; done; IFS="$OIFS" never have to leave the shell ... yay! –  Murray Jensen Apr 1 '13 at 3:05

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.