Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

So what I'm trying to do is for this: looking at the example CSV:


After the script is run, the output should look similar to:


One line of original CSV file that contained additional delimiters is now converted into 3 lines because there were three additional fields within the 4th column.

I've been working on it all day and this is what I've come up with. Will it work?


   if [ $# -ne 1 ];
     then echo "Usage: read.sh filename";
     exit 1;
   while read line
       set x $line
       while [ a -le #$]
           echo "`$1`,`$a`"
     done < $1
share|improve this question
why is this tagged as /tcsh when the script is ksh? also you need to be a lot clearer about what it is that you're trying to do. start from the beginning, describe your actual goal, not the (possibly wrong or less-than-optimal) method you've chosen to implement....there may be a much better/easier way to do it, probably involving awk – cas Sep 6 '12 at 1:47
Actually, the tcsh was a fat finger I did not see. – Marc Sep 6 '12 at 2:26
@Marc No offence, but if people ask me to do something in a cumbersome and complicated way, I will suggest an easy to understand and less comprehensive approach (such as using awk) if I know one. If you don't understand awk, you should learn it! – Bernhard Sep 6 '12 at 5:39
The job comes before the choice of tool always - one hammer does not fit all tasks. awk was designed for processing text that comes in columns, and I'd agree that it is better than shell for this task. – jw013 Sep 14 '12 at 17:38
Your single example leaves a few open questions: assuming the input is CSV, is it always the 4th field that has pipes? Are any of the fields allowed to contain commas? (Some CSV formats allow including commas via certain escape mechanisms like double-quotes) – jw013 Sep 14 '12 at 17:39

Use AWK Instead

This problem is much easier to solve with AWK. I tested this with GNU AWK; if you're using something else, you may need to tweak it a bit.

#!/usr/bin/gawk -f

BEGIN { FS = "|" }

  split( $3, array, /,/ )
  print $1 "," array[2]
  print $1 "," $2 "," array[2]
  print $1 "," $3

Store the script somewhere (e.g. parse.awk) and make sure it's executable. You can then call the script on a CSV file or on standard input, and collect your results like this:

$ echo '1,3917,3917,BGP=694|Ethernet=1610|LAG=3,Y' | parse.awk
share|improve this answer
Thank you for this suggestion. Will it be able to handle a line that is similar but has different numbers of comma and/or pipe delimited fields in it? – Marc Sep 6 '12 at 2:52
@Marc Maybe. You'll have to try it to find out whether it works with your irregular data. If you have irregular data then you may need to fix the data file, discard irregular records, or write a much more complex parser. Poorly-delimited data is always a problem for any programmatic solution. – CodeGnome Sep 6 '12 at 3:00
with a sample of only one line it's impossible to tell if it will work on all possible input lines. if there are variations then you need to show them. – cas Sep 6 '12 at 3:00
@CodeGnome thanks for your efforts. I'll take this to my friend. – Marc Sep 6 '12 at 3:17
i'm not bothered - i'm just pointing out that if you want help, then you need to provide the information required to do that. we can't read your mind and we can't see the actual input file. – cas Sep 6 '12 at 4:40

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.