0

I am a beginner to Unix shell script. I have a huge text file assume of more than 1 lakh records and each line has almost 600 characters. My requirement is to convert the flat file to csv format by putting delimiter as per required positions.

Sample File 1.txt

1234567890
9876543210

Delimiter at position : [1,3,5,9]

Expected O/p

1,23,45,6789,0
9,87,65,4321,0

I tried below code and was working when I am using 1.cfg with 3 records

1.cfg Content :

4
2
1

But as soon as I use file with 4 delimiters in config file ( 6 4 2 and 1 ) its not printing record no 2 which is 4 and printing record as 6 2 and 1

Here is my Sample Code

j=`cat 1.cfg |wc -l`
echo "Total split" $j
counter=0
set -x
for i in `cat 1.cfg`
do
counter=`expr $counter + 1`
echo "Printing value of counter " $counter

# If there is only one field in the config file
        if [ "$j" = 1 ]
        then
                COMMAND_FINAL=`echo "sed -i 's/./&,/$i' 1.txt"`
                #COMMAND_FINAL=`echo "`sed -i 's/./&,/$i' 1.txt`"`
        fi
# If there are more than one fields in the config file and for first record generating the command
        if [[ "$counter" != "$j" && "$counter" = 1 ]]
        then
                COMMAND=`echo "sed -i 's/./&,/$i;"`
                #COMMAND=`echo "`sed -i 's/./&,/$i;"`
                echo "Value of COMMAND VARIABLE is" $COMMAND
# For the 2nd fields untill 2nd last field generating the command
        elif [[ "$counter" != "$j" && "$counter" != 1 ]]
        then
                COMMAND1=`echo "s/./&,/$i;"`
                COMMAND2=$COMMAND$COMMAND1
                echo "Value of command :" $COMMAND
                echo "Value of command1 :" $COMMAND1
                echo "Value of command2 :" $COMMAND2
                #echo "If i is not 1 and i is not last Printing middle records" $COMMAND2
# For the last field generating the command
        elif [[ "$counter" = "$j" && "$j" != 1 ]]
        then
                COMMAND3=`echo "s/./&,/$i' 1.txt"`
                #COMMAND3=`echo "s/./&,/$i' 1.txt"`
                COMMAND_FINAL=$COMMAND2$COMMAND3
                echo "Final Command is " $COMMAND_FINAL
        fi
done
set -x
echo "$COMMAND_FINAL" > execute.ksh
chmod 755 execute.ksh
./execute.ksh
echo "Executing the final command"
3

With GNU awk:

awk '{$1=$1}1' FIELDWIDTHS='1 2 2 4 1' OFS=',' file

Or with GNU sed:

sed -r 's/^(.{1})(.{2})(.{2})(.{4})(.{1})$/\1,\2,\3,\4,\5/' file 

Output:

1,23,45,6789,0
9,87,65,4321,0

See: 8 Powerful Awk Built-in Variables – FS, OFS, RS, ORS, NR, NF, FILENAME, FNR

  • Thanks a lot for your help. It works and was so simple.... @Cyrus – Dhiren Aug 31 '19 at 8:43
  • Please take a look at: What should I do when someone answers my question? – Cyrus Aug 31 '19 at 8:48
  • Also can you let me know what if I want to not hard code the Fieldwidths and pass that as a argument ? I tried below code but it failed dhiren.shah7585@Dhiren /cygdrive/d/Unix_Practice $ cat 3.ksh echo "$*" params=$* awk '{$1=$1; print}' FIELDWIDTHS='$params' OFS=',' 2.txt dhiren.shah7585@Dhiren /cygdrive/d/Unix_Practice $ ./3.ksh 1 2 3 1 2 3 awk: fatal: invalid FIELDWIDTHS value, for field 1, near `$params' dhiren.shah7585@Dhiren /cygdrive/d/Unix_Practice $ – Dhiren Aug 31 '19 at 8:50
  • Use FIELDWIDTHS="$string". – Cyrus Aug 31 '19 at 8:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.