0

I have a text file which contains below string and I want to convert this into a CSV file.

I want to split below string:

location = /example/url/newsite/redirect {return 301 https://example.com/fr;}

... to values like as follows:

/example/url/newsite/redirect,301,https://example.com/fr

Currently I am using the below command

awk '{gsub(/;}/,"",$6); if ($1 == location) print $3","$6 }' redirections/*.redirections >> redirect-csv/redirect.csv

Ideally, we should not have to specify a filename. It should take all the *.redirections files in redirections/ and do a 1 to 1 mapping to generate files in redirect-csv.

redirections/
site1.redirections
site2.redirections

Run the magic command.

redirect-csv/
site1.csv
site2.csv
0

6 Answers 6

2

This kind of thing is best approached by using a regular expression to match the string against a pattern and extract the relevant parts.

echo "location = /example/url/newsite/redirect {return 301 https://example.com/fr;}" |
  sed -n 's/^location = \(.*\) {return \(3[[:digit:]]\{2\}\) \(.*\);}$/\1,\2,\3/p'

gives

/example/url/newsite/redirect,301,https://example.com/fr

Or if the fields may contain characters that are special in the CSV format (like ,, "):

perl -MText::CSV -lne '
  BEGIN{$c = Text::CSV->new}
  if (/^location = (.*) \{return (3\d\d) (.*);\}$/) {
    $c->print(STDOUT, [$1, $2, $3])
  }'
1
  • Thank you so much for the answer.
    – Yogi
    Apr 30, 2019 at 5:48
1

Following your change, here is a solution based on the first one

for file in redirections/*.redirection; do fname=$(basename $file); awk '{gsub(/;}/,"",$6); print $3" & "$5" & "$6 }' $file > redirect-csv/${fname/.redirection/.csv}; done
1
  • Thanks @darxmurf - Currently I am using the "for my_file in redirections/*.redirections; do fname=$(basename $my_file); awk -v OFS=, '{print $3,$6}' $my_file | tr -d }\; >> redirect-csv/${fname%redirections}csv done" this command but I am unable to add "if ($1 == location)" this condition in above command, do you have any suggestion for it.
    – Yogi
    Apr 30, 2019 at 11:18
0

hmm, if this line is always following the same model, one line with awk ?

echo "location = /example/url/newsite/redirect {return 301 https://example.com/fr;}" | awk '{gsub(/;}/,"",$6); print $3" & "$5" & "$6 }'

to redirect that in "myfile.csv"

awk '{gsub(/;}/,"",$6); print $3" & "$5" & "$6 }' list.txt >> myfile.csv

where list.txt contains your lines

6
  • Thanks @darxmurf, but how can I write this values to CSV using awk, do you have any suggestion for this ?
    – Yogi
    Apr 29, 2019 at 14:23
  • Well, if your lines are in "list.txt": awk '{gsub(/;}/,"",$6); print $3" & "$5" & "$6 }' list.txt >> myfile.csv
    – darxmurf
    Apr 29, 2019 at 14:26
  • Thank you so much for the answer, it really helps me a lot.
    – Yogi
    Apr 30, 2019 at 5:47
  • Can we write multiple awk commands in file & execute that file ?
    – Yogi
    Apr 30, 2019 at 8:20
  • euh, yes, just open a file, paste your awk commands line by line and then execute your file
    – darxmurf
    Apr 30, 2019 at 9:08
0

if the pattern is known and its always the same then awk and tr could help:

awk -v OFS=, '{print $3,$5,$6}' site1.redirections | tr -d }\; >> redirect-csv/site1.csv

awk will make the csv with comma as saperator and then tr will eliminate the uneeded characters. Finaly the redirection of your output will go to the needed file.

edit1: for iterate over multiple files and manipulate the file name with similar consideration that name follows a pattern:

for my_file in site*.redirections;do
awk -v OFS=, '{print $3,$5,$6}' $my_file | tr -d }\; >> redirect-csv/$(echo $my_file | cut -d. -f1).csv
done

edit2: To redirect output to a file with same name but instead of the .redirection extension to .csv extension:

for my_file in *.redirections;do
awk -v OFS=, '{print $3,$5,$6}' $my_file | tr -d }\; >> redirect-csv/${my_file%redirections}csv
done

edit3: Thanks to @darxmurf:

for my_file in *.redirections;do
awk -v OFS=, '{if ($1 == "location") print $3,$5,$6}' $my_file | tr -d }\; >> redirect-csv/${my_file%redirections}csv
done
9
  • 1
    Thank you so much for the answer, your code works as expected. but how I can pick up dynamic filename from redirections folder & pass this filename to CSV file ?
    – Yogi
    Apr 30, 2019 at 10:16
  • @Yogi have edited the answer with a potential solution that i would personlay followed.
    – igiannak
    Apr 30, 2019 at 10:22
  • Thanks @GiannakopoulosJ, I have using the below command & my redirections folder contains 89 file and name of the files are "be.total.com.redirections", "m.total.fr.redirections" & "campus.total.com.redirections" I want to convert that filename same with .CSV extension added to it I am using this code ->"for my_file in *.redirections; do awk -v OFS=, '{print $3,$6}' $my_file | tr -d }\; >> redirect-csv/$(echo $my_file).csv done" so I have tried above code but it only converts first file in redirections folder.
    – Yogi
    Apr 30, 2019 at 10:32
  • 1
    as @GiannakopoulosJ says, it's getting complicated... anyway, for your if statement, try this: {if ($1 == "location") {print $3,$6}}
    – darxmurf
    Apr 30, 2019 at 11:32
  • 1
    There are some commented lines in files which starts with "#location" so I want only those lines which starts with "location" not with "#locations". You are right in this scenario, I think I am messing this up. I should use your code with extra condition of "if ($i == "location")"
    – Yogi
    Apr 30, 2019 at 11:33
0
  1. An error -- this code, abbreviated from the OP's question:

    awk '...stuff...' r/*.red >> r-c/red.csv
    

    ...would input many files, change them, and output a single extra-long red.csv file. The >> redirect behaves rather like cat foo/* > bar/baz.

  2. Using the same abbreviations to avoid word-wrap, with basename and xargs, generate cp commands to copy the r/ files to r-c/ while changing their names, and edit those new files in-place with GNU sed:

    basename -a -s r/*.red | xargs -L 1 -I {}   cp  r/{}.red  r-c/{}.csv
    sed -i 's/^[^=]*= \|\;}$//g;s/ [^ ]* /,/;s/ /,/'  r-c/*.csv
    
0

Tried with below command and it worked fine

command:

echo "/example/url/newsite/redirect {return 301 https://example.com/fr;}"| sed "s/[{}]//g"|  sed "s/return/,/g"| sed "s/301/&,/g"| sed -r "s/\s+//g"| sed "s/;$//g"

output

echo "/example/url/newsite/redirect {return 301 https://example.com/fr;}"| sed "s/[{}]//g"|  sed "s/return/,/g"| sed "s/301/&,/g"| sed -r "s/\s+//g"| sed "s/;$//g"

/example/url/newsite/redirect,301,https://example.com/fr

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .