2

I have a lot of files and need to merge strings among them by groups of three files. Each file has ~100-500 rows and the resulting file is ~100-200Mb.

The ppc keyword generator does the same thing, but it is very slow; after 1 million keywords, it takes 2-3 hours for 1 result file.

f1

Acapella Audio Arts
Acoustic Smart

f2

australian
Australia
America

f3

Round Grille
Square Grille
grille
defence grille

result

Acapella Audio Arts australian Round Grille
Acapella Audio Arts australian Square Grille
Acapella Audio Arts australian grille
Acapella Audio Arts australian defence grille
Acapella Audio Arts Australia Round Grille
Acapella Audio Arts Australia Square Grille
Acapella Audio Arts Australia grille
Acapella Audio Arts Australia defence grille
Acapella Audio Arts America Round Grille
Acapella Audio Arts America Square Grille
Acapella Audio Arts America grille
Acapella Audio Arts America defence grille
Acoustic Smart australian Round Grille
Acoustic Smart australian Square Grille
Acoustic Smart australianе grille
Acoustic Smart australian defence grille
Acoustic Smart Australia Round Grille
Acoustic Smart Australia Square Grille
Acoustic Smart Australia grille
Acoustic Smart Australia defence grille
Acoustic Smart America Round Grille
Acoustic Smart America Square Grille
Acoustic Smart America grille
Acoustic Smart America defence grille 
etc.....
1
  • there was a little problem with dos/windows EOL, dos2unix util helps with convert
    – Michael Z
    Nov 5, 2015 at 10:12

1 Answer 1

3

Just with bash:

while IFS= read -u3 -r product; do
    while IFS= read -u4 -r region; do
        while IFS= read -u5 -r option; do
            echo "$product $region $option"
        done 5<f3
    done 4<f2
done 3<f1

This version will have less file i/o (read each file only once) so it may be faster

mapfile -t products < f1
mapfile -t regions  < f2
mapfile -t options  < f3
for product in "${products[@]}"; do 
    for region in "${regions[@]}"; do
        for option in "${options[@]}"; do
            echo "$product $region $option"
        done
    done
done
1
  • awesome, man you are awesome. i'll pray for you.
    – Michael Z
    Nov 4, 2015 at 17:16

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .