3

I have a file with the following format

INSERT INTO table1(field1,field2,field3) VALUES('values1','value2','value3');
INSERT INTO table1(field1,field2,field3) VALUES('other_values1','other_value2','other_value3');
INSERT INTO table1(field1,field2,field3) VALUES('another_values1','another_value2','another_value3');
INSERT INTO table2(table2_field1,table2_field2,table2_field3,field4) VALUES('table2_values1','table2_value2','table2_value3');
INSERT INTO table2(table2_field1,table2_field2,table2_field3,table2_field4) VALUES('other_table2_values1','other_table2_value2','other_table2_value3');
INSERT INTO table2(table2_field1,table2_field2,table2_field3,table2_field4) VALUES('another_table2_values1','another_table2_value2','another_table2_value3','another_table2_value4');

I want this output

SELECT * FROM table1 WHERE field1='values1' AND field2='values2' AND field3=='values3';
SELECT * FROM table1 WHERE field1='other_values1' AND field2='other_values2' AND field3=='other_values3';
SELECT * FROM table1 WHERE field1='another_values1' AND field2='another_values2' AND field3=='another_values3';
SELECT * FROM table2 WHERE table2_field1='table2_values1' AND table2_field2='table2_values2' AND table2_field3=='table2_values3' AND table2_field4=='table2_values4';
SELECT * FROM table2 WHERE table2_field1='table2_values1' AND table2_field2='table2_values2' AND table2_field3=='table2_values3' AND table2_field4=='table2_values4';
SELECT * FROM table2 WHERE table2_field1='table2_values1' AND table2_field2='table2_values2' AND table2_field3=='table2_values3' AND table2_field4=='table2_values4';

What I've done so far is

cat test_inserts |awk -F '[()]' '{print $1 " WHERE "$2 $4}' |sed 's/INSERT INTO /SELECT * FROM /g'

and it gives me the following output

SELECT * FROM table1 WHERE field1,field2,field3'values1','value2','value3'
SELECT * FROM table1 WHERE field1,field2,field3'other_values1','other_value2','other_value3'
SELECT * FROM table1 WHERE field1,field2,field3'another_values1','another_value2','another_value3'
SELECT * FROM table2 WHERE table2_field1,table2_field2,table2_field3,field4'table2_values1','table2_value2','table2_value3'
SELECT * FROM table2 WHERE table2_field1,table2_field2,table2_field3,table2_field4'other_table2_values1','other_table2_value2','other_table2_value3'
SELECT * FROM table2 WHERE table2_field1,table2_field2,table2_field3,table2_field4'another_table2_values1','another_table2_value2','another_table2_value3','another_table2_value4'
  • Don't cat file | awk 'stuff'; just awk 'stuff' file. – DopeGhoti Nov 15 '17 at 18:31
6

Complex AWK solution:

awk -F'[()]' '{ sub(/INSERT INTO */,"",$1); 
                printf "SELECT * FROM %s WHERE ",$1;
                len=split($2, f, ","); split($4, v, ","); 
                for (i=1; i<=len; i++) printf "%s=%s%s", f[i], v[i], (i==len? ";":" AND ");
                print "" 
              }' test_inserts
  • -F'[()]' - complex field separator
  • sub(/INSERT INTO */,"",$1) - remove INSERT INTO phrase from the 1st field (to extract a table name)
  • printf "SELECT * FROM %s WHERE ",$1 - print the start of SQL statement containing a table name
  • split($2, f, ",") - split the 2nd field by separator , to obtain field names (f becomes an array of field names)
  • split($4, v, ",") - split the 4th field by separator , to obtain field values (v becomes an array of field values)

The output:

SELECT * FROM table1 WHERE field1='values1' AND field2='value2' AND field3='value3';
SELECT * FROM table1 WHERE field1='other_values1' AND field2='other_value2' AND field3='other_value3';
SELECT * FROM table1 WHERE field1='another_values1' AND field2='another_value2' AND field3='another_value3';
SELECT * FROM table2 WHERE table2_field1='table2_values1' AND table2_field2='table2_value2' AND table2_field3='table2_value3' AND field4=;
SELECT * FROM table2 WHERE table2_field1='other_table2_values1' AND table2_field2='other_table2_value2' AND table2_field3='other_table2_value3' AND table2_field4=;
SELECT * FROM table2 WHERE table2_field1='another_table2_values1' AND table2_field2='another_table2_value2' AND table2_field3='another_table2_value3' AND table2_field4='another_table2_value4';
4

Here's an alternative in Python, in case you're into that sort of thing (more verbose but more legible than awk, at least for me):

#!/usr/bin/env python2
# -*- coding: ascii -*-
"""transform_query.py"""

import sys
import re

# Open the data file specified by the user
with open(sys.argv[1], 'r') as query_file:
    for row in query_file.readlines():

        # Regular expression to extract table name, field names, and values from each line
        match = re.search(
            r'^INSERT INTO '
                r'(?P<table>table\d+)\((?P<fields>[\w,]+)\) '
                r'VALUES\((?P<values>[^()]+)\);$',
            row.strip()
        )
        if match:

            # Store the table name (not necessary)
            table = match.group('table')

            # Split the fields string into a list
            fields = match.group('fields').split(',')

            # Split the values string into a list
            values = match.group('values').split(',')

            # Recombine the strings into a SELECT statement
            # and print the result
            print(
                "SELECT * FROM {} WHERE {};".format(
                    table,
                    ' AND '.join(
                        ['='.join([field, value]) for field, value in zip(fields, values)]
                    ),
                )
            )

Run it:

python transform_query.py query.sql

And here's the output:

SELECT * FROM table1 WHERE field1='values1' AND field2='value2' AND field3='value3';
SELECT * FROM table1 WHERE field1='other_values1' AND field2='other_value2' AND field3='other_value3';
SELECT * FROM table1 WHERE field1='another_values1' AND field2='another_value2' AND field3='another_value3';
SELECT * FROM table2 WHERE table2_field1='table2_values1' AND table2_field2='table2_value2' AND table2_field3='table2_value3';
SELECT * FROM table2 WHERE table2_field1='other_table2_values1' AND table2_field2='other_table2_value2' AND table2_field3='other_table2_value3';
SELECT * FROM table2 WHERE table2_field1='another_table2_values1' AND table2_field2='another_table2_value2' AND table2_field3='another_table2_value3' AND table2_field4='another_table2_value4';
1

If you are needing explanation, tell me and I will add comments into the code.

Requirements:

  1. The number of fields and values should match.
  2. Field names and field values should not contain these characters: ,)(; - comma, parentheses, semicolon.

gawk '{
    num = patsplit($3$4, arr, /[^,)(;]+/);
    num /= 2;

    printf("SELECT * FROM %s WHERE ", arr[1]);

    for(i = 2; i <= num; i++) {
        printf("%s=%s", arr[i], arr[num + i]);

        printf (i < num) ? " AND " : ";\n";
    }
}' input.txt

Explanation:

This string is picked as example: INSERT INTO table1(field1,field2,field3) VALUES('values1','value2','value3');

  1. num = patsplit($3$4, arr, /[^,)(;]+/);
    • patsplit(s, a, r) - split the string s into the array a on the regular expression r, and return the number of fields.
    • $3$4 - $3 is table1(field1,field2,field3), $4 is VALUES('values1','value2','value3');. So, after concatenating, we have this string: table1(field1,field2,field3)VALUES('values1','value2','value3');.
    • then, split it to arr by this regex /[^,)(;]+/. It means: all characters, except comma, parentheses, semicolon. Therefore now: arr[1] is table1, arr[2] is field1, arr[5] is VALUES, so on.
  2. num /= 2; for algorithm used in this script, the half of the num variable is needed.
  3. printf("SELECT * FROM %s WHERE ", arr[1]); - print the first element of the arr, which is the table name. In our case, it is the table1.
  4. printf("%s=%s", arr[i], arr[num + i]); - the algorithm. Print one element from the first half of the arr and corresponding element from the second half of the arr. That is, arr[2] and arr[6], arr[3] and arr[7], arr[4] and arr[8].
  5. printf (i < num) ? " AND " : ";\n"; - The ternary operator. If it is not the last iteration, print AND, else print ;\n (final semicolon and newline).

Input (I corrected your input, because it had errors - the number of field names and field values didn't match).

INSERT INTO table1(field1,field2,field3) VALUES('values1','value2','value3');
INSERT INTO table1(field1,field2,field3) VALUES('other_values1','other_value2','other_value3');
INSERT INTO table1(field1,field2,field3) VALUES('another_values1','another_value2','another_value3');
INSERT INTO table2(table2_field1,table2_field2,table2_field3,table2_field4) VALUES('table2_values1','table2_value2','table2_value3','table2_value4');
INSERT INTO table2(table2_field1,table2_field2,table2_field3,table2_field4) VALUES('other_table2_values1','other_table2_value2','other_table2_value3','other_table2_value4');
INSERT INTO table2(table2_field1,table2_field2,table2_field3,table2_field4) VALUES('another_table2_values1','another_table2_value2','another_table2_value3','another_table2_value4');

Output

SELECT * FROM table1 WHERE field1='values1' AND field2='value2' AND field3='value3';
SELECT * FROM table1 WHERE field1='other_values1' AND field2='other_value2' AND field3='other_value3';
SELECT * FROM table1 WHERE field1='another_values1' AND field2='another_value2' AND field3='another_value3';
SELECT * FROM table2 WHERE table2_field1='table2_values1' AND table2_field2='table2_value2' AND table2_field3='table2_value3' AND table2_field4='table2_value4';
SELECT * FROM table2 WHERE table2_field1='other_table2_values1' AND table2_field2='other_table2_value2' AND table2_field3='other_table2_value3' AND table2_field4='other_table2_value4';
SELECT * FROM table2 WHERE table2_field1='another_table2_values1' AND table2_field2='another_table2_value2' AND table2_field3='another_table2_value3' AND table2_field4='another_table2_value4';
  • Thanks for your answer, and yes you can add an explanation, it's useful for me and other users. – Emilio Galarraga Nov 15 '17 at 22:59
  • 1
    Note that patsplit() makes this dependent on GNU awk, and won't work for BSD's "One True" awk (at least FreeBSD). – ovirt Nov 16 '17 at 4:16
  • @user3333911 Explanation added. – MiniMax Nov 16 '17 at 18:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.