converting insert into to select

Question

I have a file with the following format

INSERT INTO table1(field1,field2,field3) VALUES('values1','value2','value3');
INSERT INTO table1(field1,field2,field3) VALUES('other_values1','other_value2','other_value3');
INSERT INTO table1(field1,field2,field3) VALUES('another_values1','another_value2','another_value3');
INSERT INTO table2(table2_field1,table2_field2,table2_field3,field4) VALUES('table2_values1','table2_value2','table2_value3');
INSERT INTO table2(table2_field1,table2_field2,table2_field3,table2_field4) VALUES('other_table2_values1','other_table2_value2','other_table2_value3');
INSERT INTO table2(table2_field1,table2_field2,table2_field3,table2_field4) VALUES('another_table2_values1','another_table2_value2','another_table2_value3','another_table2_value4');

I want this output

SELECT * FROM table1 WHERE field1='values1' AND field2='values2' AND field3=='values3';
SELECT * FROM table1 WHERE field1='other_values1' AND field2='other_values2' AND field3=='other_values3';
SELECT * FROM table1 WHERE field1='another_values1' AND field2='another_values2' AND field3=='another_values3';
SELECT * FROM table2 WHERE table2_field1='table2_values1' AND table2_field2='table2_values2' AND table2_field3=='table2_values3' AND table2_field4=='table2_values4';
SELECT * FROM table2 WHERE table2_field1='table2_values1' AND table2_field2='table2_values2' AND table2_field3=='table2_values3' AND table2_field4=='table2_values4';
SELECT * FROM table2 WHERE table2_field1='table2_values1' AND table2_field2='table2_values2' AND table2_field3=='table2_values3' AND table2_field4=='table2_values4';

What I've done so far is

cat test_inserts |awk -F '[()]' '{print $1 " WHERE "$2 $4}' |sed 's/INSERT INTO /SELECT * FROM /g'

and it gives me the following output

SELECT * FROM table1 WHERE field1,field2,field3'values1','value2','value3'
SELECT * FROM table1 WHERE field1,field2,field3'other_values1','other_value2','other_value3'
SELECT * FROM table1 WHERE field1,field2,field3'another_values1','another_value2','another_value3'
SELECT * FROM table2 WHERE table2_field1,table2_field2,table2_field3,field4'table2_values1','table2_value2','table2_value3'
SELECT * FROM table2 WHERE table2_field1,table2_field2,table2_field3,table2_field4'other_table2_values1','other_table2_value2','other_table2_value3'
SELECT * FROM table2 WHERE table2_field1,table2_field2,table2_field3,table2_field4'another_table2_values1','another_table2_value2','another_table2_value3','another_table2_value4'

Answer 1

Complex AWK solution:

awk -F'[()]' '{ sub(/INSERT INTO */,"",$1); 
                printf "SELECT * FROM %s WHERE ",$1;
                len=split($2, f, ","); split($4, v, ","); 
                for (i=1; i<=len; i++) printf "%s=%s%s", f[i], v[i], (i==len? ";":" AND ");
                print "" 
              }' test_inserts

• -F'[()]' - complex field separator
• sub(/INSERT INTO */,"",$1) - remove INSERT INTO phrase from the 1st field (to extract a table name)
• printf "SELECT * FROM %s WHERE ",$1 - print the start of SQL statement containing a table name
• split($2, f, ",") - split the 2nd field by separator , to obtain field names (f becomes an array of field names)
• split($4, v, ",") - split the 4th field by separator , to obtain field values (v becomes an array of field values)

---

The output:

SELECT * FROM table1 WHERE field1='values1' AND field2='value2' AND field3='value3';
SELECT * FROM table1 WHERE field1='other_values1' AND field2='other_value2' AND field3='other_value3';
SELECT * FROM table1 WHERE field1='another_values1' AND field2='another_value2' AND field3='another_value3';
SELECT * FROM table2 WHERE table2_field1='table2_values1' AND table2_field2='table2_value2' AND table2_field3='table2_value3' AND field4=;
SELECT * FROM table2 WHERE table2_field1='other_table2_values1' AND table2_field2='other_table2_value2' AND table2_field3='other_table2_value3' AND table2_field4=;
SELECT * FROM table2 WHERE table2_field1='another_table2_values1' AND table2_field2='another_table2_value2' AND table2_field3='another_table2_value3' AND table2_field4='another_table2_value4';

Answer 2

Here's an alternative in Python, in case you're into that sort of thing (more verbose but more legible than awk, at least for me):

#!/usr/bin/env python2
# -*- coding: ascii -*-
"""transform_query.py"""

import sys
import re

# Open the data file specified by the user
with open(sys.argv[1], 'r') as query_file:
    for row in query_file.readlines():

        # Regular expression to extract table name, field names, and values from each line
        match = re.search(
            r'^INSERT INTO '
                r'(?P<table>table\d+)\((?P<fields>[\w,]+)\) '
                r'VALUES\((?P<values>[^()]+)\);$',
            row.strip()
        )
        if match:

            # Store the table name (not necessary)
            table = match.group('table')

            # Split the fields string into a list
            fields = match.group('fields').split(',')

            # Split the values string into a list
            values = match.group('values').split(',')

            # Recombine the strings into a SELECT statement
            # and print the result
            print(
                "SELECT * FROM {} WHERE {};".format(
                    table,
                    ' AND '.join(
                        ['='.join([field, value]) for field, value in zip(fields, values)]
                    ),
                )
            )

Run it:

python transform_query.py query.sql

And here's the output:

SELECT * FROM table1 WHERE field1='values1' AND field2='value2' AND field3='value3';
SELECT * FROM table1 WHERE field1='other_values1' AND field2='other_value2' AND field3='other_value3';
SELECT * FROM table1 WHERE field1='another_values1' AND field2='another_value2' AND field3='another_value3';
SELECT * FROM table2 WHERE table2_field1='table2_values1' AND table2_field2='table2_value2' AND table2_field3='table2_value3';
SELECT * FROM table2 WHERE table2_field1='other_table2_values1' AND table2_field2='other_table2_value2' AND table2_field3='other_table2_value3';
SELECT * FROM table2 WHERE table2_field1='another_table2_values1' AND table2_field2='another_table2_value2' AND table2_field3='another_table2_value3' AND table2_field4='another_table2_value4';

Answer 3

If you are needing explanation, tell me and I will add comments into the code.

Requirements:

1. The number of fields and values should match.
2. Field names and field values should not contain these characters: ,)(; - comma, parentheses, semicolon.

---

gawk '{
    num = patsplit($3$4, arr, /[^,)(;]+/);
    num /= 2;

    printf("SELECT * FROM %s WHERE ", arr[1]);

    for(i = 2; i <= num; i++) {
        printf("%s=%s", arr[i], arr[num + i]);

        printf (i < num) ? " AND " : ";\n";
    }
}' input.txt

Explanation:

This string is picked as example: INSERT INTO table1(field1,field2,field3) VALUES('values1','value2','value3');

1. num = patsplit($3$4, arr, /[^,)(;]+/);
   • patsplit(s, a, r) - split the string s into the array a on the regular expression r, and return the number of fields.
   • $3$4 - $3 is table1(field1,field2,field3), $4 is VALUES('values1','value2','value3');. So, after concatenating, we have this string: table1(field1,field2,field3)VALUES('values1','value2','value3');.
   • then, split it to arr by this regex /[^,)(;]+/. It means: all characters, except comma, parentheses, semicolon. Therefore now: arr[1] is table1, arr[2] is field1, arr[5] is VALUES, so on.
2. num /= 2; for algorithm used in this script, the half of the num variable is needed.
3. printf("SELECT * FROM %s WHERE ", arr[1]); - print the first element of the arr, which is the table name. In our case, it is the table1.
4. printf("%s=%s", arr[i], arr[num + i]); - the algorithm. Print one element from the first half of the arr and corresponding element from the second half of the arr. That is, arr[2] and arr[6], arr[3] and arr[7], arr[4] and arr[8].
5. printf (i < num) ? " AND " : ";\n"; - The ternary operator. If it is not the last iteration, print AND, else print ;\n (final semicolon and newline).

Input (I corrected your input, because it had errors - the number of field names and field values didn't match).

INSERT INTO table1(field1,field2,field3) VALUES('values1','value2','value3');
INSERT INTO table1(field1,field2,field3) VALUES('other_values1','other_value2','other_value3');
INSERT INTO table1(field1,field2,field3) VALUES('another_values1','another_value2','another_value3');
INSERT INTO table2(table2_field1,table2_field2,table2_field3,table2_field4) VALUES('table2_values1','table2_value2','table2_value3','table2_value4');
INSERT INTO table2(table2_field1,table2_field2,table2_field3,table2_field4) VALUES('other_table2_values1','other_table2_value2','other_table2_value3','other_table2_value4');
INSERT INTO table2(table2_field1,table2_field2,table2_field3,table2_field4) VALUES('another_table2_values1','another_table2_value2','another_table2_value3','another_table2_value4');

Output

SELECT * FROM table1 WHERE field1='values1' AND field2='value2' AND field3='value3';
SELECT * FROM table1 WHERE field1='other_values1' AND field2='other_value2' AND field3='other_value3';
SELECT * FROM table1 WHERE field1='another_values1' AND field2='another_value2' AND field3='another_value3';
SELECT * FROM table2 WHERE table2_field1='table2_values1' AND table2_field2='table2_value2' AND table2_field3='table2_value3' AND table2_field4='table2_value4';
SELECT * FROM table2 WHERE table2_field1='other_table2_values1' AND table2_field2='other_table2_value2' AND table2_field3='other_table2_value3' AND table2_field4='other_table2_value4';
SELECT * FROM table2 WHERE table2_field1='another_table2_values1' AND table2_field2='another_table2_value2' AND table2_field3='another_table2_value3' AND table2_field4='another_table2_value4';