I have a table
URL_Experiment in my database (mySQL database). I have 2 million URL links in this table.
For each URL, I am checking if some particular text is present in the URL and updating the information back in my
Now, I need to make the script run fast. The easy solution that comes to my mind is:
URL_Experiment into 10 tables (each with 200,000 rows) and run the script 10 times simultaneously. This way, I can ensure that the updates happen properly. But for this, I need to save 10 scripts, each accessing the correct table (I need to rename the tables from 1 to 10).
Though the above seems a pretty neat solution, I am looking for a more sophisticated solution. Something which will execute the same script simultaneously (in parallel) as 10 processes. My concern is the execution time. I do not want the resources to be wasted when they can be utilized.