Python MySQL queue: run code/queries in parallel, but commit separately -


program 1 inserts jobs table job_table.

program 2 needs :

1. job table 2. handle job    -> needs multi-threaded (because each job involves urllib waiting time, should run in parallel) 3. insert results my_other_table, commiting result 

any (standard?) ways implement this? issue commiting inside 1 thread, commits other threads.

what happens share mysql connection between 2 threads. try creating new mysql connection inside each thread.

for program 2, @ http://www.celeryproject.org/ :)


Comments