program 1 inserts jobs table job_table.
program 2 needs :
1. job table 2. handle job -> needs multi-threaded (because each job involves urllib waiting time, should run in parallel) 3. insert results my_other_table, commiting result
any (standard?) ways implement this? issue commiting inside 1 thread, commits other threads.
what happens share mysql connection between 2 threads. try creating new mysql connection inside each thread.
for program 2, @ http://www.celeryproject.org/ :)
Comments
Post a Comment