clever fork

Do you have a question? Post it now! No Registration Necessary.  Now with pictures!

Threaded View
I want to utilize all the cpu cores as much as possible.
What I am thinking is to "break" my sql queries to as many parts as the  
number of cpu cores , and then fork every part.
Do you think it is going to work ; Do you have any other idea ; Thanks.  

Re: clever fork

"George Mpouras"
Quoted text here. Click to load it

Probably. The more interesting question is "Does this make
sense?". You could utilize more than one core for 'application
computing' in this way, however 'SQL query' usually implies RDBMS
which means that a lot of the 'computing' is done insided the
database management system which is unaffected by forks done in an
application and that I/O is necessary to move the queries to the
database and the results back to the application. I/O is also (at
best) unaffected by application forks and it is going to be a lot
slower than 'computing', hence, there's a non-negligible chance that
your application is either I/O bound or that 'the CPU-bound parts'
happen insided the RDBMS.

Re: clever fork

Quoted text here. Click to load it

Assuming a query can genuinely be broken up into independant parts (that
is, that the RDBMS doesn't end up doing all the work for each part
anyway) and that the RDBMS doesn't use multiple cores on its own, it can
be worth doing. As you say, though, there's usually no need to fork to
do this: with most DBs you can open several DB connections from a single
thread and submit one query to each connection.

However, there are still important issues here: the most important is
that the separate sessions will necessarily use separate transactions,
so there may be data-integrity problems.


Site Timeline