Out of memory errors with PDO

Do you have a question? Post it now! No Registration Necessary.  Now with pictures!

Threaded View

Hi all,

I'm doing an automated mailer system for users of our website, which
extracts a large number of members matching a specific criteria from
the database. The basic flow of the app is like the following.

1. Get all the criteria to match the users from the database,
returning records containing SQL queries.
2. Loop through each of the records and get the users matching the SQL
criteria queries.
   a. Loop through each of the user records and check if they are
already in the queue. If the record being tested already exists in the
mailer queue, they are unset from the array.
3. Re-index the array containing the user records.
4. Generate the message for all the remaining users and save them into
the queue.

Things to keep in mind are:

1. The app is using PDO with the MySQL driver.
2. A criteria SQL query can have one main table, a join that consists
of a subquery that calculates sums for a particular user, and another
regular join that detects if the user does not have a record on the
said join.
3. After extraction, the loop that checks the user array makes queries
to check if there is a user that matches one in the queue with the
same criteria. If so, this user will be removed from the user array.

What beats me is even though I've set the script to run with 512MB
worth of memory, one of the criteria queries kills the script (I'm
expecting about 20,000+ records returned from this criteria)
complaining that all memory is used up.

Any thoughts on how I should modify the query and/or code scripts, or
how to manually clean up objects in memory? The messages returned by
each method are passed by value and not by reference.

Of course I might be asking the wrong question, so don't hesitate to
help me out.

OJ Tibi

Re: Out of memory errors with PDO

Quoted text here. Click to load it

You could try an un-buffered query.

Re: Out of memory errors with PDO

O.J. Tibi wrote:
Quoted text here. Click to load it

Check in the appropriate database newsgroup to see how to improve your
SQL.  You may be able to do everything in one query.

Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.

Site Timeline