Persistence data

Do you have a question? Post it now! No Registration Necessary.  Now with pictures!

Threaded View
I have a class, which contains an associated array, which data is
fetched from DB when the class is created. (The associated array is
around 600K rows,  70MB in term of memory size.)

I need to store this variable in memory for performance reasons
(reduce massive DB lookup)

Currently I use this class from command line and it is fine. But if I
want to use this class in web context, is it possible for Apache to
share this class so I don't need to have 70MB per each Apache process.
Also, the startup loading time will contributed soem latency - I want
to reduce. (even don't want to use APC - since serialize & unserialize
also take many CPU time).

So are there any persistence data approach in PHP can solve my


Re: Persistence data

In article

Quoted text here. Click to load it

mod_php and Apache don't share memory with other web processes.  As a
separate process running php through CGI or as a thread of Apache
running mod_php, they are all separate.  The only way to have them share
data is through a file or a named pipe or some other mechanism that you
create and manage from within php.  It would be very ugly if you did it
this way and I seriously doubt you could gain much from such an effort.

You need to refactor or rethink your approach to accessing this data in
smaller chunks that are more amenable to a web environment where many
processes access a resource in php.

Or switch to java and tomcat, which will keep such things around just as
you envision.

Bottom line, php is the wrong tool to do this sort of thing.  Do it some
other way or use another tool.

DeeDee, don't press that button!  DeeDee!  NO!  Dee...
[I filter all Goggle Groups posts, so any reply may be automatically by ignored]

Re: Persistence data

howa wrote:
Quoted text here. Click to load it

   If you're not changing the data the why not dump it into a memory db
(heap, I think) on first access. Or you can tune the db to cache the data.

Registered Linux user #461062
-Have you been to yet?-

Re: Persistence data

howa wrote:
Quoted text here. Click to load it

Somehow I can hardly imagine that persistence is the problem. Do you
really need all of the 600K rows in each process? If so, you do an awful
lot of work and there is nothing to prevent it. But let me give you a
few directions of a possible solution:

- Upgrade the server speed and memory.
- If it is a complex calculation of database-stored data, consider doing
the calculation in the database, using stored procedures for instance.
- Use caching. You mention APC not being an option. I don't know each
cache type, but there are others (I use xCache mainly). But caches do
not know your class, so it is hard to store your own objects in a cache.
- Use a separate process. Write a CORBA, COM, etc. application that
remains in memory and that you can query through PHP.
- If the 600K rows are hierarchically structured, you may be able to use
some intelligence to suppress unneeded parts of the calculation or keep
in-between results for the rows that did not change.

Best regards,
Willem Bogaerts

Application smith
Kratz B.V. /

Re: Persistence data

Quoted text here. Click to load it

I was surprised to find that pushing the operation down to the
database to dereference values with multiple queries instead of
deserializing a large array from a file at run time started to pay off
around the 0.2Mb mark.

While you could write own PHP appserver daemon to process this (IIRC
there are a couple pre-wrtiten out there) but the PHP memory model
isn't at all suited to this kind of architecture (but see the new
garbage collection functionality).

Answer is to do it at the database tier.


Re: Persistence data

howa wrote:
Quoted text here. Click to load it

Why?  Does every script who accesses this class need access to 70M of data?

What is the problem you're trying to solve, anyway?
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.

Site Timeline