Exhaustive memory allocation using arrays

Do you have a question? Post it now! No Registration Necessary.  Now with pictures!

Threaded View
Why is the following code using > 100MB?

for ($i=0; $i < 1000000; $i++) {
    $value[$i] = 23.4567890123;
echo (memory_get_usage(TRUE) / (1024 * 1024)) . "MB <br>";

It looks like that each item takes 100 Byte for allocation??

thx for your help!

Re: Exhaustive memory allocation using arrays


Re: Exhaustive memory allocation using arrays

And, doubling the memory size by assigning a new value:

for ($i=0; $i < 1000000; $i++) {
    $value[$i] = "23.4567890123";
for ($i=0; $i < 1000000; $i++) {
    $value[$i] = sprintf("%f", $value[$i]);

Now php is using ~400 MB

Re: Exhaustive memory allocation using arrays

tombert.at wrote:
Quoted text here. Click to load it

I *assume* that the strings in the first loop do not actually allocate a
struct str for every instance, but instead reference a single statically
allocated instance.  The second loop is probably allocating a new struct
str for every sprintf().

Compare with:

  for ($i=0; $i < 1000000; $i++) {
      $value[$i] = "23.4567890123";
  for ($i=0; $i < 1000000; $i++) {
      $value[$i] = floatval($value[$i]);

Christoph M. Becker

Re: Exhaustive memory allocation using arrays

On Tue, 28 May 2013 23:27:13 -0700, tombert.at wrote:

Quoted text here. Click to load it

From: http://php.net/manual/en/language.types.array.php

"An array in PHP is actually an ordered map."

You may be expecting arrays where an array of 1e6 floats is held in  
memory in 1e6 sequential 8 byte locations, for a total of 8e6 bytes, but  
I suspect that if you did, you'd be very very very wrong.

It's possible that:

$arr = array_values( $arr );

May shrink your array, I don't know for sure.

It's also possible that:

$a = Array();
for ( $i = 0; i < 1e6; i++ )
    $a[] = 23.4567890123;

might produce a smaller array, simply because it's not assigning keys.  
Again, I don't know for sure.

The reason I don't know for sure is that these implementation details  
are, to me, just that. If they don't affect me, I don't need to  
understand them in the level of detail you're looking for. There's plenty  
of things I do need to understand at that level of detail, and I have  
limited time to put into gaining such understandings, so I use it where  
it's most needed. :)

Denis McMahon, denismfmcmahon@gmail.com

Re: Exhaustive memory allocation using arrays

On 29/05/13 09:22, Denis McMahon wrote:
Quoted text here. Click to load it
Exactly so. There are times when you realise that an 'excellent for some  
tasks' tool is in fact a complete dogs breakfast when applied to another  

I found, for example, that on the fly generation of graphics using PHP  
was massively CPU and RAM hungry and appeared to suffer re-rentrancy  
problems when used in multiple concurrent access: the solution was to  
generate the image data asynchronously using a C program.

I can think of many ways in which an array element dynamically assigned  
needing pointers to memory blocks and so on could easily have several  
tens of bytes overhead.

That of course is an issue with an interpreted language: it has to more  
or less avoid statically allocated memory especially if there is no way  
to declare it.


(in-ep-toc?-ra-cy) ? a system of government where the least capable to lead
are elected by the least capable of producing, and where the members of society
least likely to sustain themselves or succeed, are rewarded with goods and
services paid for by the confiscated wealth of a diminishing number of

Re: Exhaustive memory allocation using arrays

tombert.at@gmail.com wrote:
Please get a real name.

Quoted text here. Click to load it

First of all, 1 MB (megabyte) is 1'000'000 (one million, 10?) bytes.  What  
you call ?MB? here actually is 1 _MiB_ (mega-binary byte; short: mebibyte)  
or 2? bytes = 1'048'576 bytes (as you divided by 2? 2? = 2?
to arrive  
at your ?MB? value):


So your calculation

  100 ?MB? ? 1'000'000 items = 100 bytes?item

is wrong to begin with.  Assuming that you got something close to 100 as a  
result of

  memory_get_usage(TRUE) / (1024 * 1024)

each item would take about

  104'857'600 bytes / 1'000'000 = 105 bytes


Second, the value you assigned is a value of the float type.  In PHP, the  
precision of the float type, thus the number of bits required to store such  
a value, is platform-dependent:


The manual says that IEEE-754 ?double precision? floating-point values are a
common implementation (as they are in other equally dynamic programming  
languages), but we cannot assume that to be the case on your platform.  And  
you have not said which platform and PHP version you are using, so we cannot  
look into the pertinent source code.

Third, you are not just storing the float value.  You are storing a key-
value relationship:

| An array in PHP is actually an ordered map. A map is a type that
| associates values to keys.  [?]

Fourth, that PHP arrays can be associative, i. e. can have string keys, and  
that using the string representation of a numeric key accesses the same item  
as the number (ibid.), suggests implementations in which all keys are  
actually string values (as it is in other equally dynamic programming  
languages, like ECMAScript implementations; if not like that, you would have  
to store the type of the key which is even more memory-expensive), and

| The string in PHP is implemented as an array of bytes and an integer
| indicating the length of the buffer.  [?]

Fast access to array items further suggests that the keys are not only  
stored as-is (for array_keys()), but that the implementation includes a hash  

Fifth, PHP arrays do have an internal pointer ? <http://php.net/current ?  
that can be advanced: <http://php.net/next .  Combined with the dynamic size  
of PHP arrays this suggests that the items of an array are at least arranged  
in a linked list.  The existence of a function to rewind the internal  
pointer ? <http://php.net/prev ? further suggests that they are arranged in
a double-linked list.

Finally, you are not just measuring the memory that is needed for the array,  
but for executing the preceding program.  There is *at least* the memory for  
$i variable to consider.  So ?your math does not add up?.

Probably there are even more factors to consider.  See also the  
memory_get_usage() documentation and user comments.

So consider this: A PHP array could be implemented *at least* as a hash  
table (for quick item access) in which the string keys are stored in a  
linked list (so that array_keys() works), and the stored values could be in  
a double-linked list (so that internal pointer movements work).  In this  
hash table you are storing items whose plain values require *at least* 64  
bits = 8 bytes of memory each.  Add to that the memory required to run the  
program that fills the array.

I know of no native way to determine the memory it takes to store the PHP  
array; it can perhaps be obtained with a PHP profiler.  In any case, I  
presume it will be smaller than what you measured with memory_get_usage().


var bugRiddenCrashPronePieceOfJunk = (
    navigator.userAgent.indexOf('MSIE 5') != -1
    && navigator.userAgent.indexOf('Mac') != -1
)  // Plone, register_function.js:16

Site Timeline