Click here to get back home

Memory issues

 HomeNewsGroups | Search | About
 comp.lang.perl.misc    Post an article   get this group's latest topics as an RSS feed add this group's latest topics to your My MSN content add this group's latest topics to your My Yahoo content
Subject Author Date
Memory issues jm 03-29-2008
---> Re: Memory issues Joost Diepenmaa...03-29-2008
Posted by smallpond on March 29, 2008, 9:25 am
Please log in for more thread options
> Based on the fact that perl contains many memory leaks,
>
> A universal way to measure how many memory is malloced is required.
>
> Is there standard way to measure how many memory a process has
> allacated, which run with cygwin perl, active perl, and strawberry perl?
>
> This should help to localize which code makes memory leaks.


perldoc perlfaq3
See:
How can I make my Perl program take less memory?
How can I free an array or hash so my program shrinks?


Posted by jm on March 29, 2008, 10:10 am
Please log in for more thread options
smallpond a écrit :
>> Based on the fact that perl contains many memory leaks,
>>
>> A universal way to measure how many memory is malloced is required.
>>
>> Is there standard way to measure how many memory a process has
>> allacated, which run with cygwin perl, active perl, and strawberry perl?
>>
>> This should help to localize which code makes memory leaks.
>
>
> perldoc perlfaq3
> See:
> How can I make my Perl program take less memory?
> How can I free an array or hash so my program shrinks?

It is interesting, but it does not seam to solve my substitution issue.

However I does not understand this:

« Memory allocated to lexicals (i.e. my() variables)
cannot be reclaimed or reused even if they go out of scope. It is
reserved in case the variables come back into scope. Memory allocated
to global variables can be reused (within your program) by using
undef()ing and/or delete(). »

Aren't my variables local variables?
Why aren't they freed when function terminates?





Posted by smallpond on March 29, 2008, 10:27 am
Please log in for more thread options
> smallpond a =E9crit :
>
> >> Based on the fact that perl contains many memory leaks,
>
> >> A universal way to measure how many memory is malloced is required.
>
> >> Is there standard way to measure how many memory a process has
> >> allacated, which run with cygwin perl, active perl, and strawberry perl=
?
>
> >> This should help to localize which code makes memory leaks.
>
> > perldoc perlfaq3
> > See:
> > How can I make my Perl program take less memory?
> > How can I free an array or hash so my program shrinks?
>
> It is interesting, but it does not seam to solve my substitution issue.
>
> However I does not understand this:
>
> =AB Memory allocated to lexicals (i.e. my() variables)=

> cannot be reclaimed or reused even if they go out of scope. It is
> reserved in case the variables come back into scope. Memory allocat=
ed
> to global variables can be reused (within your program) by using
> undef()ing and/or delete(). =BB
>
> Aren't my variables local variables?
> Why aren't they freed when function terminates?


sub foo {
my $v =3D 5;
return $v;
}

In C, once the function terminates $v is gone and a pointer
to it will fail. In perl this reference is legal and the
space will not be reclaimed.

In your sample of code above, when you pass a string to a sub,
perl will make a copy. If you pass a reference it will not.
This isn't a memory leak in perl, it's a memory leak in your
program.

Posted by jm on March 29, 2008, 11:12 am
Please log in for more thread options
smallpond a écrit :
>> smallpond a écrit :
>>
>>>> Based on the fact that perl contains many memory leaks,
>>>> A universal way to measure how many memory is malloced is required.
>>>> Is there standard way to measure how many memory a process has
>>>> allacated, which run with cygwin perl, active perl, and strawberry perl?
>>>> This should help to localize which code makes memory leaks.
>>> perldoc perlfaq3
>>> See:
>>> How can I make my Perl program take less memory?
>>> How can I free an array or hash so my program shrinks?
>> It is interesting, but it does not seam to solve my substitution issue.
>>
>> However I does not understand this:
>>
>> « Memory allocated to lexicals (i.e. my() variables)
>> cannot be reclaimed or reused even if they go out of scope. It is
>> reserved in case the variables come back into scope. Memory allocated
>> to global variables can be reused (within your program) by using
>> undef()ing and/or delete(). »
>>
>> Aren't my variables local variables?
>> Why aren't they freed when function terminates?
>
>
> sub foo {
> my $v = 5;
> return $v;
> }
>
> In C, once the function terminates $v is gone and a pointer
> to it will fail. In perl this reference is legal and the
> space will not be reclaimed.
>
> In your sample of code above, when you pass a string to a sub,
> perl will make a copy. If you pass a reference it will not.
> This isn't a memory leak in perl, it's a memory leak in your
> program.

As you suggested, I tried to replace scalar by references, but this does
not look like saving memory (might be 10 Mbytes, I mean just the size of
the main variable):

--- results -------------------------
/tmp$ perl essai.pl && echo ok
10000001
PID TTY STAT TIME MAJFL TRS DRS RSS %MEM COMMAND
31679 pts/1 R+ 0:00 0 1022 22977 20996 4.0 perl essai.pl

10000001
PID TTY STAT TIME MAJFL TRS DRS RSS %MEM COMMAND
31679 pts/1 R+ 0:38 0 1022 150157 148372 28.7 perl essai.pl

10000001
PID TTY STAT TIME MAJFL TRS DRS RSS %MEM COMMAND
31679 pts/1 R+ 0:50 0 1022 150157 148372 28.7 perl essai.pl

10000001
PID TTY STAT TIME MAJFL TRS DRS RSS %MEM COMMAND
31679 pts/1 R+ 3:53 0 1022 198997 197212 38.1 perl essai.pl

10000001
PID TTY STAT TIME MAJFL TRS DRS RSS %MEM COMMAND
31679 pts/1 R+ 6:55 0 1022 198997 197212 38.1 perl essai.pl

ok


--- scipt ---------------------------
sub aa($)
{
my ($d) = @_;
$$d =~ s/x(.....)/$1y/g ;
$$d =~ s/x(.....)/$1z/g ;
$$d =~ s/x(.....)/$1a/g ;
$$d =~ s/x(.....)/$1b/g ;
$$d =~ s/x(.....)/$1c/g ;
return $d;
}

sub ab($)
{
my ($d) = @_;
$$d =~ s/a(.....)/$1y/g ;
$$d =~ s/b(.....)/$1z/g ;
$$d =~ s/c(.....)/$1a/g ;
$$d =~ s/y(.....)/$1b/g ;
$$d =~ s/z(.....)/$1c/g ;
return $d;
}


my $s= 'x' x (1000*1000*10) ;
$s .= "\x" ;
my $c = $s;
print length($$c) ."\n" ;
my $v = qx( ps v $$ );
print "$v\n" ;
$c = aa($c);
print length($$c) ."\n" ;
my $v = qx( ps v $$ );
print "$v\n" ;
$c = aa($c);
$c = aa($c);
$c = aa($c);
$c = aa($c);
$c = aa($c);
print length($$c) ."\n" ;
my $v = qx( ps v $$ );
print "$v\n" ;
$c = ab($c);
$c = ab($c);
$c = ab($c);
$c = ab($c);
$c = ab($c);
print length($$c) ."\n" ;
my $v = qx( ps v $$ );
print "$v\n" ;
$c = ab($c);
$c = ab($c);
$c = ab($c);
$c = ab($c);
$c = ab($c);
print length($$c) ."\n" ;
my $v = qx( ps v $$ );
print "$v\n" ;

Posted by jm on March 29, 2008, 11:53 am
Please log in for more thread options


Modifying a little bit again the script, and checking execution with
ltrace, I observed malloc is called 1871 times when free is just called
922 times.
Isn't it an issue?

I just replaced
my $s= 'x' x (1000*1000*10) ;
by
my $s= 'x' x (10) ;

and did:


/tmp$ ltrace perl essai.pl 2>&1 | sed 's/(.*//g' | sort | uniq -c |
grep 'malloc\|free'
922 free
1871 malloc



> --- scipt ---------------------------
> sub aa($)
> {
> my ($d) = @_;
> $$d =~ s/x(.....)/$1y/g ;
> $$d =~ s/x(.....)/$1z/g ;
> $$d =~ s/x(.....)/$1a/g ;
> $$d =~ s/x(.....)/$1b/g ;
> $$d =~ s/x(.....)/$1c/g ;
> return $d;
> }
>
> sub ab($)
> {
> my ($d) = @_;
> $$d =~ s/a(.....)/$1y/g ;
> $$d =~ s/b(.....)/$1z/g ;
> $$d =~ s/c(.....)/$1a/g ;
> $$d =~ s/y(.....)/$1b/g ;
> $$d =~ s/z(.....)/$1c/g ;
> return $d;
> }
>
>
> my $s= 'x' x (1000*1000*10) ;
> $s .= "\x" ;
> my $c = $s;
> print length($$c) ."\n" ;
> my $v = qx( ps v $$ );
> print "$v\n" ;
> $c = aa($c);
> print length($$c) ."\n" ;
> my $v = qx( ps v $$ );
> print "$v\n" ;
> $c = aa($c);
> $c = aa($c);
> $c = aa($c);
> $c = aa($c);
> $c = aa($c);
> print length($$c) ."\n" ;
> my $v = qx( ps v $$ );
> print "$v\n" ;
> $c = ab($c);
> $c = ab($c);
> $c = ab($c);
> $c = ab($c);
> $c = ab($c);
> print length($$c) ."\n" ;
> my $v = qx( ps v $$ );
> print "$v\n" ;
> $c = ab($c);
> $c = ab($c);
> $c = ab($c);
> $c = ab($c);
> $c = ab($c);
> print length($$c) ."\n" ;
> my $v = qx( ps v $$ );
> print "$v\n" ;

Similar ThreadsPosted
Huge Memory Load for reading into memory November 6, 2006, 7:10 pm
Get Win32 Total Physical Memory & Available Physical Memory stats October 26, 2004, 10:48 pm
2 issues with "tie" August 26, 2007, 9:32 pm
DBI Performance Issues August 25, 2006, 10:33 am
Perl script issues - Need help January 20, 2005, 9:54 pm
Known issues with Perl under Cygwin? August 26, 2005, 6:43 pm
Perldoc security issues September 12, 2005, 2:16 pm
having issues using awk and/or converting to perl January 9, 2006, 6:21 am
taint issues in utf8_heavy.pl August 23, 2006, 7:43 am
Inline replacement issues October 18, 2006, 3:53 pm

Our other projects:

Art Dolls, Fairies and Mermaids - Sunnyfaces.net

Roy's Linux, Programming and Search Engines messages

1-Script XML SitemapXML Sitemap