Click here to get back home

UNIVERSAL::can function failing under heavy load

 HomeNewsGroups | Search | About
 comp.lang.perl.modules    Post an article   get this group's latest topics as an RSS feed add this group's latest topics to your My MSN content add this group's latest topics to your My Yahoo content
Subject Author Date
UNIVERSAL::can function failing under heavy load lee 08-25-2005
Posted by lee on August 25, 2005, 11:56 am
Please log in for more thread options


I am currently working on a project that experiences intermittent
errors, that we can not find any logical answer for. Attached is a
copy of a method that will copy a given perl object....

sub _deep_copy
{
        my $self                = shift;
        my $src_ref                = $_[0];
        my $ref_type;
        my @new_array        = ();
        my %new_hash        = ();
        my $dest_ref;

        if (grep { $_ eq $src_ref } @Deep_copy_stack)
        {
                my $trace        = join "<br/>", (caller);
                $self->_croak("deep_copy detected a loop<br/>$trace");
        }
        push (@Deep_copy_stack, $src_ref);

        if ( ref( $src_ref ) )
        {
                $ref_type                                 = ref( $src_ref );

                if ($ref_type eq "HASH" )
                {
                        my $key;
                        my $value;

                        # the src_ref points to a hash. Go through each key/value pair
                        # and deep copy them over to the new_hash
                        #
                        while ( ($key, $value)        = each %$src_ref )
                        {
                                $new_hash                = $self->_deep_copy( $src_ref-> );
                        }

                        $dest_ref                                = \%new_hash;
                }
                elsif ($ref_type eq "ARRAY" )
                {
                        # the src_ref points to an array. Go through each index and
                        # deep copy them all over to the new_array
                        #
                        for ( my $ix = 0; $ix < scalar( @$src_ref ); $ix++ )
                        {
                                $new_array[$ix]                = $self->_deep_copy( $$src_ref[$ix] );
                        }

                        $dest_ref                                = \@new_array;
                }
                elsif (UNIVERSAL::isa($src_ref, 'UNIVERSAL') &&
UNIVERSAL::can($src_ref, 'copy'))
                {
                        $dest_ref                                = $src_ref->copy();
                }
                else
                {
                        print "<!-- $ref_type not supported -->\n";
                }
        }
        else
        {
                # $src_ref must be scalar. Copy it over and return it.
                $dest_ref                                        = $src_ref;
        }
        pop (@Deep_copy_stack);

        return $dest_ref;

}        # End &_deep_copy()



We have had reports of the error message (print "<!-- $ref_type not
supported -->\n") appearing for objects that we know have the copy
method, and that work at other times.

Most of our reports have come when the site is under heavy use. Has
anyone seen problems where the UNIVERSAL::isa or the UNIVERSAL::can
methods will fail under stress?

We are using perlex/Windows and mod_perl/linux I think we are seeing on
both platforms.



Posted by Jim Gibson on August 25, 2005, 2:53 pm
Please log in for more thread options



> I am currently working on a project that experiences intermittent
> errors, that we can not find any logical answer for. Attached is a
> copy of a method that will copy a given perl object....
>
> sub _deep_copy
> {
> my $self = shift;
> my $src_ref = $_[0];
> my $ref_type;
> my @new_array = ();
> my %new_hash = ();
> my $dest_ref;
>
> if (grep { $_ eq $src_ref } @Deep_copy_stack)
> {
> my $trace = join "<br/>", (caller);
> $self->_croak("deep_copy detected a loop<br/>$trace");
> }
> push (@Deep_copy_stack, $src_ref);
>
> if ( ref( $src_ref ) )
> {
> $ref_type = ref( $src_ref );
>
> if ($ref_type eq "HASH" )
> {
> my $key;
> my $value;
>
> # the src_ref points to a hash. Go through each key/value pair
> # and deep copy them over to the new_hash
> #
> while ( ($key, $value) = each %$src_ref )
> {
> $new_hash = $self->_deep_copy( $src_ref-> );
> }
>
> $dest_ref = \%new_hash;
> }
> elsif ($ref_type eq "ARRAY" )
> {
> # the src_ref points to an array. Go through each index and
> # deep copy them all over to the new_array
> #
> for ( my $ix = 0; $ix < scalar( @$src_ref ); $ix++ )
> {
> $new_array[$ix] = $self->_deep_copy( $$src_ref[$ix] );
> }
>
> $dest_ref = \@new_array;
> }
> elsif (UNIVERSAL::isa($src_ref, 'UNIVERSAL') &&
> UNIVERSAL::can($src_ref, 'copy'))
> {
> $dest_ref = $src_ref->copy();
> }
> else
> {
> print "<!-- $ref_type not supported -->\n";
> }
> }
> else
> {
> # $src_ref must be scalar. Copy it over and return it.
> $dest_ref = $src_ref;
> }
> pop (@Deep_copy_stack);
>
> return $dest_ref;
>
> } # End &_deep_copy()
>
>
>
> We have had reports of the error message (print "<!-- $ref_type not
> supported -->\n") appearing for objects that we know have the copy
> method, and that work at other times.

Is that really the error message printed? According to the code you
have posted above, the variable $ref_type should contain the return
value from ref($src_ref), which should be a string such as 'ARRAY' or
'HASH' or the name of a package (module). The error message should NOT
contain the substring '$ref_type'. If it does, then something is wrong.

If it does contain the name of a package/module, then that
package/module probably does not implement the copy() method.

>
> Most of our reports have come when the site is under heavy use. Has
> anyone seen problems where the UNIVERSAL::isa or the UNIVERSAL::can
> methods will fail under stress?

Hard as it is to accept, sometimes is just your program screwing up,
not Perl.

>
> We are using perlex/Windows and mod_perl/linux I think we are seeing on
> both platforms.
>


----== Posted via Newsfeeds.Com - Unlimited-Uncensored-Secure Usenet News==----
http://www.newsfeeds.com The #1 Newsgroup Service in the World! >100,000
Newsgroups
---= East/West-Coast Server Farms - Total Privacy via Encryption =---


Posted by lee on August 26, 2005, 7:57 am
Please log in for more thread options


Thanks for the reply and sorry for the lazy description of the error
message. The error message actually contains the name of an object.
And indeed that object does contain a copy method, and produces no
error messages under other circumstances that we have tested. We are
unable to determine why it would fail. As mentioned, most the reports
have come during hi load on the web site. I have concluded that
perhaps it is a memory problem, perhaps it is being over-written, so
that it the can method is failing or something like that. Was curious
if someone had seen something like this.



Posted by xhoster on August 26, 2005, 3:46 pm
Please log in for more thread options


> Thanks for the reply and sorry for the lazy description of the error
> message. The error message actually contains the name of an object.
> And indeed that object does contain a copy method, and produces no
> error messages under other circumstances that we have tested. We are
> unable to determine why it would fail. As mentioned, most the reports
> have come during hi load on the web site.

Is the high load due to other things running, or because very many
instances of this same code are running? If the load is due to this code,
then it is not surprising that that is when you encounter the error. If
there is a 0.0000001 chance of encountering a corner-case bug in your code,
you are more likely to encounter it when there are 10,000 cases executed
per second than when there are 10 cases per second.


> I have concluded that
> perhaps it is a memory problem, perhaps it is being over-written, so
> that it the can method is failing or something like that. Was curious
> if someone had seen something like this.

Nope. But in the last-resort else block, I would print out the
Data::Dumper on the object, not just the stringified reference. Then I
would run copy() on it anyway, to so if "can" was right about there not
being a copy method.

BTW, @Deep_copy_stack doesn't seem to be lexically scoped. I dont' see how
that would cause this particular problem, but I'd fix it anyway just in
case.

Xho

--
-------------------- http://NewsReader.Com/ --------------------
Usenet Newsgroup Service $9.95/Month 30GB


Posted by Jim Gibson on August 26, 2005, 10:15 am
Please log in for more thread options


wrote:

[snip]

> BTW, @Deep_copy_stack doesn't seem to be lexically scoped. I dont' see how
> that would cause this particular problem, but I'd fix it anyway just in
> case.

Presumably, @Deep_copy_stack is scoped outside of the _deep_copy
subroutine so that recursive calls to _deep_copy will be accessing the
same version of @Deep_copy_stack to prevent infinite recursion in the
case of a circular reference. The OP has shown us only the _deep_copy
subroutine, so we can't tell.


----== Posted via Newsfeeds.Com - Unlimited-Uncensored-Secure Usenet News==----
http://www.newsfeeds.com The #1 Newsgroup Service in the World! >100,000
Newsgroups
---= East/West-Coast Server Farms - Total Privacy via Encryption =---


Similar ThreadsPosted
tests failing for .xs routines October 4, 2005, 10:19 am
LWP connect failing - bad hostname June 7, 2006, 12:28 pm
perl-gtk installation failing on Solaris March 20, 2007, 7:28 am
Net::POP3 quit failing, cannot delete some spam email August 31, 2004, 10:13 am
Failing to install IPTables::IPv4 perl module June 30, 2005, 11:10 am
Unable to load module March 16, 2006, 12:00 am
make test failing on Berkeley DB 4.5/perl 5.8 (and 5.6) on BSD6/AMD64 November 29, 2006, 4:44 pm
Encode::Guess load ISO-8859-1 March 27, 2006, 10:09 am
Devel::Cover failing with 'bizarre copy of hash in leave' error October 21, 2004, 2:09 pm
Which module should I use in order to load files over the internet? October 5, 2006, 3:38 pm

Our other projects:

Art Dolls, Fairies and Mermaids - Sunnyfaces.net

Roy's Linux, Programming and Search Engines messages

1-Script XML SitemapXML Sitemap