Click here to get back home

DBI Performance Issues

 HomeNewsGroups | Search | About
 comp.lang.perl.misc    Post an article   get this group's latest topics as an RSS feed add this group's latest topics to your My MSN content add this group's latest topics to your My Yahoo content
Subject Author Date
DBI Performance Issues Chris H. 08-25-2006
Posted by xhoster on August 28, 2006, 5:24 pm
Please log in for more thread options
> On Mon, 28 Aug 2006 16:27:31 +0000, xhoster wrote:
>
> > You are not making the important distiction between throughput and
> > latency. Your Perl script sends the sql command, then waits to get a
> > response. While waiting, it does nothing--it doesn't use CPU and/or
> > bandwidth by "sending ahead on prospect" more update statements. So
> > you are limited by latency, not by throughput. With an insert, you can
> > circumvent this by using the MySQL-specific multiple row insert
> > statement to add many rows per network round-trip. Maybe you can use
> > the MySQL-specific "INSERT...ON DUPLICATE KEY UPDATE" syntax to
> > accomplish the same thing--just a suggestion, I've never looked into it
> > in detail myself. However, if your real situation will have the two
> > machines on the same LAN, then latency will probably not be a problem.
> > You simply can't do benchmarking on such disparate setups and expect
> > the results to be meaningful.
> >
> > Xho
>
> i understand the differences in the setup, but i dont see latency playing
> that much of a role in this issue. ping response from solaris to windows
> on the lan (live setup) is 2.73ms. ping response from my home machine to
> my colo box is 17.5ms.

Is that under load or under no load? Anyway, you reported 71 seconds for
1000 records, which is 71ms per record. 71ms is only fourfold more than
17.5 ms, so I would say that latency is definitely in the ball park to be a
problem. (I would expect MySQL to have more overhead then ping--4 times
more would not greatly surprise me.)

> my issue is that the two setups perform the same
> in regards to how slow they are getting the information to the server. i
> can understand it being slower, that's a given, but what im having an
> issue with is that it takes just as long to update 10 records remotely as
> 1,000 locally...then something is going wrong somewhere. i'd suspect it
> should be faster than that.

I'm afraid you lost me. You have a localhost msyql connection with 1000
records per second, and an over-cable-modem mysql connection with 14
records per second. Do you have a *mysql* connection on a LAN? If so,
what speed did that provide?

Xho

--
-------------------- http://NewsReader.Com/ --------------------
Usenet Newsgroup Service $9.95/Month 30GB

Posted by Peter J. Holzer on August 29, 2006, 1:59 pm
Please log in for more thread options
>>
>> [snip]
>>
>> > if I print "$1\n",
>> > the file prints just fine. But, if I do something like print "$1 after
>> > \n", the whole output is messed up. If I print "before $1\n", nothing
>> > prints at all. If I print "before $1 after\n", only after prints.
>>
>> not really sure, but could be a rogue "\r" in $1,


> There
> is a rogue carriage return (0xd) in the string

> Is there something I can do to deal with this
> situation?


Repair the corrupted file:

perl -p -i -e 'tr/\r//d' bad_file


--
Tad McClellan SGML consulting
tadmc@augustmail.com Perl programming
Fort Worth, Texas

Similar ThreadsPosted
performance and timing issues udp socket November 29, 2005, 2:36 am
SUBSTR() with replacement or lvalue performance issues November 6, 2008, 9:17 pm
2 issues with "tie" August 26, 2007, 9:32 pm
Memory issues March 29, 2008, 8:45 am
Perl script issues - Need help January 20, 2005, 9:54 pm
Known issues with Perl under Cygwin? August 26, 2005, 6:43 pm
Perldoc security issues September 12, 2005, 2:16 pm
having issues using awk and/or converting to perl January 9, 2006, 6:21 am
taint issues in utf8_heavy.pl August 23, 2006, 7:43 am
Inline replacement issues October 18, 2006, 3:53 pm

Our other projects:

Art Dolls, Fairies and Mermaids - Sunnyfaces.net

Roy's Linux, Programming and Search Engines messages

1-Script XML SitemapXML Sitemap