Click here to get back home

XML Parsing too slow
 Home | NewsGroups | Search | About
Login Password
Register Now!
 comp.lang.perl.modules    Post an article   get this group's latest topics as an RSS feed add this group's latest topics to your My MSN content add this group's latest topics to your My Yahoo content
Subject Author Date
XML Parsing too slow jabby 11-19-2005
Posted by jabby on November 19, 2005, 7:29 am
Please log in for more thread options

I have some files ranging from 6 meg to 15 meg that I need to process

to input into a database. I'm inheriting some old code using

XML::Simple and this process is taking forever (an hour or more). I

have eliminated the database as the time hog in this equation.



is there an alternative XML module I should be using?





Sent via Archivaty.com


Posted by Stephane Roux (HAbeTT) on November 19, 2005, 2:43 pm
Please log in for more thread options


> I have some files ranging from 6 meg to 15 meg that I need to process
> to input into a database. I'm inheriting some old code using
> XML::Simple and this process is taking forever (an hour or more). I
> have eliminated the database as the time hog in this equation.
>
> is there an alternative XML module I should be using?

XML::LibXML

--
|":._.:"| http://habett.com/
| (=) | http://habett.org/
| .:':. | I send the energy to my enemy


Posted by Michel Rodriguez on November 21, 2005, 11:15 am
Please log in for more thread options


jabby wrote:
> I have some files ranging from 6 meg to 15 meg that I need to process
> to input into a database. I'm inheriting some old code using
> XML::Simple and this process is taking forever (an hour or more). I
> have eliminated the database as the time hog in this equation.
>
> is there an alternative XML module I should be using?

Hi,

It really depends on the code, on why the process is slow and on the
effort you want to put into re-writing it.

Why is the code slow? Is the call to XMLin, which loads the data into a
Perl structure, slow? Or is the problem that the data takes up to much
space in memory and that the system starts swapping pages? 6/15 Megs is
not that much these days, so I am not sure the problem lies with
XML::Simple. What are you doing with the data that takes that long?

You should probably start by running the XMLin call by itself to see how
long it takes.

BTW XML::LibXML is indeed faster than XML::Parser-based modules like
XML::Simple, but its interface is a lot different, instead of working
with a Perl structure, you work with a DOM, so if most of the processing
happens once the data has already been loaded in memory, I am not sure
it will actually speed things up.

--
mirod


Similar ThreadsPosted
Net::SSH::Perl extremely slow? July 12, 2005, 8:04 pm
The IPC::ShareLite is very slow than fetch data from file ?? October 25, 2005, 7:04 pm
XML::DOM parsing pb March 9, 2006, 1:27 pm
HTML parsing March 21, 2005, 3:24 pm
CGI.pm parsing odity May 16, 2006, 10:01 am
Image data parsing October 27, 2004, 3:36 pm
Parsing OpenOffice Spreadsheets April 25, 2005, 7:23 pm
can't find xml-parsing module... May 27, 2006, 5:30 am
Lemur parsing module June 20, 2006, 10:44 pm
XML::Atom::Feed - parsing at all? January 20, 2008, 5:04 pm

Our other projects:

Art Dolls, Fairies and Mermaids - Sunnyfaces.net

Roy's Linux, Programming and Search Engines messages

1-Script XML SitemapXML Sitemap