Increase file reading efficiency

Do you have a question? Post it now! No Registration Necessary.  Now with pictures!

Threaded View
Hello, I am using the WordNet::QueryData module, "initialization step
is slow (appx. 10-15 seconds), but queries are very fast thereafter---
thousands of queries can be completed every second. " as stated by the
author, because it needs to read very big WordNet lexicon in many TEXT
files. I am new to perl, I like to know if there is a way I could
reduce the initialization time ? Thank you

Re: Increase file reading efficiency

Quoted text here. Click to load it

On my system, it only takes about 2 seconds.  So the easiest way would
be to get modern hardware :)  Or you could try to rearrange things so you
need to start up the script less and use it more for each time it is
started (for example, use mod_perl instead of CGI).

Another option would be to store the database in a form that can be
reloaded much easier, which is what the Storable module does.

This creates a serialized object in a file named "file":

$ perl
use WordNet::QueryData;
my $x= WordNet::QueryData->new();
delete $x->;
use Storable;
store $x, "file";

The delete $x->; is there because Storable can't store open
file handles.

This script would need to be run once each time WordNet
database is updated.

Then, when you go to use the database, you would no longer load
the object with WordNet::QueryData->new(), instead would load it from
the saved data:

use WordNet::QueryData;
use Storable;
$x=retrieve "file";
# now use $x just as if it was created via new

The purpose of $x->openData is to repopulate the previously deleted
$x->.  I haven't tested this rigorously.

This cuts the start-up time to 0.5 seconds, so only a 4 fold improvement.


-------------------- http://NewsReader.Com/ --------------------
The costs of publication of this article were defrayed in part by the
payment of page charges. This article must therefore be hereby marked
advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
this fact.

Re: Increase file reading efficiency

On Mar 20, 2:37 am, wrote:
Quoted text here. Click to load it

Thank you very much  Xho, I had read about Storable which increase the
reading 4 times faster than normal text file.
I will try that out as only Storable is possible for me under my web
share hosting.

Thank you

Site Timeline