Click here to get back home

Alternative to Parallel::ForkManager
 Home | NewsGroups | Search | About
Login Password
Register Now!
 comp.lang.perl.misc    Post an article   get this group's latest topics as an RSS feed add this group's latest topics to your My MSN content add this group's latest topics to your My Yahoo content
Subject Author Date
Alternative to Parallel::ForkManager nolo contendere 04-23-2008
Posted by nolo contendere on April 23, 2008, 2:29 pm
Please log in for more thread options
 Scenario:
I am expecting 3 files in a drop directory. They won't
necessarily all arrive at the same time. I want to begin processing
the each file as soon as it arrives (or as close to arrival time as is
reasonable). Would the best way to go about this be to simply have a
script that takes a filename as a parameter and marks the file as
'currently processing' when it begins to process the file (or could
move the file to a different directory)?

I could kick off 3 daemon processes looking in the drop directory, and
sleep every 5 secs, for instance.

That seems to me, to be a straightforward, if clumsy, approach. I was
wondering if there was a module that could accomplish this task more
elegantly--Parallel::ForkManager, at least in my experience, doesn't
seem entirely suited to this particular task.

Or I could code my own fork,exec,wait/waitpid.

I know TMTOWTDI, but I was seeking to benefit from others' experience,
and for a 'best practice'.

Sorry there's no tangible code; this is more of a conceptual question
I guess.

Posted by Peter Makholm on April 23, 2008, 4:51 pm
Please log in for more thread options
> I know TMTOWTDI, but I was seeking to benefit from others' experience,
> and for a 'best practice'.

If portability isn't a issue, you platform might support some kind of
monitoring of parts of the filesystem. Then you can get events when
files are created in you spool directory og moved there.

Linux::Inotify2 is a linux only-solution I'm using for a couple of
scripts. Another usable module coudl be SGI::FAM, which should be
supported on a broader range of unices.

I have been looking for something like Net::Server for spool dirs a
couple of times without finding anything really useful.

//Makholm

Posted by Ben Morrow on April 23, 2008, 5:54 pm
Please log in for more thread options

>
> > I know TMTOWTDI, but I was seeking to benefit from others' experience,
> > and for a 'best practice'.
>
> If portability isn't a issue, you platform might support some kind of
> monitoring of parts of the filesystem. Then you can get events when
> files are created in you spool directory og moved there.
>
> Linux::Inotify2 is a linux only-solution I'm using for a couple of
> scripts. Another usable module coudl be SGI::FAM, which should be
> supported on a broader range of unices.

SGI::FAM only works under Irix. I've been meaning to port it to other
systems that support fam (and gamin, the GNU rewrite) but haven't got
round to it yet. There is Sys::Gamin, but it doesn't have any tests and
doesn't appear to be maintained.

Other OS-specific alternatives include IO::KQueue for BSDish systems,
and Win32::ChangeNotify for Win32. This seems like a perfect opportunity
for someone to write an OS-independant wrapper module, but AFAIK no-one
has yet.

Ben


Posted by Peter Makholm on April 24, 2008, 4:15 am
Please log in for more thread options

> SGI::FAM only works under Irix. I've been meaning to port it to other
> systems that support fam (and gamin, the GNU rewrite) but haven't got
> round to it yet.

Never used the module myself (should have made that clear) and I have
to admit that my only reason to assume that it is usable on other
platforms is that File::Tail::FAM talk about Linux.

//Makholm

Posted by nolo contendere on April 24, 2008, 10:29 am
Please log in for more thread options
> > SGI::FAM only works under Irix. I've been meaning to port it to other
> > systems that support fam (and gamin, the GNU rewrite) but haven't got
> > round to it yet.
>
> Never used the module myself (should have made that clear) and I have
> to admit that my only reason to assume that it is usable on other
> platforms is that File::Tail::FAM talk about Linux.
>
> //Makholm

I appreciate the effort Peter, however I'm currently stuck on Solaris.

Similar ThreadsPosted
Parallel::ForkManager and Net::FTP problem? October 7, 2005, 2:19 pm
behavior of run_on_wait in Parallel::ForkManager November 2, 2004, 8:01 pm
Problem with Parallel::ForkManager on Windows Vista January 24, 2008, 4:23 am
Parallel::ForkManager not working as intended on Windows Server 2003 September 20, 2006, 9:26 am
mod_expires alternative August 2, 2004, 4:00 pm
alternative to MS word doc January 17, 2005, 6:15 pm
Alternative in regexps February 9, 2005, 7:52 pm
alternative to calltree ? February 28, 2006, 11:52 am
mime head alternative April 19, 2007, 1:42 pm
Question? Alternative to loading modules!!! January 20, 2005, 7:07 am

Our other projects:

Art Dolls, Fairies and Mermaids - Sunnyfaces.net

Roy's Linux, Programming and Search Engines messages

1-Script XML SitemapXML Sitemap