|
Posted by c.cobb on May 11, 2007, 12:43 pm
Please log in for more thread options
Given today's multi-processer computers, what's missing?
The ability to multi-process commands, of course! Say you're
a system administrator, and you have to run a set of commands
for each of 1,000+ user accounts. Why would you choose to
process each user's account one at a time??
Enter NWay. This POE-based utility will take your list of
1,000 tasks (or however many) and ensure that only a limited
number of commands will run simultaneously.
Say you have an 8-way machine. Run 8 tasks at a time. Say
you can determine that each task uses only about half of the
CPU time on a given processor. Run 16 tasks at a time.
In the first usage of this utility, over 6,500 ClearCase
Views required reformatting. The total accumulated time
to process was over 75 hours, while the total ELAPSED
time was just under 10 hours. As each reformat used up
just about 100% of a CPU, 8 tasks ran at a time, giving
almost an 8x improvement. Not bad. Not too bad at all.
There is quite a bit of flexibility on input. A simple data
file, or task list, is the input along with a few command
line options. Additional cmd-line arguments can be included
to alter defaults specified within the task list.
Error checking is done on a task-by-task basis. Logging can
be done on a task-by task basis, or combined into one large
log file. A summary of results is generated including error
count, start time, end time, elapsed and accumulated times.
If you run the tasks as a 'root' user, you can specify what
uid and/or gid will be used to actually run each task. And,
if you don't like the behavior of the command processor,
you can specify your custom Perl class either on the command
line or within the task list.
|
| Similar Threads | Posted | | ANNOUNCE: PDL-2.4.3 | August 22, 2006, 7:46 am |
| [ANNOUNCE] MultiProcFactory 0.01 | September 20, 2004, 6:29 am |
| ANNOUNCE: WWW::Webrobot 0.50 | September 29, 2004, 10:46 am |
| ANNOUNCE: WWW::Webrobot 0.51 | October 1, 2004, 9:54 am |
| ANNOUNCE: tclperl-3.1 | October 11, 2004, 4:56 am |
| [ANNOUNCE] Net::FTPSSL 0.03 | September 5, 2005, 4:54 pm |
| [ANNOUNCE] POD2::IT 0.09 | September 8, 2005, 11:06 am |
| ANNOUNCE: Teradata::SQL v0.01 | November 17, 2005, 3:23 pm |
| [ANNOUNCE] POD2::IT 0.10 | November 21, 2005, 9:44 am |
| [ANNOUNCE] wxPerl 0.26 | November 23, 2005, 9:41 pm |
|
XML Sitemap