Click here to get back home

FAQ 5.23 All I want to do is append a small amount of text to the end of a file. Do I still have to use locking?
 Home | NewsGroups | Search | About
Login Password
Register Now!
 comp.lang.perl.misc    Post an article   get this group's latest topics as an RSS feed add this group's latest topics to your My MSN content add this group's latest topics to your My Yahoo content
Subject Author Date
FAQ 5.23 All I want to do is append a small amount of text to the end of a file. Do I still have to use locking? PerlFAQ Server 06-09-2008
Posted by PerlFAQ Server on June 9, 2008, 3:03 pm
Please log in for more thread options
 This is an excerpt from the latest version perlfaq5.pod, which
comes with the standard Perl distribution. These postings aim to
reduce the number of repeated questions as well as allow the community
to review and update the answers. The latest version of the complete
perlfaq is at http://faq.perl.org .

--------------------------------------------------------------------

5.23: All I want to do is append a small amount of text to the end of a file.
Do I still have to use locking?


If you are on a system that correctly implements flock() and you use the
example appending code from "perldoc -f flock" everything will be OK
even if the OS you are on doesn't implement append mode correctly (if
such a system exists.) So if you are happy to restrict yourself to OSs
that implement flock() (and that's not really much of a restriction)
then that is what you should do.

If you know you are only going to use a system that does correctly
implement appending (i.e. not Win32) then you can omit the seek() from
the code in the previous answer.

If you know you are only writing code to run on an OS and filesystem
that does implement append mode correctly (a local filesystem on a
modern Unix for example), and you keep the file in block-buffered mode
and you write less than one buffer-full of output between each manual
flushing of the buffer then each bufferload is almost guaranteed to be
written to the end of the file in one chunk without getting intermingled
with anyone else's output. You can also use the syswrite() function
which is simply a wrapper around your systems write(2) system call.

There is still a small theoretical chance that a signal will interrupt
the system level write() operation before completion. There is also a
possibility that some STDIO implementations may call multiple system
level write()s even if the buffer was empty to start. There may be some
systems where this probability is reduced to zero.



--------------------------------------------------------------------

The perlfaq-workers, a group of volunteers, maintain the perlfaq. They
are not necessarily experts in every domain where Perl might show up,
so please include as much information as possible and relevant in any
corrections. The perlfaq-workers also don't have access to every
operating system or platform, so please include relevant details for
corrections to examples that do not work on particular platforms.
Working code is greatly appreciated.

If you'd like to help maintain the perlfaq, see the details in
perlfaq.pod.

Posted by Ben Morrow on June 9, 2008, 3:51 pm
Please log in for more thread options
>
> 5.23: All I want to do is append a small amount of text to the end of a
> file. Do I still have to use locking?
>
<snip>
> There is still a small theoretical chance that a signal will interrupt
> the system level write() operation before completion. There is also a
> possibility that some STDIO implementations may call multiple system
> level write()s even if the buffer was empty to start. There may be some
> systems where this probability is reduced to zero.

This includes all systems using the :perlio layer instead of native
STDIO.

Ben

--
I have two words that are going to make all your troubles go away.
"Miniature". "Golf".
[ben@morrow.me.uk]

Posted by brian d foy on June 10, 2008, 12:02 pm
Please log in for more thread options

> >
> > 5.23: All I want to do is append a small amount of text to the end of a
> > file. Do I still have to use locking?
> >
> <snip>
> > There is still a small theoretical chance that a signal will interrupt
> > the system level write() operation before completion. There is also a
> > possibility that some STDIO implementations may call multiple system
> > level write()s even if the buffer was empty to start. There may be some
> > systems where this probability is reduced to zero.
>
> This includes all systems using the :perlio layer instead of native
> STDIO.

Do you mean that :perlio is one of the systems where the probability is
zero for calling multiple system level writes() even if the buffer is
empty?

I don't know much about the internals. I'm not sure how to adjust the
answer.

Thanks,

Posted by Ben Morrow on June 10, 2008, 1:39 pm
Please log in for more thread options

>
> > >
> > > 5.23: All I want to do is append a small amount of text to the end of a
> > > file. Do I still have to use locking?
> > >
> > <snip>
> > > There is still a small theoretical chance that a signal will interrupt
> > > the system level write() operation before completion. There is also a
> > > possibility that some STDIO implementations may call multiple system
> > > level write()s even if the buffer was empty to start. There may be some
> > > systems where this probability is reduced to zero.
> >
> > This includes all systems using the :perlio layer instead of native
> > STDIO.
>
> Do you mean that :perlio is one of the systems where the probability is
> zero for calling multiple system level writes() even if the buffer is
> empty?

Yes. :perlio only flushes when the buffer is full (in full-buffered
mode).

> I don't know much about the internals. I'm not sure how to adjust the
> answer.

Perhaps

where this probability is reduced to zero, and this is not a concern
when using :perlio instead of your system's STDIO.

?

Ben

--
And if you wanna make sense / Whatcha looking at me for? (Fiona Apple)
* ben@morrow.me.uk *

Posted by Uri Guttman on June 9, 2008, 5:06 pm
Please log in for more thread options

PS> This is an excerpt from the latest version perlfaq5.pod, which
PS> comes with the standard Perl distribution. These postings aim to
PS> reduce the number of repeated questions as well as allow the community
PS> to review and update the answers. The latest version of the complete
PS> perlfaq is at http://faq.perl.org .

PS> --------------------------------------------------------------------

PS> 5.23: All I want to do is append a small amount of text to the end of a
file. Do I still have to use locking?


PS> If you are on a system that correctly implements flock() and
PS> you use the example appending code from "perldoc -f flock"
PS> everything will be OK even if the OS you are on doesn't
PS> implement append mode correctly (if such a system exists.) So
PS> if you are happy to restrict yourself to OSs that implement
PS> flock() (and that's not really much of a restriction) then
PS> that is what you should do.

PS> If you know you are only going to use a system that does correctly
PS> implement appending (i.e. not Win32) then you can omit the seek() from
PS> the code in the previous answer.

PS> If you know you are only writing code to run on an OS and
PS> filesystem that does implement append mode correctly (a local
PS> filesystem on a modern Unix for example), and you keep the
PS> file in block-buffered mode and you write less than one
PS> buffer-full of output between each manual flushing of the
PS> buffer then each bufferload is almost guaranteed to be written
PS> to the end of the file in one chunk without getting
PS> intermingled with anyone else's output. You can also use the
PS> syswrite() function which is simply a wrapper around your
PS> systems write(2) system call.

PS> There is still a small theoretical chance that a signal will
PS> interrupt the system level write() operation before
PS> completion. There is also a possibility that some STDIO
PS> implementations may call multiple system level write()s even
PS> if the buffer was empty to start. There may be some systems
PS> where this probability is reduced to zero.

File::Slurp has append_file which does this feature but without
locking. if you want an atomic version without locking you can slurp in
the file and write it out with the new data in atomic mode like this:

write_file( $file_name, {atomic => 1},
         read_file( $file_name, $appended_text ) ) ;

this will write it all to a new file and rename it to the original
name. it is as good as locking in most situations that need this. it
isn't super efficient if the file is getting large.

i could add lock support for append_file (which is really a call to
write_file with the 'append' option enabled). this would be added to my
long list of todo's for the slurp module.

thanx,

uri

--
Uri Guttman ------ uri@stemsystems.com -------- http://www.sysarch.com --
----- Perl Code Review , Architecture, Development, Training, Support ------
--------- Free Perl Training --- http://perlhunter.com/college.html ---------
--------- Gourmet Hot Cocoa Mix ---- http://bestfriendscocoa.com ---------

Similar ThreadsPosted
FAQ 5.21: All I want to do is append a small amount of text to the end of a file. Do I still have to use locking? December 11, 2004, 6:03 am
FAQ 5.21: All I want to do is append a small amount of text to the end of a file. Do I still have to use locking? December 22, 2004, 6:03 pm
FAQ 5.21 All I want to do is append a small amount of text to the end of a file. Do I still have to use locking? February 12, 2005, 12:03 am
FAQ 5.22 All I want to do is append a small amount of text to the end of a file. Do I still have to use locking? May 9, 2005, 11:03 am
FAQ 5.22 All I want to do is append a small amount of text to the end of a file. Do I still have to use locking? July 25, 2005, 4:03 pm
FAQ 5.22 All I want to do is append a small amount of text to the end of a file. Do I still have to use locking? September 4, 2005, 10:03 am
FAQ 5.22 All I want to do is append a small amount of text to the end of a file. Do I still have to use locking? November 26, 2005, 5:03 pm
FAQ 5.22 All I want to do is append a small amount of text to the end of a file. Do I still have to use locking? May 3, 2006, 9:03 pm
FAQ 5.22 All I want to do is append a small amount of text to the end of a file. Do I still have to use locking? August 3, 2006, 3:03 pm
FAQ 5.22 All I want to do is append a small amount of text to the end of a file. Do I still have to use locking? November 4, 2006, 9:03 pm

Our other projects:

Art Dolls, Fairies and Mermaids - Sunnyfaces.net

Roy's Linux, Programming and Search Engines messages

1-Script XML SitemapXML Sitemap