FAQ 3.17 How can I make my Perl program take less memory?

Do you have a question? Post it now! No Registration Necessary.  Now with pictures!

This message is one of several periodic postings to comp.lang.perl.misc
intended to make it easier for perl programmers to find answers to
common questions. The core of this message represents an excerpt
from the documentation provided with Perl.


3.17: How can I make my Perl program take less memory?

    When it comes to time-space tradeoffs, Perl nearly always prefers to
    throw memory at a problem. Scalars in Perl use more memory than strings
    in C, arrays take more than that, and hashes use even more. While
    there's still a lot to be done, recent releases have been addressing
    these issues. For example, as of 5.004, duplicate hash keys are shared
    amongst all hashes using them, so require no reallocation.

    In some cases, using substr() or vec() to simulate arrays can be highly
    beneficial. For example, an array of a thousand booleans will take at
    least 20,000 bytes of space, but it can be turned into one 125-byte bit
    vector--a considerable memory savings. The standard Tie::SubstrHash
    module can also help for certain types of data structure. If you're
    working with specialist data structures (matrices, for instance) modules
    that implement these in C may use less memory than equivalent Perl

    Another thing to try is learning whether your Perl was compiled with the
    system malloc or with Perl's builtin malloc. Whichever one it is, try
    using the other one and see whether this makes a difference. Information
    about malloc is in the INSTALL file in the source distribution. You can
    find out whether you are using perl's malloc by typing "perl

    Of course, the best way to save memory is to not do anything to waste it
    in the first place. Good programming practices can go a long way toward

    * Don't slurp!
        Don't read an entire file into memory if you can process it line by
        line. Or more concretely, use a loop like this:

                # Good Idea
                while (<FILE>) {
                   # ...

        instead of this:

                # Bad Idea
                @data = <FILE>;
                foreach (@data) {
                    # ...

        When the files you're processing are small, it doesn't much matter
        which way you do it, but it makes a huge difference when they start
        getting larger.

    * Use map and grep selectively
        Remember that both map and grep expect a LIST argument, so doing

                @wanted = grep <FILE>;

        will cause the entire file to be slurped. For large files, it's
        better to loop:

                while (<FILE>) {
                        push(@wanted, $_) if /pattern/;

    * Avoid unnecessary quotes and stringification
        Don't quote large strings unless absolutely necessary:

                my $copy = "$large_string";

        makes 2 copies of $large_string (one for $copy and another for the
        quotes), whereas

                my $copy = $large_string;

        only makes one copy.

        Ditto for stringifying large arrays:

                        local $, = "\n";
                        print @big_array;

        is much more memory-efficient than either

                print join "\n", @big_array;


                        local $" = "\n";
                        print "@big_array";

    * Pass by reference
        Pass arrays and hashes by reference, not by value. For one thing,
        it's the only way to pass multiple lists or hashes (or both) in a
        single call/return. It also avoids creating a copy of all the
        contents. This requires some judgment, however, because any changes
        will be propagated back to the original data. If you really want to
        mangle (er, modify) a copy, you'll have to sacrifice the memory
        needed to make one.

    * Tie large variables to disk.
        For "big" data stores (i.e. ones that exceed available memory)
        consider using one of the DB modules to store it on disk instead of
        in RAM. This will incur a penalty in access time, but that's
        probably better than causing your hard disk to thrash due to massive


Documents such as this have been called "Answers to Frequently
Asked Questions" or FAQ for short.  They represent an important
part of the Usenet tradition.  They serve to reduce the volume of
redundant traffic on a news group by providing quality answers to
questions that keep coming up.

If you are some how irritated by seeing these postings you are free
to ignore them or add the sender to your killfile.  If you find
errors or other problems with these postings please send corrections
or comments to the posting email address or to the maintainers as
directed in the perlfaq manual page.

Note that the FAQ text posted by this server may have been modified
from that distributed in the stable Perl release.  It may have been
edited to reflect the additions, changes and corrections provided
by respondents, reviewers, and critics to previous postings of
these FAQ. Complete text of these FAQ are available on request.

The perlfaq manual page contains the following copyright notice.


    Copyright (c) 1997-2002 Tom Christiansen and Nathan
    Torkington, and other contributors as noted. All rights

This posting is provided in the hope that it will be useful but
does not represent a commitment or contract of any kind on the part
of the contributers, authors or their agents.

Site Timeline