FAQ 4.41 How can I remove duplicate elements from a list or array?

Do you have a question? Post it now! No Registration Necessary.  Now with pictures!

This is an excerpt from the latest version perlfaq4.pod, which
comes with the standard Perl distribution. These postings aim to
reduce the number of repeated questions as well as allow the community
to review and update the answers. The latest version of the complete
perlfaq is at http://faq.perl.org .


4.41: How can I remove duplicate elements from a list or array?

    (contributed by brian d foy)

    Use a hash. When you think the words "unique" or "duplicated", think
    "hash keys".

    If you don't care about the order of the elements, you could just create
    the hash then extract the keys. It's not important how you create that
    hash: just that you use "keys" to get the unique elements.

            my %hash   = map { $_, 1 } @array;
            # or a hash slice: @hash{ @array } = ();
            # or a foreach: $hash = 1 foreach ( @array );

            my @unique = keys %hash;

    If you want to use a module, try the "uniq" function from
    "List::MoreUtils". In list context it returns the unique elements,
    preserving their order in the list. In scalar context, it returns the
    number of unique elements.

            use List::MoreUtils qw(uniq);

            my @unique = uniq( 1, 2, 3, 4, 4, 5, 6, 5, 7 ); # 1,2,3,4,5,6,7
            my $unique = uniq( 1, 2, 3, 4, 4, 5, 6, 5, 7 ); # 7

    You can also go through each element and skip the ones you've seen
    before. Use a hash to keep track. The first time the loop sees an
    element, that element has no key in %Seen. The "next" statement creates
    the key and immediately uses its value, which is "undef", so the loop
    continues to the "push" and increments the value for that key. The next
    time the loop sees that same element, its key exists in the hash *and*
    the value for that key is true (since it's not 0 or "undef"), so the
    next skips that iteration and the loop goes to the next element.

            my @unique = ();
            my %seen   = ();

            foreach my $elem ( @array )
                    next if $seen{ $elem }++;
                    push @unique, $elem;

    You can write this more briefly using a grep, which does the same thing.

            my %seen = ();
            my @unique = grep { ! $seen{ $_ }++ } @array;


The perlfaq-workers, a group of volunteers, maintain the perlfaq. They
are not necessarily experts in every domain where Perl might show up,
so please include as much information as possible and relevant in any
corrections. The perlfaq-workers also don't have access to every
operating system or platform, so please include relevant details for
corrections to examples that do not work on particular platforms.
Working code is greatly appreciated.

If you'd like to help maintain the perlfaq, see the details in

Site Timeline