Tests Show Problems With AV Detections

Do you have a question? Post it now! No Registration Necessary.  Now with pictures!

Yes, this is a 4-year-old story.

Still might be relevent.

In a nutshell:

   "... other companies were creating detections for the false
    submissions from Kaspersky"

----------------------------------------------------------------------

Tests Show Problems With AV Detections

http://securitywatch.pcmag.com/vulnerabilities/284133-tests-show-problems-with-av-detections

Feb 01, 2010

Here at a security press conference held by Kaspersky Lab, the company
demonstrated how some malware detections are
easily triggered by innocuous programs.

The problem arises when one vendor detects a threat. Samples are often
passed on to other vendors, through multi-scanning services like
VirusTotal. The fact that another vendor, particularly a respected one
like Kaspersky, detects a threat is enough of a reason to take a serious
look at the sample.

After suspecting such problems, Kaspersky created a test which
demonstrated the phenomenon. They wrote a series of simple and innocuous
programs, compiled them, created false detections for them in their
engine, and then submitted the files to Virustotal. Only Kaspersky
detected the files at this point.

But standard procedure with VirusTotal is that if at least one of the
products detects a submitted sample, it is submitted to the others who
didn't detect it. The idea is that they can then analyze the file and
create their own detection.

Instead, what they found was that other companies were creating
detections for the false submissions from Kaspersky. The programs create
some variables and perform simple mathematical operations on them. They
don't even touch the file system. Kaspersky provided me with the
programs and the source code.

Click on these to see some of the detections:

http://www.virustotal.com/analisis/5aee7efe6a1ad748c8f866218e42343bdbedee091a15e5931d5ccfd8b3b3b78d-1264831301
http://www.virustotal.com/analisis/0de6dfa1cc4a89c591a7d9fcbf241e4a25aadce63b187c37a18cf047c9f89772-1264867956
http://www.virustotal.com/analisis/b2a11a712d57bf24bc093174f04b86ca2eef6ce9c43198c5ff36a6577f028c45-1264867934
http://www.virustotal.com/analisis/7e79b4efded4c457be503891d6240c0676cb72d7c563e93836f3d4d57862b903-1264867923
http://www.virustotal.com/analisis/7e79b4efded4c457be503891d6240c0676cb72d7c563e93836f3d4d57862b903-1264867923
http://www.virustotal.com/analisis/0b974b85d37882e3c4160bca0a7beb917492412d365b7c59885b38bb94921c6b-1264867640

But it turns out that the fact that Kaspersky was detecting the threats
was not the only reason the others were. The real problems were the
aggressive heuristics in the products and that fact that only a static
scan was performed.

And there is something suspicious about a program that appears to do
nothing and then exits. Other vendors I communicated with on the matter
said that the behavior was not surprising and that a live on-access
detection on a system with their product installed would not be the
same. For instance, F-Secure said that "[o]n the end users Windows box,
these alerts would show up as a prompt, asking the user whether he
really trusts the program. In addition, we have massive whitelist
databases in our back-ends, so such prompts would only appear from new,
unknown applications."

I suspected that the compiler used to generate the samples might itself
be an issue, so I asked Kaspersky about it. They used the mingw
crosscompiler, a gcc version for Linux that generates Win32 binaries.
It's possible that the same source code compiled with Microsoft Visual
Studio would have generated a different reaction in the anti-malware
products, not that it should make a difference. But Kaspersky then
creates a "hello world" program with the same compiler and settings and
uploaded it to VirusTotal; hours later, even though there were no
Kaspersky detections, 2 other products called the sample "suspicious".

This problem is not entirely new; Hispasec Sistemas Lab of Spain, the
company that operates VirusTotal, wrote about it a few months ago
(original Spanish, Google translation to English). As they point out,
the volume of samples coming into company labs is so enormous that the
vast majority has to be handled by automated analysis processes, and
perhaps they are designed to be a little more paranoid than humans.

Kaspersky Lab has written an Analyst's Diary entry on the issue as well.

Site Timeline