25 min.
Is anti-virus software really woefully ineffective?
That's what a?report released in late November?by Redwood Shores, Calif., digital-security firm Imperva seems to suggest.
It claims that out of 40 top anti-virus products, less than 5 percent detected newly discovered malware, and implied that anti-virus software customers were wasting their money.
"We believe that the majority of?anti-virus products?on the market can't keep up with the rate of virus propagation on the Internet," the study stated. "[What] enterprises and consumers spend on anti-virus is not proportional to its effectiveness."
Imperva's study was widely criticized at the time for poor methodology, small?sample size?and unrealistic testing scenarios. Because of those doubts, TechNewsDaily did not report the study when it was first released.
Six weeks later, the story is enjoying a second wind, with write-ups yesterday (Jan. 1) in the influential tech blog The Register and in the business section of The New York Times.
Flawed methods?
Neither the Register nor the Times questioned Imperva's methods. Yet the questions remain.
"Not only is Imperva's sample size minutely small, but their test has been based upon an utterly flawed methodology," Graham Cluley, senior technology consultant at Sophos, an anti-virus software maker near Oxford, England, told TechNewsDaily today (Jan. 2).
"This 'study' and its conclusions are deeply flawed, wholly unreliable and massively biased,"?tweeted Rik Ferguson?of Japanese anti-virus firm TrendMicro yesterday, addressing the author of the New York Times piece.
"Kaspersky Lab believes it is necessary to draw attention to a significant drawback in Imperva's testing methodology, which makes it impossible to take these test results seriously," a representative of the Russian anti-virus software maker told TechNewsDaily.
Imperva did not initially respond to a request for comment, but instead pointed us to?a blog posting?from mid-December that addressed many of its study's critics.
"While our report acknowledged the limitations of our methodology, we believe that, fundamentally, the model for antivirus ? and not our methodology ? is flawed," the posting reads.
Later, Imperva Director of Security Strategies Rob Rachwald addressed the anti-virus industry as a whole in an email to TechNewsDaily.
"The fact that AV doesn't work is a well-known fact in security circles. Otherwise, why would so many new start-ups keep trying to create new technologies?" Rachwald asked.
"Within security teams, hardly any (competent) practioners expect to be 100 percent protected by AV and, in fact, assume they're infected and must take other precautions."
Unorthodox approach
Imperva's methods could be characterized as somewhat odd. It searched Google, underground hacker forums and its own "honey pots" (deliberately unprotected servers designed to attract malware), for new or nearly unknown pieces of malware. It ended up with 82 pieces.
Then the Imperva researchers accessed VirusTotal, a publicly accessible database of malware signatures, long numerical strings, or "hashes," unique to each piece of code. (Google recently bought VirusTotal.)
Anyone can submit a malware sample to VirusTotal to see which, of any, of the 40 or so anti-virus companies that submit their own samples to VirusTotal had already picked it up.
Imperva did exactly that. It ran the 82 malware samples it had gathered against VirusTotal's database. It found very few matches in VirusTotal. (An earlier version of this story erroneously said that Imperva had run hashes, not samples, against VirusTotal's database.)
"The initial detection rate of new viruses is nearly zero," the Imperva researchers concluded. "Though we don't recommend removing anti-virus altogether, a bigger portion of the security focus should leverage technologies that detect abnormal behavior such as unusually fast access speeds or large volume of downloads."
Jump to a conclusion?
"The test pitted various anti-virus products against a tiny collection of 82 malware samples, and still got VirusTotal to do the hard work for them," Cluley said. "And yet,?VirusTotal's own About page?clearly says that using their service for scanner testing is a 'BAD IDEA' (their capitals, not mine) and has frequently debunked testers that try to compare products by using the site."
"VirusTotal representatives clearly state that the service was not designed as a tool to perform comparative anti-virus analyses," the Kaspersky Lab representative said. "Unfortunately, this was ignored by Imperva's so-called experts, whose incorrect testing methodology resulted in incorrect conclusions."
Furthermore, modern anti-virus software already does what Imperva suggested it doesn't. Besides matching signatures, it also analyzes unknown pieces of code for behavior patterns, methods of entry, resemblance to known malware and other "heuristic" ? experience-based ? clues.
To make a real-world comparison, imagine that a team of security guards only compared strangers' faces to a file of photographs of known criminals. Such methods would work, but only up to a point, yet that's essentially what Imperva tested.
Better-trained guards would also analyze what the strangers wore, how they acted, how they communicated and how they got to the building. That's akin to what most anti-virus products do nowadays.
"Simply scanning a collection of files, no matter how large or how well-sourced, misses the point of security software entirely," Ferguson told the website of Britain's?ITPro magazine?today. "They were not exposing the products to threats in the way they would be in the wild."
Missing the forest for the trees?
Still, some digital-security experts cautioned against ignoring the overall conclusion of the Imperva study.
"Whilst there are legitimate questions surrounding the methodology, the core message that the researchers are trying to convey is correct," Steve Santorelli, director of global outreach at Team Cymru in Lake Mary, Fla., told TechNewsDaily.
"Sure, the study is flawed, but its conclusion is correct: Anti-virus is only partly useful," said Robert David Graham, co-founder and chief executive officer of Errata Security in Atlanta. "I would say the 5 percent number Imperva comes up with is low, but then, the 99 percent number anti-virus vendors claim is way too high."
"The conclusions are consistent with a growing sentiment in IT security that anti-virus, or anti-malware, at best, really doesn't help much at all," said Jeremiah Grossman, founder and chief technology officer of White Hat Security in Santa Clara, Calif. "At worst, people who purchase anti-virus products are actually paying billions of dollars annually for their computers to be less secure."
"Team Cymru ran a test several years ago and we found an average detection rate for new samples of about 30 percent," said Santorelli, who has also worked with Scotland Yard and Microsoft. "After a month, we ran the tests again, and that detection rate only went up to about 50 percent."
"One bank CISO [chief information security officer] made an interesting point to me recently:? He buys AV for legal reasons," Imperva's Rachwald said. "Why? In case someone gets infected, he can say he has AV and therefore has his legal bases covered. When you buy a product to mollify lawyers, that can't be good."
[Do You Really Need to Pay for Anti-Virus Software?]
Other tests
There are other companies that do rigorous, constant testing of anti-virus products as a major part of their business models.
"Respected testing bodies like AV-Test.org, West Coast Labs, Virus Bulletin, etc., test anti-virus products against millions of samples," Cluley said, "and do it properly by verifying that the samples they test against are truly malicious ? and avoid the many many challenges and problems that anti-virus testers have to handle."
In their response to critics, Imperva's Rob Rachwald and Tal Be'ery defended the company's methods and its conclusions. They cited a study by AV-Test that also, in their words, "reveal[ed] a worrisome security gap."
Yet the?AV-Test screen grab?that Rachwald and Be'ery posted demonstrated the opposite of Imperva's conclusions. It showed that in real-world testing of protection against zero-day malware attacks, the anti-virus industry average was 87 percent.
"AV is poor at protecting you from the latest, cutting-edge threats, but it's better than nothing, it's often free and it often comes with extra protection in addition to simple file scanning," Santorelli said.
Still, in an environment where just one mistake can have disastrous consequences, 87 percent may not be enough.
"It's not quite common just yet, but viruses and various software exploits and going to begin targeting anti-virus software directly as an attack vector to exploit systems," Grossman said. "For this reason, and others, is why you find so many computer security experts, including myself, go without anti-virus software entirely."
"Anti-virus is not a silver bullet. You cannot rely solely on anti-virus software," Santorelli added. "If you do, you will inevitably be a victim of crime."
This story was updated at 2 p.m. ET.
Copyright 2013?TechNewsDaily, a TechMediaNetwork company. All rights reserved. This material may not be published, broadcast, rewritten or redistributed.Usain Bolt 2012 Olympics Katie Ledecky Aaron Ross Sikh temple lollapalooza Nastia Liukin Gabby Douglas hair
No comments:
Post a Comment
Note: Only a member of this blog may post a comment.