And now for an astonishing secret: Some analysis firms in the IT industry sell their work for profit.
Okay, it isn’t much of a secret. But I raise it because this week NSS Labs, an independent IT security equipment testing company released its most recent test of a dozen next generation firewalls (NGFWs), appliances that are application-aware and can scan traffic. For a fee you can see the detailed results.
We don’t have that kind of money. What they did release for free, though, is an observation that CSOs and security administrators should keep in mind: Evasion techniques included in malware continue to be a challenge.
“Missing a low level evasion, such as TCP stream segmentation or IP packet fragmentation, can result in security devices failing to detect and block an entire class of exploits,” a summary of the report says. “This results in significantly lower security effectiveness. Two products experienced significant reversals in their security effectiveness scores this year: one product was 36.3 per cent less effective than last year due to missed evasions, while another increased effectiveness by 63.6 per cent after rectifying evasion weaknesses identified in last year’s test.” On the other hand 10 out of 12 products demonstrated complete evasion detection.
You’re probably wondering which two did worse than last year: I don’t know, and I don’t want this to appear to be an ad for paying NSS so you should find out which product to avoid.
The point is that selecting any security product has to be done by carefully evaluating all criteria and fitting the solution into a well-crafted multi-layer defence.
The report also notes that NGFWs are becoming more cost competitive: The average Total Cost of Ownership / Protected Mbps figure fell by half in 2014 to US$21.80 – down from an average of US$46.28 per protected Mbps from the 2013 test. Half of the products tested in 2014 had a TCO below US$20 per protected Mbps, the report says, with an overall range of US$6 – $64, down from a range of US$18 – $106 in the 2013 test. “This price movement is an early indicator that NGFW and UTM markets are converging,” it concluded.
Finally, most vendor performance claims hold up in testing: “Over half of the products outperformed their vendor-stated throughput rates during testing this year. Three vendors had products that achieved throughput rates over 25 per cnt higher than their stated rates.”