Most CISOs are cautiously optimistic about their teams’ ability to detect and stop intrusions. They understand that not all attacks can be blocked, but they think their organizations are defenceless.

Professional penetration testers, on the other hand, apparently have unlimited confidence they can sashay through the firewall — at least those who talked to cyber analytics provider Nuix at last August’s Def Con 24 conference. According to a survey released Thursday of 70 professional hackers and penetration testers at the conference,  88 per cent of  respondents said they can break through cybersecurity defences and into a targeted system within 12 hours, while 81 per cent say they can identify and take valuable data within another 12 hours.

The results are contained in a paper the company calls The Black Report 2017 (registration required).

The report isn’t scientific — it doesn’t say whether the organizations those surveyed say they have broken into are representative of entities of all sizes across all industries, nor  was there any effort to verify whether respondents were boasting. Assuming they were being honest the results should make CISOs think carefully: If pen testers think they’re this good, what do professional threat actors believe?

Among the findings:

–43 per cent believe they can compromise a target in up to six hours; another 28 per cent think they can do it in up to 12 hours;

–53 per cent admitted they sometimes encounter a system they can’t break into. Only nine per cent said it never happens;

–36 per cent said they are detected after a successful penetration about one-third of time — which means for this group two-thirds of the time they aren’t detected. Another 26 per cent said they are detected half the time. One-third said they are never detected.

–respondents said traditional countermeasures such as firewalls and antivirus almost never slowed them down but endpoint security technologies were more effective at stopping attacks;

–more than half of respondents changed their methodologies with every target, severely limiting the effectiveness of security defenses based on known files and attacks.

What’s a CISO to do with this? Nuix’s conclusion is that “many vendors are simply out of touch with the latest attack techniques and have no idea about the motivations and experiences of the attackers themselves.” I’m not so sure it’s fair to lay the blame entirely on vendors. It’s been proven that a good security awareness program is a highly effective — though admittedly not absolute — way to cut down on intrusions.

But the report does remind CISOs that good security means protecting data, not systems.