Without strong metrics that matter to the business, security awareness programs will continue to be the “bastard child of security”.
We all know how little budget security gets, and of that, security awareness programs often achieve a minimalistic amount.
Security Awareness programs are an essential component of any good security program, yet they rarely get the resources that they require to accomplish their goal. The reason seems to be, “You get the budget that you deserve, not the budget that you need.”
Security Awareness programs generally run basic phishing campaigns and push out computer-based training (CBT).
- Phishing has some potential demonstration of value.
- CBT provides broad-based training.
There might be some collateral advantage as well. However, the only tangible metrics most awareness programs provide is proof of compliance, and potentially showing that some people become less susceptible to simulated phishing.
As a CIO or CISO, to “deserve more” you need to determine metrics that show true value to the business.
To that end, I’ve broken awareness-relevant metrics into four categories:
- Compliance,
- Engagement,
- Tangible Return on Investment (ROI),
- Intangible Benefits.
Compliance Metrics
Compliance metrics involve ensuring that an organization satisfies third-party compliance needs, which, in general, means that you ensure employees complete awareness training.
The minimal amount of budget required to provide training to your employees is justified at a minimum level. No actual measurement or behavioural change.
Engagement Metrics
This is the most common category of metrics that I see used by most security awareness practitioners.
These metrics involve the general use and acceptance of the security awareness components measured by a Learning Management System (LMS).
Completion metrics, which overlap with compliance metrics, is one form.
Metrics may also include measurement of the increase in phishing attacks reported to the help desk, voluntary engagement with supplemental materials, such as if a user voluntarily takes extra awareness training, or become a champion for the department, etc.
There are two caveats with engagement metrics.
- They don’t indicate effectiveness or behavioural change for the majority of staff,
- The more engagement people undertake, the less the time they are performing their jobs.
Tangible Behavioural Improvement and ROI
This is the metric that shows employees are raising their security awareness and changing their bahaviour.
In this case, you need to measure actual security behaviours or the indications of those behaviours.
- Can people recite the criteria for a good password and practice this in everyday life?
- Do people know that they should secure sensitive information?
- Do they leave sensitive information vulnerable when they leave their desk?
Simulated phishing attacks do not demonstrate behavioural change. The issue is that while simulated phishing attacks could indicate how people might respond to real attacks, it is too easy to manipulate phishing simulations to return any level response that you want.
On the other hand, fewer malware incidents on the network are a relevant metric.
Ideally, you want to attach a financial value to behavioural changes.
For example, if malware incidents are decreased by 25%, and there is an average cost associated to malware incidents, you can calculate an estimated ROI. Likewise, if there are fewer incidents associated with compromised credentials, you can calculate an estimated ROI.
To do so, you need to collect metrics that are related to potential security failings and track those metrics. These could be measurement of attitudinal changes, cultural impact etc.
Intangible Benefits
Clearly, you want to demonstrate a financial return; however, there are other potential points of value to an organization.
For example,
- If an awareness program creates goodwill toward the security department, users may be more likely to report incidents and cooperate with other efforts.
- If users believe the awareness program generates value to them, there might be better employee retention.
- If employees are less susceptible to personal attacks, they might spend less time at work mitigating stolen identities.
- If employees utilise MFA or core passphrase methodology, the change policy can be extended to annual password change, resulting in fewer help desk tickets to remediate forgotten passwords and improved efficiency of staff.
Do you “Deserve More”?
While compliance requirements will mean that there will always be an awareness program, it is up to a security awareness practitioner to demonstrate that their efforts deserve more than the minimum funding required to achieve compliance.
The way to do this is to focus on the periodic collection of metrics that demonstrate a ROI well beyond standard engagement metrics.
Layer 8 Security
