2019.10.28
A self-policing or self-generating risk control system is one that allows for the design and adjustment of other risk controls. These are the processes by which the people who deliver a business detect problems with risk controls and adjust them, usually in concert with a risk management or internal audit function.
An excellent example of this are the quarterly risk control self assessment practices whereby process owners are given a list of controls and asked to verify that they are a) in place at all and b) effective. Typically, there is an spreadsheet with multiple tabs. On the tab, there is a description of the control, a description of the control's evidence to suit the two criteria, and a place to reference examples of the control's evidence. It might look something like this:
Quarterly process control self-test | |||||||||
Start Date | 2020.01.01 | End Date | 2020.03.31 | Report Completion Date | 2020.04.15 | ||||
Tester Name | A. Jolly Fellow | Title | Director, IT | ||||||
Reviewer Name | John Q. Oversight | Title | Internal Auditor | ||||||
AI06 | Manage Changes | ||||||||
High Level Control Objective | All changes, including emergency maintenance and patches, relating to infrastructure and applications within the production environment must be formally managed in a controlled manner. | ||||||||
Control | 328 | Logging has been enabled on all production server systems for events such as errors and outages to operating system and hardware. Log monitoring is conducted on a daily basis for production Linux and RDBMS systems by the infrastructure team. | |||||||
description of evidence | |||||||||
Time sheet entries exist for the work conducted. | |||||||||
population size | sample document location | ||||||||
120 | Records available for review in Timesheet system. | ||||||||
sample size | |||||||||
120 | |||||||||
successful tests | |||||||||
120 | |||||||||
target success rate | |||||||||
95% | |||||||||
success rate | |||||||||
100% | |||||||||
This control is in place as of | 2016.10.01 | ||||||||
justification of sample methodology | |||||||||
Number of days x number of reviews (logwatch, RDBMS, nagios) | |||||||||
This is in turn cross-indexed with a master catalog that allows the risk management team to match controls to those on the auditor's lists. In the example above, the control is labeled "AI06 Managed Changes", this is a reference to the (antique) COBIT 4 control catalog. The two lines that follow are the "high level control objective" which describes that standard's system of grouping discrete controls into a set that addresses a common set of business objectives. The control text next to the catalog reference number (328 in this case). All of the boxes that follow that describe the effectiveness of the control by listing the evidence, the count of successful control instances against total instances of the activities against which the control applies, and so on.
NOTE: I deliberately chose to use this outdated document, which has origins in my work at NikkoCiti more than a decade ago, because it shows how things were done at a certain time: with a "distributed excel database" as I've heard it sarcastically described. These early practices have been completely replaced by "GRC" software such as Resolver and Archer (which typically cost as much as a full-time equivalent staffer). Sadly I can't show any screenshots of such proprietary systems.
The example above shows a control that displays sufficient evidence of control to warrant a "pass": the control is in evidence. Where there is not evidence that the control is in place, a discussion remediation efforts to put the control in place are likely required.
This is one way in which the measurement of the controls efficacy is different from its measure of presence. Because there are endless reasons why a control might not be in evidence, and because process owners tend to automatically manage risk when a process is failing in some way, such gaps tend to raise questions about whether the control is still addressing real risk. It is at this point that the process owner–in this case the Director of IT–would work with the risk management team or the internal auditors to determine whether the control is still warranted. There are several factors including:
Of course, this review can be triggered for a control that is in evidence but is deemed to be of questionable value. As the owner of two critical business functions at that Japanese investment bank, I was routinely asked to measure quite useless things such as proving that mis-filed "incident tickets" that began with "please" were actually requests and did not have to be treated as incidents. That company being an unweildly mass of 400,000 employees in various divisions around the globe, changes took 18+ months. Naturally your organization should be able to better.
By reviewing the catalog of controls owned by each process owner at a regular cadence (such as quarterly) questions will be raised quite naturally and the risk controls will be improved upon.
It's then up to the risk or audit function to manage any internal or external auditors that may review the work. Because auditors tend to be fairly inflexible when it comes to changes in control language, managing said auditors can be at once a substantial burden on the risk practitioner but also a substantial "value add"—it's a very worthwhile activity to demonstrate that a rigidly-written control is handled by controls more relevant to the business.
What I've described above is an audit-focused process, that is to say, backward-looking. One of the incidental and perhaps non-obvious benefits of such a system is that it normalizes the concept of risk controls with process owners and/or business unit leads. This is important because it can cause the consideration of controls when new initiatives are undertaken. Because controls can include detective controls (e.g. those measures that indicate a project or process is not running to schedule or not producing the expected result) they can be applied to virtually any undertaking.
While I developed my own process for instituting what became a self-policing control environment during the course of about two years at a Canadian software firm, I eventually discovered that other people had done substantially more thorough work in the field that included clear descriptions of the thought processes behind the activities I'd undertaken. In particular, I recommend the work of Matthew Leitch, particularly his book Intelligent Internal Control and Risk management. It's now absurdly expensive but it stands alone in my experience in its applicability to control design and implementation.