managing controls for a SOC-2 audit

2017.11.05

the problem

Annual service audits are driven by a third-party with a checklist. That checklist is a series of controls designed to mitigate certain risks. While the checklist itself may evolve as versions come and go, the controls as a whole never go away. Your organization must demonstrate that all active controls are in place and effective at all times.

"At all times" is a pretty difficult thing to do. Someone has to manage every process that is touched upon by all those controls. And the way that most functional managers see their role, your audit report isn't their problem. You will be deemed responsible for delivering evidence of those controls being in place in someone else's area of the company. Compounding this problem is that it's far too late to learn of a control deviation when you're sitting with your auditors. Audit is, by its nature, backward looking: auditors ask How have you performed over the previous period? Now you're responsible for not only ensuring that business functions you don't run are behaving properly, but to manage it after the fact.

If this sounds impossible, that's because it is. You—the audit manager—will spend the rest of your days tracking down control evidence, arguing with functional managers over the meaning of words in the control, and spending weeks with your auditors trying to pass your audit.

the solution

There is a way out of this nightmare. A way that reduced my time with the auditors to as little as eight hours a year, and let me deliver five successive clean audits.

To manage audits, you have to manage control performance across an entire organization. To do that effectively and efficiently, you have to do these things:

1. Catalog the controls, and identify the functional managers who own the different processes under control.
2. Document how the controls relate to each of the processes.
3. Design a system for catching deviations in control performance before audit time.
4. Distribute the work to the functional managers.

cataloging controls

The first step is to start with the list of controls against which you'll be audited. This externally-derived regime of controls may change depending on the nature of the audit (e.g. SOC-2) or certification (PCI-DSS, ISO), but the process remains the same. You'll need to create a long spreadsheet with the controls.

This spreadsheet should include the following things:

• an identifier for your internal purposes
• the identifier currently used by your auditors (this is subject to sometimes drastic change).
• the text of the control from your auditors list
• a process manual identifier (more on this later)
• a reference to some "risks" (loss scenarios) that are mitigated by the control
• the name of the functional manager who "owns" the control

document the relevance of the controls

Controls are usually simple sentences like, "The Chief Information Security Officer shall review all active named accounts on all production servers, once per quarter." or "The Director of Human Resources shall perform a background check on all new hires, and review that background check." They have certain qualities, like naming a responsible individual, naming a concrete action, and naming a period for the activity. But outside of context, they have limited meaning: how will someone remember that these things are their responsibility; how will someone remember to do them when the moment comes?

It is important to make sure that the controls are documented in the context of the functional area where they are relevant. In the example of the human resources activity above, you'll need to make a process manual for new hires, and ensure that the control is listed in the context of the steps taken during a new hire. It's also important to indicate the loss scenarios in the process manual to provide the Director of HR with guidance on what to look for. In this case, the loss scenarios might include, "Personnel are hired to responsible positions without having been vetted during the hiring process. Those personnel, already having a criminal record, steal from the company's clients. The company is found liable for the losses experienced by the clients due to the lack of due diligence."

In this manner, all of the controls in the catalog must find a home in process manuals actually read and maintained by the functional managers.

catching deviations

The next step is to distribute the controls to each functional manager. If your organization has the resources and the institutional capability, you can use a tool like "Archer" that allows the functional managers to record their control performance in a centrally managed, database-driven environment. I have worked all manner of small, medium, and large-sized firms (Citigroup was 400,000 people when I was there) but have never seen one of those clever packages in use. Everywhere I've been, it's just been in what I've heard snidely referred to as an "Excel distributed database"—Excel workbooks distributed by email or through a shared drive volume.

How it's done, doesn't matter. It's the content of the system that matters, and it must have these components:

• The time period for the control (e.g. Jan 1, 20XX—March 31, 20XX).
• The identifier of the control (from the catalog).
• The text of the control (again, from the catalog).
• The period or trigger condition for the control (e.g. "quarterly", "annual", or "with the addition of any new production server".)
• A reference to the process manual explaining the control.
• A metric for the control: how many times the control was required.
• A measure against the metric: how many times the control was found to be in evidence.
• A description of the evidence: e.g. "Changes to the organizational structure are published to the Company in the org chart and process manuals and communicated by management in an announcement email."
• The source of evidence for the control: a list of incident tickets, for instance, or a collection of emails, or perhaps some employment records in HR's possession.
• An area for explaining any deviation, and how that will be addressed.
• An area for explaining any actual loss scenario realized, and how that is being addressed.

Once the process I'm describing here has been in place long enough to be stable, you can add some bells and whistles:

• The nature of the control (Predictive, Detective, Preventative, Corrective).
• Quantification of losses realized.
• Links to other functional areas that may experience risk as a result from the loss scenarios in this control.

All of the above can appear in a single tab in the workbook, one control per tab.

At the back of the workbook, you'll need a summary page that shows which of the controls were in place, and which were found to have deviations. It also contains the name of the functional manager and the time period being audited. And at the bottom: a place for their dated signature, and a place for yours. By countersigning the document, there's evidence that the work was done by a certain date, and by the functional manager. There's also evidence that the control performance was reviewed by you, and that any deviations and needed corrective tasks were discussed and assigned.

NOTE: It is important that the functional manager not make any changes to their assigned set of controls. I have been on both sides of this matter, as a functional manager and as an internal auditor, and I'm afraid that reason simply doesn't prevail. The decision truly lies with the service auditors, because they issue the report. They won't do so if they feel that controls have been altered or deleted and that there's a control deficiency.

distributing the work

The crucial next step is to distribute the controls to the functional managers. In the Excel workbook scenario, this means sitting down with each functional manager, and going through each of the controls in the workbook.

I strongly recommend that each functional manager be tasked with completing the workbook on a quarterly basis. It's OK that some controls are annual or "per occurrence", but assigning the work quarterly gives you several benefits:

• You resolve the impossible task of managing someone else's functional area while retaining oversight on control efficacy. The right people managing the processes being controlled.
• You catch deviations (and losses realized) long before the annual visit by the auditors. This gives you time to address those deviations.
• You teach the functional managers the entire dimension of self-governance. This makes them "risk aware" and "builds a risk culture" and other things that appear in articles in the risk periodicals. In actual practice, it does help people bear in mind that a repeatable process is an efficient and effective one.
• With each functional manager doing your legwork when it comes to gathering evidence and pointing out developments in operations processes, you cut out 99.9% of the work when the auditors are do. I've run audits where I'd simply have to gather the evidence amalgamated during our quarterly internal reviews and hand that over to the auditors as is. The auditors love it: the evidence comes with dates that are appropriate to the activity, because the evidence wasn't assembled (or created) months later; the evidence is typically much more complete and accurate; it's already filed in a sensible fashion (e.g. against the control number); and if there were deviations, these are far more likely to be properly documented and possibly addressed. This is how you can do an annual service audit in only 8 hours.

On the importance of executive support

2015.04.01

I polled a number of service organizations that undergo annual audits about the biggest contributing factors to successfully preparing for those audits. The results were interesting.

self-policing control environments

2019.10.28

An important form of risk control is one that monitors and moderates other controls; this is best run on a calendar by the people who "own the risk".