Management of Exceptions and How to Manage Them

During SOC 1 Type 2 examinations, which analyze both the design and operating effectiveness of your controls, deviations from the stated control process must be disclosed within the service auditor’s testing results, often referred to as testing “exceptions” or “deviations” as they are exceptions from the stated control activity.  The identification of at least one testing exception is a common occurrence, whether it is due to an outage, failure to document a manual process, or a simple oversight.  There are a few questions, however, that you can ask both your auditors and yourselves to help manage the exceptions.

One of the first questions you should ask when a testing exception is identified is two-fold:

Design

Testing exceptions related to the design of a control activity could result in a qualification of the entire control objective area if this issue is significant and pervasive, so it is worth having a discussion with your auditor about the given control to confirm that it properly reflects your environment.  For example, a control might state that a financial processing job is configured to run on a daily basis to support a control objective for timely data processing.  Let’s assume that your auditor inspected your job configurations and notes that the processing job is actually configured to be performed on a weekly basis.  In this instance, the control may be poorly designed for the stated objective of timely transaction processing.  Additionally, the auditor may also inspect the transaction processing policy and procedure documentation to determine whether the processing job was intended to be performed daily or weekly.  If the policy states that processing jobs occur daily, then this would be a design flaw.  However, if the policy states that processing job is intended to be performed on a weekly basis, then your auditor would likely need to discuss the discrepancy between the policy and the stated control with your management to ensure the controls are accurately stated, and if additional procedures are necessary to determine if effectively designed controls are in place to achieve the stated objective. 

Operating Effectiveness

Testing exceptions related to the operating effectiveness of a control activity could also result in a qualification of the entire control objective, if they are found to be pervasive and significant to the achievement of the objective.  A key consideration when determining the impact of the testing exception is the criticality of the failed control to the achievement of the objective, and the breadth of the exception (e.g. did it fail for 15 of 25 job processing runs or was it only one of 25).  Each of these can be discussed with your auditor, but ultimately the service auditor would need to determine if the control alone or in combination with the other effectively designed controls operated effectively to achieve the control objective.

If we revisit our example above, assume that the processing job was configured to run on a daily basis consistent with the stated control activity and the policy, however the job failed to complete for several days in a sample of days.  This breakdown in the operation of the control would result in operating testing exceptions.

After potential exceptions are uncovered by the auditors, a vetting process should take place to determine if any further information could provide sufficient evidence for the sample in question.  If the exceptions are confirmed, the next stage involves the exception wording that will appear in the report.  Exception wording appears in the section reserved for information provided by the service auditor (this is typically in matrix format in a dedicated section of the SOC 1 report) and would disclose the nature and extent of the testing deviations.

Two great questions that can be asked of your auditor at this point are:

1. Q: Can any additional testing be done to show that the exceptions noted were not pervasive?

A: In some instances where testing identifies a single exception in the original sample, auditors can increase their sample size to determine if additional deviations in operation are present.  If the additional sample size finds no further exceptions, the disclosure about the one exception will remain, however, the control activity may be deemed to have been operating effectively.

2. Q: Can any subsequent testing be performed to show that a given exception was resolved after it was noted during the audit?

A: Continuing with our example above, consider if the auditor performed additional testing and found that the job was in fact, manually re-run within hours after the initial failure and the computer operator noted the successful completion in a system log for each date previously noted.  This subsequent testing essentially shows resolution of the previously noted exception.

The last question that should be asked after exception wording has been decided upon is whether or not you’d like to insert a management response to the exception (typically found in a dedicated section of the report).  Management responses to testing exceptions are not covered by the service auditor’s opinion, but often times provide useful information regarding the causative factors of the noted exception to the users of the report. 

In regards to inserting a management response to an exception there are often two schools of thought:

1. Some organizations feel that by responding to testing exceptions, they only draw further attention to the exception that occurred.

2. Some organizations feel that an explanation is warranted in the circumstances, which may include a description of the causative factors and/or corrective plans for remediation

As noted, there are quite a few things to consider when determining if a response should be made.  Understanding the users of your report and their informational needs will largely determine your selected course of action. 

After an exception is noted by your auditor and confirmed, regardless of the way in which it is represented in the SOC 1 report, it is important to reflect upon the root cause of the exception.  Oftentimes exceptions can be useful in improving business operations or information technology practices; or at the very least, help to reduce the chances of future exceptions.