Get the Pen Test Report You Deserve: A Leader’s Guide

You think you’re close to picking the right team. Your goals align, and you think the team is of sufficient quality. But, there’s one aspect that can be easily overlooked – yet it may ultimately determine whether the exercise was worth conducting. 

Your Team: Reporting

The report should always contain, at a minimum, the testing conducted, as well as any findings and remediation steps. However, one area where providers often fall short is effectively demonstrating the level of effort invested. This key component serves to provide confidence that your budget was spent wisely, and confidence that your security was assessed diligently. Finally, we strongly believe in over-communication: weekly status updates, detailing current findings and roadblocks, are just as important as the final report.

To help identify potential issues when selecting a provider, we’ve compiled a list of questions. Asking these questions can help uncover pitfalls you might encounter with a specific assessor:

How soon can I expect the final report?

Even if you’ve agreed on a test timeline, don’t assume the final report will be delivered on the last day of testing. Keep this in mind when you are scheduling your pen test and verify when final delivery is expected.

How often will your team provide official updates?

Ideally, the team will provide updates as they occur – especially with respect to any high-risk findings or impediments. However, establishing a required weekly status update will place more onus on the team to notify you of findings and impediments as they arise.

How soon will you notify us of significant findings?

For high-risk findings, we recommend requesting notification as soon as reasonably possible, ideally within 24 hours. For lower-risk findings, we think it’s acceptable to be notified within a business week.

Do you report findings that are un-exploitable/implausible?

Some findings, particularly from automated tools, represent “best practices” rather than actual vulnerabilities. Others may qualify as vulnerabilities but may be implausible in real-world scenarios – such as when no exploit exists for the identified vulnerability. Whether to address these issues ultimately depends on several components: your risk tolerance, business impact, threat profile, and any applicable compliance requirements.

Some organizations may prefer all findings to be reported, regardless of impact threshold, while others may adopt a more strategic approach to better prioritize their limited remediation resources. Ultimately, it’s up to the provider to determine whether a finding meets their reporting threshold – but it’s always beneficial to level-set these thresholds with your provider.

Will your report contain evidence of findings? And remediation steps?

A report that doesn’t show how to reproduce a finding, step-by-step, is of limited value to your technical teams. Likewise, some teams may be uncertain of how to remediate a specific issue and need clear next steps. Establish replication and remediation steps as a baseline expectation for your provider.

Will there be details inside of the report explaining the work actually performed?

Depending on your organization’s security program maturity and the defined scope, it’s entirely possible that there will be zero findings to report. In that scenario, it can be challenging to determine whether the absence of findings reflects a mature platform or a lack of thoroughness in the provider’s assessment. In these instances, you should not expect a basic report with little detail. The provider should go above and beyond by detailing what methods were attempted and explaining why they were unsuccessful.

What will your team do if we discover you missed a finding?

An unfortunate reality of pen testing is that findings can occasionally be missed. We stand by the fact there’s no pen test out there that guarantees the identification of 100% of all known and unknown vulnerabilities – if your provider claims otherwise, we strongly recommend choosing another provider.

There are many instances where a finding can be missed. Ideally, the finding wasn’t truly missed – it's not unusual for “missed findings” to be introduced into the scope after the pen test is completed. However, it’s still possible that the finding was overlooked due to a lack of knowledge on the tester’s part. Or, perhaps a new CVE was issued for the vulnerability at the end of the engagement. Concerning possibilities such as an entire portion of the scope being missed, are also possible: a mis-filled scoping document or an oversight by the tester. In any case, if the error was caused by the provider, we'd expect to see some actionable next steps from the provider to improve the relationship, advance their practice, and avoid future misses.

Conclusion

Reporting is a critical component of any pen test engagement. The report should clearly outline findings, provide remediation steps, and serve as evidence of the work performed. Additionally, the provider should deliver periodic updates, highlight impediments, and notify you of high-risk issues as quickly as reasonably possible. By asking these questions, we believe you’ll be better equipped to determine whether a provider is the right fit for your organization. If you think our industry-leading approach aligns with your goals, we encourage you to fill out our brief pen test scoping questionnaire, and a pen test leader will reach out!