How to Prepare for Your Schellman Penetration Test

You know the phrase, “hindsight is 20/20.” It’s a well-trodden lament that can apply to almost everything—a failed job interview, an embarrassing misstep during a conversation, an insistence that the professor won’t include information from his footnotes in the final exam while studying.

Everything always looks clearer after the fact—compliance projects included. There’s always part of the process that could have gone smoother, or a hurdle that you could’ve avoided had you just known.

For those of you seeking or preparing for a penetration test (a.k.a., pen test), we’d like to offer such helpful foresight. Just last year, Schellman completed over 200 penetration tests , many of which were for clients who asked for more than one service. With all that experience—from the last year and more—we have a great understanding of what separates a successful from a not-so-successful pen test.

To achieve the former, a lot of the action you need to take precedes the start of your pen test with Schellman. In this article, we’ll detail six common issues our team often faces when starting new client engagements—both before they start, and during the process.

Using this insight, you’ll be able to prepare better and pave a smoother way forward during your experience with us.

6 of the Biggest Problems to Avoid Ahead of Your Pen Test

1. Failure to Authorize the Pen Test

Unfortunately, the largest problem we run into is also associated with the most critical component—the Pen Test Authorization Letter. As this document defines the scope of the engagement, without this document completed and signed, we cannot begin active testing.

In our experience, several things can cause delays here:

• Key personnel being out of the office
• Uncertainty on who has the authority to sign or which hosts should be in the defined scope
• A need for multiple reviews and approvals, such as a product team, a chief information security officer (CISO), and/or legal department

These factors and their effect on the Authorization Letter can derail your pen test before it even begins. The more delay at the start like this, the less time we have to perform the services you need within the allotted time. But more than a time crunch, things can even cascade to the point where we don’t have enough time to conduct a full assessment.

To avoid this issue caused by failure to sign, consider the following:

• Know and alert individuals that need to sign and/or review.
• Keep a look out for communication from Schellman regarding the Authorization Letter.
• When you receive it, prioritize getting the Authorization Letter to the correct individuals.
• Validate the scope with your technical team(s).

2. Dysfunctional Testing Environment

Particularly when dealing with web application pen tests, we have been put into situations where the testing environment was not ready. This can happen for a variety of reasons, but here are some common reasons for that:

• Incomplete quality assurance (QA) validation of the pen test environment
• Firewall rules blocking access to the testing environment
  • If you know that your application requires allowing specific IP addresses inbound, ensure that your network or security team allows access to Schellman’s source IP addresses provided in the Authorization Letter.
  • In specific scenarios where your application is not exposed to the Internet, ensure that Schellman receives a hardware device and/or VPN instructions.
• Minimal resource allocation of CPU or RAM
  • Application resources must be adequate, and pen testers need more than the average developer or user. For instance, the foundation behind a quality web application pen test is to balance three things:
• Adequate application coverage
• Growing web application sizes and features
• An increasing list of attack scenarios
  • It’s not unusual for this balancing act to generate over 100,000 requests to the application throughout the pen test. So, if you know that your application features some form of resource bottleneck—like high CPU or RAM usage—ensure ahead of the test that the application servers have enough provisioned.
• Pen test environment not stood up
  • This may be separate from your production environment, and if so, supporting infrastructure may not be in place or specific application configurations may no longer be valid.
  • Moreover, sometimes, the team coordinating the pen test and the team responsible for standing up the pen test environment are not in sync. Validate expectations with your system administrators before final approval of a pen test date. If an environment is not stood up, we have nothing to test! 

To avoid all these issues with your pen test environment, have a technical point of contact validate that the application and major features function as expected ahead of the start of testing.

3. Failure to Provide Credentials

For authenticated web application pen tests, we require you to provide us with at least two sets of credentials for each tenant:

• Application administrator (higher privileged)
• Application user (lower privileged)

In some cases, we can register our own accounts, but in others, you’ll need to provide them. Failure to provide both a higher and lower privileged user account can significantly delay the web application portion of your pen test.

To prevent such setbacks, before the start of your pen test, coordinate with your technical team to create these accounts and provide us with the credentials via our secure platform, AuditSource. Once we receive the credentials, we’ll validate them as soon as possible and get ready to start.

4. Failure to Allow Inbound E-mails or Execution of Payloads

When dealing with specific attack vectors, some scenarios may require specific technical controls to be bypassed. Why? Because while technical controls can be quite effective in blocking these specific attacks, there are some tests where they are not the subject, such as:

• Social engineering exercises (usually e-mail-based phishing)
  • These are meant to test how your employees will respond when presented with a convincing phishing attack, not the effectiveness of your e-mail security product or web proxy.
• Assumed breach internal network pen tests (usually a binary payload executed on an internal corporate device)
  • Similarly, the end goal of this assessment is not to test your anti-virus (AV) or endpoint detection and response (EDR) solutions; rather, it jumps into a post-exploitation scenario where a breach has occurred and an attacker has a foothold on your workstation to determine what they can do next. 

We recognize that you may have policies in place which do not typically permit the execution of such files. Your controls are there for a reason, after all. But an attacker—such as an advanced persistent threat (APT)—has the luxury of unlimited time to figure a way to bypass them. However, a penetration test is a timeboxed assessment, giving pen testers a limited window—that time is best spent demonstrating the impact of what could happen if your technical controls were to fail.

Another important note here: If you’re performing a pen test to support a FedRAMP initiative, the latest FedRAMP Penetration Test Guidance document issued by the project management office (PMO) now clearly defines the expected behavior of a cloud service provider (CSP) regarding this topic.

5. Poorly Defined Scope

While conducting reconnaissance of your provided scope, we may run into hosts that are not defined within the Authorization Letter. In the vast majority of cases, these hosts have been correctly left out, though sometimes one or two may have been missed for inclusion.

When that happens, we’ll reach out to our point of contact to validate. Please understand that if we then modify the scope, that could mean additional fees and require more testing time depending on how many new hosts are added.

To reduce such impediments, validate with your technical team(s) that the scope that will be provided within the Authorization Letter is accurate and that all in-scope hosts are provided and correct.

How to Set Your Penetration Test Scope

6. Lack of Communication

Sometimes, we run into:

• Delayed responses; or
• Missing appropriate stakeholders on communications.

To improve communication, we have a few recommendations:

• Ensure that representatives for all support teams are on all weekly status calls.
• Exchange cell/office phone numbers between all team members involved.
• Create an online, shared, real-time chat method (Slack or Teams channel).

What Happens if You’re Not Properly Prepared for Your Pen Test?

All of these factors can throw serious wrenches into the engagement. If our team decides that things have been affected to the point that the pen test is not adequate due to these issues that are outside of our control, please be forewarned that one or more of the following can occur:

• A reschedule of the pen test (which may be weeks in the future depending on our availability)
• Clear distinction of what could not be tested within the pen test report
• Additional fees necessary to complete the pen test at a future date
• Potential compliance impacts 

Getting Ready for Your Schellman Pen Test

As ominous as all that sounds, the truth is that the vast majority of our clients do an amazing job in their preparation. And now, you won’t need hindsight either because you’re armed with information to help you do the same in avoiding all these potential obstacles.

So that you set yourself in the best possible way in your setup for your pen test, make sure you read our other content that can help with that:

• What Is Schellman’s Penetration Test Project Process?
• The Intricacies in Pen Test Timing