A Breakdown of FedRAMP Penetration Test Guidance 3.0

For the first time since 2017, the FedRAMP Project Management Office (PMO) has updated the Penetration Testing Guidance document. 

For Cloud Service Providers (CSPs) seeking FedRAMP Authority to Operate (ATO), that’s important news. What will be more important is ensuring you understand the changes. Through direct conversations and various surveys, Schellman’s Pen Test Team provided feedback to the PMO—feedback this update took into consideration. During that time, we gleaned considerable insight that we’d now like to pass on to help you.

In this article, we’ll do two things: we’ll provide a rudimentary summary of what’s new in the guidance and we’ll also dive more technically into the six attack vectors that must be performed. Whether you’re looking for a brief overview of what’s new or a detailed deconstruction, you’ll have a better idea of what is required for a FedRAMP pen test. 

What’s New in the FedRAMP Pen Test Guidance 3.0

Here’s a high-level overview of the update:

• Renamed attack vectors
• The Internal network attack vector is now merged with the External network attack vector, with an updated focus on “Internal Threat Models” 
  • 3PAOs and CSPs are now urged to consider types of insider threats and determine how to best minimize these threats, though the guidance is purposely written as vague, allowing for many ways to accomplish this objective. 
• Clarification on testing client-side applications
• All testing must now be performed in the production environment (no staging/QA)
• More details on what is expected of CSPs regarding Attack Vector 1: External to Corporate (Phishing):
  • You must allow phishing attacks past all technical controls in place to see how users would respond.
  • Any data submitted within the campaign, real or not, is to be considered a failure of the test.
  • You must provide security awareness training and password rotation for any users that fall victim to the phish at the end of the campaign. 

Taking all that into account, we’ll delve a little deeper into specific attacks and how they’ll work. If you’ve engaged Schellman before as your third-party assessment organization (3PAO), some of the following might seem familiar. Our team has always approached the FedRAMP pen test attack vectors as real-world attackers to demonstrate real impact to our clients—this updated guidance indicates that the FedRAMP PMO now aligns with that mindset.

As a reminder, six different attack vectors are in scope of a FedRAMP penetration test, each designed to simulate a different, realistic threat. To dive deeper into each attack vector, keep scrolling or choose from the list below:

1. External to Corporate (Phishing)
2. 2. External to CSP Target System (External Network / Insider Threat Assessment)
3. Tenant to CSP Management System (Web Application)
4. Tenant-to-Tenant (Web Application)
5. Mobile Application to Target System (iOS and Android)
6. Client-Side Application and/or Agents to Target System

Attack Vector 1: External to Corporate (Phishing)

Attack Type: Execution of a social engineering (phishing) attack targeting your system administrators and managing personnel who may influence system administrators.

(It is possible to expand beyond credential harvesting e-mail-based attacks—discuss different possible scenarios involving script or file execution during the planning stage.)

How Does It Work: Your employees will be tested with a “worst case scenario”—a phishing attack has made it to their inbox, what will they do next? Will they fall victim to a sophisticated social engineering attack?

• You will provide a list of employee names and e-mail addresses to make up the target list.
• The phishing campaign must be allowed through any technical preventive controls such as e-mail filters or web content filtering proxies.
• Any data submitted within the campaign, real or not, will be considered a failure of the test. 

Follow-Up Action: Upon completion of the campaign, You must provide security user awareness training to those employees who fell victim to the phish and provide evidence that their credentials have been rotated. 

Schellman’s approach to sampling with phishing: If more than 200 employees fall within the targeted list, a sampling approach of down-selecting to 200 employees will be utilized. 

Attack Vector 2: External to Target System (External and Internal FedRAMP Boundary)

Attack Type: This attack vector includes two different types of network assessments:

• External threats: An Internet-based attack as an uncredentialed attacker attempting to gain unauthorized access into the FedRAMP boundary.
• Internal threats: Attempted exploitation of weak permissions/access controls and poor customer separation measures—e.g., improper network segmentation and poor implementation of security controls—as well as abuse of system services. 

External Threats

How Does It Work: As an unauthenticated attacker on the Internet, the Pen Test Team will perform active reconnaissance, vulnerability scanning, and manual testing to identify and exploit any vulnerabilities on Internet-facing hosts within the production FedRAMP boundary. 

During testing, all external endpoints are understood and all passive or active blocking security devices—such as web application firewalls and or software-based security controls—are bypassed.

Internal Threats

How Does It Work: During your FedRAMP boundary overview call, penetration testers will join the call alongside the FedRAMP Manual Controls Team and work with you to understand how you currently authenticate into the boundary while also reviewing the types of attack paths that could be accessible from different “Internal Threat Models” (Guidance Section 3.1.2). 

More specifically, we will:

• Discuss the possibility of vulnerabilities or misconfigurations impacting the FedRAMP authorization boundary.

• Assess the controls implemented to prevent an attacker from pivoting from one internal network segment to another.

• Talk through other types of issues that a commercial vulnerability scanner may not identify (assuming that you will have already performed authenticated vulnerability scans within the boundary).

Apart from controls that are inherited or are interconnected, e.g., boundary authentication and authorization mechanisms that leverage your corporate Active Directory (AD) deployment, networks and hosts outside of the FedRAMP boundary will not be in scope for this insider threat assessment. In this case, the corporate AD infrastructure would be considered in scope and all other corporate assets would remain out of scope.

Attack Vector 3: Tenant to CSP Management System (Web Application)

Attack Type: A full application penetration test that attempts to access your management systems through misconfiguration, a flaw in system design, abuse of intended function, low-code or no-code software deployment, and/or command line interface (CLI). Intended to identify any opportunity that privileged customer accounts would have to compromise your underlying system architecture.

How Does It Work: You would provide your Pen Test Team with privileged level accounts to applications within the production environment, which they would then use to facilitate and identify scenarios where the attacker may go from unauthenticated access to authenticated access to privileged level access. You must provide the highest level of permissions available to customers to conduct this kind of test.

While you might prefer to evaluate a tenant within your development/test environments, these are rarely identical to the production deployment, and so they cannot be used as a valid representation for the FedRAMP penetration test vectors.

Attack Vector 4: Tenant-to-Tenant (Web Application)

Attack Type: A full application test where your Pen Test Team will attempt to use provisional access one of your customers might have to compromise another client.

How Does It Work: You will need to provide two full production customer tenants and their granted access methods that mirror those used by your customers. Your environment must also be set up to test all aspects, including authentication, data access, user permissions, and sessions.

As an authenticated user of the application, the Pen Test Team will focus on gaining access to the other tenant’s data, attempting vertical and horizontal privilege escalation. They’ll also seek to identify and exploit vulnerabilities in the application that could potentially lead to gaining access to the other tenant.

Attack Vector 5: Mobile Application to Target System (iOS and Android)

Attack Type: Simulation of a mobile application user attempting to access your target system or your target system’s mobile application. 

How Does It Work: Your Pen Test Team will assess the in-scope Android and/or iOS application(s) and review how it/they handle(s) authorization, as well as cache functionality, data storage, encryption, logging, and other functionality to identify potential local vulnerabilities that could lead to a breach in your environment. 

(If your app contains SSL pinning or root/jailbreak detection, please provide builds that have these protections disabled to expedite testing.) 

Attack Vector 6: Client-Side Application and/or Agents to Target System

Attack Type: Investigation to ensure any client-side applications do not introduce any new vulnerabilities to the host operating system (OS) nor do they insecurely store sensitive data locally. 

How Does It Work: You’ll list any in-scope client-side applications, i.e., components installed locally within a customer environment. These must be included in your FedRAMP authorization boundary and tested. The Pen Test Team will download and install those applications/components on a Virtual Machine (VM)—snapshots of the VM will be made before and after the install. In addition, any traffic will be proxied and reviewed.

• If these applications are essential for your customer's use to interact with the environment or application, they must be included in your authorization boundary and tested as part of your system boundary security assessment. These may include (though not exclusively):
  • Appliances
  • Browser extensions
  • Thick clients
  • Agents
• However, if they’re optional use, these may be included in your tested authorization boundary if your customers and you agree they should be.
  
  

Next Steps for Your FedRAMP Pen Test

All of these will need to be performed as part of your FedRAMP preparation moving forward. (The Schellman Pen Test Team will be implementing this methodology on all FedRAMP projects beginning August 22, 2022.)

However, if a specific attack vector cannot be performed, it will be noted in the SAR as a deviation from the Penetration Testing Guidance. Moreover, understand that a 3PAO might see non-conformance to testing a particular attack vector as a High Risk finding in the SAR Risk Exposure Table (RET).

If you feel that testing the attack vector would negatively impact your production system, you can submit a non-conformance justification for why it cannot be tested to an Authorizing Official (AO), but that may result in delaying your FedRAMP authorization since the AO will need to understand and agree to the deviation or non-conformance. 

Such a technical breakdown of these six attack vectors and their details is a lot to take in, so we encourage you to reach out to our Pen Test Team should you have any questions about these changes.