Compliance Penetration Testing: Which Frameworks Actually Require It

Regulatory position current as of 3 September 2026. Confirm applicability with your assessor or counsel.

Three of the frameworks driving penetration testing purchases name it explicitly. Three do not, and the difference decides whether you are buying a mandate or a best practice. PCI DSS Requirement 11.4 requires it directly, with external and internal testing at least annually and after significant change, segmentation validation, and a tester who is organizationally independent. NYDFS 23 NYCRR 500.05 requires annual testing plus vulnerability assessments twice a year. The FTC Safeguards Rule makes an annual test the fallback where you do not run continuous monitoring. HIPAA never uses the term. SOC 2 does not require it in the Trust Services Criteria. NIST SP 800-171 does not mandate it either. Auditors and assessors frequently expect testing under all three of those anyway, which is a different thing from a requirement and should be bought differently. We recommend you establish which category you are in before scoping anything.

Overview

• A scan is not a penetration test. They are separate obligations with different frequencies, providers, and outputs, and conflating them is the most common error in this space.
• Three frameworks name it, three do not. PCI DSS, NYDFS, and the FTC Safeguards Rule are explicit. HIPAA, SOC 2, and NIST 800-171 are not.
• Independence is a stated requirement in PCI DSS. Your provider’s relationship to the environment matters, not just its skill.
• Automated output does not satisfy PCI DSS. The standard describes penetration testing as a highly manual process.
• The report is the product. Scope statement, control mapping, evidence, and a retest are what an assessor actually needs.

The 5 Why’s

This is written for compliance officers, IT directors, and security leads at organizations between roughly one hundred and a few thousand employees who have been told they need a penetration test and are working out what to buy. The instruction usually arrives without the detail that determines the scope.

The requesting party varies and so does the answer. A payment brand or acquiring bank points at PCI DSS. A New York financial regulator points at Part 500. An FTC-covered financial institution faces the Safeguards Rule. An auditor preparing a SOC 2 report asks for testing as evidence rather than as a requirement. A healthcare organization is usually responding to its own risk analysis obligation or to a customer questionnaire. A defense supplier is often responding to a prime’s expectation rather than to a control in NIST 800-171.

Sector conditions shape the scope. Organizations handling cardholder data have the most prescriptive obligations of any group here and the least room for interpretation. Financial services firms in the New York market face an explicit annual cadence with retention requirements attached, relevant across our financial services work. Healthcare organizations work through the Security Rule’s evaluation standard, which is broader and vaguer, an area we also address in our healthcare practice. Manufacturers and defense suppliers increasingly face testing expectations imposed by customers rather than regulators, which means the customer defines what counts.

The consequence of buying the wrong thing is paying for a technically competent test that your assessor cannot accept as evidence, then paying again inside your audit window.

A Scan Is Not a Penetration Test, and a Penetration Test Is Not a Scan

This confusion costs money in both directions, and it is worth resolving before anything else.

A vulnerability scan is automated. It compares your systems against known weaknesses and produces a list by severity. It is fast, inexpensive, and best run frequently. A penetration test is human-led. A tester attempts to achieve an objective in your environment, chaining weaknesses, exercising logic flaws, and proving what an attacker could actually reach. PCI SSC describes penetration testing as a highly manual process, and that framing has a practical consequence: an engagement that was substantially an automated scan with a narrative wrapped around it can be technically useful and still fail as PCI DSS evidence.

Where this bites hardest is PCI DSS, because the standard imposes both obligations separately. Quarterly external scanning sits under Requirement 11.3 and, for the external scan, must be performed by an Approved Scanning Vendor. Penetration testing sits under Requirement 11.4 on a different cadence. Organizations regularly buy quarterly scans and believe they have satisfied 11.4, or buy an annual penetration test and believe it covers the scanning obligation. Neither holds, and a QSA will identify the gap.

The same distinction matters outside PCI DSS even where the language is looser. An auditor asking for evidence that your vulnerability management program identifies and remediates exploitable weaknesses will not accept a scan report as proof that anything was exploitable. That is what a test demonstrates. Getting this right is most of what we do in scoping conversations on penetration testing, because the wrong purchase is usually a scoping failure rather than a provider failure.

Which Frameworks Explicitly Require Penetration Testing?

Three, and each specifies something different, so the requirement rather than the framework name should drive your scope.

PCI DSS, Requirement 11.4. The most prescriptive regime in this set. It requires a documented testing methodology covering network and application layers, external testing at least every twelve months and after any significant change, internal testing on the same basis, correction of exploitable findings with testing repeated to verify the fix, and validation that segmentation controls actually isolate the cardholder data environment. Service providers carry additional segmentation testing on a six-month cycle, and multi-tenant providers face a separate obligation to confirm logical separation between customer environments. All of the future-dated version 4.0 requirements became mandatory on 31 March 2025, and assessments conducted now align to version 4.0.1. Two points catch organizations out: the tester must be organizationally independent, and PCI SSC does not define what counts as a significant change, which makes that trigger a judgment call your QSA will review.

NYDFS, 23 NYCRR 500.05. Annual penetration testing of information systems based on identified risks, plus vulnerability assessments twice a year. The second amendment introduced tiered obligations by organization size, and the annual testing requirement was retained for covered entities other than those qualifying as limited under employee and revenue thresholds. Results must be retained as part of your documented cybersecurity program, which means the report is an artifact you keep rather than a project you complete.

FTC Safeguards Rule, 16 CFR 314.4(d). Conditional in a way most summaries miss. The rule asks for continuous monitoring, or, where you do not have it, annual penetration testing plus vulnerability assessments every six months. The annual test is the fallback rather than the default, which means organizations with genuine continuous monitoring may satisfy the requirement without an annual test at all. The rule took full effect for non-banking financial institutions in June 2023 and reaches a wide range of organizations significantly engaged in financial activities.

Applicability is where organizations most often go wrong, particularly in insurance, where the business of insurance answers to state insurance authorities rather than to the FTC, making the relevant rules the state model law or, in New York, Part 500. Organizations subject to more than one of these should scope once and map the evidence to each, since a single well-scoped engagement can serve multiple obligations if the report carries the right control mapping. Organizations that qualify for a tiered exemption should confirm it in writing rather than assuming, because the thresholds turn on measures that change as a company grows.

What we recommend you do about it:

• Identify the specific citation, not the framework. Requirement 11.4 and Requirement 11.3 are different purchases.
• Confirm your applicability tier. Exemptions and thresholds exist and they change with headcount and revenue.
• Check whether continuous monitoring gets you out of an annual test. Under the Safeguards Rule that path exists.
• Ask your assessor what they will accept before scoping. They are the audience for the report.
• Diary the significant-change trigger, not just the annual date. Under PCI DSS that is where findings originate.

Which Frameworks Do Not, and What Do Auditors Expect Anyway?

Three, and in each case testing is common practice rather than a control you can be cited for omitting. That distinction matters commercially, because vendors sell against these frameworks as though the requirement were explicit.

HIPAA. The Security Rule does not use the term penetration testing. The hook is the evaluation standard at 45 CFR 164.308(a)(8), which requires periodic technical and non-technical evaluation of your safeguards in response to environmental or operational change, alongside the risk analysis obligation. A penetration test can serve that evaluation well and is not the only way to satisfy it. Worth noting that the proposed overhaul of the Security Rule, which would have introduced explicit testing and scanning cadences, remains at the proposed stage and is not current law, so anyone selling you compliance against those provisions is selling against a rule that does not yet exist. Our healthcare work reflects the current rule rather than the proposal.

SOC 2. Not required by the Trust Services Criteria. In practice it is the most useful evidence available that your vulnerability management controls identify and remediate exploitable weaknesses, and most auditors expect to see it. What they actually want is a set of artifacts: a scope statement, evidence of tester independence, the methodology used, severity ratings, remediation timelines, and proof of closure, plus a demonstration that findings feed patching, change management, and incident response.

NIST SP 800-171 and CMMC. The 110 requirements do not include penetration testing, and a Level 2 assessment does not require you to have had one. Testing may still be worth doing, and primes sometimes ask for it contractually, but the obligation in that case comes from your customer rather than from the framework.

How much to spend here depends on who is asking. Organizations pursuing a SOC 2 report should time testing to the audit window, early enough to remediate and retest so the auditor sees the post-fix state. Organizations under HIPAA with a mature risk analysis process may get more from targeted testing of the systems handling protected health information than from a broad annual engagement. Organizations facing a customer expectation rather than a regulatory one should ask that customer what evidence satisfies them, because the answer is sometimes a completed questionnaire or an assessment rather than a test. And any organization told it needs testing for a framework in this second group is entitled to ask which control requires it, since the honest answer is that none of them do.

What we recommend you do about it:

• Ask which control requires the test. For these three frameworks there is not one, and a provider should say so.
• Time SOC 2 testing to the audit window. Early enough to fix and retest before field work.
• Under HIPAA, tie testing to your risk analysis. The evaluation standard is the hook, so the test should reference it.
• For customer-driven requests, ask what evidence they accept. Sometimes it is not a test at all.
• Do not buy against the proposed HIPAA rule. It is not final and may be narrowed or withdrawn.

What Makes a Report Count as Compliance Evidence?

Independence, manual work, an explicit scope, control mapping, and a retest. A technically excellent report missing any of those can fail as evidence, and that failure is discovered at the worst point in the cycle.

Independence is a stated requirement under PCI DSS 11.4.2, and it has an implication worth being direct about: a provider that manages the environment it is testing is not organizationally independent of it. That constrains who can perform your PCI testing, and it is a reason to separate the party that runs your environment from the party that tests it, whatever the framework. We will tell clients when that separation applies to us, in the same way we do on compliance engagements generally.

The rest is documentation discipline. A scope statement should say what was tested and what was excluded and why, because a test that quietly omitted your most important system is worse than none. Control mapping ties each finding to the specific requirement it bears on, which is what allows a QSA or an auditor to accept the report without additional work. Evidence should include reproduction detail sufficient for your team to verify a fix. And retest matters more than most buyers realise, particularly under PCI DSS, where 11.4.4 requires that exploitable findings be corrected and testing repeated to verify the correction, so an engagement with no retest provision leaves you short of the requirement. Ask to see a redacted sample report before contracting, since sample quality predicts yours better than any capability discussion.

What we recommend you do about it:

• Confirm independence against the framework, not against good intentions. Under PCI DSS it is a stated control.
• Require control mapping in the deliverable. Findings tied to the citation your assessor checks.
• Require retest and reissue. Under PCI DSS the verification of corrections is part of the requirement.
• Get an explicit scope statement including exclusions. What was not tested is as important as what was.
• Ask for a redacted sample before you sign. It is the single best predictor of what you will receive.

How to Scope This Properly

Work backward from the citation rather than forward from the service. That single change avoids most of the waste in this category.

Start by getting the requirement in writing from whoever asked: which framework, which control, what scope, what evidence format, and by when. If the answer is a framework in the second group above, ask which control requires a test, because none of them do, and the honest answer changes what you should buy and how much.

Then separate scanning from testing in your plan, since several frameworks require both on different cadences and neither substitutes for the other. Then check independence against the framework’s own language, and separate the party that manages your environment from the party that tests it wherever a standard requires it.

Then judge the deliverable rather than the engagement. Scope statement with exclusions, control mapping to the citation, evidence you can act on, and a retest that puts the post-fix state in front of your assessor. Ask for a redacted sample before you contract, and tell your provider your audit field work date on the scoping call so testing, remediation, and retest sequence correctly around it.

If you have been told you need a penetration test and cannot name the control behind it, that is the place to start rather than a proposal. Schedule a consultation to work through which requirement actually applies to you.