On August 25, 2026, the Cybersecurity and Infrastructure Security Agency published “A Tale of Two SOCs,” a detailed comparison of simultaneous red team assessments against two critical infrastructure organizations. The results are a useful reminder for every enterprise security team: having security tools is not the same as being secure.

CISA used similar attacker tradecraft against both organizations. In both environments, the red team eventually achieved full domain compromise and reached sensitive business systems and cloud resources. The major difference was how each organization detected and responded to the activity. Organization A failed to identify and contain the intrusion. Organization B detected the initial compromise, isolated affected systems, and forced the assessment into an “assume breach” model so testing could continue.

That contrast gets to the heart of why penetration testing matters.

A vulnerability scanner can identify outdated software, exposed services, weak configurations, and known CVEs. Those findings are valuable, but they do not necessarily tell an organization how an attacker could combine multiple weaknesses into a meaningful attack path. A penetration test goes further by validating whether weaknesses can actually be used to gain access, escalate privileges, move laterally, or reach critical information.

CISA’s assessment of Organization A provides a strong example. The red team found a web application using default credentials and used it to send phishing emails from an internal address. After gaining access to workstations, the team performed Active Directory discovery and identified weaknesses involving Machine Account Quota and misconfigured Active Directory Certificate Services templates. Those issues helped the red team increase its access and move through the environment.

None of these weaknesses should be viewed in isolation. The real risk came from how they connected.

This is why enterprise penetration testing should focus on attack paths instead of producing a long list of scanner findings. Testers should ask what happens after initial access. Can a standard user enumerate sensitive systems? Can credentials be recovered from workstations or configuration files? Can an attacker abuse certificate services, service accounts, application permissions, or cloud identities? Can endpoint controls detect the behavior? Most importantly, will anyone respond?

CISA found that alert quality played a major role in the different outcomes. Organization A had so much alert noise that meaningful activity was lost among routine events and false positives. Organization B had better baselines and more finely tuned monitoring, allowing suspicious activity to stand out. CISA also highlighted organizational silos, unclear escalation procedures, excessive cloud permissions, long-lived credentials, and insufficient controls around workload identities.

Those findings should influence how enterprises scope their own security testing.

A mature penetration test should evaluate more than the external perimeter. It should include identity architecture, Active Directory, privilege escalation paths, segmentation, endpoint defenses, cloud services, application permissions, and incident response. Testing should also validate whether existing controls behave as expected. An EDR platform that generates an alert is useful. An alert that nobody investigates is not a functioning security control.

Just as important, findings should result in remediation and retesting. Fixing a single vulnerability without addressing the surrounding identity, monitoring, or segmentation weaknesses may leave the same attack path open through another route. A useful engagement ends with prioritized evidence, clear remediation guidance, and validation that the highest-risk paths have actually been closed.

Organizations should also treat penetration testing as part of an ongoing security program rather than a once-a-year compliance exercise. Environments change constantly. New applications are deployed, permissions accumulate, employees change roles, vendors receive access, and cloud integrations expand. Each change can create new attack paths.

The lesson from CISA’s recent assessment is simple: security cannot be measured by the number of tools deployed. It should be measured by how effectively an organization can prevent, detect, contain, and recover from realistic attacks.

Penetration testing provides the controlled environment to answer that question before a real attacker does.

Sources:
CISA: A Tale of Two SOCs, August 25, 2026
Cybersecurity Dive: CISA red team tests expose major defensive differences
Image by Pexels from Pixabay