Why passing a compliance audit doesn't mean you are secure

There is a conversation that comes up almost every time we walk into a new company. The IT lead proudly shows the ISO 27001 certificate hanging on the wall or the favourable report from the latest ENS review, before asking what point there is in looking at anything else if everything is already in order.

The short answer is that piece of paper does not say what most people think it says.

What a compliance audit looks at

A compliance audit verifies that the company has the controls documented and applied according to a specific framework. ISO 27001, ENS, NIS2, PCI DSS, SOC 2. Each one has its own list of requirements and its own way of reviewing them, but they all share the same underlying idea, which is to verify that processes, policies and records actually exist.

The compliance auditor will ask for your access management policy, will review a sample of user onboardings and offboardings, will check that there is an up-to-date asset inventory, will look at logs over a reasonable window and will sign off an opinion stating that all of that is implemented in a coherent way.

What that auditor will not do is try to break in.

What an offensive audit looks at

When we run a pentest, or an offensive security audit, what we test is precisely that. We try to break in. With permission, with an agreed scope and with an expert team, but we get in the way anyone with an interest in doing so without permission would get in.

We do not care that you have the strictest password policy in your sector if in practice we find a service account with the password sitting inside an Active Directory description field. We do not care that the documentation says VPN access is hardened if there is a single account without a second factor, configured three years ago because of an urgency, that no one has touched since.

Compliance measures intent. An offensive audit measures reality.

Why this happens so often

Companies that combine both types of review are the exception, not the norm. Most of them run the audit a client, an insurer or a regulator requires. The chapter closes there. The security team breathes easy for twelve months and management files the report away in a drawer.

What happens during that year is invisible from the documentation. New services appear without going through the inventory, SaaS tools are signed up and never reviewed, access stays open for employees who have already left, someone copies a database to a shared drive to run a migration and leaves it there. None of that breaks compliance because compliance is not watched every day. But all of that is exactly what an attacker finds when they arrive.

What we see in practice

We have audited companies with a valid ISO 27001 where an unprivileged user could read the administrators’ passwords in plain text inside a maintenance script.

We have seen environments with compliance frameworks in place where the Active Directory domain fell in under two hours from an unprivileged position, exploiting a misconfiguration inherited from years back.

We have found backups perfectly documented according to policy, stored on a network path readable by every employee in the domain.

None of those findings broke compliance. All of them broke the company, had anyone with bad intentions reached them before we did.

How the two approaches complement each other

We are not saying compliance is unnecessary. It is necessary, often mandatory. It brings a governance discipline that few companies have without a framework forcing them to apply it.

What we are saying is that compliance answers a different question from the one many companies believe it is answering. It tells you whether you have the measures the standard expects. It does not tell you whether those measures hold up when someone tests them.

An offensive audit answers precisely that second question. It goes after the gaps that exist between policy and reality, the services that appeared after the last review, the default configurations no one adjusted, the accounts left created by mistake.

When both approaches are run in the same year, over the same perimeter, things come up that neither would have caught on its own.

Three questions before you call yourselves secure

If you have a compliance audit coming up or recently completed, before you file the report it is worth raising three questions in the internal meeting.

The first is whether anyone has genuinely tried to break into the critical systems over the past year. Not simulated on a spreadsheet, not at a workshop table. Break in.

The second is whether the result of that exercise was turned into concrete and verified fixes, or whether it stayed in a PDF.

The third is, if the answer to the previous two is no, what would prevent that visit from happening before someone who does not announce themselves shows up.

That is the difference between having a report and being secure.

---

If you want us to put the real security of your organisation to the test, write to [email protected] and we will outline a scope tailored to your context.