October 18, 2026

Penetration test vs vulnerability scan

A vulnerability scan is a piece of software that checks your systems against a list of known problems: out-of-date versions, missing patches, weak encryption settings. A penetration test is a person trying to break into your app the way an attacker would, using scanners along the way for coverage. Scans are cheap and worth running every month. But when a customer, partner or auditor asks for a penetration test, a scan report won’t satisfy them, and they can tell the difference.

What a scan is good at

Scanners are good at the things they understand. One will tell you that a server is running a web framework with a known flaw, or that an old encryption protocol is still switched on, and it will check hundreds of hosts in an hour without getting bored. Run one regularly.

What a scan can’t do

A scanner doesn’t know what your app is for, so it can’t tell when the app is doing exactly what it was built to do, for the wrong person. One of the most common serious findings in a web app is one customer being able to see another customer’s data by changing a number in a URL. The app returns a normal page with a normal status code. To a scanner, nothing happened.

The same goes for business logic. Applying a discount code twice, skipping a payment step, approving your own expense claim: these are bugs in what the app is meant to do, and no list of known flaws contains them.

Scanners also can’t chain things together. In our sample report, a management console with a default password led to a shell on the server, and a kernel bug on that server led to full control of it. A scanner might flag the console and the kernel version as two separate items. It takes a person to see that together they’re a path from the public internet to root.

AI features are out of reach too. A chatbot leaking its instructions or another user’s documents, or an agent being talked into misusing a tool it’s allowed to call, depends on what you connected the model to, so none of it is in a scanner’s database.

And a scanner can’t tell you what matters. It rates findings by a generic score. A “medium” that exposes customer data matters more than a “high” on a server with nothing on it, and only a person who understands your app can say which is which. Scanners also report problems that aren’t there, and someone has to check each one. If that someone is your developer, you’ve paid for a report that makes work.

Why the customer asked for a pentest

Security questionnaires, SOC 2 auditors and enterprise procurement teams ask for a penetration test because they want evidence that a person tried to get in and wrote down what happened. Their checklist usually says so in as many words: an independent third party, a named methodology, a scope, findings with severities and proof that the findings were fixed. A scanner export has none of that, and the people reading it review these documents for a living. Send one back and they ask again for the real thing, with the deadline closer than it was.

How to tell which one you’ve been sold

Some “penetration tests” on the market are a scan with a cover page. Three questions settle it.

1. Did a person do the testing, and for how many days? A test is priced in days of someone’s time. If nobody can say how many, it was a scan.
2. Do the findings have reproduction steps? A tester writes down what they did. A scanner writes down a database entry.
3. Does the report say what the app does? A tester had to understand it to test it. A scanner didn’t.

How we do it

Every test is done by hand over a fixed number of days agreed before we start. Rob scopes it, does most of the testing himself and reviews every finding before it goes in the report. We run scanners too, for coverage, and the report says which findings came from a tool and which from a person. The report has a summary your customer can read and detail your developers can act on, and once you’ve fixed things, the retest is free.

If you’ve been asked for a penetration test and aren’t sure what size of job it is, tell us what you’ve built and we’ll quote it after a short call. If you want to know what you’d get back, here’s what’s in the report.