Web Vulnerability Scanners Comparison –
Netsparker Confirmed a Market Leader

How Are Tests Performed?

Shay Chen and his team built The Web Application Vulnerability Scanner Evaluation Project (WAVSEP), a testbed that they scan to see how every scanner performs. In it the scanners are tested against realistic setups, with crawling the most basic HTML website to identifying security vulnerabilities typically found in modern Single Page Applications (SPA). The WAVSEP is an open source project and new tests are incorporated every year. You can download it from the WAVSEP GitHub repository.

Testing Vulnerability Management & Assessment Solutions

This year Shay and his team went a step further. They have been installing and integrating DAST solutions in real-life enterprise SSDLC (Secure Software Development Lifecycle) processes to get a better understanding of how they can expand the WAVSEP testbed and test the scanners. So the have implemented automated vulnerability scanners in financial, hi-tech and telecom organizations.

They wanted to test more than security vulnerability detection rates. They wanted to see how these tools can really help business improve their vulnerability management and triaging processes, and their information security programme. As Shay himself explains:

Some of these experiences led us to develop test cases aimed to inspect issues in proclaimed features that we noticed didn't work as expected in actual implementations, and some to the creation of comparison categories that are apparently crucial for real-world implementations.

The Negative Impact of False Positives

Shay and his team also talked about the importance of accurate scan results in the report, after their first-hand experience with scanners in real-life environments. Quoting from the official benchmark results:

Weeding out a reasonable amount of false positives during a pentest is not ideal, but could be performed with relative ease. However, thousands upon thousands of false positives in enterprise SSDLC periodic scan scenarios can take their toll.

False positives occur in scan results to the detriment of the web application security industry. So much so, that large organizations, that have hundreds or even thousands of web applications, limit their efforts to a handful of mission-critical websites and ignore the rest. I was quite shocked to learn this, though it is unsurprising because many hacks and data leaks that happen every year.

False Positives Make Scaling Up Web Security Impossible

If a solution reports false positives, it is impossible – unless you have an army of people – to scale up your efforts and secure all your web applications. Even if you have the budget for such an undertaking, there is still the troublesome problem of human error.

This is why we developed Netsparker's proprietary Proof-Based Scanning^TM, technology that automatically verifies detected vulnerabilities – proving they are real flaws, and not false positives. The benefits of such technology are plentiful, and since the scan results are accurate, you can easily scale up your efforts. In a real-life environment, with thousands of web applications, you can start the vulnerability triage process and fix them within a matter of hours.

Evaluation Criteria

In the 2017/2018 benchmark tests, Shay and his team included several previously uncovered aspects of scanners and new tests to check the detection capabilities of previously uncovered vulnerabilities. This included OS Command Injection, and repurposing XSS via RFI tests that can also be used for Server Side Request Forgery (SSRF) evaluation.