SUPPORT

24/5 Hotline Support Service

+44 (0)20 3588 3841

Open a Support Ticket

support@netsparker.com

Troubleshooting Inconsistent Web Security Scan Results

Netsparker is a black box web vulnerability and security scanner. It works by emulating an actual attacker, sending a number of HTTP requests to a target website and waiting for the responses from the web application in order to identify any vulnerabilities or other security issues.

Netsparker can send a large number of requests within minutes. Since it is all automated, sometimes there are factors that could affect the results of the scan, which also means two scans of the same website can have different results. Here is a list of things that can have an impact on your scan results. The next time you encounter inconsistent scan results, you can use this list to troubleshoot the problem:

  • Server-side problems
  • Connection problems

Server-side Problems

This table lists and explains the potential server-side problems, which may block the scanner's requests or result in the web application not responding to the scanner's requests. If the scanner does not get a response for any of its HTTP requests, it cannot determine if there is a vulnerability on the target web application, resulting in different scan results.

Problem

Description

Server load problem

If the server is out of resources, the web application might not be responding to all the HTTP requests from the scanner, which will generate timeouts. This can happen because Netsparker affects the web applications in different ways, so you might notice an unusual CPU load on the application server, database server, and other components.

HTTP Error 500 / Internal Server Errors

These problems occur randomly due to a session state problem, database server load or current CPU load, for example.

Session Killed

The session gets killed during the scan because of application memory recycle, server restart or error recovery features in the web servers, and the scanner can't recreate the session.

Different caching algorithms

Depending on the caching system, like a reverse proxy, used on the web application, the same request might get different responses each time.

Web Application Firewalls, IPS and other similar security software blocking the scanner's HTTP requests

If there is a mechanism blocking Netsparker’s HTTP requests, such as firewalls, IPSes, or a load-balancer, Netsparker won’t be able to access the web application properly. These systems might prevent some specific attack payloads and headers Netsparker uses, or limit the number of requests. In order to overcome this issue, you should whitelist your IP, that you are making scans, on your system, or should create a firewall rule for allowing Netsparker’s custom header, which is “X-Scanner: Netsparker”.

Connection Problems

This section lists the potential connection problems.

Problem

Description

Reverse proxy or proxy connection failures

Sometimes, the target can be deployed on a local or private server that requires a proxy connection. Such servers don’t allow to reach out them without providing a proxy connection. Likewise, the machine that Netsparker is installed in might need a proxy to connect to the internet.

Temporary internet connection failures

When Netsparker is not able to access the website, the problem may arise from a temporary internet connection failure. The machine Netsparker installed or the target server might be affected from these connection corruptions.

HTTP Timeouts (due to server-side or communication related problems)

If the target server has issues related to server performance, it might respond very slowly. If its capabilities are not enough for an automated tool, then probably it will no longer respond.

Load balancers

These will almost definitely cause problems and different scan results.

Ideally in such a setup, you should scan the website directly, by bypassing the load balancer.

How Can You Improve the Scan Results?

If you are getting inconsistent scan results, you can try the following suggestions.

Suggestion

Description

Decrease the Scan Speed

This is done by reducing the number of concurrent connections the scanner opens with the web application.

To decrease the scan speed open the Scan Policy Editor, navigate to the HTTP options and adjust the Concurrent Connections slider. You can also adjust the scan speed during the scan.

Monitoring the application and database server

This is done by making sure they are not under a heavy load.

If you notice a spike in resources, try to determine which pages the scanner is scanning, which might also help you solve an underlying problem.

Ensure that there is minimum interference between the Netsparker scanner and target website.

Network components such as proxies, web application firewalls, network firewalls, intrusion prevention or detection systems can all slow down the connection and drop or block requests.

Diagnosing the Problem

Here are some tips to help you troubleshoot the issue and solve it.

Check the HTTP Request and Response for the vulnerabilities that were identified in only one of the scans:

  1. Do all the responses look as expected?
  2. Do you always get the same response when you send the same request multiple times? You can test this by exporting the request to the HTTP Request Builder tool.
  3. Was the vulnerable link, file or parameter identified in the scan in which the vulnerability was not reported? You can check this by reviewing the Sitemap and Crawled URLs List (CSV) Report in both scans.
  4. Check the sitemap, which is populated during the crawling stage, and shows users all crawled pages and parameters. If something is listed in the Sitemap, Netsparker will attack it. This ensures that all pages and parameters are covered successfully.
  5. Watch all requests by Viewing the HTTP Requests and Response of an Issue if you need to get a detailed view of the scan. If you notice that Netsparker has missed a page or parameter, you can use the proxy feature to show these missing resources or import these requests. 

If you are unable to resolve the issue, contact support@netsparker.com.

Netsparker

Dead accurate, fast & easy-to-use Web Application Security Scanner

GET A DEMO