SUPPORT

24/5 Hotline Support Service

+44 (0)20 3588 3841

Open a Support Ticket

support@netsparker.com

Scan Scope

The Scan Scope allows you to define which parts of the target web application should be crawled. This in turn dictates what will be scanned, because unless a page, parameter or any other object is first crawled, it will not be scanned.

Sometimes, you need to limit the scope of the scan. For example, if you want to scan a web application that uses data from external sources, you can configure the scanner to follow and scan the external sources (or not).

By default Netsparker scanners do not follow and scan data from external sources.

Another typical scenario is when you want to scan a web application that is installed in a subfolder, or only one section of a web application. For example, if the web application you want to scan is installed at http://www.example.com/app1 and you do not want the scanner to scan anything else from the http://www.example.com domain, you can configure the Scan Scope to restrict the scan to that subfolder.

Defining the URL in the Scan Scope

Consider these sample URLs:

  • http://example.com
  • https://example.com
  • http://example.com:81
  • http://test.example.com

Even though they share the same FQDN (Fully Qualified Domain Name), each URL is treated as a separate URLs:

  • The second one using a different protocol from the first one (HTTPS)
  • The third one is running on port 81
  • The last one has a different subdomain (test.example.com)

So, even if the root domain (example.com) is redirected to the www subdomain (www.example.com), you should enter www.example.com to scan that website. If you enter example.com as your target URL, Netsparker will not send requests to the www subdomain since it is out of scope.

Configuring the URL Path

There are three options available:

Entered Path and Below

When you select Entered Path and Below, Netsparker will only crawl and attack the target path and all the URLs under that path. If you enter the URL https://example.com/testfolder/ the following URLs will be crawled:

  • https://example.com/testfolder/test.php
  • https://example.com/testfolder/test/modify.php
  • https://example.com/testfolder/test/

The following URLs will not be crawled:

  • http://example.com/test.php; this URL is not under the given target.
  • http://test.example.com; this URL is of a different domain.
If you do not enter a trailing slash in the target URL, Netsparker assumes that the target URL ends with the last available slash in the URL and will alert you with the notification as illustrated.

Only Entered URL

When you select Only Entered URL, Netsparker will only crawl the target URL and no external links are followed. This function is useful if you want to only test one page and all the parameters in that page without testing the whole web application. If you enter https://example.com/testfolder/test.php the following URLs will be crawled:

  • https://example.com/testfolder/test.php
  • https://example.com/testfolder/test.php?id=1

The following URLs will not be crawled:

  • https://example.com/testfolder/register.php; the URL path is different than the one in the target URL
  • http://example.com/testfolder/test.php; the protocol is different. Target URL was HTTPS.
If you enter http://example.com/test, URLs such as http://example.com/testx will also be crawled. In this case the second URL is scanned because it contains the target URL.

Whole Domain

When you select Whole Domain, Netsparker will start crawling and scanning the target URL and all URLs beginning with the same hostname, regardless of the scheme and port number. Therefore if you enter https://example.com/testfolder/test.php the following URLs will be tested:

  • https://example.com/index.php
  • http://example.com/register/
  • https://example.com/testfolder/test.php
  • http://example.com/testfolder/test/test.php?id=1
  • http://example.com:81

Configuring the Scan Scope

You can configure the scan scope in both Netsparker editions.

For further information, see Scan Scope for Netsparker Enterprise, Scan Settings – Scope for Netsparker Standard and Advanced Scan Scope Settings.

How to Configure the Scan Scope in Netsparker Enterprise

  1. Log in to Netsparker Enterprise.
  2. From the main menu, click Scans, then New Scan. The New Scan window is displayed.
  3. Click the Scan Scope tab.

  1. In the Target URL field, enter a URL (see Defining the URL in the Scan Scope).
  2. Select one of the following tabs (see Configuring the URL Path):
    • Entered Path and Below
    • Only Entered URL
    • Whole Domain
  3. Click Delete () next to any item, if required.
  4. Click New Regx Pattern to create a new field, if required.
  5. Enable Include or Exclude, which will be applied to the configuration you have just entered (see Scanning Subdirectories).
  6. Configure the remaining fields as required.

  1. Select Disallowed HTTP Methods from the dropdown, if required.

  1. Click Launch.

How to Configure the Scan Scope in Netsparker Standard

  1. Open Netsparker Standard.
  2. In the Home tab, click New. The Start a New Website or Web Service Scan dialog is displayed.
  3. Click the Scope tab.

  1. In the Target Website or Web Service URL field, enter the URL (see Defining the URL in the Scan Scope).
  2. In the Scope dropdown, select an option (see Configuring the URL Path):
    • Entered Path and Below
    • Only Entered URL
    • Whole Domain

  1. Include or delete RegEx, as required.
  2. In Exclude URLs with RegEx, enable Include or Exclude (see Scanning Subdirectories).
  3. Select Disallowed HTTP Methods from the dropdown, if required.

  1. Complete the remaining Scan Options and Authentication, as required (see Netsparker Standard Scan Option Fields and Configuring Form Authentication in Netsparker Standard).
  2. Click Start Scan.

Filtering the URLs in the Scan Scope

It is possible to exclude or include URLs in the Scan Scope using regular expressions. By default, the Exclude option is selected and there are three predefined regular expressions, which are used to exclude URLs that might end an authenticated session. When Netsparker finds a URL that matches one of these regular expressions, it will not crawl or scan the page to prevent session logout.

When you use the Include option the Netsparker scanners will ONLY crawl and scan the URLs that match those regular expressions.

How to Filter URLs in the Scan Scope in Netsparker Enterprise

  1. Log in to Netsparker Enterprise.
  2. From the main menu, click Scans, then New Scan. The New Scan window is displayed.
  3. Click the Scan Scope tab. The Include URLs with RegEx fields are displayed.
  4. In New RegEx Pattern, enable Include or Exclude.

  1. Click Launch.

How to Filter URLs in the Scan Scope in Netsparker Standard

  1. Open Netsparker Standard.
  2. From the Start a New Website or Web Service Scan dialog, in the Scan Setting menu, click Scope.
  3. In Exclude URLs with RegEx, enable Include or Exclude.

  1. Click Start Scan.

Writing Regular Expressions to Include/Exclude URLs

You do not need to be knowledgeable about regular expressions to filter URLs. All you need to know is that there are a few special characters that when used in a regular expression and are not part of it you must escape with backslash. These characters are ()|.*+-?

So if the URL for which you want to write a regex contains one of those characters, just escape. Read the Wikipedia article on Regular Expressions for more information.

Example of How to Filter URLs with RegEx

In a typical logged in session is a link on all pages that allows the user to log out, such as:

<a href="session-end.php">Logout</a>

If Netsparker crawls this link during the scan, it will end the session. To ensure the scanner scans all the pages, you need to exclude that URL from the scan. To do so we need to write a regular expression to match the URL session-end.php. Since it contains special characters (hyphen and dot) that need to be escaped the regular expression should be:

session\-end\.php

Notice the backslash being used to escape the - and the . characters. If on the other hand you want to make sure Netsparker always crawls and scans such URL, use the same regular expression and tick the option Include.

Scan Scope Exceptions

It is important to point out that there are some exceptions during which Netsparker will ignore the Scan Scope configuration. These are highlighted below:

  • During authentication: most of the time successful or failed login attempts are redirected to a page which can be out of scope. In this case, the scanner would still need to crawl the page to check whether or not the authentication succeeded. For this reason, Netsparker does not check the Scan Scope configuration during authentication requests.
  • The target URL to scan is never checked against the scope. Only the crawled pages crawled from the target URL are checked.
  • The scanner will request JavaScript files that are located on external domains (common in a CDN setup) while performing JavaScript (DOM) Simulation (parsing) and DOM XSS attacks irrelevant of the Scan Scope configuration.

Scan Scope Examples

Here are two scan scope examples:

Scanning a Subdirectory

Target Website Setup:

  • The web application URL is http://example.com
  • You want to scan http://example.com/admin/

Scan Scope Configuration

  • Scope: Entered Path and Below
  • Target URL: http://example.com/admin/
  • Include Regex (Optional): /admin

How to Scan a Subdirectory on the Target Website in Netsparker Enterprise

  1. Log in to Netsparker Enterprise.
  2. From the main menu, click Scans, then New Scan. The New Scan window is displayed.
  3. Click the Scan Scope tab.
  4. Enter the Target URL (e.g. http://example.com/admin/).
  5. Click Enter Path and Below.

  1. Enable Include in New RegEx Pattern.
  2. Click New RegEx Pattern. A new Include URLs with RegEx field is displayed.
  3. Enter the subdirectory in the new field (e.g. /admin).

  1. Click Launch.

How to Scan a Sub-Category on the Target Website in Netsparker Standard

  1. Open Netsparker Standard.
  2. In the Home tab, click New. The Start a New Website or Web Service Scan dialog is displayed.
  3. In Scan Settings, click the Scope tab.
  4. Enter the Target URL or Web Service URL (e.g. http://example.com/admin/).

  1. From the Scope dropdown, select Enter Path and Below.

  1. In Exclude URLs with RegEx, enable Include.
  2. Enter the subdirectory in the new field (e.g. /admin).

  1. Click Start Scan.

Excluding a Subdirectory

Target Website Setup:

  • The web application URL is http://example.com
  • You want to exclude http://example.com/admin/

Scan Scope Configuration

  • Scope: Entered Path and Below
  • Target URL: http://example.com/
  • Exclude Regex: /admin

How to Exclude a Subdirectory on the Target Website in Netsparker Enterprise

  1. Log in to Netsparker Enterprise.
  2. From the main menu, click Scans, then New Scan. The New Scan window is displayed.
  3. Click the Scan Scope tab.
  4. Enter the Target URL (e.g. http://example.com/admin/).
  5. Click Enter Path and Below.
  6. Enable Exclude in New RegEx Pattern.
  7. Click New RegEx Pattern. A new Include URLs with RegEx field is displayed.
  8. Enter the subdirectory in the new field (e.g. /admin).
  9. Click Launch.

How to Exclude a Subdirectory on the Target Website in Netsparker Standard

  1. Open Netsparker Standard.
  2. In the Home tab, click New. The Start a New Website or Web Service Scan dialog is displayed.
  3. In Scan Settings, click the Scope tab.
  4. Enter the Target URL or Web Service URL (e.g. http://example.com/admin/).
  5. From the Scope dropdown, select Enter Path and Below.
  6. In Exclude URLs with RegEx, enable Exclude.
  7. Enter the subdirectory in the new field (e.g. /admin).
  8. Click Start Scan.
Netsparker

Dead accurate, fast & easy-to-use Web Application Security Scanner

GET A DEMO