Netsparker Desktop has a built-in proxy that allows you to manually crawl a target and scan it. Manual crawling is a process that is used to scan parts of a web application that for some reason cannot be crawled automatically. During a manual crawl the scanner will only scan the URLs that you feed through the proxy, though you can also combine both the automated and manual crawl as this posts explains. Manual crawling is typically used when:
- You want to scan sections of the web application that were not automatically crawled,
- You want to scan a limited number of URLs and parameters,
- You want to launch a Controlled Attack.
Manual Crawling with Netsparker Desktop
To do a manual crawl with Netsparker Desktop web application security scanner follow this procedure:
Step 1: Configure a Browser to Proxy the Traffic through the Scanner
By default when Netsparker’s proxy is started all the popular browsers such as Internet Explorer, Google Chrome and Mozilla Firefox will automatically proxy traffic through it. Therefore you do not need to manually configure the browser’s proxy settings.
If not, open the scanner’s Options from the Tools drop down menu, click to the Internal Proxy node and tick the option Register as System Proxy. If you are using a browser that is not automatically proxying the traffic through Netsparker’s proxy, configure it to proxy the traffic to port 10010, Netsparker’s proxy default port.
Tip: When the proxy is started the listening port will be shown on the Proxy button.
Step 2: Start Netsparker Desktop Scanner in Proxy Mode
Once the browser is configured start the Netsparker Desktop web application security scanner and specify a target URL in the Start a New Website or Web Service Scan dialog. The target URL will be used to filter the requests received from the web browser. Therefore if for example you want to scan http://php.testsparker.com, enter such URL and if you browse pages from other domains Netsparker will not add them to the Scan Scope.
Note: All the requests captured from proxy will also be filtered according to the configured Scan Scope.
To start Netsparker’s proxy select Manual Crawl (Proxy Mode) from the Start Scan drop down button in the Start a New Website or Web Service Scan dialog.
Step 3: Start Browsing the Pages You Want to Scan
You can start browsing the pages you want the scanner to scan via your web browser. At this stage you can notice the pages being added to the sitemap as you browse them.
Scan the Manually Crawled Pages
Once you have crawled all the pages click the Resume button from Netsparker’s toolbar or press F5 so the scanner proceeds with attacking the pages listed in the sitemap.
Using Both Automated and Manual Crawling in a Web Security Scan
To crawl a website automatically and then add URLs from a manual crawl select the option Crawl and Wait from the Start Scan drop down button in the Start a New Website or Web Service Scan dialog.
In this mode Netsparker will crawl the website automatically and will stop before starting the attack phase, allowing you to switch on the proxy by clicking the Start Proxy button in the toolbar. At this stage follow the procedure above to configure a browser to proxy the traffic through Netsparker Desktop and browse the pages you want to add to the sitemap for the scan.