104 lines
3.9 KiB
Markdown
104 lines
3.9 KiB
Markdown
## Description
|
|
|
|
This module is a http crawler, it will browse the links recursively from the
|
|
web site. If you have loaded a database plugin and connected to a database,
|
|
this module will report web pages and web forms.
|
|
|
|
## Vulnerable Application
|
|
|
|
You can use any web application to test the crawler.
|
|
|
|
## Options
|
|
|
|
**URI**
|
|
|
|
Default path is `/`
|
|
|
|
**DirBust**
|
|
|
|
Bruteforce common url path, default is `true` but may generate noise in reports.
|
|
|
|
**HttpPassword**, **HttpUsername**, **HTTPAdditionalHeaders**, **HTTPCookie**
|
|
|
|
You can add some login information
|
|
|
|
**UserAgent**
|
|
|
|
Default User Agent is `Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)`
|
|
|
|
## Verification Steps
|
|
|
|
1. Do: ```use auxiliary/scanner/http/crawler```
|
|
2. Do: ```set RHOST [IP]```
|
|
3. Do: ```set RPORT [PORT]```
|
|
4. Do: ```set URI [PATH]```
|
|
4. Do: ```run```
|
|
|
|
## Scenarios
|
|
|
|
### Example against [WebGoat](https://github.com/WebGoat/WebGoat)
|
|
|
|
```
|
|
msf> use auxiliary/scanner/http/crawler
|
|
msf auxiliary(crawler) > set RHOST 127.0.0.1
|
|
msf auxiliary(crawler) > set RPORT 8080
|
|
msf auxiliary(crawler) > set URI /webgoat/
|
|
msf auxiliary(crawler) > set DirBust false
|
|
msf auxiliary(crawler) > run
|
|
[*] Crawling http://127.0.0.1:8008/webgoat/...
|
|
[*] [00001/00500] 302 - 127.0.0.1 - http://127.0.0.1:8008/webgoat/ -> /webgoat/login.mvc
|
|
[*] [00002/00500] 200 - 127.0.0.1 - http://127.0.0.1:8008/webgoat/login.mvc
|
|
[*] FORM: POST /webgoat/j_spring_security_check;jsessionid=8B1EAF2554B60EFC93A52AFCA4B6C202
|
|
[-] [00003/00500] 404 - 127.0.0.1 - http://127.0.0.1:8008/webgoat/images/favicon.ico
|
|
[*] [00004/00500] 200 - 127.0.0.1 - http://127.0.0.1:8008/webgoat/plugins/bootstrap/css/bootstrap.min.css
|
|
[*] [00005/00500] 200 - 127.0.0.1 - http://127.0.0.1:8008/webgoat/css/font-awesome.min.css
|
|
[*] [00006/00500] 200 - 127.0.0.1 - http://127.0.0.1:8008/webgoat/css/animate.css
|
|
[*] [00007/00500] 302 - 127.0.0.1 - http://127.0.0.1:8008/webgoat/j_spring_security_check;jsessionid=8B1EAF2554B60EFC93A52AFCA4B6C202 -> /webgoat/login.mvc;jsessionid=8B1EAF2554B60EFC93A52AFCA4B6C202?error
|
|
[*] [00008/00500] 200 - 127.0.0.1 - http://127.0.0.1:8008/webgoat/login.mvc;jsessionid=8B1EAF2554B60EFC93A52AFCA4B6C202?error
|
|
[*] FORM: GET /webgoat/login.mvc
|
|
[*] FORM: POST /webgoat/j_spring_security_check;jsessionid=8B1EAF2554B60EFC93A52AFCA4B6C202
|
|
[*] [00009/00500] 200 - 127.0.0.1 - http://127.0.0.1:8008/webgoat/css/main.css
|
|
[*] [00010/00500] 302 - 127.0.0.1 - http://127.0.0.1:8008/webgoat/start.mvc -> http://127.0.0.1:8008/webgoat/login.mvc
|
|
[*] [00011/00500] 200 - 127.0.0.1 - http://127.0.0.1:8008/webgoat/login.mvc
|
|
[*] FORM: POST /webgoat/j_spring_security_check
|
|
[*] Crawl of http://127.0.0.1:8008/webgoat/ complete
|
|
[*] Auxiliary module execution completed
|
|
```
|
|
|
|
## Follow-on: Wmap
|
|
|
|
As you see, the result is not very user friendly...
|
|
|
|
But you can view a tree of your website with the Wmap plugin. Simply run :
|
|
|
|
```
|
|
msf auxiliary(crawler) > load wmap
|
|
msf auxiliary(crawler) > wmap_sites -l
|
|
[*] Available sites
|
|
===============
|
|
|
|
Id Host Vhost Port Proto # Pages # Forms
|
|
-- ---- ----- ---- ----- ------- -------
|
|
0 127.0.0.1 127.0.0.1 8080 http 70 80
|
|
|
|
|
|
msf auxiliary(crawler) > wmap_sites -s 0
|
|
|
|
[127.0.0.1] (127.0.0.1)
|
|
└── webgoat (7)
|
|
├── css (3)
|
|
│ ├── animate.css
|
|
│ ├── font-awesome.min.css
|
|
│ └── main.css
|
|
├── j_spring_security_check;jsessionid=8B1EAF2554B60EFC93A52AFCA4B6C202
|
|
├── login.mvc
|
|
├── login.mvc;jsessionid=8B1EAF2554B60EFC93A52AFCA4B6C202
|
|
├── plugins (1)
|
|
│ └── bootstrap (1)
|
|
│ └── css (1)
|
|
│ └── bootstrap.min.css
|
|
├── start.mvc
|
|
└── j_spring_security_check
|
|
|
|
```
|