Discovering The Hidden Web

How To Perform Pentest Recon Using GoBuster.

The first step to start attacking a web application is to perform recon on your target. Recon refers to the process in which hackers and penetration testers dig deep into an application to gather information and discover additional content that is not normally exposed to the user.

An important step of recon is to discover hidden content like obscure subdomains, secret directories, and virtual hosts. Today, let’s talk about a recon tool that helps us accomplish these goals: GoBuster.

GoBuster is a tool for brute-forcing to discover subdomains, directories and files (URIs), and virtual hostnames on target web servers.

Installing GoBuster

Let’s start by installing GoBuster! You can find GoBuster’s project page here.

You most likely will not need to build the project from source. On most Linux distributions, you can install GoBuster via the apt-get command:

apt-get install gobuster

Otherwise, if you have a Go environment ready to go, you can use:

go get github.com/OJ/gobuster dns

Using GoBuster

Now that you have the program installed, let’s jump right into performing recon using GoBuster! GoBuster has three available modes: “dns”, “dir” and “vhost”. They are used to brute-force subdomains, directories and files, and virtual hosts respectively.

DNS mode

The DNS mode is used for DNS subdomain brute-forcing. You can use it to find subdomains for a given domain. In this mode, you can use the flag “-d” to specify the domain you want to brute force and “-w” to specify the wordlist you want to use.

gobuster dns -d <target domain> -w <wordlist>

You can use your own custom wordlists for this, but a good option is to use wordlist published online. For example, the Seclists Github Repository has a pretty extensive wordlist for subdomain brute-forcing:danielmiessler/SecLists.

Dir mode

The Dir mode is used to find additional content on a specific domain or subdomain. This includes hidden directories and files.

In this mode, you can use the flag “-u” to specify the domain or subdomain you want to brute force and “-w” to specify the wordlist you want to use.

gobuster dir -u <target url> -w <wordlist>

You can find a list of web content wordlists to use here:danielmiessler/SecLists.

Vhost mode

Lastly, you can use the Vhost mode to find virtual hosts of a target server.

Virtual hosting is when organizations host multiple domain names on a single server or cluster of servers. This allows one server to share its resources with multiple hostnames. Finding virtual hostnames on a server can reveal additional web content belonging to an organization.

gobuster vhost -u <target url> -w <wordlist>

For brute-forcing virtual hosts, you can use the same wordlists as brute-forcing subdomains via the DNS mode.

Advanced options

Here are some additional options that you might find useful. GoBuster has plenty of advanced options that you can use to specify its behavior. To view the options for each mode, you can run:

gobuster help <mode>

For example, in dir mode, you can brute-force files with specific file extensions using the -x flag:

gobuster dir -u <target url> -w <wordlist> -x .php

For dir and vhost modes, you can use -k to skip SSL certificate verification and suppress SSL errors:

gobuster dir -u <target url> -w <wordlist> -k

And for both dir and vhost modes, you can even use the -c flag to specify the cookies that should accompany your requests:

gobuster dir -u <target url> -w <wordlist> -c 'session=123456'

Happy Hacking!

Good recon skills are one of the keys to being successful as a hacker or a penetration tester. And GoBuster is a simple and powerful tool to add to your recon toolkit! Happy hacking and good luck!