This room explains how Search Engines work and leveraging them into finding hidden content.
To access the room you can click here: https://tryhackme.com/room/googledorking
Task 1 - Ye Ol’ Search Engine
No answer needed.
Task 2 - Let’s Learn About Crawlers
Name the key term of what a “Crawler” is used to do
ANSWER: Index
What is the name of the technique that “Search Engines” use to retrieve this information about websites?
ANSWER: Crawling
What is an example of the type of contents that could be gathered from a website?
ANSWER: Keywords
Task 3 - Enter: Search Engine Optimisation
No answer needed.
Task 4 - Beepboop - Robots.txt
Where would “robots.txt” be located on the domain “ablog.com”
ANSWER: ablog.com/robots.txt
If a website was to have a sitemap, where would that be located?
ANSWER: /sitemap.xml
How would we only allow “Bingbot” to index the website?
ANSWER: User-agent: Bingbot
How would we prevent a “Crawler” from indexing the directory “/dont-index-me/”?
ANSWER: Disallow: /dont-index-me/
What is the extension of a Unix/Linux system configuration file that we might want to hide from “Crawlers”?
ANSWER: .conf
Task 5 - Sitemaps
What is the typical file structure of a “Sitemap”?
ANSWER: XML
What real life example can “Sitemaps” be compared to?
**ANSWER:**Map
Name the keyword for the path taken for content on a website
ANSWER: Route
Task 6 - What is Google Dorking?
What would be the format used to query the site bbc.co.uk about flood defences
ANSWER: site: bbc.co.uk flood defences
What term would you use to search by file type?
ANSWER: filetype:
What term can we use to look for login pages?
ANSWER: intitle: login