A Beginners Guide to OSINT
OSINT - Open Source Intelligence that refers to a collection of data/information by exploiting publicly available resources. It is used for digital intelligence and investigation process that uses cyber tools to find strategic information in open sources that are obtained legally and ethically.
Author Daina McFarlane
OSINT has been around since the beginning of time and no one can pinpoint when it started. Numerous research states that “OSINT was introduced during World War II as an intelligence tool used mostly by nations security agencies”. The Internet started January 1, 1983, and ever since, the Internet has revolutionized and turned the world into a massive village of information.
The importance of OSINT gathering has become a necessity due to the explosive growth of the Internet and the huge volume of valuable digital data that is produced at a constant rate for organizations to use such as government departments, non-government departments and business corporations at their disposal. Since OSINT is publicly accessible sources of information found online or offline, anyone can conduct information gathering using available tools and techniques.
Is OSINT Valuable for an Investigation? Yes, OSINT is valuable because of its less rigorous processing and exploitation processes and timeline to gather information than more technical intelligence disciplines such as HUMINT – Human intelligence , SIGINT – Signal intelligence , MASINT – Measurement and signature intelligence and GEOINT – Geospatial intelligence.
Everyone leaves digital traces of their information; you just need to know how to find them. Three main method of collecting OSINT sources of information are passive, semi-passive and active. The usage of either one depends on the scenario and how deep the data need to be collected. Passive is the most used type as it targets only publicly available resources. Another name used by us security professionals for passive information gathering is reconnaissance. Semi-passive gathers information by sending limited traffic to target servers, investigating lightly without launching any alarm on the target’s side. Active gathers information by interacting directly with the system. Five steps of the OSINT cycle consists of Planning, Gathering, Analysis, Dissemination and Feedback. Due to the overwhelming sea of information, reconnaissance is broken down into 5 sub-phases refer to the OSINT process:
Source Information – the initial phase where the individual identifies potential sources from which information may be gathered from. Sources are documented and detailed notes are written down for later use.
Data Harvesting – information is collected and harvested from the selected sources and other sources that are discovered throughout this phase.
Data Processing and Integration – harvested information is processed for actionable intelligence by searching for information that may assist in the investigation.
Data Analysis – the individual performs data analysis of the processed information using OSINT analysis tools.
Results Delivery – the final stage in which OSINT analysis is completed, and the findings are presented/reported to other members of the team.
OSINT Tools and Techniques
A plethora of OSINT tools are available, both free, and commercial. The focus will be on the most popular tools that are used in the OSINT process. The key thing to know is that OSINT process is about using bits and pieces of information and running that information through a particular tool to discover more information about a person or entity.
Google Searching and Dorking
Google Searching or simply Google, as you know is a web search engine and its main purpose is to search for text in publicly accessible documents offered by web servers. The first investigation tool is the search operators and other advanced operators are located here: Search Operators
Google Dorking is known as Google hacking which are advanced search strings used within a web browser. Checkout the Google hacking database here: Google Hacking Database
Common operators are:
Intitle: return documents that mentioned words in the page title
Inurl: restrict the results to documents containing that word in the URL
Filetype: used to find filetypes
Ext: used to identify files with specific extensions such as .log
Intext: search for specific text on the page
Wikipedia define WHOIS as a query response protocol that is widely used for querying databases that store the registered users or assignees of an internet resource such as DNS, IP address block or an autonomous system. Check it out here: WHOIS Lookup & Domain Lookup
Spokeo is a search engine that search by name, phone, address or email to confidentially lookup information about people revealing all types of publicly available information such as public records, criminal records, school records etc. For example, searching for John Brown show many results, so the more specific you are the lesser and better the output.
There are many other similar websites like Spokeo such as OSINT Framework, Family Tree Now, Pipl, ThatsThem, US Search, Zabasearch, Radaris and many others. IntelTechniques.com provide online training, podcast and books. They are useful to use to see if any of your private information, that is potentially damaging information is not posted for everyone to see. Frankly, it is somewhat a rather difficult task to keep your own private information off the Web.
DataSploit is found within Kali or BlackArch Linux and is used to collect targeted data on a particular domain, email, username or phone number and then organizes the results coherently in HTML and JSON reports or text files. The information DataSploit attempt to find are credentials, api-keys, tokens, subdomains, domain history, legacy portals etc. Recon-ng and theHarvester is also another excellent and useful tool that is also built in Kali Linux. The advantage of theHarvester over Recon-ng is that it is faster and simpler to use.
The popular OSINT tool is Shodan that is specifically designed for Internet-connected devices including ICS, IoT, video game systems and more. Shodan GUI have more functionality and is used to view live camera feeds and can visually depict geographically where vulnerabilities are located throughout the world. It gives a huge footprint of devices connected online and is a gold mine for researchers to see the exposed assets. An example of a use case is testing for default passwords.
The Community Edition (CE) of Maltego is free and is developed by Paterva and is an inbuilt tool in Kali Linux. Maltego helps to perform a significant reconnaissance against targets using several built-in transforms as well as the capability to write custom ones. A user must register on the Paterva site before Maltego can be used. It can footprint Internet infrastructure used on social networking sites and collect information about the people who use it. Maltego will query DNS records, whois records, search engines, social networks, various APIs and extract metadata that is used to find correlational relationships between names, email addresses, aliases, groups, companies/organizations, websites, domains, DNS names, netblocks, IP addresses, affiliations, documents and files.
There are so many other tools that you can explore such as Automater and Sublist3r. Other search engines specifically for the Dark Web such as DeepDotWeb, Hidden Wiki, OnionScan and Tor Scan to find useful information. The collection of OSINT information is only limited by your imagination. Any tool can be tweaked and use once you have the basic understanding. Even though OSINT could easily let you feel scared and it’s scured; Don’t Be! Instead, use the knowledge from OSINT to safeguard your information. For continuous learning checkout SANS OSINT Summit 2021 that happened on February 11-12 for awesome resources. Happy learning OSINT!
About the Author: Daina McFarlane is a passionate about cybersecurity, a security advocate, continuous learner and youth mentor. She currently serves with ISACA Vancouver Chapter as a mentorship coordinator with the mentorship team. Her mission is to educate as many people about cybersecurity.