OPEN SOURCE INTELLIGENCE TOOLS AND RESOURCES …

[Pages:510]OPEN SOURCE INTELLIGENCE TOOLS AND RESOURCES HANDBOOK

2020

OPEN SOURCE INTELLIGENCE TOOLS AND RESOURCES HANDBOOK

2020

Aleksandra Bielska

Noa Rebecca Kurz, Yves Baumgartner, Vytenis Benetis

2

Foreword

I am delighted to share with you the 2020 edition of the OSINT Tools and Resources Handbook. Once again, the Handbook has been revised and updated to reflect the evolution of this discipline, and the many strategic, operational and technical challenges OSINT practitioners have to grapple with.

Given the speed of change on the web, some might question the wisdom of pulling together such a resource. What's wrong with the Top 10 tools, or the Top 100? There are only so many resources one can bookmark after all. Such arguments are not without merit. My fear, however, is that they are also shortsighted. I offer four reasons why.

To begin, a shortlist betrays the widening spectrum of OSINT practice. Whereas OSINT was once the preserve of analysts working in national security, it now embraces a growing class of professionals in fields as diverse as journalism, cybersecurity, investment research, crisis management and human rights. A limited toolkit can never satisfy all of these constituencies.

Second, a good OSINT practitioner is someone who is comfortable working with different tools, sources and collection strategies. The temptation toward narrow specialisation in OSINT is one that has to be resisted. Why? Because no research task is ever as tidy as the customer's requirements are likely to suggest.

Third, is the inevitable realisation that good tool awareness is equivalent to good source awareness. Indeed, the right tool can determine whether you harvest the right information. It follows that the more tools you have in your portfolio, the more flexible your OSINT capabilities are likely to be.

Finally, the process of compiling this handbook is an intelligence exercise in its own right, alerting us to where we are at as a community and the challenges we are likely to face going forward ? not least of which are disinformation, the fracturing of the internet, the proliferation of niche social media platforms, and the urgent need for better tools to monitor and analyse the content they provide.

Our hope is that this Handbook helps you plug any gaps in your collection efforts and alerts you to the many tools and resources you can consult for better intelligence, including on the discipline of OSINT itself. Of course, the Handbook makes no claim to being complete. Something new will doubtless pop-up tomorrow, and I suspect there are hundreds of tools we haven't even come across at all. With your help, we can address any omissions.

In any event, the usual caveats apply. While every effort has been made to test and validate these resources, we caution that today's wonder tool may be tomorrow's security liability. Thus, whenever testing a new script, application or browser extension, please do so in a secure environment to minimise any risk to your assets or operations.

3

I would like to thank my colleagues at i-intelligence for their efforts in compiling this Handbook. Much of the credit belongs to Aleksandra Bielska who, as our Head of Training, evaluates hundreds of tools every year to determine which ones make it into our courses. I am also indebted to Yves Baumgartner and Vytenis Benetis for routinely updating us on new solutions. Thanks too are also due to Noa Kurz who worked tirelessly over the summer to help us get this document ready. I close by acknowledging the contributions of the many students, practitioners and Twitter followers we've interacted with over the past two years. Your generosity, insights, suggestions and support are much appreciated. Chris Pallaris Director, i-intelligence GmbH

4

Table of Contents

Foreword............................................................................................................................. 3 Search Tools ..................................................................................................................... 17

General Search .......................................................................................................................... 17 Meta Search ............................................................................................................................... 18 Visual Search and Clustering Search Engines ......................................................................... 20 Decentralised Search Engines.................................................................................................. 21 Charitable Search Engines........................................................................................................ 22 Code Search .............................................................................................................................. 23 File and FTP Search................................................................................................................... 25 Open Database, Directories and Services Search .................................................................. 26 Similar Sites Search ................................................................................................................... 27 Children-Friendly Search Engines............................................................................................ 28 National Search Engine Directories......................................................................................... 29 Custom Search........................................................................................................................... 30 Specialty Search Engines .......................................................................................................... 31 Search Tools............................................................................................................................... 32 Social Media ..................................................................................................................... 34 Major Social Networks .............................................................................................................. 34 Social Media Search and Monitoring ...................................................................................... 37 Working with Hashtags ............................................................................................................. 39 Finding Influencers and Other Social Media Users ................................................................ 41 Social Media Management and Content Discovery............................................................... 42 Social Media Analytics .............................................................................................................. 43 Social Media Tools: Twitter ...................................................................................................... 44 Social Media Tools: Facebook ................................................................................................. 49 Social Media Tools: Instagram ................................................................................................. 51

5

Social Media Tools: Pinterest ................................................................................................... 55 Social Media Tools: Reddit ....................................................................................................... 56 Social Media Tools: Voat .......................................................................................................... 59 Social Media Tools: VKontakte................................................................................................. 60 Social Media Tools: LinkedIn.................................................................................................... 61 Social Media Tools: Github ...................................................................................................... 63 Social Media Tools: TikTok ....................................................................................................... 65 Messaging and Chat Tools ....................................................................................................... 66 Social Media Tools: Snapchat .................................................................................................. 68 Social Media Tools: WhatsApp ................................................................................................ 69 Social Media Tools: Skype ........................................................................................................ 70 Social Media Tools: Kik ............................................................................................................. 71 Social Media Tools: Telegram .................................................................................................. 72 Social Media Tools: IRC ............................................................................................................ 74 Dating Apps and Sites .............................................................................................................. 75 Fitness Apps and Communities ............................................................................................... 76 Working with Blogs .......................................................................................................... 77 Blog Search ................................................................................................................................ 77 Working with Forums ....................................................................................................... 78 Forum, Newsgroup and Discussion Board Search Tools....................................................... 78 Working with Chan Boards.............................................................................................. 79 Chan Search Tools..................................................................................................................... 79 Working with Q&A Sites .................................................................................................. 80 Q&A Sites................................................................................................................................... 80 People Investigations....................................................................................................... 81 People Search Tools ................................................................................................................. 81 Name Research.......................................................................................................................... 84 Public Records ........................................................................................................................... 85 Document Checking ................................................................................................................. 87 Ancestry Research ..................................................................................................................... 88 Missing People .......................................................................................................................... 89 Address and Contact Information Search ............................................................................... 91 Contact Detail Recovery Apps ................................................................................................. 93

6

Investigating Phone Numbers.................................................................................................. 95 Cell Tower Search...................................................................................................................... 99 E-mail Search ........................................................................................................................... 100 Validating E-mails .................................................................................................................... 102 Analysing E-mails..................................................................................................................... 104 Analysing E-mail Headers ....................................................................................................... 105 Matching E-mails to Social Media Profiles ............................................................................ 106 IP Grabbers and E-mail Trackers ............................................................................................ 107 Username Check ..................................................................................................................... 108 CV and Resume Search ........................................................................................................... 110 Expert Search........................................................................................................................... 111 Business Research .......................................................................................................... 113 Company Search Tools ........................................................................................................... 113 Business Registers Lists and Directories................................................................................ 117 Job Search Resources ............................................................................................................. 118 Company Reviews ................................................................................................................... 120 Salary Search............................................................................................................................ 121 Patent Research ....................................................................................................................... 123 Trademark Search.................................................................................................................... 125 Tenders..................................................................................................................................... 127 Charity, NGO, and Nonprofit Research................................................................................. 129 Researching Leaks ................................................................................................................... 131 Real Estate Research ...................................................................................................... 132 Real Estate Research ............................................................................................................... 132 Online Marketplaces ...................................................................................................... 135 Classifieds and Online Marketplaces..................................................................................... 135 Cryptocurrency and Banking Research ........................................................................ 138 Cryptocurrency Research ........................................................................................................ 138 Banking and Credit Card Information ................................................................................... 141 Currency Conversion ............................................................................................................... 143 Legal Research ............................................................................................................... 144 Legal Research Tools and Resources .................................................................................... 144

7

Researching Countries ................................................................................................... 147 Country Profiles ....................................................................................................................... 147

Researching Terrorism, Crime, and Cybersecurity ...................................................... 148 Researching Terrorism ............................................................................................................ 148 Researching Crime .................................................................................................................. 151 Researching Drug Trade ......................................................................................................... 153 Researching Money Laundering and Corruption ................................................................. 154 Researching Human Trafficking ............................................................................................. 155 Requests for User Information: Information for Law Enforcement...................................... 158 Researching Cybersecurity ..................................................................................................... 159 Military Research ..................................................................................................................... 161 Crisis Alerts and Travel Advisory ............................................................................................ 163

Researching Stolen Items .............................................................................................. 164 Researching Stolen Items ....................................................................................................... 164

Transportation ................................................................................................................ 166 Airplanes, Airports and Flight Trackers ................................................................................. 166 Drones ...................................................................................................................................... 169 Cars ........................................................................................................................................... 171 Railways .................................................................................................................................... 173 Public Transport....................................................................................................................... 176 Ships ......................................................................................................................................... 178 Container and Cargo Tracking ............................................................................................... 181

Web Intelligence ............................................................................................................ 182 Researching Domains ............................................................................................................. 182 IP / IPv6 ..................................................................................................................................... 187 SEO Tools ................................................................................................................................ 191 Analysing Unique IDs .............................................................................................................. 193 Subdomain Enumeration ........................................................................................................ 194 Identifying Backlinks................................................................................................................ 195 Site Mapping Tools ................................................................................................................. 196 Identifying Publish Date.......................................................................................................... 197 Extracting Files ........................................................................................................................ 198 Analysing URLs ........................................................................................................................ 199

8

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download