Monitoring 21st Century OSINT Sources

The OSINT OPSEC Tool

Right now, people are leaking a whole lot of info online...

Some of them want to get fired...

or give us their credit cards...

or their passports...

So why should you be concerned about that?

About Me

• Full time Application/Systems Support

• Part time Systems Administrator/Engineer

• Interest in security started after attending Kiwicon II

What is OPSEC?

• OPSEC = Operations Security

• Identifying secrets that helps “the bad guys”

• To maintain good OPSEC; keep quiet

Why should you care about OPSEC?

• Everyone has secrets

• That includes organizations you are a part of:- businesses- governments- non-profits

OPSEC, OSINT and social media

• OSINT = Open Source Intelligence

• Publicly available sources

• 20th Century: newspapers, radio, television

• 21st Century: the Internet; social media

What can we find from 21st Century OSINT sources?

Security related Info

• In 2009, a US Congressman tweeted his locations during a trip in Iraq

• Serious potential consequences

Political Info

• In September an attempt was made to extort Mitt Romney with his tax returns

• Demand was posted on Pastebin

• Though a hoax; the idea still remains

Business Info

• Employees may leak information

• New projects, mergers etc

Internal IT Infrastructure Info

• StackExchange used bySys Admins/Network Engineers etc

• Code snippets posted

• Configurations posted

The Idea

• Monitor 21st Century OSINT sources, and their users

• Send alerts when something of interest is found

• Easy overview of multiple sites

• Map out hits

• Open source tool; data sovereignty

Existing Solutions?

• Google Alerts; not real time enough; especially with social media

• Twitter/Facebook search engines; no real-time alerting

• Free alert services; not fast enough; not enough sources

• Commercial solutions; not free

FBI Interest

• The FBI is (obviously) interested in monitoring social media

• In January this year they released a Request For Information (RFI)

• “...to determine the capability of industry to provide an Open Source and social media alert, mapping and analysis application solution”.

So, the tool...

What’s monitored?

Source Keyword(s) User + Keyword(s)

Twitter Yes Yes

Reddit No Yes

StackOverflow No Yes

ServerFault No Yes

Facebook Yes No

PasteBin Yes No

Wordpress Yes No

Behind the scenes

• Python to scrape the sources

• Sources are constantly queried; if keyword(s) found in content: user gets alerted via email

• MySQL DB

Behind the scenes II

• StackExchange API doesn’t return everything

• BeautifulSoup scrapes post data

• Pastebin provides no native search API

• Modified PasteLert (andrewmohawk.com)

Source Native API?

Twitter Yes

Reddit Yes

StackOverflow Yes*

ServerFault Yes*

Facebook Yes

Wordpress Yes

PasteBin No

Real-time?

• Depends on the API

• Pastebin was real-time...

• User hit sources; 10 minute delay per user (with 10 users)

• Keyword hit sources; 1 hour delay per keyword (with 12 keywords)

Use cases

• Organizations can use this to check that their members are not leaking information

• Recon to gather info on internal systems

• During a pentest in case employees mention anything valuable

Potential interesting finds pt I

• Proprietary source code

Potential interesting finds pt II

• Private keys

• Your company customer database

Thanks

• bogan for the original idea

• pipes, metlstorm and Adrian Hayes for feedback

• The crue, volunteers and sponsors for another wicked ‘con

Feedback/Source/Contact

• @hyprwired

• github.com/hyprwired/osint-opsec-tool

• brendan.a.jamieson@gmail.com