Social Media and Open Source Intelligence

Image Source: http://www.expertsystem.com/wp-content/uploads/2016/06/osint-big-data.jpg 1

Value of OSINT

Information does not have to be secret to be valuable.  Open Source Information is unclassified, available and one of the largest sources of information available

In many cases not leveraged to it's full potential due to a number of issues including volume, language, format and insight (the ability to understand what has been collected)

2

Recent History of OSINTThe Foreign Broadcast Information Service (FBIS) was created in 1941 to access and exploit OSINT in relation to World War II. A classic example of their value and success is reflected in the price of oranges in Paris as an indicator of whether railroad bridges had been bombed successfully.

The recent history of OSINT began in 1988 when General Alfred M. Gray, Jr., Commandant of the Marine Corps, called for a redirection of US intelligence away from the collapsing Soviet Union and toward non-state actors and Third World zones of instability. Additionally, he pointed out that most of the intelligence which needs to be known could be obtained via OSINT, and recommended a substantive increase in resources for this aspect of the intelligence collection spectrum of sources.

3Source: https://en.m.wikipedia.org/wiki/Open-source_intelligence

Recent Perspective on OSINT“Classified sources and methods will always have value in our agency and to our customers, but we cannot always view unclassified information as supplemental”

“Moving forward the reverse is more likely to be true - that which is exquisite but classified will supplement an ever broader and richer unclassified base”

NGA Director Robert Cardillo

4Source: CSIRNET.com “Open Source Intel” by Adam Stone, May 2016

Contents• OSINT vs SOCMINT • Aggregation vs Live Connection • Deep and Dark Web • Peer to Peer Networks • Common Issues • Types of Analysis • Practical Applications

5

OSINT vs SOCMINTOpen Source Intelligence (OSINT) – Information gathered from unrestricted or public sources. Typically not focused on Social Media Information (i.e. person or group centric) though often encompasses information feeds that are resident within some social media platforms.

Social Media Intelligence (SOCMINT) – Information gathered from Social Media Platforms. It is person and group centric and focuses on networks, messaging, and social network analysis.

6

Social Media VolumesTweets - 500 Million / Day

Active Users / Monthly*

Facebook (1.79 billion) - Avg ~155 friends/user**

WhatsApp (500 million)

Twitter (284 million)

Instagram (200 million)

7

*Source: https://zephoria.com/top-15-valuable-facebook-statistics/ ** Source: http://expandedramblings.com/index.php/by-the-numbers-17-amazing-facebook-stats/

Social MediaWhile not “vetted”, can be significantly faster, with greater volume and more granular than other forms of intelligence

8

Aggregation vs Live Connection• Aggregation Services focus on

content, metadata, and trends analysis.

• They analyse historical information and are helpful when searching by content and sentiment.

• They offer limited utility for granular network analysis as the entire network is not captured at the time content is.

• Example – GNIP, Voyager, Babel Street, Pathar.

• Live Connection Services focus on Individuals, Groups, and Networks.

• They analyse networks and content as they occur at the time of capture and are helpful when searching by named individual or individuals associated with a group.

• The application of Social Network Analytics yields insight into the position and roles of individuals.

• Example – X1, SNAPD, API Direct.

9

The Open Source TerrainIt’s not just Social Media

• News • Blogs • Marketplaces • Wikis • Business • Reviews • Events …

Image Source: https://conversationprism.com/ 10

OSINT News Media Example: GDELT (Global Database of Events, Language and Tone)

GDELT monitors print, broadcast, and web news media in over 100 languages from across every country in the world to keep continually updated on breaking developments anywhere on the planet. Its historical archives stretch back to January 1, 1979 and update every 15 minutes. Through its ability to leverage the world's collective news media, GDELT moves beyond the focus of the Western media towards a far more global perspective on what's happening and how the world is feeling about it.

11

Dark Web

Unindexed by Search Engines

Shrouded behind password/paywall

protection

Technological barrier to entry,

requiring software for

access

Deep Web

Surface Web

12

Comparing the Surface Web - Deep/Dark Web

• Crowded social commentary • News reports • Opinion pieces • Mainstream perspectives

• Dialogue between threat actors • Black market products and services • Malicious tactics, techniques, and

procedures (TTPs) • Weapons and training manuals • Illicit community perspectives

13

What’s on the Deep & Dark Web?

• Drugs • Counterfeit Items • Stolen Goods • Weaponry • Identities • Credit Cards • Malware / Source

Code

14

Early Identification of Radicalization

15

Early Identification of Radicalization

16

Leveraging Deep / Dark Web

It is possible to generate Actionable Intelligence through Deep and Dark Web monitoring that allows your organization to proactively protect itself and not rely solely on signatures, behaviors, or indicators from other breaches.

17

Analysis of Peer to Peer networks for Terror Materials

Protocol! Client+examples!

Ares+ Ares%Galaxy,%Warez%P2P%

Bi3orrent+ Azureus,%BitComet,%BitTornado,%Transmission,%Vuze,%µTorrent%

Direct+Connect+ StrongDC,%DC++,%%

Open+Fas3rack+ OpenFT,%GIFT,%KcEasy,%KGNitro%

eDonkey+ eMule,%AMule%

Gnutella+ BearShare,%GTKGgnutella,%LimeWire,%%Shareaza,%Phex,%Frostwire%

Gnutella+2+ Shareaza,%Gnucleus,%%

18

Peer to Peer Network Functionality

19

Discovery / Tracking of Terrorist Materials on Peer to Peer Networks

20

Examination of the individual materials

21

Common Issues with OSINT• Volume: Too much to read

• Language: Not just English

• Format: Largely Unstructured in many cases

• Insight: Understanding what we’ve collected

22

VolumeSince it’s often impossible to “save everything”, one approach to addressing this is “streaming analaytics”, providing the ability to run analytics across the data as it moves to determine if this information is valuable

23

LanguageA large amount of content we’re collecting will not be in english therefore we need to translate that content to make it easier to understand

24

FormatFrequently the content we collect will be unstructured, so we need a means to “understand” what’s been written by creating a “mental model” of the information by extracting entities, links and properties

25

Insight

Once we have collected and processed all this information, what does it all mean?

To understand what we’ve collected and processed we may require multiple forms of analysis to gain insight into what we have collected and what it means to us.

26

Link Analysis

27

Evaluating relationships (connections) between nodes

28

• Relationships: Obvious, Non Obvious

• Anchor Points: Home, Work, Religion, relationships, support infrastructure

• Behavior: Rational, Irrational, Open, Closed norms and variation

• Constraints: Political, cultural, physical

• Communication: Social networks to understand the conscious/cognitive dimension of Human Geography

• Movement: Travel Home, Work, Relationships, as a human activity

• Culture: Norms / variation like language, religion, economy, gov’t

• Challenges: Alliances & relations including close, casual, competitive, adversarial

Social Network Analysis

Relationship AnalysisFusing and Correlating information to identify relationships that have not been disclosed

29

Identity AnalysisUsing the “disclosed” information to identify alternate identities that may be in use

30

Geo-Spatial Analysis

31

Understanding information in a geo-spatial context in relation to other known information

Temporal AnalysisUnderstanding how events relate over time

32

Frequency/Statistical Analysis

33

OSINT Law Enforcement ApplicationsEarly Identification of Self Radicalization - Identifying individuals as they begin the path to radicalization through the analysis of the radicalization materials

Tax evader detection - Web content such as news, blogs and social networks, especially data concerning famous companies and celebrities are of great value. Using OSINT applications customized for the analyst’s requirements, it’s possible to retrieve precise and contextual information from big data on the events, behaviors, lifestyles, activities, and professional and personal relationships of a single target.

Online counterfeit (i.e. pharmaceuticals, clothing, jewelry). Text mining and OSINT applications can extract the key elements present within a posting such as: vendor alias, email, telephone number, brand, product, etc. This data is then compared to identify the typical characteristics of a counterfeit product (i.e. available only in certain sizes) and to highlight the correlation between the data (i.e. a connection between vendors and/or repeated events between vendors).

34Content Source: http://www.expertsystem.com/osint-applications-3-examples/

OSINT Military Applications• Target Analysis: discusses how OSINT might fulfill team needs in the absence of classified intelligence support, to create a detailed description and vulnerability assessment, evaluate the natural environment and the human environment, and carry out route planning.

• Terrain Analysis: uses OSINT to establish key factors relevant to special aviation and covert ground movement, in part by leveraging commercial charts, commercial imagery, and alternatives for terrain reconnaissance including unmanned aerial vehicles and indigenous scouts.

• Civil Affairs: can use OSINT in relation to human intelligence (understanding the demographics, the socio-economic environment, displaced persons, and crime, among other topics); to technical intelligence about the local command & control, communications, computing, and intelligence environment, the infrastructures of transportation, power, and finance; to welfare intelligence (water, food, medical); cultural intelligence about protected or restricted targets, and liaison intelligence.

• Weather Analysis: uses OSINT as a means of rapidly getting to the basics of temperature, visibility and timing of sun and moon, wind, and inclement weather.

35Source: http://www.oss.net/dynamaster/file_archive/060409/5432a5e19def62b82684a111fe03f899/STEELE%20OSINT%20FOR%20HANDBOOK%203.3%20Chapter.doc

OSINT Civil ApplicationsVisa Processing: Evaluating OSINT and Social Media sources to determine if a visa application requires additional scrutiny

Employee Vetting: Leveraging OSINT to evaluate the veracity of the information you have been provided

Know your customer: Financial Responsibility to understand who you are doing business with

36

Thank You

37

OSINT Referenceshttps://inteltechniques.com/links

https://www.toddington.com/resources/

http://www.onstrat.com/osint/

http://i-sight.com/resources/101-osint-resources-for-investigators/

https://www.cia.gov/library/center-for-the-study-of-intelligence

38