michael schiebel - security analytics journey - a year's lesson learned

1 © Hortonworks Inc. 2011 – 2016. All Rights Reserved

A year's lesson learnedMichael SchiebelCybersecurity Strategist3/30/2016

Security analytics journey


Today’s talk regards:Community Driven,Open Source Projects

3 © Hortonworks Inc. 2011 – 2016. All Rights Reserved © Hortonworks Inc. 2011 – 2015. All Rights Reserved

To be 100% Open you must influence the roadmap

We Employ the Committers--one third of all committers to the Apache® Hadoop™ project, and a majority in other important projects

Our Committers Innovateand expand both Open Enterprise Hadoop and Apache NiFi

We Influence the Hadoop Roadmapby communicating important requirements to the community through our leaders

A PA C H E H A D O O P C O M M I T T E R S


Agenda • Who am I?• A year in review• Core lessons• Apache Nifi• Apache Metron• Putting it all together• Next steps• Questions


Who am I? +15 years information security experience

– Financial & Healthcare– Incident Response– Computer Forensics & Malware analysis– Application Security– Security Architecture– Security Executive

• Strategy• Business case development

Hortonworks Cybersecurity Strategist– Your Advocate– Hortonworks’ vision for Apache Metron– Evangelize an open source approach

Cybercrime is largest transfer of wealth in the history of civilization.We need an open solution available to everyone.

Only together can we solve this problem.


A year in review• Asked to create an open source organization• Met the Hortonworks folks• OpenSOC forked

• Apache Foundation accepted• 12/15 Apache (Incubating) Metron created• ~30 committers• Multi-company support

• Decided to join Hortonworks• Apache Nifi!

• The single (IMHO) most important open source project in 2015.

• WYSIWYG data ingest and transformation• Data Provenance(Chain of custody)

2015 Infosec Summ

it


Core Lesson – Data Ingestion Finding and getting data is hard Secure delivery of data is hard

Traditional tools generate tons of data as signature based events.Analytics needs access to raw activity dataDon’t underestimate the challenge – it’s a people and process problem


Core Lesson – Don’t be the new shiny

Focus on the value proposition– Show how it makes the job easier and more efficient at each phase of implementation.– How current state isn’t effective.– Metrics, Metrics, Metrics

Don’t hype the technology – this can be viewed as scary or untried– Analytics is mature – predates computers– Unlike other technologies, analytics shows the evidence the prediction is based on.– It’s just math.

“Why are you building some new Star Trek thing? We need to focus on the basics.”


Core Lesson – Avoid YADB (Yet another dashboard)

It’s about the people– Make the tool fit the people and process– Does this make their job easier?

Integration and workflow are key Focus on automated detection and response


Apache Nifi WYSIWYG Data Ingest Secure data

transmission Chain of custody Data enrichment

pipeline


PCAP

NETFLOW

DPI

Network Tap

IDS

AV

EMAIL

FIREWALL

HOST LOGS

PARSE

NORMALIZE

USER

ASSET

GEO

WHOIS

CONN

TAG

VALIDATE

PROCESS

ENRICH

STIX

Flat Files

Aggregators

Model As AService

Cloud Services

LABEL

Real-TimeSearch

InteractiveDashboards

DataModelling

KnowledgeGraphs

PCAPStore

IntegrationLayer

PCAPReplay

SecurityLayer

WorkflowEngine

RulesEngine

Apache Metron


Putting it all together


Next Steps Call to action:

– Continue to build community: Apache Metron, Apache Nifi– Become a committer– Help write documentation– Try out projects and provide feedback and suggestions

Help develop open data schema– How can we extend to solve other security problems? (FAIR actuarial tables…)

Other opens source projects– Workflow– UI– Sensors– Automated Response– Container/DevOps extensions


Questions?Blog: http://www.hortonworks.com/blog/author/mschiebel/Email: [email protected]: @mgschiebel

http://www.hortonworks.com/blog/author/mschiebel/

mailto:[email protected]

mailto:[email protected]

michael schiebel - security analytics journey - a year's lesson learned

Technology