Transcript
Page 1: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

Fruit: Why you so low? Network Recon 2011AD

Hack.lu 2011

Page 2: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

Oh, Hi.

● I'm Metlstorm (Adam to my mum)● Based in Wellington, New Zealand

● I hack stuff. ● Usually with python, bacon, vim, unix and beer.

● Roll with Brett Moore's Insomnia Security● Previously of Immunity,

Security-Assessment.com

● On (double-award winning) weekly infosec news podcast Risky.biz

Page 3: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

Proprietor, Kiwicon (est 2007)

Page 4: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,
Page 5: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

^^^^ Still the best dressed hacker, even while in NZ!

Page 6: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,
Page 7: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,
Page 8: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

Triforce Journey

● This talk is nominally about Network Reconnaissance● But really, its about a journey

● Three, entertwined journies● The LHKF project● Network reconnaissance as a whole● My journey, as a hacker

Page 9: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

Network Reconnaissance

● Traditional tools● Portscanners, banner grabbers, fingerprinting● Netcat, some-worm.c, commercial tools● Nmap 5.x == state of the art; fast, flexible, app-

layer, scriptable

● Distributed● Unicorn scan, RIP Jack.

● Modern tools● Flexible, protocol layer scanning● Searchable web interface

Page 10: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

Hang on, isnt this just V-A

● Well, yes. But have you tried asking Qualys to scan a Class B?● Not only is it expensive, but your machine will die

rendering the 50000 page pdf report, ha ha.

● Ditto nessus or whatever● Metasploit + DB might...

● But even New Zealand has 6.8M IPs. :/

● None of the tools scale well

Page 11: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

So I Wrote Another One

● Geo-targeted network recon data acquisition system● With a web interface● Automated, fire-and-forget-and-go-to-the-pub

operation● That scales properly

Page 12: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

Changelog

● v1.0 “Low Hanging Kiwifruit” for Kiwicon ]I[● 580k hosts in 6.2M IPs (.nz)

● v2.0 “Low Scuttling Chillicrab” for SyScan 2010● 360k hosts in 4.8M IPs (.sg)● New acquisition engine

● V2.1 “Now with added Luxembourg” ● (also I accidentally a whole Belgium)● 840k (.nz) + 414k (.be) + 52k (.lu) ● New db schema, search engine

Page 13: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,
Page 14: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,
Page 15: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,
Page 16: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,
Page 17: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,
Page 18: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

What's it good for?

● Target location● Exploit-centric targeting (script kiddie-ing)● Pre-seeding your “warhol worm”● Scope expansions

● National sitrep● In lieu of data breach disclosure laws● Security Consultancy● Lulz...

Page 19: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,
Page 20: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

The Innards

● v1.0 was an exercise to see how plausible it was to “just scan everything and grep”

● Nmap, python ghetto-queue, lotsa shellscripts, and manglethis2that.py glued together with some 1980s style curses gui.

● It looked something like this:

Page 21: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

Re-enactment

Page 22: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,
Page 23: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

The Innards

● Which worked surprisingly well● And taught me the necessary lessons about

how to scale it up● v2.0 re-engineered the acquisition portion

● (pretty much a coupla weekend's work)● looks something like this

metlstrm@lhkf:~$ python>>> from lhkf.acquisition import scanCountry>>> scanCountry(“lu”, [22,23,25,80,110...])

Page 24: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

Message Bus

MongoDB

Queue

Bulk ScannerPool

App ScannerPool

Disk GrindingPool

Queue Queue

The Internets

MongoDB

lhkf.scanCountry(“sg”, [21,22,23,25,80...])

Webserver

TargetGeneration

GeoIP

Page 25: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

(Enterprise) Architecture

● Hip, cloud web2.0 stylin'● MongoDB “nosql” main data store● Erlang/RabbitMQ message bus● Python/Celery MQ/Job dispatch engine

● Workflow rules to sort everything out

● PostgreSQL for relational data● Python/Django frontend● GridFS distributed filestore for bulk data (e.g.

images)

Page 26: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

Target Selection

● What's a country in cyberspace?● Domains that end in .nz/.lu/.be?● Netblocks announced at some domestic peering

exchanges?● Address registry allocations?● GeoIP?

● They're all valid answers, you just gotta pick● I chose GeoIP; outsource the problem to maxmind● Misses out dns names hosted overseas● Thats okay; simplifies our “jurisdictional issues”

Page 27: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

Acquisition

● High rate nmap TCP SYN scans, tuned well● Tried with unicorn scan; if anything its too fast, and

sadly unmaintained● Typically sit at 4kpps (16 Class C/sec...)● Pushing 30kpps makes my ISP sad :(

● Custom python protocol aware banner grabbing framework● plug in python libs, external binaries, Xservers,

whatever necessary to get app data● ~20 specific protocols at present, including

“graphical banners”

Page 28: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

Correlation

● With DNS PTR● Address registry “whois” info● DNS

● With DNS CNAME / A / MX / NS (NZ zone files)● Bing ip: lookup “unlimited API calls” :)

● Store all historical data to track changes over time

Page 29: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

Storage

● (580k + 360k hosts) * avg 15 ports/host + applayer data ~= 1.4B rows. per scan refresh

● Classic data-mine style problem● Dataset is search/read heavy, very insert light, near

zero updates.● Optimise for retreival; denormalise, index.● Relational DB wrong solution.

● MongoDB “document store” database● Auto sharding/replicating to scale out● Easy as hell to use

Page 30: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

Open Cast Data Mining

● There is just, well, a lot of it. What do you want?● Old unix boxen?● Things with self-signed certs? Wildcard certs?● Cisco Switches? Blade chassis?● SunRPC services? Writable SMB shares?● .gov/.mil/.spooks?

● Search by● Banners, SSL Cert DN, 302 targets, <title>, and

other protocol stuff (smb, ldap, mysql, mssql....)

Page 31: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

IDS Avoidance

● Corps spend mega fat-cash on IDSes and Security Operations Centres● So best be careful to avoid them, right?

● One port at a time across the whole country, randomise● Tune for detection rate across above average

netblock size (say, /16)

Page 32: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

IDS Who-Gizzashit

● Scanning .nz● 7 abuse@ mails

● Scanning .sg● 1 abuse@ mail● And it was hilarious!

– (the “eCop” detected my “horizontal and vertical” scans!)

● Scanning .lu, .be● No abuse mails :D

Page 33: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

“Hack the planet!”

Page 34: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

IDS Baiting

● So, noone's watching, right? Hack the planet?● Not quite. People are watching.● Just check out the DNS PTR backscatter if you

don't believe me.

● Portscans just aren't interesting in 2010AD● So how do we make 'em interesting?

● Pro Tip #437: Don't have a few beers on Friday night, then do this ......

Page 35: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

...in-addr.arpa. IN PTR scanner03.ccip.govt.nz.

ewps.

Page 36: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

Yeeeah, about that...

● ...don't.● My poor ISP got a call from the spooks at 0910

Monday morning,● Poor spooks probably had to fill out all sorts of

forms, in triplicate.

● So apparently people are watching :)● Hi there!

IN PTR not.really.the.CCIP.terribly.sorry.about.the.confusion.

Page 37: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

But Not Good For

● Actually doing something about it● I did try, for a while● But like software full disclosure, it's a waste of time.

● The Digital Pearl Harbour?● Open it up! Use it for hacker tourism!● Invite all the .tr and .br kidz to come own us all up!● All the low-hanging shit gets owned, it hurts for a

bit, but eventually herd health will improve● Be a stronger, better high-tech economy● … yeah, no. :/

Page 38: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

Breakin' the Law● Portscanning & preauth banner grabbing is

pretty much legal in most jurisdictions● I obey all warning banners telling me to disconnect● Scanner is tuned to avoid causing DoS to any

single IP or netblocok

● Aggregating & searching public data is legal● Providing info that can be used to “access in

excess of your authority” is possibly illegal in .nz, but there's no case law (and is also stupid)

● Making this data illegal only helps the badguys● Because they already have it.

Page 39: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

However

● I've chosen at this time not to make LHKF general public access● Instead, providing access on a case-by-case to

infosec industry people, CERTs, .gov, and anyone who sounds legit enough to me.

● Like you guys, amirite? (l: haxor.lu p: giraffe)

● I spose I could monetise it, but that sounds like actual work instead of fun

● And besides, there is already a public one of these...

Page 40: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

What About Shodan● Shodan is the same thing, but with breadth

rather than depth focus, and public● 4 ports (21,22,23,80) ● Whole world as target

● LHKF approx contemporary with Shodan● Shodan went public ~4 days before LHKF did at

Kiwicon 3

● In terms of raw data, about similar size● My .sg + .nz ~= shodan's * in host/port tuples● But: .nz: shodan: 24k hosts, LHKF: 580k

● Shodan's interface is much more hip, web2.0

Page 41: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

So What Does It All Mean

● Search engines are a force multiplier● Public data + aggregation & search = power

● Building a system like this is easy, fun and entirely too feasible● Engineering time is a few weekends

● If I have, others have● If you're a cyber*.mil and you don't have one of

these, you're doing your cyber-thing wrong.

Page 42: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

But isn't portscanning stuff just so 1997AD?

Page 43: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

Network Recon

● Recon matters● Active recon (scanning) less than it used to

– Easy to do● Passive recon (sniffing, traffic analysis) more than it

used to– (And not N-IDS/IPS)– Scales up well if you're a telco, IX, or intelligence agency

Page 44: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

Passive

● Sniff for C&C, data exfiltration from your net to detect compromise● Something in your organisation is owned; anything

else is statistically infeasible

● Acquire botnet data from someone● DNS sinkholes (ala Shadowserver)● Darknets (ala CYMRU)● Other shady crowds (Endgame, CyberEIS,

Damballa, Unveillance)

● Pretty much the only new tool in the defence arsenal lately

Page 45: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

Targeting

● Targeting is under-estimated;● Look at both Francois & Fred, Phillippe yesterday;

both are powerful attack classes, facilitated by targeting.

Assertion: ● Targeting info approaches 0day in value.

● This is one of the things that made me stop and think...

Page 46: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

Endgame.us pricelist from HBGary's mailspool

(big kthx to aaron barr for awesome passwd management)

Page 47: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,
Page 48: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,
Page 49: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

Value

● 25 x 0day = $2.5M● Botnet telemetry = $2M● Active recon info = $2M

● And you get these all correlated.

Page 50: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

Target Acquisition

● Targeting is the main function

Warehouse all the info, so you can search one db for each new tasking/target/mission

● Find the thing you need to own– Target org, its ISP, its outsourcer, its bank, its arms

vendor, its scada vendor...● Or the thing you already own (same diff, really;

given incremental cost of owning something)– Or the thing some botnet owns, and that you can buy or

steal

Page 51: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

Vector

● 0day are bad weapons● Shelf life hard to predict● Every time you use it, you risk burning it

● Utilising botnets makes more sense● More predictable/stable/weaponisable● Can outsource the crime to herders, JIT acquire● More efficient use of 0day (10s of k new hosts for a

flash 0day, vs blowing your USB 0day on a single stuxnetting)

Page 52: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

End game

● A large scale recon map relating:● Target organisations● Their trust partners● Vulnerability● Existing compromises to reuse

● == massive force multiplier

Page 53: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

The Personal Journey

● I'm a trad hacker; unix, networks, enterprise apps, trust expansion

● The world has changed around us● Its not about “this box is vulnerable to statdx”

– Its “your operational patch management policy is bad”

● I thought scanning whole countries was pretty bad-ass 4-5 years ago. ● I was wrong. It's passé. Everyone does it. ● But why its relevant now is … “cyber”.

Page 54: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

Cyber, the verb.

● Cyber changes everything● Traditional private sector infosec - AV, pentests,

code reviews, arch reviews, policy -● Is irrelevant in the world of Stuxnet, of massive

state-sponsored cyber-espionage, of Diginotar, of multi-terabit of BGP rerouting into .cn.

● We simply cannot defend against multi-million dollar offensive tech budgets

● Plus, all the talent, bugs, info is being vacuumed up into the cyber-mil-industrial complex– And if you dont...

Page 55: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,
Page 56: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

So I whittled a giraffe

I hope you like it.

www.lowhangingkiwifruit.com● Go explore .lu, .be and .nz. ● Creds are:

● Login: haxor.lu / Pass: giraffe● It'll be live for a week or two

● Be good, don't use your powers for evil● The performance will probably suck with

everyone using it, so be patient too

Page 57: Fruit: Why you so low? - insomniasec · Python/Django frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

KTHX & QuestionsGood luck. You'll need it.

metlstorm (at) storm.net.nzAlso, come to Kiwicon V in Wellington, New Zealand

Nov 5-6 2011


Top Related