Fruit: Why you so low? Network Recon 2011AD
Hack.lu 2011
Oh, Hi.
● I'm Metlstorm (Adam to my mum)● Based in Wellington, New Zealand
● I hack stuff. ● Usually with python, bacon, vim, unix and beer.
● Roll with Brett Moore's Insomnia Security● Previously of Immunity,
Security-Assessment.com
● On (double-award winning) weekly infosec news podcast Risky.biz
Proprietor, Kiwicon (est 2007)
^^^^ Still the best dressed hacker, even while in NZ!
Triforce Journey
● This talk is nominally about Network Reconnaissance● But really, its about a journey
● Three, entertwined journies● The LHKF project● Network reconnaissance as a whole● My journey, as a hacker
Network Reconnaissance
● Traditional tools● Portscanners, banner grabbers, fingerprinting● Netcat, some-worm.c, commercial tools● Nmap 5.x == state of the art; fast, flexible, app-
layer, scriptable
● Distributed● Unicorn scan, RIP Jack.
● Modern tools● Flexible, protocol layer scanning● Searchable web interface
Hang on, isnt this just V-A
● Well, yes. But have you tried asking Qualys to scan a Class B?● Not only is it expensive, but your machine will die
rendering the 50000 page pdf report, ha ha.
● Ditto nessus or whatever● Metasploit + DB might...
● But even New Zealand has 6.8M IPs. :/
● None of the tools scale well
So I Wrote Another One
● Geo-targeted network recon data acquisition system● With a web interface● Automated, fire-and-forget-and-go-to-the-pub
operation● That scales properly
●
Changelog
● v1.0 “Low Hanging Kiwifruit” for Kiwicon ]I[● 580k hosts in 6.2M IPs (.nz)
● v2.0 “Low Scuttling Chillicrab” for SyScan 2010● 360k hosts in 4.8M IPs (.sg)● New acquisition engine
● V2.1 “Now with added Luxembourg” ● (also I accidentally a whole Belgium)● 840k (.nz) + 414k (.be) + 52k (.lu) ● New db schema, search engine
What's it good for?
● Target location● Exploit-centric targeting (script kiddie-ing)● Pre-seeding your “warhol worm”● Scope expansions
● National sitrep● In lieu of data breach disclosure laws● Security Consultancy● Lulz...
The Innards
● v1.0 was an exercise to see how plausible it was to “just scan everything and grep”
● Nmap, python ghetto-queue, lotsa shellscripts, and manglethis2that.py glued together with some 1980s style curses gui.
● It looked something like this:
Re-enactment
The Innards
● Which worked surprisingly well● And taught me the necessary lessons about
how to scale it up● v2.0 re-engineered the acquisition portion
● (pretty much a coupla weekend's work)● looks something like this
metlstrm@lhkf:~$ python>>> from lhkf.acquisition import scanCountry>>> scanCountry(“lu”, [22,23,25,80,110...])
Message Bus
MongoDB
Queue
Bulk ScannerPool
App ScannerPool
Disk GrindingPool
Queue Queue
The Internets
MongoDB
lhkf.scanCountry(“sg”, [21,22,23,25,80...])
Webserver
TargetGeneration
GeoIP
(Enterprise) Architecture
● Hip, cloud web2.0 stylin'● MongoDB “nosql” main data store● Erlang/RabbitMQ message bus● Python/Celery MQ/Job dispatch engine
● Workflow rules to sort everything out
● PostgreSQL for relational data● Python/Django frontend● GridFS distributed filestore for bulk data (e.g.
images)
Target Selection
● What's a country in cyberspace?● Domains that end in .nz/.lu/.be?● Netblocks announced at some domestic peering
exchanges?● Address registry allocations?● GeoIP?
● They're all valid answers, you just gotta pick● I chose GeoIP; outsource the problem to maxmind● Misses out dns names hosted overseas● Thats okay; simplifies our “jurisdictional issues”
Acquisition
● High rate nmap TCP SYN scans, tuned well● Tried with unicorn scan; if anything its too fast, and
sadly unmaintained● Typically sit at 4kpps (16 Class C/sec...)● Pushing 30kpps makes my ISP sad :(
● Custom python protocol aware banner grabbing framework● plug in python libs, external binaries, Xservers,
whatever necessary to get app data● ~20 specific protocols at present, including
“graphical banners”
Correlation
● With DNS PTR● Address registry “whois” info● DNS
● With DNS CNAME / A / MX / NS (NZ zone files)● Bing ip: lookup “unlimited API calls” :)
● Store all historical data to track changes over time
Storage
● (580k + 360k hosts) * avg 15 ports/host + applayer data ~= 1.4B rows. per scan refresh
● Classic data-mine style problem● Dataset is search/read heavy, very insert light, near
zero updates.● Optimise for retreival; denormalise, index.● Relational DB wrong solution.
● MongoDB “document store” database● Auto sharding/replicating to scale out● Easy as hell to use
Open Cast Data Mining
● There is just, well, a lot of it. What do you want?● Old unix boxen?● Things with self-signed certs? Wildcard certs?● Cisco Switches? Blade chassis?● SunRPC services? Writable SMB shares?● .gov/.mil/.spooks?
● Search by● Banners, SSL Cert DN, 302 targets, <title>, and
other protocol stuff (smb, ldap, mysql, mssql....)
IDS Avoidance
● Corps spend mega fat-cash on IDSes and Security Operations Centres● So best be careful to avoid them, right?
● One port at a time across the whole country, randomise● Tune for detection rate across above average
netblock size (say, /16)
IDS Who-Gizzashit
● Scanning .nz● 7 abuse@ mails
● Scanning .sg● 1 abuse@ mail● And it was hilarious!
– (the “eCop” detected my “horizontal and vertical” scans!)
● Scanning .lu, .be● No abuse mails :D
“Hack the planet!”
IDS Baiting
● So, noone's watching, right? Hack the planet?● Not quite. People are watching.● Just check out the DNS PTR backscatter if you
don't believe me.
● Portscans just aren't interesting in 2010AD● So how do we make 'em interesting?
● Pro Tip #437: Don't have a few beers on Friday night, then do this ......
...in-addr.arpa. IN PTR scanner03.ccip.govt.nz.
ewps.
Yeeeah, about that...
● ...don't.● My poor ISP got a call from the spooks at 0910
Monday morning,● Poor spooks probably had to fill out all sorts of
forms, in triplicate.
● So apparently people are watching :)● Hi there!
IN PTR not.really.the.CCIP.terribly.sorry.about.the.confusion.
But Not Good For
● Actually doing something about it● I did try, for a while● But like software full disclosure, it's a waste of time.
● The Digital Pearl Harbour?● Open it up! Use it for hacker tourism!● Invite all the .tr and .br kidz to come own us all up!● All the low-hanging shit gets owned, it hurts for a
bit, but eventually herd health will improve● Be a stronger, better high-tech economy● … yeah, no. :/
Breakin' the Law● Portscanning & preauth banner grabbing is
pretty much legal in most jurisdictions● I obey all warning banners telling me to disconnect● Scanner is tuned to avoid causing DoS to any
single IP or netblocok
● Aggregating & searching public data is legal● Providing info that can be used to “access in
excess of your authority” is possibly illegal in .nz, but there's no case law (and is also stupid)
● Making this data illegal only helps the badguys● Because they already have it.
However
● I've chosen at this time not to make LHKF general public access● Instead, providing access on a case-by-case to
infosec industry people, CERTs, .gov, and anyone who sounds legit enough to me.
● Like you guys, amirite? (l: haxor.lu p: giraffe)
● I spose I could monetise it, but that sounds like actual work instead of fun
● And besides, there is already a public one of these...
What About Shodan● Shodan is the same thing, but with breadth
rather than depth focus, and public● 4 ports (21,22,23,80) ● Whole world as target
● LHKF approx contemporary with Shodan● Shodan went public ~4 days before LHKF did at
Kiwicon 3
● In terms of raw data, about similar size● My .sg + .nz ~= shodan's * in host/port tuples● But: .nz: shodan: 24k hosts, LHKF: 580k
● Shodan's interface is much more hip, web2.0
So What Does It All Mean
● Search engines are a force multiplier● Public data + aggregation & search = power
● Building a system like this is easy, fun and entirely too feasible● Engineering time is a few weekends
● If I have, others have● If you're a cyber*.mil and you don't have one of
these, you're doing your cyber-thing wrong.
But isn't portscanning stuff just so 1997AD?
Network Recon
● Recon matters● Active recon (scanning) less than it used to
– Easy to do● Passive recon (sniffing, traffic analysis) more than it
used to– (And not N-IDS/IPS)– Scales up well if you're a telco, IX, or intelligence agency
Passive
● Sniff for C&C, data exfiltration from your net to detect compromise● Something in your organisation is owned; anything
else is statistically infeasible
● Acquire botnet data from someone● DNS sinkholes (ala Shadowserver)● Darknets (ala CYMRU)● Other shady crowds (Endgame, CyberEIS,
Damballa, Unveillance)
● Pretty much the only new tool in the defence arsenal lately
Targeting
● Targeting is under-estimated;● Look at both Francois & Fred, Phillippe yesterday;
both are powerful attack classes, facilitated by targeting.
Assertion: ● Targeting info approaches 0day in value.
● This is one of the things that made me stop and think...
Endgame.us pricelist from HBGary's mailspool
(big kthx to aaron barr for awesome passwd management)
Value
● 25 x 0day = $2.5M● Botnet telemetry = $2M● Active recon info = $2M
● And you get these all correlated.
Target Acquisition
● Targeting is the main function
Warehouse all the info, so you can search one db for each new tasking/target/mission
● Find the thing you need to own– Target org, its ISP, its outsourcer, its bank, its arms
vendor, its scada vendor...● Or the thing you already own (same diff, really;
given incremental cost of owning something)– Or the thing some botnet owns, and that you can buy or
steal
Vector
● 0day are bad weapons● Shelf life hard to predict● Every time you use it, you risk burning it
● Utilising botnets makes more sense● More predictable/stable/weaponisable● Can outsource the crime to herders, JIT acquire● More efficient use of 0day (10s of k new hosts for a
flash 0day, vs blowing your USB 0day on a single stuxnetting)
End game
● A large scale recon map relating:● Target organisations● Their trust partners● Vulnerability● Existing compromises to reuse
● == massive force multiplier
The Personal Journey
● I'm a trad hacker; unix, networks, enterprise apps, trust expansion
● The world has changed around us● Its not about “this box is vulnerable to statdx”
– Its “your operational patch management policy is bad”
● I thought scanning whole countries was pretty bad-ass 4-5 years ago. ● I was wrong. It's passé. Everyone does it. ● But why its relevant now is … “cyber”.
Cyber, the verb.
● Cyber changes everything● Traditional private sector infosec - AV, pentests,
code reviews, arch reviews, policy -● Is irrelevant in the world of Stuxnet, of massive
state-sponsored cyber-espionage, of Diginotar, of multi-terabit of BGP rerouting into .cn.
● We simply cannot defend against multi-million dollar offensive tech budgets
● Plus, all the talent, bugs, info is being vacuumed up into the cyber-mil-industrial complex– And if you dont...
So I whittled a giraffe
I hope you like it.
www.lowhangingkiwifruit.com● Go explore .lu, .be and .nz. ● Creds are:
● Login: haxor.lu / Pass: giraffe● It'll be live for a week or two
● Be good, don't use your powers for evil● The performance will probably suck with
everyone using it, so be patient too
KTHX & QuestionsGood luck. You'll need it.
metlstorm (at) storm.net.nzAlso, come to Kiwicon V in Wellington, New Zealand
Nov 5-6 2011