Fruit: Why you so low? - insomniasec frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,
Post on 18-May-2018
Embed Size (px)
Fruit: Why you so low? Network Recon 2011AD
Oh, Hi. I'm Metlstorm (Adam to my mum)
Based in Wellington, New Zealand I hack stuff.
Usually with python, bacon, vim, unix and beer. Roll with Brett Moore's
Insomnia Security Previously of Immunity,
Security-Assessment.com On (double-award winning) weekly infosec
news podcast Risky.biz
Proprietor, Kiwicon (est 2007)
^^^^ Still the best dressed hacker, even while in NZ!
Triforce Journey This talk is nominally about Network
Reconnaissance But really, its about a journey
Three, entertwined journies The LHKF project Network reconnaissance as a whole My journey, as a hacker
Network Reconnaissance Traditional tools
Portscanners, banner grabbers, fingerprinting Netcat, some-worm.c, commercial tools Nmap 5.x == state of the art; fast, flexible, app-
layer, scriptable Distributed
Unicorn scan, RIP Jack. Modern tools
Flexible, protocol layer scanning Searchable web interface
Hang on, isnt this just V-A Well, yes. But have you tried asking Qualys to
scan a Class B? Not only is it expensive, but your machine will die
rendering the 50000 page pdf report, ha ha. Ditto nessus or whatever Metasploit + DB might...
But even New Zealand has 6.8M IPs. :/ None of the tools scale well
So I Wrote Another One Geo-targeted network recon data acquisition
system With a web interface Automated, fire-and-forget-and-go-to-the-pub
operation That scales properly
Changelog v1.0 Low Hanging Kiwifruit for Kiwicon ]I[
580k hosts in 6.2M IPs (.nz) v2.0 Low Scuttling Chillicrab for SyScan 2010
360k hosts in 4.8M IPs (.sg) New acquisition engine
V2.1 Now with added Luxembourg (also I accidentally a whole Belgium) 840k (.nz) + 414k (.be) + 52k (.lu) New db schema, search engine
What's it good for? Target location
Exploit-centric targeting (script kiddie-ing) Pre-seeding your warhol worm Scope expansions
National sitrep In lieu of data breach disclosure laws Security Consultancy Lulz...
The Innards v1.0 was an exercise to see how plausible it
was to just scan everything and grep Nmap, python ghetto-queue, lotsa shellscripts,
and manglethis2that.py glued together with some 1980s style curses gui.
It looked something like this:
The Innards Which worked surprisingly well And taught me the necessary lessons about
how to scale it up v2.0 re-engineered the acquisition portion
(pretty much a coupla weekend's work) looks something like this
(Enterprise) Architecture Hip, cloud web2.0 stylin' MongoDB nosql main data store Erlang/RabbitMQ message bus Python/Celery MQ/Job dispatch engine
Workflow rules to sort everything out PostgreSQL for relational data Python/Django frontend GridFS distributed filestore for bulk data (e.g.
Target Selection What's a country in cyberspace?
Domains that end in .nz/.lu/.be? Netblocks announced at some domestic peering
exchanges? Address registry allocations? GeoIP?
They're all valid answers, you just gotta pick I chose GeoIP; outsource the problem to maxmind Misses out dns names hosted overseas Thats okay; simplifies our jurisdictional issues
Acquisition High rate nmap TCP SYN scans, tuned well
Tried with unicorn scan; if anything its too fast, and sadly unmaintained
Typically sit at 4kpps (16 Class C/sec...) Pushing 30kpps makes my ISP sad :(
Custom python protocol aware banner grabbing framework plug in python libs, external binaries, Xservers,
whatever necessary to get app data ~20 specific protocols at present, including
Correlation With DNS PTR Address registry whois info DNS
With DNS CNAME / A / MX / NS (NZ zone files) Bing ip: lookup unlimited API calls :)
Store all historical data to track changes over time
Storage (580k + 360k hosts) * avg 15 ports/host +
applayer data ~= 1.4B rows. per scan refresh Classic data-mine style problem
Dataset is search/read heavy, very insert light, near zero updates.
Optimise for retreival; denormalise, index. Relational DB wrong solution.
MongoDB document store database Auto sharding/replicating to scale out Easy as hell to use
Open Cast Data Mining There is just, well, a lot of it. What do you want?
Old unix boxen? Things with self-signed certs? Wildcard certs? Cisco Switches? Blade chassis? SunRPC services? Writable SMB shares? .gov/.mil/.spooks?
Search by Banners, SSL Cert DN, 302 targets, , and
other protocol stuff (smb, ldap, mysql, mssql....)
IDS Avoidance Corps spend mega fat-cash on IDSes and
Security Operations Centres So best be careful to avoid them, right?
One port at a time across the whole country, randomise Tune for detection rate across above average
netblock size (say, /16)
IDS Who-Gizzashit Scanning .nz
7 abuse@ mails Scanning .sg
1 abuse@ mail And it was hilarious!
(the eCop detected my horizontal and vertical scans!) Scanning .lu, .be
No abuse mails :D
Hack the planet!
IDS Baiting So, noone's watching, right? Hack the planet?
Not quite. People are watching. Just check out the DNS PTR backscatter if you
don't believe me. Portscans just aren't interesting in 2010AD
So how do we make 'em interesting?
Pro Tip #437: Don't have a few beers on Friday night, then do this ......
...in-addr.arpa. IN PTR scanner03.ccip.govt.nz.
Yeeeah, about that... ...don't. My poor ISP got a call from the spooks at 0910
Monday morning, Poor spooks probably had to fill out all sorts of
forms, in triplicate.
So apparently people are watching :) Hi there!IN PTR
But Not Good For Actually doing something about it
I did try, for a while But like software full disclosure, it's a waste of time.
The Digital Pearl Harbour? Open it up! Use it for hacker tourism! Invite all the .tr and .br kidz to come own us all up! All the low-hanging shit gets owned, it hurts for a
bit, but eventually herd health will improve Be a stronger, better high-tech economy yeah, no. :/
Breakin' the Law Portscanning & preauth banner grabbing is
pretty much legal in most jurisdictions I obey all warning banners telling me to disconnect Scanner is tuned to avoid causing DoS to any
single IP or netblocok Aggregating & searching public data is legal Providing info that can be used to access in
excess of your authority is possibly illegal in .nz, but there's no case law (and is also stupid)
Making this data illegal only helps the badguys Because they already have it.
However I've chosen at this time not to make LHKF
general public access Instead, providing access on a case-by-case to
infosec industry people, CERTs, .gov, and anyone who sounds legit enough to me.
Like you guys, amirite? (l: haxor.lu p: giraffe) I spose I could monetise it, but that sounds like
actual work instead of fun And besides, there is already a public one of
What About Shodan Shodan is the same thing, but with breadth
rather than depth focus, and public 4 ports (21,22,23,80) Whole world as target
LHKF approx contemporary with Shodan Shodan went public ~4 days before LHKF did at
Kiwicon 3 In terms of raw data, about similar size
My .sg + .nz ~= shodan's * in host/port tuples But: .nz: shodan: 24k hosts, LHKF: 580k
Shodan's interface is much more hip, web2.0
So What Does It All Mean Search engines are a force multiplier
Public data + aggregation & search = power Building a system like this is easy, fun and
entirely too feasible Engineering time is a few weekends
If I have, others have If you're a cyber*.mil and you don't have one of
these, you're doing your cyber-thing wrong.
But isn't portscanning stuff just so 1997AD?
Network Recon Recon matters
Active recon (scanning) less than it used to Easy to do
Passive recon (sniffing, traffic analysis) more than it used to (And not N-IDS/IPS) Scales up well if you're a telco, IX, or intelligence agency
Passive Sniff for C&C, data exfiltration from your net to
detect compromise Something in your organisation is owned; anything
else is statistically infeasible Acquire botnet data from someone
DNS sinkholes (ala Shadowserver) Darknets (ala CYMRU) Other shady crowds (Endgame, CyberEIS,
Damballa, Unveillance) Pretty much the only new tool in the defence
Targeting Targeting is under-estimated;
Look at both Francois & Fred, Phillippe yesterday; both are powerful attack classes, facilitated by targeting.
Assertion: Targeting info approaches 0day in value.
This is one of the things that made me stop and think...
Endgame.us pricelist from HBGary's mailspool(big kthx to aaron b