Fruit: Why you so low? - insomniasec frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

Download Fruit: Why you so low? - insomniasec  frontend ... But why its relevant now is ... The performance will probably suck with everyone using it,

Post on 18-May-2018




0 download

Embed Size (px)


  • Fruit: Why you so low? Network Recon 2011AD 2011

  • Oh, Hi. I'm Metlstorm (Adam to my mum)

    Based in Wellington, New Zealand I hack stuff.

    Usually with python, bacon, vim, unix and beer. Roll with Brett Moore's

    Insomnia Security Previously of Immunity, On (double-award winning) weekly infosec

    news podcast

  • Proprietor, Kiwicon (est 2007)

  • ^^^^ Still the best dressed hacker, even while in NZ!

  • Triforce Journey This talk is nominally about Network

    Reconnaissance But really, its about a journey

    Three, entertwined journies The LHKF project Network reconnaissance as a whole My journey, as a hacker

  • Network Reconnaissance Traditional tools

    Portscanners, banner grabbers, fingerprinting Netcat, some-worm.c, commercial tools Nmap 5.x == state of the art; fast, flexible, app-

    layer, scriptable Distributed

    Unicorn scan, RIP Jack. Modern tools

    Flexible, protocol layer scanning Searchable web interface

  • Hang on, isnt this just V-A Well, yes. But have you tried asking Qualys to

    scan a Class B? Not only is it expensive, but your machine will die

    rendering the 50000 page pdf report, ha ha. Ditto nessus or whatever Metasploit + DB might...

    But even New Zealand has 6.8M IPs. :/ None of the tools scale well

  • So I Wrote Another One Geo-targeted network recon data acquisition

    system With a web interface Automated, fire-and-forget-and-go-to-the-pub

    operation That scales properly

  • Changelog v1.0 Low Hanging Kiwifruit for Kiwicon ]I[

    580k hosts in 6.2M IPs (.nz) v2.0 Low Scuttling Chillicrab for SyScan 2010

    360k hosts in 4.8M IPs (.sg) New acquisition engine

    V2.1 Now with added Luxembourg (also I accidentally a whole Belgium) 840k (.nz) + 414k (.be) + 52k (.lu) New db schema, search engine

  • What's it good for? Target location

    Exploit-centric targeting (script kiddie-ing) Pre-seeding your warhol worm Scope expansions

    National sitrep In lieu of data breach disclosure laws Security Consultancy Lulz...

  • The Innards v1.0 was an exercise to see how plausible it

    was to just scan everything and grep Nmap, python ghetto-queue, lotsa shellscripts,

    and glued together with some 1980s style curses gui.

    It looked something like this:

  • Re-enactment

  • The Innards Which worked surprisingly well And taught me the necessary lessons about

    how to scale it up v2.0 re-engineered the acquisition portion

    (pretty much a coupla weekend's work) looks something like this


  • Message Bus



    Bulk ScannerPool

    App ScannerPool

    Disk GrindingPool

    Queue Queue

    The Internets


    lhkf.scanCountry(sg, [21,22,23,25,80...])




  • (Enterprise) Architecture Hip, cloud web2.0 stylin' MongoDB nosql main data store Erlang/RabbitMQ message bus Python/Celery MQ/Job dispatch engine

    Workflow rules to sort everything out PostgreSQL for relational data Python/Django frontend GridFS distributed filestore for bulk data (e.g.


  • Target Selection What's a country in cyberspace?

    Domains that end in .nz/.lu/.be? Netblocks announced at some domestic peering

    exchanges? Address registry allocations? GeoIP?

    They're all valid answers, you just gotta pick I chose GeoIP; outsource the problem to maxmind Misses out dns names hosted overseas Thats okay; simplifies our jurisdictional issues

  • Acquisition High rate nmap TCP SYN scans, tuned well

    Tried with unicorn scan; if anything its too fast, and sadly unmaintained

    Typically sit at 4kpps (16 Class C/sec...) Pushing 30kpps makes my ISP sad :(

    Custom python protocol aware banner grabbing framework plug in python libs, external binaries, Xservers,

    whatever necessary to get app data ~20 specific protocols at present, including

    graphical banners

  • Correlation With DNS PTR Address registry whois info DNS

    With DNS CNAME / A / MX / NS (NZ zone files) Bing ip: lookup unlimited API calls :)

    Store all historical data to track changes over time

  • Storage (580k + 360k hosts) * avg 15 ports/host +

    applayer data ~= 1.4B rows. per scan refresh Classic data-mine style problem

    Dataset is search/read heavy, very insert light, near zero updates.

    Optimise for retreival; denormalise, index. Relational DB wrong solution.

    MongoDB document store database Auto sharding/replicating to scale out Easy as hell to use

  • Open Cast Data Mining There is just, well, a lot of it. What do you want?

    Old unix boxen? Things with self-signed certs? Wildcard certs? Cisco Switches? Blade chassis? SunRPC services? Writable SMB shares? .gov/.mil/.spooks?

    Search by Banners, SSL Cert DN, 302 targets, , and

    other protocol stuff (smb, ldap, mysql, mssql....)

  • IDS Avoidance Corps spend mega fat-cash on IDSes and

    Security Operations Centres So best be careful to avoid them, right?

    One port at a time across the whole country, randomise Tune for detection rate across above average

    netblock size (say, /16)

  • IDS Who-Gizzashit Scanning .nz

    7 abuse@ mails Scanning .sg

    1 abuse@ mail And it was hilarious!

    (the eCop detected my horizontal and vertical scans!) Scanning .lu, .be

    No abuse mails :D

  • Hack the planet!

  • IDS Baiting So, noone's watching, right? Hack the planet?

    Not quite. People are watching. Just check out the DNS PTR backscatter if you

    don't believe me. Portscans just aren't interesting in 2010AD

    So how do we make 'em interesting?

    Pro Tip #437: Don't have a few beers on Friday night, then do this ......

  • IN PTR


  • Yeeeah, about that... ...don't. My poor ISP got a call from the spooks at 0910

    Monday morning, Poor spooks probably had to fill out all sorts of

    forms, in triplicate.

    So apparently people are watching :) Hi there!IN PTR


  • But Not Good For Actually doing something about it

    I did try, for a while But like software full disclosure, it's a waste of time.

    The Digital Pearl Harbour? Open it up! Use it for hacker tourism! Invite all the .tr and .br kidz to come own us all up! All the low-hanging shit gets owned, it hurts for a

    bit, but eventually herd health will improve Be a stronger, better high-tech economy yeah, no. :/

  • Breakin' the Law Portscanning & preauth banner grabbing is

    pretty much legal in most jurisdictions I obey all warning banners telling me to disconnect Scanner is tuned to avoid causing DoS to any

    single IP or netblocok Aggregating & searching public data is legal Providing info that can be used to access in

    excess of your authority is possibly illegal in .nz, but there's no case law (and is also stupid)

    Making this data illegal only helps the badguys Because they already have it.

  • However I've chosen at this time not to make LHKF

    general public access Instead, providing access on a case-by-case to

    infosec industry people, CERTs, .gov, and anyone who sounds legit enough to me.

    Like you guys, amirite? (l: p: giraffe) I spose I could monetise it, but that sounds like

    actual work instead of fun And besides, there is already a public one of


  • What About Shodan Shodan is the same thing, but with breadth

    rather than depth focus, and public 4 ports (21,22,23,80) Whole world as target

    LHKF approx contemporary with Shodan Shodan went public ~4 days before LHKF did at

    Kiwicon 3 In terms of raw data, about similar size

    My .sg + .nz ~= shodan's * in host/port tuples But: .nz: shodan: 24k hosts, LHKF: 580k

    Shodan's interface is much more hip, web2.0

  • So What Does It All Mean Search engines are a force multiplier

    Public data + aggregation & search = power Building a system like this is easy, fun and

    entirely too feasible Engineering time is a few weekends

    If I have, others have If you're a cyber*.mil and you don't have one of

    these, you're doing your cyber-thing wrong.

  • But isn't portscanning stuff just so 1997AD?

  • Network Recon Recon matters

    Active recon (scanning) less than it used to Easy to do

    Passive recon (sniffing, traffic analysis) more than it used to (And not N-IDS/IPS) Scales up well if you're a telco, IX, or intelligence agency

  • Passive Sniff for C&C, data exfiltration from your net to

    detect compromise Something in your organisation is owned; anything

    else is statistically infeasible Acquire botnet data from someone

    DNS sinkholes (ala Shadowserver) Darknets (ala CYMRU) Other shady crowds (Endgame, CyberEIS,

    Damballa, Unveillance) Pretty much the only new tool in the defence

    arsenal lately

  • Targeting Targeting is under-estimated;

    Look at both Francois & Fred, Phillippe yesterday; both are powerful attack classes, facilitated by targeting.

    Assertion: Targeting info approaches 0day in value.

    This is one of the things that made me stop and think...

  • pricelist from HBGary's mailspool(big kthx to aaron b