an effective spam protection system

An Effective Spam Protection System

Wael Alnemer 100416646

Introduction : What would you do to stop spam?

Before :

Firewalls have basic rules

Job is done

Internal Network, why should I care !?, spam is coming from outside

Software : maybe Windows Server / Exchange or Linux/ Spamassassin... Does it make any difference ?

Imagine : you are responsible to protect a small business network against spam .

Exchange Server

Internal Firewall External Firewall

?? ? ?

?

?

?

Introduction : Think again

Wait ... Have you thought of

Budget ? .. Maybe they can’t afford your solution

Content-based filter limitations (e.g.,Spamassassin) ?

Spam is not always unsolicited message, it could carry malware as well.

Hardware and other resources limitations ?

Was your solution itself secured ?

Network Infrastructure ? .. Maybe the environment is heterogeneous

Do you know that your opponents have successfully hacked a bigger organization than yours (e.g., Sony Pictures Entertainment )

Job is NOT done

??????

Introduction : Strategic Planning

Don’t surrender Change your Approach

Strategic plan : Set goals, then determining the required actions to achieve these goals, and mobilizing whatever resource in your sphere of control to execute the actions.

The heart of your defense strategy should based on knowing your opponents’ techniques, intentions, maneuvers and tricks.

You can know about them from different resources

Honeybot projects

www.honeynet.org

Security labs

Research papers

Cybercriminal takes up to 20 different measures to hide his identity.

Gathering information about your enemy

waelnemer alnemer alnerwelcome home the project must �nish this time

waelnemer alnemer alnemerwelcome home , the �g is a little messy

Kaspersky labsOpenDNS security labs

C. Dietrich, C. Rossow, F. Freiling, H. Bos,M. Steen,and N. Pohlmann,” On Botnets that use DNS for Command and Control”

Increase the rate of readiness

It’s not a matter of if you’re going to be compromised, the real questions are, when and how long it takes you to notice.

The best strategy to prepare for cyber-attack, is to eliminate the element of surprise.

Why use Spamassassin as first line of defense ,when 75% of all spam could easily be dismissed by ACL.

Why use Windows OS as a platform for your defenses, when statistically, the majority of malware were designed to exploit Windows OS vulnerabilities.

Why not using high port numbers , whenever possible, to hide our services from being detected.

?

?

?

The whole picture

Update

Hot-fix

Logs/Audit Debugging

Ethi

cal H

acki

ngSe

curi

ty to

ols

New

idea

sSe

curi

ty L

abs

Hot

fix,

patc

hes

and

upd

ates

Secu

rity

Pol

icy

Incr

ease

P

rote

ctio

n D

isco

ver

Secu

rity

hol

esR

earr

ange

P

rior

itie

sEff

ecti

veIn

vest

igat

ion

Experience

Honeybot Research

Res

ourc

es

Benefi

ts and practical usage

Background: SMTP

SMTP (Simple Mail Transfer Protocol )is a text-based protocol, in which a mailsender communicates with a mail receiver by issuing command.

If you think SMTP is inherently insecure, you are right.

Today Extended SMTP RFC [2821] is what we use.

Conceived in 1982 RFC [821].

Simple so it can be deployed on a huge scale, and on various platforms.

To put it simply : No SMTP = No email.

Thunderbird / telnet exim.mailexample.out 25

Code : 220 , Hello

Code : 250 , This is a list of my services

EHLO : Hello, I support service extension requests

DATA

Hello , Blah blah blah.

MAIL FROM: [email protected]

Code : 250 , OK

RCPT TO: [email protected]

Code : 250 , Accepted

Code : 250 , OK

Code : 354 , Start mail input; end with “ . ”

code : 0221 mailexample.out Service closing transmission channel

QUIT

Sess

ion

initi

atio

nCl

ient

Initi

atio

nM

ail t

rans

actio

nsSe

ssio

n te

rmin

atio

n

Email delivery network

SMTP : 587SMTP : 25

MUA MSA/MTA

MTA MDA MUA

SMTP : 25

SMTP : 25SMTP : 25

MX records

DNS:53DNS Server

Internet

TCP port 587 was dedicated for SMTP mail submission RFC[6406]

Mail User Agent (MUA)

Mail Submission Agent (MSA)

Mail Transfer Agent (MTA)

Mail delivery Agent (MUA)

Many ISP block port 25, as part of an effort to reduce the amount of spam that is sent through their networks.

SMTP authentication Windows Domain controller

Act

ive

Dir

ecto

ry

Linux/Ubuntu

Exim

Deliver Tickets

Access LDAP as a service LDAP/TLS Bind

Received SMTP command

SSSD

KDC LDAP

Users

Keytab

OPEN

PAM

SASL

LDAP

NSS

Kerberos 5

Enterprise SubCA

SIDUID,GID

SMTP Auth

RECP TO

2

SMTP Session

1

SMTP authentication based on SASL (Simple Authentication and Security Layer) concept RFC [4422]

Authentication mechanisms supported by SASL

PLAIN and LOGIN base64 encoded.

CRAM-MD5 RFC [2195].

Cyrus SASL the GNU SASL.

GSSAPI, geared for Kerberos V5 RFC [4752].

In this project we’ve deployed LDAP/LTS against AD.

1

2 System Security Services Daemon (SSSD) to verify the recipient’s existence.

SMTP with TLS

Code : 250 , This is a list of my services including STARTTLS

Code : 220 , Go ahead


Clie

nt In

itia

tion

Mai

ltr

ansa

ctio

ns


negotiating TLS-encrypted connection

STARTTLS

STARTTLS SMTPSClient Server

AUTH LOGIN



Sess

ion

init

iati

on stunnel exim.mailexample.out 465

negotiating TLS-encrypted connection

Thunderbird : 465

AUTH LOGIN

Certification Authority

TCP Session establishment

CA

Server

Certificates In this project were self-signed issued by MS (Root CA - Subordinate CA) format

The certificates were used to establish SMTP-TLS and LDAP-TLS.

There is no way for client’s OS to verify self-signed certificates, hencethe trust must be made in advance, by importing the related server certificate and store it in client machine.

Botnet is a group of compromised computers (Bots), exploited without their owners realizing that their computers areperforming additional tasks.

Botnet under the command and control of a malicious botmaster.

80% of all spam in 2010 were sent from botnets.

Other cybercriminals’ techniques are not trivial, but rather less critical.

Botnet is the most vicious technology at cybercriminal disposal.

Botnets is a very serious security issue; almost all Governments have shown a profound concern about it.

Spamming Methods

Botmaster

C&C ServerAccess

Commands Co

mm

and

s

Commands

1

2

3 5

4

Personal identities.

Credit card.

Bank information .

Platform for information dispersion:

Platform for other purposes :

Platform for collecting sensitive information

Distribute spam ,and launch DoS attacks

Click fraud

Botnet threats

1

2

3

5

4

Botnet protocols :

Botnet topology :

Centralized : bot needs C&C server(s) to establish reliable channels, and receive the commands from it.

Distributed: no need for C&C server, bot acts as clients and server .

Internet Relay Chat(IRC)

HTTP and DNS (first fully DNS based botnet discovered in 2011)

Botnet topology and protocols

C&C

Exploiting the already established infrastructure.

Eggdrop was the first bot, developed in 1993 for good intention.

Camouflage their genuine intentions.

Difficult to be detected,easily vanished into daily traffics.

1

2

2

1

Typical Bot Life Cycle

1

2

34

5

6DNS Server

Botmaster

C&CBot software

DNSIRC

HTTP

Scan to discover and exploit a vulnerable host.

Download and install a copy of the bot software.

DNS lookup.

Declaring it’s readiness to C&C.

Botmaster sends his commands to the C&C server.

C&C server forwards the commands to all bots.

1

2

3

5

4

6

Fast-Flux Service Networks : Botnets Facilitator

DNS

Flux Agents/proxies

Botmaster Control Center

Web Server

IP 1

IP 2

IP 3

IP 4

1 3

4

5

6

7

2

67.10.117.xxx

66.229.133.xxx

74.67.113.xxx

70.244.2.xxx

ns.ouit.ca

.ca root

FstFx.ouit.ca

Ask IP address of the DNS responsible for ouit.ca.

FFSN used to hide the real source of bogus websiteseven C&C real IP address

Get IP address of ns.ouit.ca

Ask Authoritative DNS forIP address of bogus FstFx.ouit.ca.

IP address belong to one of the flux agent pool with veryshort TTL .

Initiate a communication with the alleged web-server through proxy.

Flux agent requests the contents of FstFx.ouit.ca from the web-server

Flux agent redirects the response from the genuine web-server to the victim

1

2

3

5

4

6

7

Anti-spam measures

Many different anti-spam measures have evolved over the years Laws and regulations (e.g.,Canada’s Anti-Spam Law (CASL))

Behavioral measures

Economic measures

Technological measures.

The technological approach in fighting the spams

Content-based filter (e.g., Spamassassin)

Network-level Anti-spam techniques(e.g., Mailbox dispatcher)

There are three lists in Mail Box Dispatcher: Blacklist, Whitelist, and Greylist.

Blacklists : Can come in many forms, when they are DNS-based; they are called Domain Name System Blacklists (DNSBLs).

Data has to be distributed among MTAs from specific provider (e.g., Spamhaus).

DNS would returned a specific A record If the host was in the list(e.g.,Spamhaus would return 127.0.0.2).

Greylist : Is temporary in nature.

Has two time-out settings. Any sender who didn’t retry or attempts a retry too soon will be refused.

Mailbox dispatcher

Mailbox dispatcher : process summery

Is sender onBlack list

Is sender onGrey list

Is sender onWhite list

Sender

delivery

Ask the senderto Try again later

Time passed since last

Add sender toGery list , Reject

Yes

Yes

Yes

No

Too Soon

No

resp

onse

- To

o Lo

ng

No

No

Accept Delivery

Content-based filter Signature-based filter

Work like Anti-virus software , Assign a signature to well known spam.

Signature is a unique identifier obtained by assigning a value to each character in the email,all values are totaled, creating the spam signature.

Bayesian filtering

Scans the contents of each message searching for words, phrases and formats common to Spammers. The search then assigns a score.

Is forward-looking, it has the abilities to predict whether the email is spam or not, based on probability.

Fewer emails passing the filter for inspection mean more time the filterneeds to build its own database and be effective.

Exim Access Control List

After the firewall, Exim Access Control Lists (ACL) is the first line of defense in our attempt to fight spam.

It might considered as most sophisticated and flexible mechanism forSMTP real-time filtering , though it’s more than firewall set of rules than a filter governed by probabilities or signatures.

With ACL we can force the spammer to obey the rules.

When String expansions combine with ACL ,the result is a powerful weapon against spammers.

String expansion required user’s familiarity with regular expression (e.g., Perl, JavaScript).

The main idea of ACLs is to control Exim’s behavior when it receives certain SMTP commands.

ACL structure

Action verb

Sample : Sample : Statement #1

SMTP Commands

Statement #2

acl_smtp_connect

acl_smtp_data

acl_smtp_helo

acl_smtp_rcpt

acl_smtp_auth

RCPT_acl

Prede�ned variable

deny

discard

drop

require

defer

warn

accept

Special variables

Condition

Modi�er

Modi�er

String expansions

controldelayLog messagesError messages

$acl_c$acl_m

authenticateddomainsdnslists

verify

Expansion operators

Expansion items

Expansion conditionsExpansion variables

Action verb

Condition

deny

domains = *.dom.example

! verify = recipient

message = can’t verify recipient

${hmac${certextract${�lter${lookup

${base62:<digits>}${domain:<string>}

${md5:<string>}${randint:<n>}

isip {<string>}ge {<string1>}{<string2>

def:<variable name>

$domain

match {<string1>}{<string2>}

$home$host$interface_address

ACL Com

plexityA

CL statement

The email system should be able to eliminate spam.

Email system should work properly under pressure and heavy network traffic.

Email system should be simple in design, affordable yet resilient and easy to maintain.

The email system should be heterogeneous.

And finally the email system itself should be protected against viruses and misuse.

System objectives

The email system should handle large volume of traffic with minimum delay.

First and second Objectives

Sp m

Yes

No

SpamPositive

SMTP session

Pipe transport

Triggered by Unknown sender

Check the sender

Check the sender

Inbound SMTP

Blacklist

Mail Box Dispatcher

ACL

LiteSQL

Greylist

Whitelist

VirusPositive

?

?

?

Greylisted Mail DB

Known resenders DB

hybrid anti-spam filter

Next stagein mail delivery

Why Anti-virus and other content examiner filter were pushed to the final stage ?

Content-based filter (e.g., spamassassin) are memory hog, consume a lot of CPU bandwidth and time.

Reduce the rate of false positive generated by spamassassin –or any content examiner filter –dramatically.

ACL is lightweight network-level anti-spam countermeasure, with other network-level techniques; they are responsible of rejecting the majorityof the spam,without consuming a lot of system resources.

Due to Spamassassin’s high resources consumption it might leave the system vulnerable to DoS attack.

First and second Objectives

?

Third Objective

LiteSQL

Greylisted Mail DB

Known resenders DB

Greylist

Mail Box Dispatcher

Trigger

ACL

?

Greylisting is responsible for most of the delay.

Being less restricted by triggering Greylisting less often (Caution : Unknown senders will be in the whitelist)

Resender-database list the hosts that are known to retry sending (caution : tedious job )

Forth and Fifth Objectives

Unfortunately, it’s difficult to design a network that is simultaneously simple and heterogeneous (e.g., SSSD, Open LDAP, Kerberos)

It’s not easy to design an affordable network without reducing theeffectiveness of fighting the spam especially at the home front

Why the network has to be heterogeneous in the first place ?

Our design consisted of two important parts:

Internal network : MS Exchange is very popular ; widely used for intranets.

DMZ : Taking advantage of Exim Internet gateway capabilities, flexibility and ACL.

Linux is a free software Microsoft is more customer support orientedMS windows servers are easier to install and configureLinux servers is more customizable than MS servers

?

Sixth Objective

Linux is highly customizable; kernel could be modified to accomplish only certain tasks related to the mail services, hence better protection and smaller TCB (Trusted Computing Base ).

Updating Exim regularly with the latest releases and patches (e.g.,Exim 4.32 header_syntax function buffer overflow)

System Architecture Windows Domain controller

Act

ive

Dir

ecto

ry

Linux/Ubuntu

To M

S E

xch

ange

Ser

ver

DNS

sssd_pam module

sssd_nss module

SMTP

: ST

ART

TLS

Deliver Tickets

192.168.1.10

192.168.8.100

192.168.8.50

Ticket is neededAccess LDAP as a serviceLDAP/TLS Bind

SSSD

Sharing files and services

KDC LDAP

Users

OPEN

PAM

SASL

LDAP

NSS

Kerberos 5

RR AccessService

Router

Stand-AloneRoot-CA

Enterprise SubCA

Pass_to_Exchange

SMTP Transport

ACL

Local smtp

SID UID,GID

Keytab

ldap.conf

krb5.conf

sssd.conf

exim4.conf

Mail Box Dispatcher

PIPE Transport

Sp m

RouterPass_to_SPMFlt

Exim

SSSD will access LDAP as a service, hence requires Kerberos service ticket

Spamassassin check won’t take place in Exim itself but as a separate and independent process

192.168.1.20 192.168.1.10 192.168.8.50 192.168.8.100

Exchange Server 2013DNS Server

VM 1 VM 2 VM 3

Iptables

192.168.1.0/24 192.168.8.0/24

192.168.8.150

Internal Firewall External Firewall

DMZInternal Network

Windows Server 2012 FirewallRouting and Remote Access ServiceCerti�cation Authority

Exim Server OpenLDAP MIT Kerberos

SSSD

Resembling the network by virtual machines

Testing the network infrastructure

Windows Domain Controller validate user’s credentials

Telnet:SMTP instructions

Testing the network infrastructure...continued

Successful delivery to the recipient

Conclusion and discussion

Defense lines were placed to take action in sequence.

In a busy network, we would imagine giving high priority to tasks involving fighting the spam on network-level, and more time in updating the anti-virus on personal level.

DNS functions and purposes would have severely underestimated in fighting the spam, had we chosen to focus on one anti-spam techniques.

Most of the complexity comes from Linux/MS windows interoperability;one have to build the infrastructure only once.

highlights of our approach

Question ?

?

an effective spam protection system

Documents