Email Protocols and Email Protocols and TroubleshootingTroubleshooting

Brandon CheckettsBrandon Checketts

Components of an Email Components of an Email SystemSystem

Mail Transport Agent (MTA)Mail Transport Agent (MTA) Mail User Agent (MUA)Mail User Agent (MUA) Local Delivery Agent (LDA)Local Delivery Agent (LDA)

..others??..others??

Common Email ProtocolsCommon Email Protocols

Sending Mail:Sending Mail: SMTP (Simple Mail Transport Protocol)SMTP (Simple Mail Transport Protocol)

Servers include Sendmail, Postfix, Exim, QmailServers include Sendmail, Postfix, Exim, Qmail

Receiving MailReceiving Mail IMAP (Internet Message Access Protocol)IMAP (Internet Message Access Protocol) POP3 (Post Office Protocol v3)POP3 (Post Office Protocol v3)

Servers Include Dovecot, Courier, QmailServers Include Dovecot, Courier, Qmail

SMTP DesignSMTP Design

Delivers a message from one machine to Delivers a message from one machine to anotheranother

Became popular in the 1980’s (as Became popular in the 1980’s (as complement to UUCP)complement to UUCP)

Used for outgoing messages from a sender Used for outgoing messages from a sender to their outgoing mail serverto their outgoing mail server

Communication between mail servers on Communication between mail servers on the Internetthe Internet

Typically listens on TCP Port 25 Typically listens on TCP Port 25 and also on 587 and 465and also on 587 and 465

SMTP ServersSMTP Servers

SendmailSendmail Widely available, complicated to configure (M4 macros, Widely available, complicated to configure (M4 macros,

etc)etc) The term ‘sendmail’ is used in multiple contextsThe term ‘sendmail’ is used in multiple contexts

PostfixPostfix Widely available, semi-straightforwardWidely available, semi-straightforward

EximExim Generaly available, semi-complicated to configureGeneraly available, semi-complicated to configure

QmailQmail Generally available, completely different than most Generally available, completely different than most

other *nix serversother *nix servers

(These are my opinions - your mileage may vary)(These are my opinions - your mileage may vary)

SMTP UsesSMTP Uses

If ever configuring a mail server, try to separate these two types of If ever configuring a mail server, try to separate these two types of servicesservices

Outgoing Mail ServerOutgoing Mail Server Should have some kind of authenticationShould have some kind of authentication Queue messages when receiving server is unavailableQueue messages when receiving server is unavailable Sends bounce message to sender after retrying deliverySends bounce message to sender after retrying delivery

Incoming Mail Server (or MX server)Incoming Mail Server (or MX server) Receives incoming messages from the InternetReceives incoming messages from the Internet Delivers message to a mailboxDelivers message to a mailbox (Should never send a bounce)(Should never send a bounce)

Basic SMTP CommandsBasic SMTP Commands

HELO (and EHLO)HELO (and EHLO) MAIL FROMMAIL FROM RCPT TORCPT TO DATADATA QUITQUIT Response CodesResponse Codes

2xx (Success)2xx (Success) 4xx (Temporary Failure)4xx (Temporary Failure) 5xx (Permanent Failure5xx (Permanent Failure

Simple SMTP SessionSimple SMTP Session [root@xpgrinders ~]# [root@xpgrinders ~]# telnet filter1.roundsphere.com 25telnet filter1.roundsphere.com 25 Trying 64.18.6.14...Trying 64.18.6.14... Connected to filter1.roundsphere.com.Connected to filter1.roundsphere.com. Escape character is '^]'.Escape character is '^]'. 220 Postini ESMTP <snip…>rtisements.220 Postini ESMTP <snip…>rtisements. ehlo example.comehlo example.com 250-Postini says hello back250-Postini says hello back 250-STARTTLS250-STARTTLS 250-8BITMIME250-8BITMIME 250 HELP250 HELP mail from:<sender@example.com>mail from:<sender@example.com> 250 Ok250 Ok rcpt to:<recipient@destination.com>rcpt to:<recipient@destination.com> 250 Ok250 Ok datadata 354 Feed me354 Feed me Subject: This is a testSubject: This is a test From: "Brandon Testing" <user@example.com>From: "Brandon Testing" <user@example.com> To: "Brandon Testing Again" <recipient@destination.com>To: "Brandon Testing Again" <recipient@destination.com>

this is my messagethis is my message .. 250 Thanks250 Thanks quitquit 221 Catch you later221 Catch you later

Simple SMTP Session (Big Simple SMTP Session (Big #1)#1) [root@xpgrinders ~]# [root@xpgrinders ~]# telnet telnet

filter1.roundsphere.com 25filter1.roundsphere.com 25 Trying 64.18.6.14...Trying 64.18.6.14... Connected to filter1.roundsphere.com.Connected to filter1.roundsphere.com. Escape character is '^]'.Escape character is '^]'. 220 Postini ESMTP <snip…>rtisements.220 Postini ESMTP <snip…>rtisements. ehlo example.comehlo example.com 250-Postini says hello back250-Postini says hello back 250-STARTTLS250-STARTTLS 250-8BITMIME250-8BITMIME 250 HELP250 HELP mail from:<sender@example.com>mail from:<sender@example.com> 250 Ok250 Ok

Simple SMTP Session (Big Simple SMTP Session (Big #2)#2) rcpt to:<recipient@destination.com>rcpt to:<recipient@destination.com>

250 Ok250 Ok datadata 354 Feed me354 Feed me Subject: This is a testSubject: This is a test From: "Brandon Testing" <sender@example.com>From: "Brandon Testing" <sender@example.com> To: "Brandon Testing" <recipient@destination.com>To: "Brandon Testing" <recipient@destination.com>

This is my messageThis is my message .. 250 Thanks250 Thanks quitquit 221 Catch you later221 Catch you later

Weaknesses and ExtensionsWeaknesses and Extensions

No Sender AuthenticationNo Sender Authentication SMTP AuthSMTP Auth

UnencryptedUnencrypted SSL and TLSSSL and TLS

Text-OnlyText-Only MIMEMIME

Bounces Bounces SPAM SPAM

SPF and DKIMSPF and DKIM

POP3 ProtocolPOP3 Protocol

Retrieves messages from a mail serverRetrieves messages from a mail server Typically, messages are downloaded to Typically, messages are downloaded to

your mail client, and deleted from the your mail client, and deleted from the serverserver

Designed for use with dial-up connections Designed for use with dial-up connections when people were intermittently when people were intermittently connectedconnected

Listens on Port 110 (with Secure POP Listens on Port 110 (with Secure POP generally on port 995)generally on port 995)

Sample POP3 SessionSample POP3 Session [root@xpgrinders ~]# [root@xpgrinders ~]# telnet mail.roundsphere.com 110telnet mail.roundsphere.com 110 Trying 206.71.88.102...Trying 206.71.88.102... Connected to mail.roundsphere.com.Connected to mail.roundsphere.com. Escape character is '^]'.Escape character is '^]'. +OK Dovecot ready.+OK Dovecot ready. user user@testing.comuser user@testing.com +OK+OK pass letmeinpass letmein +OK Logged in.+OK Logged in. listlist +OK 1 messages:+OK 1 messages: 1 4821 482 .. retr 1retr 1 +OK 482 octets+OK 482 octets <snip>Message Headers</snip><snip>Message Headers</snip>

This is my short messageThis is my short message

.. quitquit +OK Logging out.+OK Logging out.

IMAP ProtocolIMAP Protocol

Listens on port 143 (IMAP/SSL on port Listens on port 143 (IMAP/SSL on port 993)993)

Mail stays on the server. Mail Client Mail stays on the server. Mail Client caches information locallycaches information locally

Extremely useful for multiple users, Extremely useful for multiple users, multiple machines, Webmail, etcmultiple machines, Webmail, etc

Searches are done on the serverSearches are done on the server

Sample IMAP SessionSample IMAP Session [root@xpgrinders ~]# [root@xpgrinders ~]# telnet mail.roundsphere.com 143telnet mail.roundsphere.com 143 Trying 206.71.88.102...Trying 206.71.88.102... Connected to mail.roundsphere.com.Connected to mail.roundsphere.com. Escape character is '^]'.Escape character is '^]'. * OK Dovecot ready.* OK Dovecot ready. A1 LOGIN user@testing.com letmeinA1 LOGIN user@testing.com letmein A1 OK Logged in.A1 OK Logged in. A2 SELECT InboxA2 SELECT Inbox * FLAGS (\Answered \Flagged \Deleted \Seen \Draft)* FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags

permitted.permitted. * 1 EXISTS* 1 EXISTS * 0 RECENT* 0 RECENT * OK [UIDVALIDITY 1225333589] UIDs valid* OK [UIDVALIDITY 1225333589] UIDs valid * OK [UIDNEXT 2] Predicted next UID* OK [UIDNEXT 2] Predicted next UID A2 OK [READ-WRITE] Select completed.A2 OK [READ-WRITE] Select completed. A3 FETCH 1 BODY[HEADER]A3 FETCH 1 BODY[HEADER] * 1 FETCH (BODY[HEADER] {454}* 1 FETCH (BODY[HEADER] {454} <snip> Message Header Delivered</snip><snip> Message Header Delivered</snip> A3 OK Fetch completed.A3 OK Fetch completed. A4 LOGOUTA4 LOGOUT * BYE Logging out* BYE Logging out A4 OK Logout completed.A4 OK Logout completed. Connection closed by foreign host.Connection closed by foreign host.

SPAM & PhishingSPAM & Phishing

SMTP has no built-in way to verify the SMTP has no built-in way to verify the legitimacy of the messagelegitimacy of the message

Anybody can say they are anybody elseAnybody can say they are anybody else SMTP is far too prolific to try to replace itSMTP is far too prolific to try to replace it

(demonstrate sending an email as (demonstrate sending an email as PayPal)PayPal)

Fighting SpamFighting Spam

GreylistingGreylisting Content FilteringContent Filtering

This can get to be incredibly CPU This can get to be incredibly CPU intensiveintensive

DNS-based BlacklistsDNS-based Blacklists

Consider Appliances and Outsourced Consider Appliances and Outsourced ServicesServices

SPF and SenderIDSPF and SenderID

Concept is to validate the path the message Concept is to validate the path the message tooktook

SPF Record published in DNS gives a list of SPF Record published in DNS gives a list of the servers authorized to send email for a the servers authorized to send email for a given domaingiven domain

Fairly Simple to createFairly Simple to create SPF Record Looks Like:SPF Record Looks Like:

v=spf1 a a:mail.domain.com ~allv=spf1 a a:mail.domain.com ~all

SPF Wizard at http://www.openspf.org/SPF Wizard at http://www.openspf.org/

DomainKeys / DKIMDomainKeys / DKIM

Cryptographic Hash to sign messagesCryptographic Hash to sign messages Public Key and policy information is Public Key and policy information is

distributed via DNSdistributed via DNS Private key is used to sign the message, and Private key is used to sign the message, and

certain headers (From, To, Subject, etc)certain headers (From, To, Subject, etc) Recipients use public key to verify Recipients use public key to verify

authenticity of the messageauthenticity of the message Verifies a legitimate sender, and is not Verifies a legitimate sender, and is not

concerned about the path it took to get there.concerned about the path it took to get there. Fairly complicated to set-upFairly complicated to set-up

Sample DKIM HeaderSample DKIM Header

DKIM-Signature: v=1; a=rsa-sha1; DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=brandonchecketts.com; c=relaxed; d=brandonchecketts.com;

h= subject:to:message-id:date:from; h= subject:to:message-id:date:from; s=mail; s=mail; bh=t4KP+oqmtYCr/TS32vUW oYU+80M=; bh=t4KP+oqmtYCr/TS32vUW oYU+80M=;

b=E2UmQVoQVm+XMopufIQ6bjnfN9as7R6b=E2UmQVoQVm+XMopufIQ6bjnfN9as7R6R7x8ipJpLn/Xm+SM/fvt4 R7x8ipJpLn/Xm+SM/fvt4 lV81G2Bt1hisa3V2SP+emw2ecpImC27o+olV81G2Bt1hisa3V2SP+emw2ecpImC27o+olMA1XshTARGdUepTFWermUZ0WAaIt4 lMA1XshTARGdUepTFWermUZ0WAaIt4 rWwqv+hpVd/r3RNkRmS+kNZv5uZYQ5PerWwqv+hpVd/r3RNkRmS+kNZv5uZYQ5PeulOHMBHvH4Q5R9XDWIe6MiU= ulOHMBHvH4Q5R9XDWIe6MiU=

For More InformationFor More Information

Wikipedia!Wikipedia!

If Time PermitsIf Time Permits

Demonstrate Encrypted SMTP/POP Demonstrate Encrypted SMTP/POP SessionsSessions

Demonstrate Postfix / Dovecot Config Demonstrate Postfix / Dovecot Config FilesFiles

Discuss Webmail?Discuss Webmail? Making Sense out of message Making Sense out of message

headersheaders