1

Lecture #3Electronic Mail Protocols

HAIT

Summer 2005

Shimrit Tzur-David

2

Message Transfer

• The message transfer system is concerned with relaying messages from the originator to the recipient.

• The simplest way to do this is to establish a transport connection from the source machine to the destination machine and then just transfer the message.

• After examining how this is normally done, we will examine some situations in which this does not work and what can be done about them.

3

Mail Servers

Mail Servers • mailbox contains incoming

messages for user• message queue of outgoing (to

be sent) mail messages• SMTP protocol between mail

servers to send email messages– client: sending mail server– “server”: receiving mail

server

mailserver

useragent

useragent

useragent

mailserver

useragent

useragent

mailserver

useragent

SMTP

SMTP

SMTP

4

SMTP—The Simple Mail Transfer Protocol

• Uses TCP to reliably transfer email message from client to server, port 25

• Direct transfer: sending server to receiving server• Three phases of transfer

– handshaking (greeting)– transfer of messages– closure

• Command/response interaction– commands: ASCII text– response: status code and phrase

5

Sequence of Events

• After establishing the TCP connection to port 25, the sending machine, operating as the client, waits for the receiving machine, operating as the server, to talk first.

• The server sends a line of text giving its identity and telling whether it is prepared to receive mail. If it is not, the client releases the connection and tries again later.

• If the server is willing to accept e-mail, the client announces whom the e-mail is coming from and whom it is going to. If such a recipient exists at the destination, the server gives the client the go-ahead to send the message.

• The client sends the message and the server acknowledges it. • When all the e-mail has been exchanged in both directions, the

connection is released.

6

Scenario: Alice sends message to Bob

4) SMTP client sends Alice’s message over the TCP connection

5) Bob’s mail server places the message in Bob’s mailbox

6) Bob invokes his user agent to read message

1) Alice uses UA to compose message “to” bob@someschool.edu

2) Alice’s UA sends message to her mail server; message placed in message queue

3) Client side of SMTP opens TCP connection with Bob’s mail server

useragent

mailserver

mailserver user

agent

1

2 3 4 56

7

Sample SMTP interactionS: 220 hamburger.edu SMTP service ready

C: HELO crepes.fr S: 250 Hello crepes.fr, pleased to meet you C: MAIL FROM: <alice@crepes.fr> S: 250 alice@crepes.fr... Sender ok C: RCPT TO: <bob@hamburger.edu> S: 250 bob@hamburger.edu ... Recipient ok C: DATA S: 354 Enter mail, end with "." on a line by itself C: Do you like ketchup? C: How about pickles? C: . S: 250 Message accepted for delivery C: QUIT S: 221 hamburger.edu closing connection

8

Try SMTP Interaction for Yourself

• telnet servername 25• see 220 reply from server

• enter HELO, MAIL FROM, RCPT TO, DATA, QUIT commands

• The commands above let you send email without using email client (reader)

9

Mail Message Format

SMTP: protocol for exchanging email msgs

Format:• header lines, e.g.,

– To:– From:– Subject:different from SMTP

commands!

• body– the “message”, ASCII

characters only

header

body

blankline

10

Message Format: Multimedia Extensions

• MIME: multimedia mail extension.• Additional lines in msg header declare MIME

content type

From: alice@crepes.fr To: bob@hamburger.edu Subject: Picture of yummy crepe. MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Type: image/jpeg

base64 encoded data ..... ......................... ......base64 encoded data

multimedia datatype, subtype,

parameter declaration

method usedto encode data

MIME version

encoded data

11

Weak Spots of SMTP

• Even though the SMTP protocol is completely well defined, a few problems can still arise. – Message length - some older implementations

cannot handle messages exceeding 64 KB. – Timeouts - if the client and server have different

timeouts, one of them may give up while the other is still busy, unexpectedly terminating the connection.

12

ESMTP

• To get around some of these problems, extended SMTP (ESMTP) has been defined.

• Clients wanting to use it should send an EHLO message instead of HELO initially.

• If this is rejected, then the server is a regular SMTP server, and the client should proceed in the usual way.

• If the EHLO is accepted, then new commands and parameters are allowed.

13

Mail access protocols

• SMTP: delivery/storage to receiver’s server• Mail access protocol: retrieval from server

– POP: Post Office Protocol, port TCP/110• authorization (agent <-->server) and download

– IMAP: Internet Mail Access Protocol, port 150• more features (more complex)• manipulation of stored messages on server

– HTTP: Hotmail , Yahoo! Mail, etc.

useragent

sender’s mail server

useragent

SMTP SMTP accessprotocol

receiver’s mail server

14

POP3 Situation where both sender and receiver having a permanent connection to the Internet

Situation where the sender is on-line but the receiver is not

15

POP3 Protocol

• POP3 begins when the user starts the mail reader. • The mail reader calls up the ISP and establishes a

TCP connection with the message transfer agent at port 110.

• Once the connection has been established, the POP3 protocol goes through three states in sequence:– Authorization - deals with having the user log in.

– Transactions - deals with the user collecting the e-mails and, for example, marking them for deletion.

– Update - causes the e-mails to be deleted.

16

POP3 Protocol – Cont.

authorization phase• client commands:

– user: declare username– pass: password

• server responses– +OK– -ERR

transaction phase, client:• list: list message numbers• retr: retrieve message by

number• dele: delete• quit

C: list S: 1 498 S: 2 912 S: . C: retr 1 S: <message 1 contents> S: . C: dele 1 C: retr 2 S: <message 1 contents> S: . C: dele 2 C: quit S: +OK POP3 server signing off

S: +OK POP3 server ready C: user bob S: +OK C: pass hungry S: +OK user successfully logged on

17

Gentle Point

• POP3 protocol supports the ability to download a specific message or set of messages and leave them on the server.

• Most e-mail programs just download everything and empty the mailbox.

• This behavior means that in practice, the only copy is on the user's hard disk. If that crashes, all e-mail may be lost permanently.

18

Sequence of Events

1. Alice creates a message for Bob using some e-mail program (i.e., user agent) and clicks on an icon to send it.

2. The e-mail program hands the message over to the message transfer agent on Alice's host.

3. The message transfer agent sees that it is directed to bob@xyz.com so it uses DNS to look up the MX record for xyz.com. This query returns the DNS name of xyz.com's mail server.

4. The message transfer agent now looks up for the IP address of this machine using DNS again.

5. It then establishes a TCP connection to the SMTP server on port 25 of this machine. Using an SMTP command sequence analogous, it transfers the message to Bob's mailbox and breaks the TCP connection.

19

Sequence of Events – Cont.

6. Bob boots up his PC, connects to his ISP, and starts his e-mail program.

7. The e-mail program establishes a TCP connection to the POP3 server at port 110 of the ISP's mail server machine.

8. Bob's e-mail program runs the POP3 protocol to fetch the contents of the mailbox to his hard disk.

9. Once all the e-mail has been transferred, the TCP connection is released. Why?

20

IMAP – Internet Message Access Protocol

• What happens if a user opens his mailbox in more than one place?

• Many people have a single e-mail account at work or school and want to access it from work, from their home PC, from their laptop when on business trips, etc.

• While POP3 allows this, since it normally downloads all stored messages at each contact, the result is that the user's e-mail quickly gets spread over multiple machines, more or less at random, some of them not even the user's.

21

IMAP Policy

• IMAP assumes that all the e-mail will remain on the server indefinitely in multiple mailboxes.

• IMAP provides extensive mechanisms for reading messages or even parts of messages, a feature useful when using a slow modem to read the text part of a multipart message with large audio and video attachments.

• Since the working assumption is that messages will not be transferred to the user's computer for permanent storage, IMAP provides mechanisms for creating, destroying, and manipulating multiple mailboxes on the server.

• In this way a user can maintain a mailbox for each correspondent and move messages there from the inbox after they have been read.

22

A comparison of POP3 and IMAP

Feature POP3 IMAP

Where is emails stored?

Where is emails read?

Connect time required

Use of server resources

Multiple mailboxes

User control over downloading

Partial message downloads

Are disk quotas a problem?

Simple to implement

Keeps user state

User’s PC Server

Off-line On-line

Little Much

Minimal Extensive

No Yes

Little Great

No Yes

No Could be

Yes No

YesNo

23

More about POP3

• Previous example uses “download and delete” mode.

• Bob cannot re-read e-mail if he changes client.

• “Download-and-keep”: copies of messages on different clients

24

Delivery Features

• Many systems provide additional features for processing of incoming e-mail.

• For example: the ability to set up filters. – These are rules that are checked when e-mail comes in or

when the user agent is started.

• Each rule specifies a condition and an action. – Rule examples - any message received from the boss goes

to mailbox number 1, any message from a select group of friends goes to mailbox number 2, and any message containing certain objectionable words in the Subject line is discarded without comment.

25

Delivery Features – Cont.

• Filter - automatically categorizes incoming e-mail as either important or spam (junk e-mail) and stores each message in the corresponding mailbox.

• Such filters typically work by first checking to see if the source is a known spammer.

• They usually examine the subject line. If hundreds of users have just received a message with the same subject line, it is probably spam. Other techniques are also used for spam detection.

• Forward incoming e-mail to a different address.• Installation of a vacation daemon.

26

Web-Mail• Some Web sites, for example, Hotmail and Yahoo, provide e-

mail service. • They have normal message transfer agents listening to port 25

for incoming SMTP connections. • To contact, say, Hotmail, you need to establish a TCP

connection over which SMTP commands can be sent in the usual way.

• When the user goes to the e-mail Web page, a form is presented in which the user is asked for a login name and password.

• When the user clicks on Sign In, the login name and password are sent to the server, which then validates them.

• The server finds the user's mailbox and builds a listing page, only formatted as a Web page in HTML. The Web page is then sent to the browser for display.