Electronic Mail

Electronic Mail• Most heavily used application on any network• Uses client-server architecture

– Electronic mail client accepts mail from user and delivers to server on destination computer

– Many variations and styles of delivery

• Simple Mail Transfer Protocol (SMTP)– TCP/IP

– Delivery of simple text messages

• Multi-purpose Internet Mail Extension (MIME)– Delivery of other types of data

– Voice, images, video clips

SMTP Concept

User Agents and Mail Transfer Agents

Relay MTAs

Mail Gateways

The Entire E-mail System

SMTP• Simple Mail Transfer Protocol is standard

application protocol for delivery of mail from source to destination

• Built on top of TCP: provides reliable delivery of data

• SMTP does not normally use intermediate mail servers for sending mail

• SMTP uses persistent connections• Other functions:

– E-mail address lookup– E-mail address verification

• Example protocol exchange

Sample SMTP interaction

C: c:\Telnet hamburger.edu 25 S: 220 hamburger.edu C: HELO ce.sharif.edu S: 250 Hello ce.sharif.edu, pleased to meet you C: MAIL FROM: <ali@ce.sharif.edu > S: 250 ali@ce.sharif.edu... Sender ok C: RCPT TO: <bob@hamburger.edu> S: 250 bob@hamburger.edu ... Recipient ok C: DATA S: 354 Enter mail, end with "." on a line by itself C: Do you like ketchup? C: How about pickles? C: . S: 250 Message accepted for delivery C: QUIT S: 221 hamburger.edu closing connection

SMTP• RFC 821• Not concerned with format of messages or data

– Covered in RFC 822 (see later)

• SMTP uses info written on envelope of mail – Message header

• Does not look at contents – Message body

• Except:– Standardize message character set to 7 bit ASCII

Basic Operation• Mail created by user agent program (mail client)

– Message consists of:• Header containing recipient’s address and other info

• Body containing user data

• Messages queued and sent as input to SMTP sender program– Typically a server process (daemon on UNIX)

Mail Message Contents• Each queued message has:

– Message text• RFC 822 header with message envelope and list of recipients

• Message body, composed by user

– A list of mail destinations• Derived by user agent from header

• May be listed in header

• May require expansion of mailing lists

• May need replacement of mnemonic names with mailbox names

• If BCCs indicated, user agent needs to prepare correct message format

SMTP Sender• Takes message from queue• Transmits to proper destination host

– Via SMTP transaction

– Over one or more TCP connections to port 25

• Host may have multiple senders active• When delivery complete, sender deletes

destination from list for that message• When all destinations processed, message is

deleted

Optimization• If message destined for multiple users on a given

host, it is sent only once– Delivery to users handled at destination host

• If multiple messages ready for given host, a single TCP connection can be used– Saves overhead of setting up and dropping connection

Possible Errors• Host unreachable• Host out of operation• TCP connection fail during transfer• Sender can re-queue mail

– Give up after a period

• Faulty destination address– User error

– Target user changed address

– Redirect if possible

– Inform user if not

SMTP Protocol - Reliability • Used to transfer messages from sender to receiver

over TCP connection• Attempts to provide reliable service• No guarantee to recover lost messages• No end to end acknowledgement to originator• Error indication delivery not guaranteed• Generally considered reliable

SMTP Receiver• Accepts arriving message• Places in user mailbox or copies to outgoing

queue for forwarding• Receiver must:

– Verify local mail destinations

– Deal with errors• Transmission

• Lack of disk space

• Sender responsible for message until receiver confirm complete transfer– Indicates mail has arrived at host, not user

SMTP Forwarding• Mostly direct transfer from sender host to receiver

host• May go through intermediate machine via

forwarding capability– Sender can specify route

– Target user may have moved

Conversation• SMTP limited to conversation between sender and

receiver• Main function is to transfer messages• Rest of mail handling beyond scope of SMTP

– May differ between systems

Figure 3.9 SMTP Mail Flow

SMTP System Overview• Commands and responses between sender and

receiver• Initiative with sender

– Establishes TCP connection

• Sender sends commands to receiver• e.g. HELO<SP><domain><CRLF>• Each command generates exactly one reply• e.g. 250 requested mail action ok; completed

SMTP Replies• Leading digit indicates category

– Positive completion reply (2xx)

– Positive intermediate reply (3xx)

– Transient negative completion reply (4xx)

– Permanent negative completion reply (5xx)

Operation Phases• Connection setup• Exchange of command-response pairs• Connection termination

Connection Setup• Sender opens TCP connection with receiver• Once connected, receiver identifies itself

– 220 <domain> service ready

• Sender identifies itself– HELO

• Receiver accepts sender’s identification– 250 OK

• If mail service not available, step 2 above becomes:– 421 service not available

Mail Transfer• Sender may send one or more messages to

receiver• MAIL command identifies originator

– Gives reverse path to used for error reporting– Receiver returns 250 OK or appropriate fail/error

message

• One or more RCPT commands identifies recipients for the message– Separate reply for each recipient

• DATA command transfers message text– End of message indicated by line containing just period

(.)

Closing Connection• Two steps• Sender sends QUIT and waits for reply• Then initiate TCP close operation• Receiver initiates TCP close after sending reply to

QUIT

Electronic Mailboxes and Addresses• E-mail users have an electronic mailbox into

which mail is deposited• Usually associated with computer account; one

user may have different electronic mailboxes• User accesses mail with a mail reader program• Electronic mailbox is identified by an e-mail

addressmailbox@computer

Internet Mail Addressing• User portion or mailbox portion is site specific

– Joe– Seyyed_hamid_1221– 1234.5678

• Host portion or computer portion is domain name• Source mail client

– Resolves destination name using DNS – Contacts mail delivery server at destination– Copies mail to server

• Destination mail server– Interprets user name according to local mailbox addresses– Places mail in appropriate mailbox

E-mail message format• Simple two-part format

– Header includes delivery information

– Body carries text of message

• Header and body separated by blank line

E-mail Headers• Lines of text in format keyword: information• keyword identifies information; information can

appear in any order• Essential information:

– To: list of recipients– From: sender– Cc: list of copy recipients

• Useful information– Reply-to: different address than From:– Received-by: for debugging

• Frivolous information:– Favorite drink: lemonade– Phase-of-the-moon: gibbous

Data in E-mail• Original Internet mail carried only 7-bit ASCII

data• Couldn’t contain arbitrary binary values; e.g.,

executable program• Techniques for encoding binary data allowed

transport of binary data• Uuencode: three 8-bit binary values as four ASCII

characters (6 bits each)– Also carries file name and protection information

– Incurs 33% overhead

– Requires manual intervention

Multipurpose Internet Mail Extension (MIME)

• Extension to RFC822• SMTP can not transmit executables

– Uuencode and other schemes are available• Not standardized

• Can not transmit text including international characters (e.g. â, å, ä, è, é, ê, ë)– Need 8 bit ASCII

• Servers may reject mail over certain size• Translation between ASCII and EBCDIC not standard• SMTP gateways to X.400 can not handle none text data in

X.400 messages• Some SMTP implementations do not adhere to standard

– CRLF, truncate or wrap long lines, removal of white space, etc.

MIME• Multipart Internet Mail Extensions - Extends and

automates encoding mechanisms• Allows inclusion of separate components – programs,

pictures, audio clips – in a single mail message• Sending program identifies the components so

receiving program can automatically extract and inform mail recipient– Header includes

MIME-Version: 1.0Content-Type: Multipart/Mixed; Boundary=Mime_separator

– Separator line gives information about specific encodingContent-Type: text/plain

MIME (cont.)• MIME is extensible – sender and receiver agree on

encoding scheme• MIME is compatible with existing mail systems

– Everything encoded as ASCII

– Headers and separators ignored by non-MIME mail systems

• MIME encapsulates binary data in ASCII mail envelope

MIME

Overview of MIME• Five new message header fields

– MIME version

– Content type

– Content transfer encoding

– Content Id

– Content Description

• Number of content formats defines• Transfer encoding defined

Content Types• Text body• Multipart

– Mixed, Parallel, Alternative, Digest

• Message– RFC 822, Partial, External-body

• Image– jpeg, gif

• Video– mpeg

• Audio– Basic

• Application– Postscript– octet stream

MIME Transfer Encodings• Reliable delivery across wide largest range of

environments

• Content transfer encoding field– Six values

– Three (7bit, 8bit, binary) no encoding done• Provide info about nature of data

• Quoted-printable– Data largely printable ASCII characters

– Non-printing characters represented by hex code

• Base64– Maps arbitrary binary input onto printable output

• X-token– Named nonstandard encoding

Figure 3.10 Printable Encoding of Binary Data into Radix-64 Format

Programs as mail recipients• Can arrange for e-mailbox to be associated with a

program rather than a user’s mail reader• Incoming mail automatically processed as input to

program• Example – mailing list subscription administration• Can be used to implement client-server processing

– Client request in incoming mail message

– Server response in returned mail reply

Mailing Lists and Forwarders• E-mail addresses can be attached to programs as

well as electronic mailboxes• Mail exploder or mail forwarder resends copies of

message to e-mail addresses in mailing list– UNIX mail program sendmail provides mail aliases

– Mailing list processor, e.g. listserv, can also interpret subscription management commands

Mail Gateways• Mailing list processing may take significant

resources in large organizations• May be segregated to a dedicated server computer:

mail gateway– Provides single mail destination point for all incoming

mail

– Can use MX records in DNS to cause all mail to be delivered to gateway

Mail gateways and forwarding• Users within an organization may want to read

mail on local or departmental computer• Can arrange to have mail forwarded from mail

gateway• Message now makes multiple hops for delivery• Hops may be recorded in header• Forwarded mail may use proprietary (non-SMTP)

mail system

Mail Gateways and E-mail Addresses• Organization may want to use uniform naming for

external mail• Internally, may be delivered to many different

systems with different naming conventions• Mail gateways can translate e-mail addresses

Mailbox Access• Where should mailbox be located?• Users want to access mail from most commonly

used computer• Cannot always use desktop computer as mail

server– Not always running

– Requires multitasking OS

– Requires local disk storage

• Can TELNET to remote computer with mail server

Mail Access Protocols• Typically users do NOT run a server on their local

PC• User agent accesses a mailbox from a shared mail

server• Mail access protocol transfers mail from server to

local PC• 2 popular protocols:

– POP3 (Post Office Protocol v3)

– IMAP (Internet Mail Access Protocol)

• Can’t use SMTP because getting messages is a pull operation & SMTP is a push protocol.

Mail Access Protocol• Instead of TELNEt, use protocol that accesses

mail on remote computer directly• TCP/IP suite includes Post Office Protocol (POP)

for remote mailbox access– Computer with mailboxes runs POP server

– User runs POP client on local computer

– POP client can access and retrieve messages from mailbox

– Requires authentication (password)

– Local computer uses SMTP for outgoing mail

Email Protocols

POP3 and SMTP

POP3

• Three phases– Authorization

– Transaction

– Update

• During a POP3 session, the server maintains state information

• POP3 downloads mail messages to the local machine – not good for “nomadic user”

Dialup Access and POP• POP useful for dialup connection

– User’s computer not always connected

– Can download all mail at once and read off-line

– Can compose mail off-line and mail in one connection

POP3 protocol

authorization phase• client commands:

– user: declare username– pass: password

• server responses– +OK– -ERR

transaction phase, client:• list: list message numbers• retr: retrieve message by

number• dele: delete• quit

C: list S: 1 498 S: 2 912 S: . C: retr 1 S: <message 1 contents> S: . C: dele 1 C: retr 2 S: <message 1 contents> S: . C: dele 2 C: quit S: +OK POP3 server signing off

S: +OK POP3 server ready C: user bob S: +OK C: pass hungry S: +OK user successfully logged on

POP3• Stands for Post Office Protocol – Version 3• RFC in 1993• POP3 “is intended to permit a workstation to

dynamically access a maildrop on a server host in a useful fashion. Usually, this means that the POP3 protocol is used to allow a workstation to retrieve mail that the server is holding for it.“

POP3• Doesn’t provide extensive mail operations on the

server• Normally mail is downloaded, then deleted• Deletion doesn’t really occur until QUIT• Uses port 110

Minimal POP3 Commands• USER name – user’s name

• PASS string – user’s password

• QUIT – quits, and updates the mailbox

• STAT – status

• LIST [msg] – lists the mail messages (#’s only)

• RETR msg – retrieves a mail message

• DELE msg – deletes a message

• NOOP – No operation

• RSET – resets the mail to its original state

Example POP3 Transaction

Server

Establish the Socket

Server Client

Socket (110)PrimaryThread

PrimaryThread

Server is OK

Server Client

Socket (110)PrimaryThread

PrimaryThread

+OK POP3 server ready

Establish the Socket

Server Client

Socket (110)PrimaryThread

PrimaryThread

USER bob

This user is OK

Server Client

Socket (110)PrimaryThread

PrimaryThread

+OK

Send the Password

Server Client

Socket (110)PrimaryThread

PrimaryThread

PASS ITSD4304

The Password is OK

Server Client

Socket (110)PrimaryThread

PrimaryThread

+OK

Ask how the Server is Feeling…

Server Client

Socket (110)PrimaryThread

PrimaryThread

STAT

I’m Fine, Thanks!

Server Client

Socket (110)PrimaryThread

PrimaryThread

+OK 2 320

Got Any Mail?

Server Client

Socket (110)PrimaryThread

PrimaryThread

LIST

Yup…

Server Client

Socket (110)PrimaryThread

PrimaryThread

+OK 2 messages (320 octets)

Message One

Server Client

Socket (110)PrimaryThread

PrimaryThread

1 120

Message Two

Server Client

Socket (110)PrimaryThread

PrimaryThread

2 200

End

Server Client

Socket (110)PrimaryThread

PrimaryThread

.

Get Message One

Server Client

Socket (110)PrimaryThread

PrimaryThread

RETR 1

Message One Coming!

Server Client

Socket (110)PrimaryThread

PrimaryThread

+OK 120 octets

Message One Coming!

Server Client

Socket (110)PrimaryThread

PrimaryThread

<the POP3 server sends message 1>

End of Message 1

Server Client

Socket (110)PrimaryThread

PrimaryThread

.

Get Message One

Server Client

Socket (110)PrimaryThread

PrimaryThread

DELE 1

Message 1 Gone!

Server Client

Socket (110)PrimaryThread

PrimaryThread

+OK message 1 deleted

Note: it’s not really gone…

Get Message One

Server Client

Socket (110)PrimaryThread

PrimaryThread

QUIT

QUIT – Commits/Updates

Server Client

Socket (110)PrimaryThread

PrimaryThread

+OK bob POP3 server signing off

Note: now it’s gone

Socket Disposed

Server Client

PrimaryThread

PrimaryThread

IMAP• Internet Mail Access Protocol• Allows users to manipulate remote mailboxes• Must maintain info about user (e.g., folder

hierarchy) even after the user quits• IMAP server is always in 1 of 4 states:

– Nonauthenticated– Authenticated– Selected– Logout

POP3 (more) and IMAP

More about POP3

• Previous example uses “download and delete” mode.

• Bob cannot re-read e-mail if he changes client

• “Download-and-keep”: copies of messages on different clients

• POP3 is stateless across sessions

IMAP

• Keep all messages in one place: the server

• Allows user to organize messages in folders

• IMAP keeps user state across sessions:– names of folders and

mappings between message IDs and folder name

Security• SMTP, POP3 and IMAP are plain text protocols• Secured version available to protect the user

– SMTPS

– POP3s

– Need to be configured both on server and client

Stunnel

• Name of an application/service– Even if the mail server application does not implement

secure version

• What does it do?– gathers the encrypted message from client

– Decrypts the data

– Sends plain text data using a local connection to insecure port

• Server application considers it a normal client

STunneling

Server Client

Stunnel(on server)Listening to port 995

Send decrypted data to port 25 of the server

Encrypted data(Command/mail body)

Web interface

• Nowadays there are many web interface user agents

• They allow accessing messages from any client with only a web browser

• However the interface is not as sophisticated as special mail agents

• But it is very useful if you need to check your emails from different computers

Summary for Email

• Electronic mail allows quick, asynchronous communication across entire Internet

• Can attach e-mail addresses to programs for processing– Mailing lists– Other client-server applications

• Simple Mail Transfer Protocol (SMTP) is Internet standard for mail delivery

• Mail gateways– Provide uniform user addressing outside organizations– Translate from Internet mail (e.g. SMTP) to proprietary systems

• Post Office Protocol (POP) and Internet Mail Access Protocol (IMAP) allow remote access to electronic mailboxes.