peter eicher, product manager [email protected]
Post on 18-Dec-2015
219 views
TRANSCRIPT
WWW.SYBARI.COM
Agenda
• Anti-Spam Challenges• Typical Anti-spam solutions
– Content filter, heuristics, Bayesian
• RPD™ (Recurrent Pattern Detection)Patent Pending Technology
• Implementation and Management• ASD Evaluation Mode
WWW.SYBARI.COM
Two Unique Anti-Spam Issues• The growing number of spam attacks
– Over 500,000 unique spam attacks detected in our service center each DAY
– Compare to virus technology (1000 per month)– Need for a real-time solution with minimal IT
involvement
• For the first time with a security product, the user must be involved in the decision-making– Spam is not black and white– Need flexibility to fine-tune solution to client’s
needs
WWW.SYBARI.COM
The Problem
• “Spam is a rapidly growing problem for all email users. The traffic is doubling every 4 months, as are the associated costs”
• Today : 40-60% of all e-mail is Spam
• Unique spam attacks have increased 200% in 2002 (Osterman Research)
• A study shows that the annual cost of spam is : $8.9-billion for U.S. corporations (Forrester research)
• Typical user receives 14,500 spam emails each year
WWW.SYBARI.COM
53.9
39.7
28.4
19.413.5
Market Trend: The Volume of SpamTotal spam messages/day (Billions)
Graph source: The Radicati Group, Inc. 2003
Spammers will continue to improve infiltration tactics…
…and demand will grow for a real-time
adaptive solution
2003:
Damages exceed $500 per-employee, annually
2004:
Enterprises cannot afford staying unprotected
Early 2002:
Annoyance-only level
2003
2004
2005
2006
2007
WWW.SYBARI.COM
• Most anti-spam solutions rely on a combination of content filtering, heuristic scanning and/or Bayesian filtering
• These techniques have numerous flaws• Spam detection rarely higher than 70%
without extensive administrator attention
• False positives extremely high
Typical Anti-spam Solutions
WWW.SYBARI.COM
• Useful as a content management tool– Prevent certain words/topics from being
sent to or from your employees• However, both inefficient and
unsuccessful for spam management– Requires continuous administrator
attention (multiple hours per day)– Simple spelling tricks defeat content
filtering• Examples: $ave, V*i*a*gr*a, Chëὰρ
– There are 105 variations available just for the letter A!
– Results in numerous false positives• Impossible to use in certain industries
Content Filtering
WWW.SYBARI.COM
• Think your administrators can keep up? Here’s a few ways to spell Viagra…
Content Filtering
V I @ G R A , [email protected], \./iagra, Viiagra, V?agr?, V--i--a--g--r-a, V!agra, V1agra, VI.A.G.R.A, vi@gra, vIagr.a, via-gra, Via.gra, Vriagra, Viag*ra, vi-agra, Vi-ag.ra, v-iagra, Viagr-a, V^I^A^G^G^A, V'i'a'g'r'a', V*I*A,G,R.A, VI.A.G.R.A..., Viag\ra!, Vj@GRA, V-i:ag:ra, V'i'a'g'r'a, V/i;a:g:r:a, V i a g r @, V+i\a\g\r\a, Viag[ra, V?agra, V;I;A*G-R-A, V-i-a-g-r-a, V*I*A*G*R*A , V-i-@-g-r-a, VI@AGRA, Vi@gr@, \/^i^ag-ra, VlAGRA, V\i\a.g.r.a, V1@GRA, v_r_i_a_g_r_a, V\i\a:g:r:a, V^i^a^g^r^a, V-i-@-g-r-@, Viag(ra.
WWW.SYBARI.COM
Heuristic Scanning
• A “scoring” technique that looks at thousands of “characteristics” to determine spam and creates a score– Level of “spaminess” must be constantly
adjusted• Used in many spam products• Well understood by spammers
– Spammer websites allow “testing” of spam vs. heuristic scanners
• Extremely performance intensive– Every detection is a new event that doesn’t
benefit from previous detections• Very high false positive rate
– A “best guess” solution
WWW.SYBARI.COM
Bayesian Filtering
• A learning system that uses statistical analysis of vocabulary• Lists of “good” and “bad” words
• Requires active user participation to be effective
• Can be very effective for individual user• Far less effective in an enterprise setting
– One user’s choice can negate another’s• Deliberately attacked by spammers
– “Invisible” random text lowers spam score by increasing count of “good” words
• High rates of false positives
WWW.SYBARI.COM
Five anti-spam challenges
1. Catching spam and spammer evolution• Need a high detection rate today• Solution must overcome tomorrow’s spammers
2. What defines “spam” for the end-user?• Unsolicited emails – considered spam by almost
everyone• Solicited commercial email – may or may not be
considered spam• ‘Opt-out’ and unsubscribing are often tricky and users
have been trained to avoid this• Anti-spam should handle all of these situations
3. Reaching a near-zero false positive level without compromising the detection level
WWW.SYBARI.COM
Five anti-spam challenges
4. Real-time updates & filtering• Blocking from the first minute of an
attack• Remove the “window of vulnerability”
created by scheduled filter updates• Improving anti-virus filtering
5. International efficacy• Languages, encoding methods &
double-byte can cut the effectiveness of content-based detection to zero
WWW.SYBARI.COM
Outsmarting Spam
All messages in a spam outbreak have a repetitive component – the
attack “pattern”
… and Sybari ASD knows how to trace it!
WWW.SYBARI.COM
First, some statistics• The ASD Service Center detects on average over
600,000 unique spam attacks per day– Based on statistics from 12/07/03 to 1/06/04– High of 799,000 to low of 340,000
December 29, 2003
0
5000
10000
15000
20000
25000
30000
Time of Day
Uni
que
outb
reak
s
Actual new outbreaks per hour from 12/29/03
WWW.SYBARI.COM
The ASD Spam Detection Engine• Located at the ASD Service Centers,
monitoring over 15 million message signatures daily
• Automatically detects the repetitive component of each spam outbreak
• Uses Recurrent Pattern Detection technology, or RPDTM
– Powered by Commtouch Software• Identifies the identical or approximate
patterns appearing in spam– Statistical analysis determines spam– Spam “signatures” created based on detection
WWW.SYBARI.COM
Recurrent Pattern Detection• Identical match and approximate match
techniques detect spam attacks– Every spam attack has some element of similarity– Checks sender, subject line, body
Classification system,
statistical analysis
SPAM!
SPAM!
Valid mailMail Signatures
WWW.SYBARI.COM
The ASD Spam Detection Engine• Based on message prevalence, mail is
rated as “not spam,” “bulk mail” or “confirmed spam”
• Bulk Mail and Confirmed Spam can be handled differently
• Spam is “confirmed” by human monitors to ensure complete confidence in rejecting confirmed spam messages
WWW.SYBARI.COM
RPDTM Benefits
1. 95%+ detection rate - detects solicited & unsolicited spam
2. No false positive mistakes due to “suspicious” content in legitimate person to person messages• Does not rely on specific words• Critical for industries that use many “spam”
words – financial, real estate, medical, retail, marketing, etc.
3. Immune to constantly evolving spammer tactics• Relies on the one factor that remains
consistent for all spam – it is sent in volume
WWW.SYBARI.COM
RPDTM Benefits
4. The fastest spam detection technology:
• Blocks spam from the first minutes of an outbreak
• Real-time spam signature updates ensure the highest detection levels
5. Content-agnostic – detects spam in:
• All languages• All encoding methods, and double-byte• All file formats
WWW.SYBARI.COM
Internet
Internet
Service Center/Gateway Interaction
• Real time signature updates from Service Center
Internet
Internet
Good
Bulk
Spam
Recurrent Pattern Detection
Signature Database – over six million sigs
===
Classifier InboxInbox
Tag, Junk Folder or Reject
Tag, Junk Folder or Reject
Local Signature Cache
Local Match
If unknown
Service Center
Data Center
Local detection first, remote detection as
needed.
WWW.SYBARI.COM
Implementation and Management
WWW.SYBARI.COM
ASD Implementation• Installed on
Windows 2000/2003 server
• Installs on SMTP Gateway or Exchange server
• Supports Exchange 5.5, 2000, 2003
• Uses SQL MSDE database
Sample deployment scenario
• Directory integration allows controlled deployment– One user/group at a time
WWW.SYBARI.COM
The ASD Gateway and Service Center
Policy Flow and Spam Management Options
WWW.SYBARI.COM
ASD Gateway Administration• Centralized administrator control of
system-wide block and accept rules– Spam can be rejected, quarantined or
sent to user• Maintains database of individual user
preferences for delegated control• Easy to use browser interface
:Strong anti-spam-filtering capabilities, flexible deployment options; easy to set up and manage.
KEY PERFORMANCE INDICATORS
USABILITY EXCELLENT
MANAGEABILITY EXCELLENT
WWW.SYBARI.COM
Gateway Administration
Lists blocked mail received from specific Domain or From field
WWW.SYBARI.COM
Gateway Administration
Approve (white list) –
all future mail from sender
will be allowed
Reject (black list) – all future mail from sender will be rejected and
treated based on group/rule settings
Quarantine – all future mail from sender will be sent to site Quarantine
based on group/rule settings
User Decision – all future mail
from sender will be sent to user
for decision
WWW.SYBARI.COM
Gateway Administration
WWW.SYBARI.COM
Gateway AdministrationSpam is
identified as Confirmed or
Bulk
Three actions for confirmed or suspected spam
• User Decision – send to Junk Mail folder
• Site Quarantine – send to quarantine for administrator decision
• Reject – reject message
Because spam is fluid and attacks happen quickly, mail with “low” or “moderate” chance of being spam can be held until Service Center is
re-polled.
WWW.SYBARI.COM
Gateway Status report
WWW.SYBARI.COM
Gateway Status report
An overview of system status
Total number of Block and Approve rules created by
users and Admins
Total number of Users and Users in Exception group
Total number of spam messages in given time
period, and percentage of emails considered spam
WWW.SYBARI.COM
Gateway General Traffic Reporting
An overview of system traffic
Total messages, spam and non-spam,
processed by policy or detection
Number of messages
approved and blocked
WWW.SYBARI.COM
About the ASD Junk Mail folder• Users make their own spam decisions• Users can white-list desired messages or
black-list unwanted messages with one click– No need to impose system wide blocks– Completely private and secure– Relieves admin from constant decision making
• The Junk Mail folder is automatically created in the user’s Outlook client– Does not disrupt the user experience
• Junk Folder is self-cleaning, based on administrator defined life cycle
WWW.SYBARI.COM
What the User Sees…
WWW.SYBARI.COM
What the User Sees…
Approve Sender: all further emails from this sender
go to Inbox
Block Sender: all further emails
from this sender will be blocked at
the Gateway
Policy Manager: allows user to
review and change existing rules, write new
rules
WWW.SYBARI.COM
What the User Sees…
• The Policy Manager allows end users to modify or create rules
• Provides support for POP3 accounts (clients that are not MS Outlook)
WWW.SYBARI.COM
Non-Junk Folder users
• Users who don’t use or want a Junk Folder can have spam “tagged” with admin-defined prefix– For example, Outlook Express users or
other POP3 clients
• A second ASD user group is defined in the Directory Services to support users that do not want/need a Junk Folder– Created using a simple utility
WWW.SYBARI.COM
About the Site Quarantine
• Administrator can direct spam to a Quarantine folder rather than the Junk Mail folder
• Spam and/or suspected spam can be sent to the Quarantine folder– Depends on administrator settings
• Administrator takes actions on quarantined messages– Reject message– Approve: release to user’s inbox– User Decision: send to user’s Junk Mail folder
WWW.SYBARI.COM
Quarantine Folder
Approve sender – mail is delivered to
end user Inbox
Reject sender – mail is deleted
User Decision – mail is delivered to user’s Junk
Mail folder
WWW.SYBARI.COM
ASD Evaluation Mode
• Run ASD in “Spam Analyzer” mode• Detects spam without taking any
actions– No Junk Folders created– No stamping of email– End users are unaffected/unaware
• Administrators receive full report data on number of spam messages detected, spam domains, etc.
• Understand ROI potential of ASD
WWW.SYBARI.COM
Summary – Sybari Advanced Spam Defense (ASD)
• Manages spam as a background service– Minimal IT maintenance– External Service Center scales to increasing volume– Global view of Internet traffic
• Gives IT control over inbound e-mail– Integrates directly into e-mail system– Fine-tune sensitivity when needed – Enforcement of enterprise policies
• Keeps responsibility in the hands of end users– Only they know the real definition of spam for them– Reduces false positives and non-delivery complaints– Preserves confidentiality and security