monitoring, privacy, and confidentiality

Copyright © 1997-2004 Coradiant Inc. Do not copy or distribute.

Monitoring, privacy, and confidentiality

Monitoring, privacy, and confidentiality

Respecting privacy without crippling IT

Alistair CrollChief Strategy Officer, Coradiant inc.

Copyright © 1997-2004 Coradiant, Inc. Do not copy or distribute. 2

The golden days…(with apologies to the New Yorker)The golden days…(with apologies to the New Yorker)

““The best thing about the Internet is they The best thing about the Internet is they don’t know you’re a dog.”don’t know you’re a dog.”

Tom Toles. Tom Toles. Buffalo NewsBuffalo News, April 4, 2000., April 4, 2000.


The reality …The reality …

“You’re a four-year-old German Shepherd-Schnauser mix, likes to shop for rawhide chews, 213 visits to Lassie website, chatroom conversation 8-29-99 said third Lassie was the hottest, downloaded photos of third Lassie 10-12-99, e-mailed them to five other dogs whose identities

are…” Tom Toles. Tom Toles. Buffalo NewsBuffalo News, April 4, 2000., April 4, 2000.


The short versionThe short version

•Monitoring and privacy hate one another– Monitoring requires detailed forensics

– Privacy requires anonymity, restrictions

•This argument is playing out on the web– Web needs better monitoring than normal apps

– Much room for mischief on the Internet

•Real-User Monitoring is central to the debate!– Deep user activity visibility, but potential for evil

•Strategies exist to ensure that compliance and transaction monitoring play nice together


The good news:Web Real-User Monitoring

The good news:Web Real-User MonitoringChanging the way companies measure, improve, and operate their web systems

… a crash course


Aggressive move to web-based applicationsAggressive move to web-based applications

•First B2C applications– Over 20% of G2000 revenues now come from web-

based channels

– Customer self-service growing dramatically

•Then B2B with partners and suppliers– Can cut costs by up to 90%

– Enables just-in-time, accountability, etc.

•Now for internal applications– $0.37 of every 2002 IT dollar spent on web EAI

– Since 2001, all enterprise SW vendors have web UI

– Enterprise software is $170B, all going web


Near-unanimousindustry adoption of web technologyNear-unanimousindustry adoption of web technology

•SSL as a VPN fabric

•HTTP as a transport for traffic– RPC

– SOAP and XML

– Thin- and fat-client applications

•URI formats for most addresses


Side effect: Protocol consolidation changing measurement toolsSide effect: Protocol consolidation changing measurement tools

•At layer 3, IP convergence

•At layer 4, TCP and UDP

•At layer 5, transactional integrity– SSL provides server or mutual authentication,

message confidentiality and integrity

– HTTP state machine provides ???????????

•At layer 6/7 security and business process– Application session ID

•Bottom line: We rely on fewer protocols carry more traffic, so can focus on these and examine transactions deeply


But performance unpredictable, nonstandardBut performance unpredictable, nonstandard

• The worst 5% of tests show up to 40 times average delay

• Worse for real users—these are synthetic test averages!


The web is orneryThe web is ornery


How do we take back control?How do we take back control?

•We need to see each unique visit– Every user is different

– So “brownouts” and slowdowns vary

•We need open access to every step of a user interaction

– This is usually the root cause of problems• Less than 2% of errors are hardware• More than 30% are application-related

•We need visibility into all parties involved– Desktop, last mile, ISP, backbone, WAN, firewall,

load-balancer, web, app, database, EDI partners


Real User Monitoring rapidly displacing traditional measurement methodsReal User Monitoring rapidly displacing traditional measurement methods

•Existing tools inaccurate, misleading, costly

–Synthetic tests miss errors, add load• See < 1% of user errors,• 82.3% of slowdowns found by end users• Only 2-4% of users report a problem

–Agents complicate, hide service quality• Site slow while platforms fine, agents impact performance,

miss problems

–Logfiles useless when servers down• Don’t store needed data; affect performance

•Over 50% of outages not found by management tools

– End users, help desk warnings instead•Estimated at 7% of worldwide APM market by 2007


A typical problem resolution cycle in complex environmentsA typical problem resolution cycle in complex environments

Typical


Problem resolution for RUMProblem resolution for RUM

Typical

WithRUM


The downside: PrivacyThe downside: Privacy

Maintaining the trust and mitigating the risk


Bad things happen on the webBad things happen on the web

•Many users from many organizations

–Customers, partners, internal users

•Few good privacy or trust standards

–Mostly e-business focused–Spam, spyware, trojans undermine users’ faith

•Huge potential liabilities

–Fraud/ID theft, extortion, and privacy violations

•Legislation that’s hard to enforce

–Many government edicts–Limited ability to comply and do our jobs

•The people who fix the systemsmay not be allowed to look at them!


Let’s look at just identity theft:Out of 215M US residents in 2002Let’s look at just identity theft:Out of 215M US residents in 2002

• 9.9M people victimized, cost $47B in 2002 (US FTC)• identity thieves stole nearly $100M from financial firms in 2003

– $6,767 per victim

1Source: Identity Theft Survey Report conducted by Synovate for the FTC (March-April 2003). 2Based on the U.S. population age 18 and over (215.47 million) as of July 1, 2002 (Source: Population Division, U.S. Census Bureau; Table NA-EST2002-ASRO-01).

New Accounts & Other Frauds

Other Existing Accounts

Existing Credit Card Only

Total Victimization

3.2 million victims (1.5%)2

1.5 million victims

(0.7%)

5.2 million victims

(2.4%)

(4.6%)

Vic

tim

s in

M

illio

ns

0

5

10


What happensWhat happens

•“The ‘Really Bad People’ pay ‘ethically challenged’ techies to do their dirty work”

– Microsoft

•Identity theft–Capturing login data and posing as a user

•Key theft–Stealing encryption information to forge digital

signatures•Exposing vulnerabilities

–Hackers can see how the app works•Regulatory Violations

–Divulging private information–Violating secrecy and trade acts


Where is it most likely to happen?Where is it most likely to happen?

• Threat more likely to come from insiders– Employees with access to large financial databases– Shift from single individuals to mass amounts of information– Insider bribery increasingly common

– Joanna Crane, FTC

• “The greatest threats [are] from employees, partners, and other trusted insiders with authorized access to a company's networks, systems, and proprietary info.”

– Informationweek, 2003

• Up to 80%of all security violations are committed (or facilitated) by employees

– Faulkner, May 2003

• Primarily from health-care-related institutions, and secondly from financial institutions

– Michigan State University’s identity theft research center


Unstructured data (logfiles)are the most dangerous, least protectedUnstructured data (logfiles)are the most dangerous, least protected

• In large corporations, unstructured data doubles every 2 months

• Unstructured data is in too many places

Source: Goldman-Sachs

“The accumulation of data through technology has outpaced our policies and procedures to protect it. The technology is there, but we're not using it.”

James H. VaulesNational Fraud Center Inc.


How much do privacy and fraudcost us today?How much do privacy and fraudcost us today?

•90% of respondents breached in 12 months

•80% acknowledged financial losses– $455,848,000 in quantifiable losses

– $170,827,000 theft of proprietary information

– $115,753,000 in financial fraud

•74% said Internet frequent attack point– 33% said internal systems

* Source: 2002 CSI/FBI Computer Crime and Security Survey


What does the future hold?What does the future hold?

•By 2006, 20-30% of G1000 enterprises will suffer financial exposure because of mistakes in customer privacy management

•By 2006, a large enterprise’s typical costs to recover from mistakes in customer privacy management will be $5-20M/year

Data Source: Radicati Group, Gartner


The bottom lineThe bottom line

•Like it or not, you will have to make sure monitoring tools comply with legislative and organizational privacy standards


What legislation will you have to follow?What legislation will you have to follow?

•The Fourth Amendment forms the basis of a “right to privacy,” the right to be left alone

– Justice Brandeis

•Now the law is enforcing it:

•SB 1386 (“Safe harbor”)•Sarbanes Oxley•HIPAA•SEC disclosure rules•Finance•FDA CFR Part 11

•FISMA•Gramm-Leach-Bliley•PIPEDA•EU Data Directive•Basel II•COPPA & FERPA•Many national laws


How restrictive are these?Consider SB-1386 (“Safe harbor”)How restrictive are these?Consider SB-1386 (“Safe harbor”)

•California Senate Bill 1386– Legislates Identity Theft– Applies to all organizations with information about

California residents

• In July 2003, all companies had 9 months to comply

•Protects combinations of Name and– SSN – Credit-card with PIN – Driver’s license number

• If breached you must notify everyone who might have been affected

– No notification required if encrypted (MD5) or blanked


What does the law look for?What does the law look for?

•Inadvertent release of sensitive personal information from weak procedures is illegal

• The real test: were there reasonable procedures in light of the sensitivity of the information to prevent such breaches?

• What constitutes reasonable and appropriate procedures is linked directly to the sensitivity of the information collected by the company

• Companies cannot wait for a breach to occur; they must take reasonable steps to guard against reasonably anticipated vulnerabilities

– J. Howard Beales, III, DirectorBureau of Consumer Protection

Federal Trade Commission, June 2004


What’s “reasonable”?What’s “reasonable”?

•Collected fairly and lawfully

•Relevant and not excessive

•Used for its intended purpose

•Accurate and up-to-date

•Kept no longer than necessary

•Used only by appropriate people

•Reasonably safe and secure


What if I wait for it to break?What if I wait for it to break?

•The FTC can rectify misrepresentations– Administrative orders and civil penalties up to

$12,000 a day for violations (US Dept. of Commerce)

•HIPPA wrongful disclosure penalties– $50,000 and/or imprisonment for up to 1 year

– $100,000 and/or imprisonment for up to 5 years if under false pretenses

– $250,000 and/or imprisonment for up to 10 years if intent to sell information


Can’t we all just get along?


We need to monitor to do our jobs,and for the web, RUM is itWe need to monitor to do our jobs,and for the web, RUM is it

•Downtime reduction– An hour of “hard down” downtime costs $50K - $1M

– But less obvious “brownouts” cost too• Organizational time to resolve

• Lost revenue

• Use of less efficient channels

•SLA attainment– Fueled by a bad history, companies demand

guarantees and refunds

•Performance improvement– You can’t improve what you can’t measure


Matching identity to activity or contentmakes privacy problems happenMatching identity to activity or contentmakes privacy problems happen

•Collecting any personally-identifiable information (PII)

– “If you don’t log in, they can’t steal you”

– Not really true (Verizon ruling links IP to identity)

•Revealing transactions that were not intended to be public

[email protected] visited diseases/cancer/info.html

5000 shares of XMPL sold for $50 apiece

West Coast team entered large deal in accounting app

Etc.


RUM exposes users and their activity to let us measure web performanceRUM exposes users and their activity to let us measure web performance

•Decrypt the session

•See all aspects of the transaction– What pages were visited

– Every form filled out

– Every URI and cookie

– Even raw HTML goes past the device


So we need to be responsibleSo we need to be responsible

•Ethical concerns

•Industry and legislative compliance

•Exposure to lawsuits and costs


The three roles in privacy policyand how we apply them to RUMThe three roles in privacy policyand how we apply them to RUM

Source: Synomos, Inc./Zero Knowledge

Store only measurements


Datavaulting

Datavaulting

Trust employees

Trust employees

Strip offsensitive

data

Strip offsensitive

data

EnvironmentalSafety

EnvironmentalSafety


Monitoring responsibly:Store measurements onlyMonitoring responsibly:Store measurements only

•Store aggregate performance measurements at regular intervals

•Pros– Good for performance (you know a function is slow)

•Cons– Doesn’t provide drill-down and problem resolution

– Is it always the same user? Server? Content? Host?

•As soon as you associate identity with activity, you run afoul of privacy




Monitoring responsibly:Data vaulting and retentionMonitoring responsibly:Data vaulting and retention

•Store access policies alongside performance data– Only authorized users can access data

•Keep data for a limited time only– Forced deletion makes information useful only for a short

while, limiting liability

•Pros– Robust, accountable, audit trail– Simple to implement– Access to every byte

•Cons– Complex to manage; forklift upgrade to existing systems– Makes trending difficult– Still exposes the organization– No way to prevent offline storage– Examples: Ingrian networks

Datavaulting

Datavaulting


Monitoring responsibly:Scrub what’s confidentialMonitoring responsibly:Scrub what’s confidential

•Apply rules about what must be hidden at collection time

– Delete it or one-way-encrypt it

•Pros– Simple to implement with some tools

– Easy to explain and defend politically

•Cons– Not all data is available for analysis

– Need to know what to scrub beforehand

– App teams must keep monitoring teams informed

Strip offsensitive

data

Strip offsensitive

data


Monitoring responsibly:Trust employeesMonitoring responsibly:Trust employees

•Pros– Very simple

•Cons– Very naive

Trust employees

Trust employees


Monitoring responsibly: Environmental SafetyMonitoring responsibly: Environmental Safety

•Having monitoring systems behave responsibly is part of the equation

•Protecting the monitoring systems from misuse, theft, compromise is essential as well

– System security • We have entered the age of hardened appliances• Hardened means physically tamper-proof

and attack-resilient

– Application security• Subject monitoring application to auditable and

secure coding practices

– Rigorous configuration policy management process– Auditable policy, systems, workflows, and processes– Physical security for systems and sites

EnvironmentalSafety

EnvironmentalSafety


Some conclusionsSome conclusions


ConclusionsConclusions

•Monitoring and privacy have to get along

•Increased application-layer visibility makes the problem worse

•Ethical, legislative and business motivations for responsible monitoring abound

•Expect auditors to knock on the computer measurement door soon


[email protected]

monitoring, privacy, and confidentiality

Documents