Upload File

data mining algorithms - stanford university · data mining cs102 data mining looking for patterns...

  • Home
  • Documents
  • Data Mining Algorithms - Stanford University · Data Mining CS102 Data Mining Looking for patterns in data Similar to unsupervised machine learning •Popularity predates popularity
39
CS102 Data Mining Data Mining Algorithms CS102 Spring 2020

Upload: others

Post on 16-Aug-2020

14 views

Category:

Documents


0 download

Report

TRANSCRIPT

Page 1: Data Mining Algorithms - Stanford University · Data Mining CS102 Data Mining Looking for patterns in data Similar to unsupervised machine learning •Popularity predates popularity

CS102Data Mining

Data Mining Algorithms

CS102Spring 2020

Page 2: Data Mining Algorithms - Stanford University · Data Mining CS102 Data Mining Looking for patterns in data Similar to unsupervised machine learning •Popularity predates popularity

CS102Data Mining

Data Tools and Techniques

§ Basic Data Manipulation and AnalysisPerforming well-defined computations or asking well-defined questions (“queries”)

§ Data MiningLooking for patterns in data

§ Machine LearningUsing data to build models and make predictions

§ Data VisualizationGraphical depiction of data

§ Data Collection and Preparation

Page 3: Data Mining Algorithms - Stanford University · Data Mining CS102 Data Mining Looking for patterns in data Similar to unsupervised machine learning •Popularity predates popularity

CS102Data Mining

Data MiningLooking for patterns in data

Similar to unsupervised machine learning• Popularity predates popularity of machine learning• “Data mining” often associated with specific data

types and patterns

We will focus on “market-basket” data• Widely applicable (despite the name)

And two types of data mining patterns• Frequent item-sets• Association rules

Page 4: Data Mining Algorithms - Stanford University · Data Mining CS102 Data Mining Looking for patterns in data Similar to unsupervised machine learning •Popularity predates popularity

CS102Data Mining

Other Data and Patterns

Other types of data• Networks/graphs• Streams• Text (“text mining”)

Other patterns• Similar items• Structural patterns in large graphs/networks• Clusters, anomalies

Specific techniquesfor each one

Page 5: Data Mining Algorithms - Stanford University · Data Mining CS102 Data Mining Looking for patterns in data Similar to unsupervised machine learning •Popularity predates popularity

CS102Data Mining

(In)Famous Early Success Stories

Victoria’s Secret

Walmart

Beer & Diapers

Page 6: Data Mining Algorithms - Stanford University · Data Mining CS102 Data Mining Looking for patterns in data Similar to unsupervised machine learning •Popularity predates popularity

CS102Data Mining

Market-Basket Data

Originated with retail data• Each shopper buys “market basket” of groceries• Mine data for patterns in buying habits

General definition• Domain of items• Transaction - one or more items occurring

together• Dataset – set of transactions (usually large)

Page 7: Data Mining Algorithms - Stanford University · Data Mining CS102 Data Mining Looking for patterns in data Similar to unsupervised machine learning •Popularity predates popularity

CS102Data Mining

Market-Basket Examples

Items Transaction

Page 8: Data Mining Algorithms - Stanford University · Data Mining CS102 Data Mining Looking for patterns in data Similar to unsupervised machine learning •Popularity predates popularity

CS102Data Mining

Market-Basket Examples

Items Transaction

Groceries

Page 9: Data Mining Algorithms - Stanford University · Data Mining CS102 Data Mining Looking for patterns in data Similar to unsupervised machine learning •Popularity predates popularity

CS102Data Mining

Market-Basket Examples

Items Transaction

Groceries Grocery cart

Page 10: Data Mining Algorithms - Stanford University · Data Mining CS102 Data Mining Looking for patterns in data Similar to unsupervised machine learning •Popularity predates popularity

CS102Data Mining

Market-Basket Examples

Items Transaction

Groceries Grocery cartOnline goods

Page 11: Data Mining Algorithms - Stanford University · Data Mining CS102 Data Mining Looking for patterns in data Similar to unsupervised machine learning •Popularity predates popularity

CS102Data Mining

Market-Basket Examples

Items Transaction

Groceries Grocery cartOnline goods Virtual shopping cart

Page 12: Data Mining Algorithms - Stanford University · Data Mining CS102 Data Mining Looking for patterns in data Similar to unsupervised machine learning •Popularity predates popularity

CS102Data Mining

Market-Basket Examples

Items Transaction

Groceries Grocery cartOnline goods Virtual shopping cart

University courses

Page 13: Data Mining Algorithms - Stanford University · Data Mining CS102 Data Mining Looking for patterns in data Similar to unsupervised machine learning •Popularity predates popularity

CS102Data Mining

Market-Basket Examples

Items Transaction

Groceries Grocery cartOnline goods Virtual shopping cart

University courses Student transcript

Page 14: Data Mining Algorithms - Stanford University · Data Mining CS102 Data Mining Looking for patterns in data Similar to unsupervised machine learning •Popularity predates popularity

CS102Data Mining

Market-Basket Examples

Items Transaction

Groceries Grocery cartOnline goods Virtual shopping cart

University courses Student transcriptUniversity students

Page 15: Data Mining Algorithms - Stanford University · Data Mining CS102 Data Mining Looking for patterns in data Similar to unsupervised machine learning •Popularity predates popularity

CS102Data Mining

Market-Basket Examples

Items Transaction

Groceries Grocery cartOnline goods Virtual shopping cart

University courses Student transcriptUniversity students Party

Page 16: Data Mining Algorithms - Stanford University · Data Mining CS102 Data Mining Looking for patterns in data Similar to unsupervised machine learning •Popularity predates popularity

CS102Data Mining

Market-Basket Examples

Items Transaction

Groceries Grocery cartOnline goods Virtual shopping cart

University courses Student transcriptUniversity students Party

Movies

Page 17: Data Mining Algorithms - Stanford University · Data Mining CS102 Data Mining Looking for patterns in data Similar to unsupervised machine learning •Popularity predates popularity

CS102Data Mining

Market-Basket Examples

Items Transaction

Groceries Grocery cartOnline goods Virtual shopping cart

University courses Student transcriptUniversity students Party

Movies Person

Page 18: Data Mining Algorithms - Stanford University · Data Mining CS102 Data Mining Looking for patterns in data Similar to unsupervised machine learning •Popularity predates popularity

CS102Data Mining

Market-Basket Examples

Items Transaction

Groceries Grocery cartOnline goods Virtual shopping cart

University courses Student transcriptUniversity students Party

Movies PersonSymptoms

Page 19: Data Mining Algorithms - Stanford University · Data Mining CS102 Data Mining Looking for patterns in data Similar to unsupervised machine learning •Popularity predates popularity

CS102Data Mining

Market-Basket Examples

Items Transaction

Groceries Grocery cartOnline goods Virtual shopping cart

University courses Student transcriptUniversity students Party

Movies PersonSymptoms Patient

Page 20: Data Mining Algorithms - Stanford University · Data Mining CS102 Data Mining Looking for patterns in data Similar to unsupervised machine learning •Popularity predates popularity

CS102Data Mining

Market-Basket Examples

Items Transaction

Groceries Grocery cartOnline goods Virtual shopping cart

University courses Student transcriptUniversity students Party

Movies PersonSymptoms PatientMenu items

Page 21: Data Mining Algorithms - Stanford University · Data Mining CS102 Data Mining Looking for patterns in data Similar to unsupervised machine learning •Popularity predates popularity

CS102Data Mining

Market-Basket Examples

Items Transaction

Groceries Grocery cartOnline goods Virtual shopping cart

University courses Student transcriptUniversity students Party

Movies PersonSymptoms PatientMenu items Restaurant customer

Page 22: Data Mining Algorithms - Stanford University · Data Mining CS102 Data Mining Looking for patterns in data Similar to unsupervised machine learning •Popularity predates popularity

CS102Data Mining

Market-Basket Examples

Items Transaction

Groceries Grocery cartOnline goods Virtual shopping cart

University courses Student transcriptUniversity students Party

Movies PersonSymptoms PatientMenu items Restaurant customer

Words

Page 23: Data Mining Algorithms - Stanford University · Data Mining CS102 Data Mining Looking for patterns in data Similar to unsupervised machine learning •Popularity predates popularity

CS102Data Mining

Market-Basket Examples

Items Transaction

Groceries Grocery cartOnline goods Virtual shopping cart

University courses Student transcriptUniversity students Party

Movies PersonSymptoms PatientMenu items Restaurant customer

Words Document

Page 24: Data Mining Algorithms - Stanford University · Data Mining CS102 Data Mining Looking for patterns in data Similar to unsupervised machine learning •Popularity predates popularity

CS102Data Mining

Data Mining Algorithms

Frequent Item-Sets – sets of items that occur frequently together in transactions• Groceries bought together• Courses taken by same students• Students going to parties together• Movies watched by same people

Association Rules – When certain items occur together, another item frequently occurs with them• Shoppers who buy phone + charger also buy case• Students who take Databases also take Machine

Learning• Diners who order curry and rice also order bread

Page 25: Data Mining Algorithms - Stanford University · Data Mining CS102 Data Mining Looking for patterns in data Similar to unsupervised machine learning •Popularity predates popularity

CS102Data Mining

Frequent Item-SetsSets of items that occur frequently

together in transactions

ØHow large is a “set”?ØWhat does “frequently” mean?

Page 26: Data Mining Algorithms - Stanford University · Data Mining CS102 Data Mining Looking for patterns in data Similar to unsupervised machine learning •Popularity predates popularity

CS102Data Mining

Frequent Item-SetsSets of items that occur frequently

together in transactions

ØHow large is a “set”?Usually specify a minimum min-set-sizePossibly also a maximum max-set-size

ØWhat does “frequently” mean?Notion of support

Page 27: Data Mining Algorithms - Stanford University · Data Mining CS102 Data Mining Looking for patterns in data Similar to unsupervised machine learning •Popularity predates popularity

CS102Data Mining

Support

Support for a set of items S in a dataset of transactions is the fraction of the transactions containing S:

Specify support-threshold for frequent item-setsOnly return sets wheresupport > support-threshold

# of transactions containing S____________________________________

total # of transactions

Page 28: Data Mining Algorithms - Stanford University · Data Mining CS102 Data Mining Looking for patterns in data Similar to unsupervised machine learning •Popularity predates popularity

CS102Data Mining

Your TurnTransactions:

T1: milk, eggs, juiceT2: milk, juice, cookiesT3: eggs, chipsT4: milk, eggsT5: milk, juice, cookies, chips

What are the frequent item-sets if:• min-set-size = 2 (no max-set-size)• support-threshold = 0.3

Support:# of transactions containing S

____________________________________

total # of transactions

Page 29: Data Mining Algorithms - Stanford University · Data Mining CS102 Data Mining Looking for patterns in data Similar to unsupervised machine learning •Popularity predates popularity

CS102Data Mining

Computing Frequent Item-Sets“Apriori” algorithm

Efficiency relies on the following property:

Or the inverse:

If S is a frequent item-set satisfyingsupport-threshold t, then every subset of S is alsoa frequent item-set satisfying support-threshold t.

If S is not a frequent item-set satisfyingsupport-threshold t, then no superset of S can bea frequent item-set satisfying support-threshold t.

Page 30: Data Mining Algorithms - Stanford University · Data Mining CS102 Data Mining Looking for patterns in data Similar to unsupervised machine learning •Popularity predates popularity

CS102Data Mining

Association RulesWhen a set of items S occurs together,

another item i frequently occurs with themS à i

ØHow large is a “set”?ØWhat does “occurs together” mean?ØWhat does “frequently occurs with them” mean?

Page 31: Data Mining Algorithms - Stanford University · Data Mining CS102 Data Mining Looking for patterns in data Similar to unsupervised machine learning •Popularity predates popularity

CS102Data Mining

Association RulesWhen a set of items S occurs together,

another item i frequently occurs with themS à i

ØHow large is a “set”?Usually specify a minimum min-set-size for SPossibly also a maximum max-set-size for S

ØWhat does “occurs together” mean?ØWhat does “frequently occurs with them” mean?

Page 32: Data Mining Algorithms - Stanford University · Data Mining CS102 Data Mining Looking for patterns in data Similar to unsupervised machine learning •Popularity predates popularity

CS102Data Mining

Association RulesWhen a set of items S occurs together,

another item i frequently occurs with themS à i

ØHow large is a “set”?Usually specify a minimum min-set-size for SPossibly also a maximum max-set-size for S

ØWhat does “occurs together” mean?Notion of support

ØWhat does “frequently occurs with them” mean?Notion of confidence

Page 33: Data Mining Algorithms - Stanford University · Data Mining CS102 Data Mining Looking for patterns in data Similar to unsupervised machine learning •Popularity predates popularity

CS102Data Mining

Support and ConfidenceSupport for association rule S à i in a dataset of transactions is fraction of transactions containing S:

Confidence for association rule S à i in a dataset of transactions is the fraction of transactions containing S that also contain i:

# of transactions containing S____________________________________

total # of transactions

# of transactions containing S and i__________________________________________

# of transactions containing S

Page 34: Data Mining Algorithms - Stanford University · Data Mining CS102 Data Mining Looking for patterns in data Similar to unsupervised machine learning •Popularity predates popularity

CS102Data Mining

Support and Confidence

Specify support-threshold and confidence-threshold for association rules

Only return rules where:support > support-threshold andconfidence > confidence-threshold

Page 35: Data Mining Algorithms - Stanford University · Data Mining CS102 Data Mining Looking for patterns in data Similar to unsupervised machine learning •Popularity predates popularity

CS102Data Mining

Your TurnTransactions:

T1: milk, eggs, juiceT2: milk, juice, cookiesT3: eggs, chipsT4: milk, eggsT5: milk, juice, cookies, chips

What are the association rules S à i if:• min-set-size = 1 (no max-set-size)• support-threshold = 0.5• confidence-threshold = 0.5

# of transactions containing S____________________________________

total # of transactions

# of transactions containing S and i__________________________________________

# of transactions containing SConfidence:

Support:

Reminder: support andconfidence must be >

threshold, not ≥

Page 36: Data Mining Algorithms - Stanford University · Data Mining CS102 Data Mining Looking for patterns in data Similar to unsupervised machine learning •Popularity predates popularity

CS102Data Mining

Computing Association Rules

1. Use frequent item-sets to find left-hand sides S satisfying support threshold

2. Then extend to find right-hand sides S à isatisfying confidence threshold

NOT a property:

If S à i is an association rule satisfyingsupport-threshold t and confidence-threshold c,

and S’Í S, then S’ à i is an an associationrule satisfying support-threshold t and

confidence-threshold c.

Why Not?

Page 37: Data Mining Algorithms - Stanford University · Data Mining CS102 Data Mining Looking for patterns in data Similar to unsupervised machine learning •Popularity predates popularity

CS102Data Mining

Association Rules: LiftAssociation rule S à i might have high confidence because item i appears frequently, not because it’s associated with S.

Confidence for association rule S à i in a dataset of transactions is the fraction of transactions containing S that also contain i :

# of transactions containing S and i__________________________________________

# of transactions containing S

Lift

divided by the overall frequency of i :,

#trans containing S and i #trans containing i______________________________ ÷ _______________________

#trans containing S total #trans

Page 38: Data Mining Algorithms - Stanford University · Data Mining CS102 Data Mining Looking for patterns in data Similar to unsupervised machine learning •Popularity predates popularity

CS102Data Mining

Lift: ExamplesTransactions:

T1: milk, eggs, juiceT2: milk, juice, cookiesT3: eggs, chipsT4: milk, eggsT5: milk, juice, cookies, chips

juice à cookies Lift = (2/3) ÷ (2/5) = 10/6 = 1.67

eggs à milk Lift = (2/3) ÷ (4/5) = 10/12 = 0.83

Lift:#trans containing S and i #trans containing i

______________________________ ÷ _______________________

#trans containing S total #trans

Lift = 1: no associationLift > 1: associationLift < 1: anti-association

Page 39: Data Mining Algorithms - Stanford University · Data Mining CS102 Data Mining Looking for patterns in data Similar to unsupervised machine learning •Popularity predates popularity

CS102Data Mining

Data Mining Algorithms

CS102Spring 2020


CS590D: Data Mining Chris Clifton - Purdue University · Data Mining: Classification Schemes • General functionality – Descriptive data mining – Predictive data mining ... –

Data Mining-Graph Mining

UNIT - I Data Mining. UNIT - I Introduction : Fundamentals of data mining, Data Mining Functionalities, Classification of Data Mining systems, Major issues

Introduction to Introduction to Data Mining Data Mining

Data mining and privacy preserving in data mining

Data Mining: What is Data Mining?

Introduction to Data Miningstxavierstn.edu.in/ict_ppts/Computer Science/Jenila Vincent/part1.pdf · Define data mining Data mining vs. databases Basic data mining tasks Data mining

1 Data Mining Chapter 26. 2 Chapter 1. Introduction Motivation: Why data mining? What is data mining? Data Mining: On what kind of data? Data mining functionality

Text Mining - Data Mining

EE3J2 Data Mining EE3J2 Data Mining

September 4, 20151 Chapter 1. Introduction Motivation: Why data mining? What is data mining? Data Mining: On what kind of data? Data mining functionality

Data Mining BY JEMINI ISLAM. Data Mining Outline: What is data mining? Why use data mining? How does data mining work The process of data mining Tools

COMMENT MINING, POPULARITY PREDICTION, AND SOCIAL …€¦ · COMMENT MINING, POPULARITY PREDICTION, AND SOCIAL NETWORK ANALYSIS by Salman Jamali A Thesis Submitted to the Graduate

MINING EDUCATIONAL DATA USING DATA MINING … · Educational data mining includes machine learning and data mining techniques. Data related to ... , density based, grid based, model

Galaxy: Towards Scalable and Interpretable Explanation on ...ding/papers/08588786.pdfclimate data and growing popularity of data mining techniques [1], qualitative understanding between

Introduction to Data Mining - homepages.math.uic.eduhomepages.math.uic.edu/~jyang06/stat486/R/DataMining_JYang2014.pdfFundamentals of Data Mining Typical Data Mining Tasks Data Mining

Lecture 2: Data Mining 1. Roadmap What is data mining? Data Mining Tasks – Classification/Decision Tree – Clustering – Association Mining Data Mining

Visual Data Mining: An Overview What is Visual Data Mining? Survey of techniques Data Visualization Visualizing Data Mining Results Visual Data Mining

Data Mining - Brigham Young University · 2020. 10. 22. · Data Mining Popularity lRecent Data Mining explosion based on: lData available –Transactions recorded in data warehouses

DATA MINING AND ANALYSIS - doc.lagout.org Mining/Data Mining and Analysis... · DATA MINING AND ANALYSIS The fundamental algorithms in data mining and analysis form the basis

Data Mining vs. Statistics Pavel Brusilovsky. 2 Objectives 2 Intro to Data Mining Data Mining vs. Statistics Data Mining vs. Text Mining Applications

Web Mining – Data Mining im Internet Mining – Data Mining im Internet Vorlesung SS 2014 ... Web Mining is Data Mining for Data on the World-Wide Web Text Mining: Application of

Data Mining. The Mining Analogy Data mining gains its name to some degree its popularity, by playing off a meaning that the data you have stored is much

October 18, 2015 Data Mining: Concepts and Techniques 1 DATA MINING Motivation: Why data mining? What is data mining? Data Mining: On what kind of data?

Data Mining LECTURE # 01 Introduction to Data Mining

Data Mining Chapter 1 Introduction -- Basic Data Mining Tasks -- Related Concepts -- Data Mining Techniques

Cryptocurrency Mining Malware - Sucuri Security · 4 The Rising Popularity of Cryptocurrencies Since the introduction of Bitcoin in 2009, the popularity and adoption of cryptocurrencies

Data Mining & Texte Mining Partie I : Data Mining Chapitre

Course : Data mining - Lecture : Mining data streams

ICT619 Intelligent Systems Topic 6: Data Mining. ICT6192 Data Mining Introduction Business Applications of Data Mining Data Mining Activities

Applied Data Mining - Lagout Mining/Applied Data Mining [Xu... · The book reviews applied data mining from theoretical basis to ... data mining application arenas, ... research topics

What is Data Mining? Data Mining Motivation Data Mining Applications Applications of Data Mining in CRM Data Mining Taxonomy Data Mining Techniques

Gufran Ahmad. Contents What is Data Mining? Data Mining / KDD process Different aspects of Data Mining Why Data Mining? Data Mining in Business Examples

Data Mining Taylor Statistics 202: Data Mining

Massive Data Analytics Data Mining Introductiondisi.unitn.it/.../MassiveDataAnalytics/slides/DataMiningIntro-2in1.pdfMassive Data Analytics Data Mining Introduction ... Data mining,