interactive machine learning appendix

Department of Computer Science

Interactive Machine Learning (Appendix) Zitao Liu

[email protected]

School of Arts of and Sciences



Examples: CueFlik(CHI2008)

• Goal: allows users to quickly create their own rules for re-ranking images based on their visual characteristics.

• Workflow: user provides CueFlik with examples of images that each rule should match, is shown the rule that CueFlik has learned, and then continues to provide CueFlik with examples of images that should be matched or rejected until they are satisfied with the learned rules.

• ML Model: Nearest Neighbor; similarity measure; active learning.

• Specialties: None.


Examples: CueFlik(CHI2008)

Positive examples

Negative examples


Examples: CueT(CHI2011)

• Goal: learns from the triaging decisions of operators and help them quickly and accurately triage alarms in a highly dynamic environment.

• Workflow: constantly updating and learning: alarms arrive, CueT provides recommendations and operators make decisions and CueT learns from it.

• ML Model: Nearest Neighbor; Adaptive similarity measure (online metric learning).

• Specialties: When to generate a new ticket(class)


Examples: CueT(CHI2011)

Alarm

view

Ticket

view

Severity

Similarity

visualization


Examples: Apolo(CHI2011)

• Goal: guides the user to incrementally and interactively explore large network data(demonstrated by scientific literature).

• Workflow: iterative exploration: browse examples -> create groups(labels) -> label examples -> browse new examples…

• ML Model: belief propagation.



Examples: Apolo(CHI2011)


Examples: Info. Extraction(CHI2009)

• Goal: proposes a novel synergistic method for jointly amplifying community content creation and learning based information extraction.

• Workflow: automatically extracts potential infobox and invites visitors to manually examined and explicitly confirmed it.

• ML Model: Doc/Sentence level classifiers; Information Extraction(CRF).



Examples: Info. Extraction(CHI2009)

info.

box

dialog

confirmation


Examples: EnsembleMatrix(CHI2009)

• Goal: presents a graphical view of confusion matrices to help users understand relative merits of various classifiers and helps users build combination multiclass classifiers.

• Workflow: users can directly manipulate the visualizations in order to build combination multiclass classifiers(Partition and Reweight).

• ML Model: ensemble classifier.

• Specialties: none.


Examples: EnsembleMatrix(CHI2009)

Main ensemble classifier each classifier

linear combination widget

Heat map

representation


Examples: ManiMatrix(CHI2010)

• Goal: provides controls and visualizations that enable system builders to refine the behavior of classification systems in an intuitive manner.

• Workflow: users refine parameters of a confusion matrix (modify decision boundaries of classifiers) via an interactive cycle of reclassification and visualization.

• ML Model: assigning sets of costs of misclassification in a confusion matrix. Costs of misclassification are non-uniform across classes.

• Specialties: decision-theoretic analysis(𝑅𝑖𝑠𝑘𝑗 = 𝑝𝑖 ⋅ 𝐶𝑜𝑠𝑡𝑖𝑗𝑐𝑖=1 )


Examples: ManiMatrix(CHI2010)

6 cloudy days

were misclassified

as rainy

changes after

user’s click

lock the cell

with a desired

direction

desired direction of

the value changes

(increasing)


Examples: ReGroup(CHI2012)

• Goal: helps people create custom, on-demand groups in online social networks.

• Workflow: ReGroup observes a person’s normal interaction of adding members to a group, it learns a probabilistic model of group membership in order to suggest both additional members and group characteristics for filtering a friend list. It continually update its membership model based on interactive user feedback.

• ML Model: Naïve Bayes(probability of each friend being a member of the group). Re-trained every time a person adds friends to a group.

• Specialties: obtain implicit negative examples; unlearnable groups; missing data


Examples: ReGroup(CHI2012)

Top 5 relevant group

characteristics

Relevant friend

suggestions

A static, hierarchical

list of all feature value

filters


Examples: Visual-FSSEM(KDD2000)

• Goal: guides the feature selection procedures and enable a deeper understanding of the data in unsupervised setting.

• Workflow: performs a greedy sequential forward or backward search for features as measured by the chosen performance criterion(scatter separability, maximum likelihood, cluster entropy and probability of error) Users can select initial feature subset and limit the pool features to search from. The user selects the direction forward or backward in every step and the number of steps.

• ML Model: LDA(linear discriminant analysis) for visualization; mixture multivariate Gaussians for clustering.

• Specialties: EM local minima;

interactive machine learning appendix

Data & Analytics