modern machine learning: new challenges and connections maria-florina balcan
TRANSCRIPT
![Page 1: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/1.jpg)
Modern Machine Learning: New Challenges and
Connections
Maria-Florina Balcan
![Page 2: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/2.jpg)
Machine Learning is Changing the World
“A breakthrough in machine learning would be worthten Microsofts” (Bill Gates, Microsoft)
“Machine learning is the hot new thing”(John Hennessy, President, Stanford)
“Web rankings today are mostly a matter of machine learning” (Prabhakar Raghavan, VP Engineering at Google)
![Page 3: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/3.jpg)
The World is Changing Machine Learning
New applications Explosion of data
![Page 4: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/4.jpg)
Modern applications: massive amounts of raw
data.Only a tiny fraction can be annotated by human experts.
Billions of webpages ImagesProtein sequences
Modern ML: New Learning Approaches
![Page 5: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/5.jpg)
Modern applications: massive amounts of raw
data.
Modern ML: New Learning Approaches
Learning AlgorithmExpert
• Semi-supervised Learning, (Inter)active Learning.
Techniques that best utilize data, minimizing need for expert/human intervention.
![Page 6: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/6.jpg)
Modern applications: massive amounts of
data distributed across multiple locations.
Modern ML: New Learning Approaches
![Page 7: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/7.jpg)
Modern ML: New Learning Approaches
• scientific data
Key new resource communication.
• video data
E.g.,
Modern applications: massive amounts of
data distributed across multiple locations.
![Page 8: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/8.jpg)
The World is Changing Machine Learning
Resource constraints. E.g.,
• Computational efficiency (noise tolerant algos) • Communication
• Human labeling effort
• Statistical efficiency
• Privacy/Incentives
New approaches. E.g.,
• Semi-supervised learning
• Distributed learning
• Interactive learning
• Multi-task/transfer learning
• Never ending learning
• Community detection
![Page 9: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/9.jpg)
d log(1/)
A2
equilibriad(1/)
ÂErr(h)
The World is Changing Machine Learning
My research: foundations, robust algorithms.
Applications: vision, comp. biology, social networks.
New approaches. E.g.,
• Semi-supervised learning
• Distributed learning
• Interactive learning
• Never ending learning
• Community detection
• Multi-task/transfer learning
![Page 10: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/10.jpg)
Machine Learning is Changing the WorldMy research: MLT way to model and tackle
challenges other fields. E.g.,
Game TheoryApproximation Algorithms
Matroid Theory
Machine Learning Theory
Discrete Optimization
Mechanism Design
Control Theory
![Page 11: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/11.jpg)
• Machine Learning Lenses in Other Areas
Outline of the talk
• Modern Learning Paradigms
• Interactive Learning
• Submodularity, implications to Matroid Theory, Algorithmic Game Theory, Optimization
;
V
• Discussion, Other Exciting Directions
![Page 12: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/12.jpg)
Supervised Learning• E.g., which emails are spam and which are important.
Not spam spam
• E.g., classify objects as chairs vs non chairs.
Not chair
chair
![Page 13: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/13.jpg)
Labeled Examples
Learning Algorithm
Expert / Oracle
Data Source
c* : X ! {0,1}
h : X ! {0,1}
(x1,c*(x1)),…, (xm,c*(xm))
• Algo sees (x1,c*(x1)),…, (xm,c*(xm)), xi i.i.d. from D
Distribution D on X
Statistical / PAC learning model
(x1,…,x
m)
+
+
--++--
-
• Does optimization over S, finds hypothesis h 2 C.• Goal: h has small error, err(h)=Prx 2 D(h(x) c*(x))
• c* in C, realizable case; else agnostic
![Page 14: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/14.jpg)
14
Two Main Aspects in Classic Machine Learning
Algorithm Design. How to optimize?Automatically generate rules that do well on observed
data.
Generalization Guarantees, Sample Complexity
Confidence for rule effectiveness on future data.
E.g., Boosting, SVM, etc.
![Page 15: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/15.jpg)
Interactive Machine Learning
• Active Learning
• Learning with more general queries; connections
![Page 16: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/16.jpg)
Active Learning
face
O
O
O
Expert Labeler
raw data
Learning Algorith
m
Unlabeled
data
Classifier
not face
![Page 17: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/17.jpg)
Active Learning in Practice
• Text classification: active SVM (Tong & Koller, ICML2000).
• e.g., request label of the example closest to current separator.
• Video Segmentation (Fathi-Balcan-Ren-Regh, BMVC 11).
![Page 18: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/18.jpg)
w
+-
Exponential improvement.
• Sample with 1/ unlabeled examples; do binary search.-
Active: only O(log 1/) labels.
Passive supervised: (1/) labels to find an -accurate threshold.
+-
Active Algorithm
• Canonical theoretical example [CAL92, Dasgupta04]
Provable Guarantees, Active Learning
![Page 19: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/19.jpg)
• First noise tolerant algo [Balcan, Beygelzimer, Langford,
ICML’06]
[Hanneke07, DasguptaHsuMontleoni’07, Wang’09 , Fridman’09, Koltchinskii10, BHW’08, BeygelzimerHsuLangfordZhang’10, Hsu’10, Minsker’12, Ailon’12, …]
Generic (any class), adversarial label noise.
Substantial follow-up on work.
Lots of exciting activity in recent years.
My work:
Provable Guarantees, Active Learning
• First computationally efficient algos[Balcan-Long COLT’13] [Awasthi-Balcan-Long STOC’14]
![Page 20: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/20.jpg)
Margin Based Active Learning
• Realizable: exponential improvement in label complex. over passive [only O(d log 1/) labels to find w error ].
[Balcan-Long COLT’13] [Awasthi-Balcan-Long STOC’14]
Learning linear separators, when D logconcave in Rd.
• Agnostic: poly-time AL algo outputs w with err(w) =O(´), ´ =err(best lin. sep).
+
+
--++--
-
• Unexpected Implications for Passive Learning!
• Computational complexity: improves significantly on the best guarantees known for passive [KKMS’05], [KLS’09].
• Sample complexity: resolves open question about ERM.
![Page 21: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/21.jpg)
Margin Based Active-Learning, Realizable Case
Draw m1 unlabeled examples, label them, add them to W(1).iterate k = 2, …, s
• find a hypothesis wk-1 consistent with W(k-
1).• W(k)=W(k-1).• sample mk unlabeled samples x
satisfying |wk-1T ¢ x| · k-1
• label them and add them to W(k).
w1
1
w2
2
w3
![Page 22: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/22.jpg)
Log-concave distributions: log of density fnc concave.• wide class: uniform distr. over any convex set, Gaussian,
etc.
Theorem If then err(ws)· D log-concave in Rd.
after
Active learning Passive learning
rounds using
label requests label requests
unlabeled examples
labels per round.
Margin Based Active-Learning, Realizable Case
![Page 23: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/23.jpg)
Analysis: Aggressive Localization
Induction: all w consistent with W(k), err(w) · 1/2k
w
w*
![Page 24: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/24.jpg)
Analysis: Aggressive Localization
Induction: all w consistent with W(k), err(w) · 1/2k
wk-1
w
k-1
w*
Suboptimal
wk-1
w
k-1
w*
![Page 25: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/25.jpg)
Analysis: Aggressive Localization
Induction: all w consistent with W(k), err(w) · 1/2k
wk-1
w
k-1
w*
· 1/2k+1
![Page 26: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/26.jpg)
Analysis: Aggressive Localization
Induction: all w consistent with W(k), err(w) · 1/2k
wk-1
w
k-1
w*
· 1/2k+1
Enough to ensure
Need only labels in round k.
![Page 27: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/27.jpg)
Localization in concept space.
Margin Based Active-Learning, Agnostic Case
Draw m1 unlabeled examples, label them, add them to W.
Localization in instance space.
iterate k=2, …, s • find wk-1 in B(wk-1, rk-1) of small
tk-1 hinge loss wrt W.• Clear working set.• sample mk unlabeled samples x
satisfying |wk-1 ¢ x| · k-1 ;• label them and add them to W.
end iterate
• Analysis: exploit localization & variance analysis control the gap between hinge loss and 0/1 loss in each round.
![Page 28: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/28.jpg)
Improves over Passive Learning too!
Passive Learning
Prior Work Our Work
Malicious
Agnostic
Active Learning[agnostic/malicious]
[KLS’09]
NA
[KLS’09]
err(w)err(w)
err(w) err(w)
err(w)
![Page 29: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/29.jpg)
Improves over Passive Learning too!
Passive Learning
Prior Work Our Work
Malicious
Agnostic
Active Learning[agnostic/malicious]
Info theoretic optimal
[KKMS’05]
[KLS’09]
[KKMS’05]
NAInfo theoretic optimal
Slightly better results for the uniform distribution case.
err(w)
err(w)
err(w)
err(w)
err(w)
err(w)
![Page 30: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/30.jpg)
Useful for active and passive learning!
Localization both algorithmic and analysis tool!
![Page 31: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/31.jpg)
Important direction: richer interactions with the expert.
Learning AlgorithmExpert
Fewer queries
Natural interaction
Better Accuracy
![Page 32: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/32.jpg)
Learning Algorith
m
raw data
Expert
Labeler
Classifier
New Types of InteractionClass Conditional
Query
Mistake Query
Learning Algorithm
raw data
Expert Labeler
)Classifier
dog cat penguin
wolf
![Page 33: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/33.jpg)
Class Conditional & Mistake Queries
• Used in practice, e.g. Faces in IPhoto.• Lack of theoretical understanding.
• Realizable (Folklore): much fewer queries than label requests.
[Balcan-Hanneke COLT’12]
Tight bounds on the number of CCQs to learn in the presence of noise (agnostic and bounded noise)
Our Work
![Page 34: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/34.jpg)
Important direction: richer interactions with the expert.
Learning AlgorithmExpert
Fewer queries
Natural interaction
Better Accuracy
![Page 35: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/35.jpg)
3.5
2.4
9.1
Interaction for Faster Algorithms
E.g., clustering protein sequences by function
Expert an expensive computer program.
Voevodski-Balcan-Roglin-Teng-Xia, [UAI 10, SIMBAD 11, JMLR’12]
• Strong guarantees under natural stability condition: produce accurate clusterings with only k one-vs-all queries.
• BLAST to computes similarity to all other objects in dataset (one-vs-all queries).
• Better accuracy than state of the art techniques using limited info on standard datasets (pFam and SCOP datasets).
![Page 36: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/36.jpg)
Voevodski-Balcan-Roglin-Teng-Xia [UAI 10, SIMBAD 11, JMLR’12]
Better accuracy than state of the art techniques using limited info on standard datasets (pFam and SCOP datasets).
Interaction for Faster Algorithms
![Page 37: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/37.jpg)
Interactive Learning
• First noise tolerant poly time, label efficient algos for high dim. cases.[BL’13] [ABL’14]
• Learning with more general queries.
• Active & Differentially Private [Balcan-Feldman, NIPS’13]
[BH’12]
• Communication complexity, distributed learning.
Cool Implications:
• Sample & computational complexity of passive learning
[Balcan-Blum-Fine-Mansour, COLT’12] Runner UP Best Paper
Summary:
Related Work:
![Page 38: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/38.jpg)
Learning Theory Lenses on Submodularity
![Page 39: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/39.jpg)
Submodular functions• Discrete fns that model laws of diminishing returns.
• Optimization, operations research
• Algorithmic Game Theory
• Machine Learning
[Lehman-Lehman-Nisan’01], ….
[Bilmes’03] [Guestrin-Krause’07], …
• Social Networks
[Kleinberg-Kempe-Tardos’03]
![Page 40: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/40.jpg)
[Balcan&Harvey, STOC 2011 & Necktar Track, ECML-PKDD 2012]
Novel structural results
New Lenses on Submodularity
Learning Submodular Functions
Alg. Game TheoryEconomics
Matroid Theory
Discrete Optimization
Applications: Economics, Social Networks, …
![Page 41: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/41.jpg)
Submodular functions
xS Tx
+
+
Large improvement
Small improvement
For T µ S, xS, f(T [ {x}) – f(T) ¸ f(S [ {x}) – f(S)
• V={1,2, …, n}; set-function f : 2V ! R submodular if
E.g.,
+
![Page 42: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/42.jpg)
Submodular functions
• Concave Functions Let h : R ! R be concave. For each S µ V, let f(S) = h(|S|)
• Vector Spaces Let V={v1,,vn}, each vi 2 Rn.
For each S µ V, let f(S) = rank(V[S])
More examples:
• Coverage Fns Let A1,,An be sets.For each S µ V, let f(S) = ||
![Page 43: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/43.jpg)
Learning submodular functions
• Influence Function in Social Networks• V = set of nodes.• f(S) = expected influence when S is the originating set of nodes in the difusion.
• Valuation Functions in Economics • V = all the items you sell.• f(S) = valuation on set of items S.
![Page 44: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/44.jpg)
PMAC model for learning real valued functions
Distribution D on 2[n]
Labeled Examples
Learning Algorithm
Expert / Oracle
Data Source
Alg.outputs f : 2[n] R+g :2[n] R+
(S1,f(S1)),…, (Sm,f(Sm))
• Algo sees (S1,f(S1)),…, (Sm,f(Sm)), Si i.i.d. from D, produces g.
Probably Mostly Approximately Correct
• With probability ¸ 1-± we have PrS[g(S) · f(S)· ® g(S)]¸ 1-²
![Page 45: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/45.jpg)
Learning submodular functions [BH’11]
No algo can PMAC learn the class of submodular fns with approx. factor õ(n1/3).
Theorem: (General lower bound)
Corollary: Matroid rank fns do not have a concise, approximate representation.
Surprising answer to open question in Economics of
Theorem: (General upper bound)
Poly time alg. for PMAC-learning (w.r.t. an arbitrary distribution) with an approx. factor ®=O(n1/2).
• Even if value queries allowed; even for rank fns of matroids.
Noam NisanPaul Milgrom
![Page 46: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/46.jpg)
A General Lower Bound
Construct a hard family of matroid rank functions.
A1 A2 ALA3
X
X X
Low=log2n
High=n1/3
X
… … …. ….
L=nlog log n
No algo can PMAC learn the class of non-neg., monotone, submodular fns with an approx. factor õ(n1/3).
Theorem
Vast generalization of partition matroids.
![Page 47: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/47.jpg)
Exploit Additional Properties
• E.g., product distribution
• BH’11, Lipschitz submodular
• FV’13 general submodular
• Learning valuation fns from AGT and Economics.
• Learning influence function in social networks.
![Page 48: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/48.jpg)
Learning Valuation FunctionsTarget dependent learnability for classes from Algorithmic Game Theory and Economics
Additive µ OXS µ GS µ Submodular µ XOS µ Subadditive
XOR
OR
({1}, $2) ({2}, $5)
Romania SwitzerlandOR
({2}, $6)({1}, $10)({3}, $5)
[Balcan-Constantin-Iwata-Wang, COLT 2012]
Theorem: (Poly number of XOR trees)
O(n²) approximation in time O(n1/²).
![Page 49: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/49.jpg)
Learning the Influence FunctionInfluence function in social networks: coverage
function.
[Du, Liang, Balcan, Song 2014]
• Maximum likelihood approach; promising performance
• Memetracker Dataset, blog data cascades : “apple and jobs”, “tsunami earthquake”, “william kate marriage”
![Page 50: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/50.jpg)
[Balcan&Harvey, STOC 2011 & Necktar Track, ECML-PKDD 2012]
Novel structural results
New Lenses on Submodularity
Learning Submodular Functions
Algo Game TheoryEconomics
Matroid Theory
Discrete Optimization
Follow-up work: Algo Game Theory, Machine Learning, Discrete Optimization, Privacy.
![Page 51: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/51.jpg)
Discussion. Other Exciting Directions
![Page 52: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/52.jpg)
• Foundations for Modern ML and Applications
Themes of My Research
Game TheoryApprox.
Algorithms
Matroid Theory
Machine Learning Theory
Discrete Optimization
Mechanism Design
Control Theory
• Learning Lens on Other Fields, Connections
![Page 53: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/53.jpg)
Foundations for Modern Paradigms
Resource constraints. E.g.,
• Computational efficiency (noise tolerant algos) • Communication
• Human labeling effort
• Statistical efficiency
• Privacy/Incentives
New approaches. E.g.,
• Semi-supervised learning
• Distributed learning
• Interactive learning
• Never ending learning
• Community detection
• Multi-task/transfer learning
![Page 54: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/54.jpg)
Analysis Beyond the Worst Case• Identify ways to capture structure of real world
instances, analysis beyond the worst case.
• E.g., circumvent NP-hardness barriers for objective based clustering (e.g., k-means, k-median)
• Perturbation resilience: small changes to input distances shouldn’t move optimal solution by much.
• Approximation stability: near-optimal solutions shouldn’t look too different from optimal solution.
0.7
2.1
[BBG’09] [BB’09] [BRT’10] [BL’12]
![Page 55: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/55.jpg)
Analysis Beyond the Worst Case
• Understand characteristics of practical tasks that makes them easier to learn.
• Develop learning tools that take advantage of such characteristics.
Major Direction:
• Identify ways to capture structure of real world instances, analysis beyond the worst case.
![Page 56: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/56.jpg)
Understanding & Influencing Multi-agent Systems
What do we expect to happen?
More realistic: view agents as adapting, learning entities.Major Question: Can we “guide” or “nudge” dynamics to high-quality states quickly [using minimal intervention]?
Traditional: analyze equilibria (static concept)
![Page 57: Modern Machine Learning: New Challenges and Connections Maria-Florina Balcan](https://reader030.vdocuments.site/reader030/viewer/2022013011/5517fc8f550346d5568b5027/html5/thumbnails/57.jpg)
• Foundations for Modern ML and Applications
Themes of My Research
Game TheoryApprox.
Algorithms
Matroid Theory
Machine Learning Theory
Discrete Optimization
Mechanism Design
Control Theory
• Learning Lens on Other Fields, Connections