allison gilmore, data scientist, ayasdi at mlconf sf - 11/13/15

The Shape of DataAllison Gilmore

Principal Data Scientist

November 13, 2015

2Company Confidential & Proprietary

Data has shape.Shape has meaning.

You already know this.

Company Confidential & Proprietary

Shape as Organizing Principle


Geometry or Topology?Geometry : Metric Topology : Locality

≅ ≅


Topological Summaries Capture Shape

Lens


Enhancing Traditional Methods

Company Confidential & Proprietary 9


Nodes are groups of similar data points.

Edges connect similar nodes.

Node position on the screen does not matter.


Enhancing Traditional Methods

PCA sees 3 clusters.

Using PCA coordinates as lenses, we can see more.


Topological Summary Shows 4 Clusters


Disease State & Model Choice

David Schneider, Stanford Microbiology and Immunology


Topological Model for Total Knee Replacement

Low length of stay

Low to moderate length of stay

Long length of stay


Carepaths for Total Knee Replacement

16


Beating* the Curse of Dimensionality

* I mean, there are always conditions.Niyogi, Smale, and Weinberger, A Topological View of Unsupervised Learning from Noisy Data,SIAM J. of Computing 20(2011) 646-663. http://math.uchicago.edu/~shmuel/noise.pdf

If a dataset is supported near a manifold, its key topological features can be detected from a sample whose size is independent of the dimension of ambient space.

Doesn’t matter!Dimension d < N

19Company Confidential & Proprietary

[email protected]

www.ayasdi.com

mailto:[email protected]

http://www.ayasdi.com/


Understanding Shape Improves Models

20

HighLow

Ground Truth Fraud Model Predicted Fraud

HighLow


Topology Guides Model Creation

allison gilmore, data scientist, ayasdi at mlconf sf - 11/13/15

Technology