Probabilistic Convergence and Bounds

Junier Oliva10-701

2/19/2013

Useful Inequalities

• Markov’s Inequality:

• Chebyshev’s Inequality:

Convergence in Distribution

• Notation:• Definition:

Convergence in Pth Mean, Lp norm

• Notation:• Definition:• Intuition:

Almost Sure Convergence

• Notation:• Definition:• Intuition:

Convergence in Probability• Notation:• Definition:

• Intuition:

…

Relation Among Convergences

Convergence in Quadratic Mean => Convergence in Probability

Convergence in Quadratic Mean => Convergence in Probability

Convergence in Quadratic Mean => Convergence in Probability

Convergence in Quadratic Mean => Convergence in Probability

Convergence in Quadratic Mean => Convergence in Probability

Convergence in Quadratic Mean => Convergence in Probability

Convergence in Probability => Convergence in Distribution

Convergence in Probability => Convergence in Distribution

Convergence in Probability => Convergence in Distribution

Convergence in Probability => Convergence in Distribution

Convergence in Probability => Convergence in Distribution

Convergence in Probability => Convergence in Distribution

Convergence in Probability => Convergence in Distribution

Convergence in Probability => Convergence in Distribution

Convergence in Probability => Convergence in Distribution

Convergence in Probability => Convergence in Distribution

Convergence in Probability => Convergence in Distribution

Convergence in Probability => Convergence in Distribution

Hoeffding’s Bound

Union Bound

Empirical Optimization Set-up

Bounding Worst Case With High Probability

Bounding Worst Case With High Probability

Bounding Worst Case With High Probability

Bounding Worst Case With High Probability

Bounding Worst Case With High Probability

Bounding Worst Case With High Probability

Nearly Optimal Estimator

Nearly Optimal Estimator

Nearly Optimal Estimator

Nearly Optimal Estimator

Nearly Optimal Estimator

All Together

All Together

Thank You

Convergence in Probability => Convergence in Distribution

Convergence in Probability => Convergence in Distribution

Bounding Worst Case With High Probability

Bounding Worst Case With High Probability

Nearly Optimal Estimator