a new bound for discrete distributions based on maximum entropy † gzyl h., ‡ novi inverardi p.l....

A new bound for discrete distributionsA new bound for discrete distributionsbased on Maximum Entropybased on Maximum Entropy

††Gzyl H., Gzyl H., ‡‡Novi Inverardi P.L. and Novi Inverardi P.L. and ‡‡Tagliani A.Tagliani A.

†† USB and IESA - Caracas (Venezuela)USB and IESA - Caracas (Venezuela)‡‡ Department of Computer and Managements SciencesDepartment of Computer and Managements Sciences

University of Trento (Italy)University of Trento (Italy)

CNRS, Paris, France, July 8-13, 2006

MaxEnt 2006MaxEnt 2006

A new bound for discrete distributions based on Maximum Entropy 2

Our aim is to Our aim is to comparecompare some classical bounds for nonnegative integer-valued some classical bounds for nonnegative integer-valued random variables for estimating the survival probability random variables for estimating the survival probability

and a new tighter and a new tighter boundbound obtained through the obtained through the Maximum EntropyMaximum Entropy method method constrained by constrained by fractional momentsfractional moments given by given by

We will show the superiority of this last bound with respect to the others.We will show the superiority of this last bound with respect to the others.

Aim of the paperAim of the paperMaxEnt 2006MaxEnt 2006


Most used candidates as an Most used candidates as an upper bound upper bound of are:of are:

• the the Chernoff’s boundChernoff’s bound (Chernoff (1952)) (Chernoff (1952))

• the the moment boundmoment bound (Philips and Nelson (1995)) (Philips and Nelson (1995))

• the the factorial moment boundfactorial moment bound

These three bounds come from the These three bounds come from the Markov InequalityMarkov Inequality and involve only and involve only integer integer momentsmoments or or moment generating functionmoment generating function ( ( mgf mgf ).).

Some classical distribution boundsSome classical distribution boundsMaxEnt 2006MaxEnt 2006


Given two distribution F and G Given two distribution F and G sharing the first 2Q momentssharing the first 2Q moments, the following , the following distribution bound is well known in literature (Akhiezer (1965)), distribution bound is well known in literature (Akhiezer (1965)),

where ,where ,

andand

is the is the Hankel matrixHankel matrix..For any real value of For any real value of x,x, QQ(x)(x) represents the maximum mass can be represents the maximum mass can be

concentrated at the point concentrated at the point xx under the condition that under the condition that 2Q2Q moments are met; moments are met; being being QQ(x)(x) the reciprocal of a polynomial of degree the reciprocal of a polynomial of degree 2Q 2Q in in x,x, the bound goes the bound goes

to zero at the rate to zero at the rate x x --2Q 2Q asas x x goes to infinity: this behavior gives relatively goes to infinity: this behavior gives relatively sharp sharp tail informationtail information but no much on the but no much on the central partcentral part of the distribution of the distribution..

Akhiezer’s boundAkhiezer’s boundMaxEnt 2006MaxEnt 2006


Improving Akhiezer’s boundImproving Akhiezer’s bound

The main question isThe main question is

how to choosehow to choose

the the approximantapproximant distributiondistribution sharing the sharing the same 2Qsame 2Q moments of moments of F(x)F(x) which allows which allows

the bound improvement.the bound improvement.

A key role in achieving this improvement is played by the A key role in achieving this improvement is played by the MaxEnt techniqueMaxEnt technique..

constrained by constrained by integerinteger moments moments

MaxEntMaxEnt

constrained by constrained by fractionalfractional moments moments



Improving Akhiezer’s bound:Improving Akhiezer’s bound: integer momentsinteger moments

IfIf X X is a discrete r.v. with is a discrete r.v. with pmfpmf with assigned with assigned

is the is the MaxEntMaxEnt MM--approximant approximant of of PP based on based on

the firstthe first M M integer moments integer moments

thenthen combiningcombining

andand

we prove that the Akhiezer’s bound can be replaced by the following we prove that the Akhiezer’s bound can be replaced by the following uniformuniform

boundbound

This bound is This bound is tightertighter than Akhiezer’s bound because . than Akhiezer’s bound because .

Further, Further, HH[[PP] is unknown but, it can be estimated through ] is unknown but, it can be estimated through Aitken D2- method. .



Improving Akhiezer’s bound:Improving Akhiezer’s bound: fractional momentsfractional moments IfIf

• X X is a is a non negativenon negative discrete r.v. with discrete r.v. with pmf pmf

• a sequence of a sequence of M+1M+1 fractional momentsfractional moments

with chosen according towith chosen according to

• is the is the MaxEntMaxEnt MM--approximant approximant of of P,P, based on based on

the previous the previous MM fractional moments fractional moments

thenthen the chain of inequalities, similarly in the case of integer moments, gives the chain of inequalities, similarly in the case of integer moments, gives

Numerical evidence proves that , even for small Numerical evidence proves that , even for small MM, then the , then the Akhiezer’s bound can be replaced by the following new Akhiezer’s bound can be replaced by the following new uniformuniform and and computablecomputable boundbound



Improving Akhiezer’s bound: a new boundImproving Akhiezer’s bound: a new bound

The proposed bound is sharper than Akhiezer’s bound in the The proposed bound is sharper than Akhiezer’s bound in the centralcentral part of part of the distribution and vice versa, the Akhiezer’s bound is sharper than it in the the distribution and vice versa, the Akhiezer’s bound is sharper than it in the tailstails. . CombiningCombining Akhiezer’s and our Akhiezer’s and our MaxEntMaxEnt bound, we have a bound, we have a sharpersharper upper upper

bounds, valid for bounds, valid for X X ≥ ≥ 00 and and MM which guarantees : which guarantees :

where is obtained from through a convergence where is obtained from through a convergence accelerating process, so that may be assumed. Here the accelerating process, so that may be assumed. Here the maximum value allowed of maximum value allowed of QQ stems from the number of given moments or stems from the number of given moments or from numerical stability requirements.from numerical stability requirements.



How to calculate fractional moments?How to calculate fractional moments?

As seen, the As seen, the fractional momentsfractional moments play the role of building blocks of the play the role of building blocks of the

proposed procedureproposed procedure..

But, But, how tohow to calculatecalculate them? them?

Several scenarios will be analyzed, depending on the available information on Several scenarios will be analyzed, depending on the available information on

the distribution of . This latter is assumed given by a finite or infinite the distribution of . This latter is assumed given by a finite or infinite

sequence of moments and/or by the sequence of moments and/or by the mgfmgf. .

Here we present three cases:Here we present three cases:

a)a) both and are known;both and are known;

b)b) assigned and existing assigned and existing mgfmgf ; ;

c)c) known, known, RR finite or infinite finite or infinite



Case a): both and are knownCase a): both and are knownMaxEnt 2006MaxEnt 2006


Case b) assigned and existing mgf Case b) assigned and existing mgf MaxEnt 2006MaxEnt 2006


Case c) known, R finite or Case c) known, R finite or infinite infinite



A numerical example (1)A numerical example (1)MaxEnt 2006MaxEnt 2006


THANK YOU THANK YOU for your attention!for your attention!


Density reconstruction via ME techniqueDensity reconstruction via ME techniqueMaxEnt 2006MaxEnt 2006


Density reconstruction via ME techniqueDensity reconstruction via ME technique

But it is not the unique choice!But it is not the unique choice!



An uniform boundAn uniform bound MaxEnt 2006MaxEnt 2006


Aitken Aitken 22-method-method MaxEnt 2006MaxEnt 2006


How to choose fractional moments?How to choose fractional moments? MaxEnt 2006MaxEnt 2006


Fractional momentsFractional moments MaxEnt 2006MaxEnt 2006


Comparing and combining boundsComparing and combining boundsMaxEnt 2006MaxEnt 2006

a new bound for discrete distributions based on maximum entropy † gzyl h., ‡ novi inverardi p.l....

Documents

new bound

akhiezers bound maxent

bound improvement

bound goes

new tighter bound

moment bound philips

following distribution

integer moments maxent