certificates of positivity: theory, practice, and...

Introduction psd ternary quartics PSD, not SOS polynomials Polya’s Theorem Finding SOS decompositions Certificates of positivity on semialgebraic sets Applications to Optimization Rational sums of squares Rational certificates of positivity

Certificates of Positivity: Theory, Practice, andApplications

Vicki Powers

Dept. of Mathematics and Computer ScienceEmory University, Atlanta, GA

Currently: National Science Foundation

Triangle Lectures in Combinatorics, NC State, RaleighSeptember 21, 2013

Vicki Powers Certificates of Positivity: Theory, Practice, and Applications


1 Introduction

2 psd ternary quartics

3 PSD, not SOS polynomials

4 Polya’s Theorem

5 Finding SOS decompositions

6 Certificates of positivity on semialgebraic sets

7 Applications to Optimization

8 Rational sums of squares

9 Rational certificates of positivity



In theory, theory andpractice are the same. But inpractice, they are different.

– Albert Einstein



Introduction

Let R[X ] := R[X1, . . . ,Xn] and let∑

R[X ]2 denote the set ofsums of squares in R[X ]. If f ∈

∑R[X ]2, we say f is sos.

For S ⊆ Rn, we write f ≥ 0 (f > 0) on S to mean f takes onlynonnegative (strictly positive) values on S . If f ≥ 0 on Rn, we sayf is positive semidefinite, or psd.

Clearly f sos implies f is psd – the square of a real number ispositive. This simple fact and generalizations of it underlie a largebody of theoretical and computational work on certificates ofpositivity.



Suppose S ⊆ Rn, f ∈ R[X ], and f ≥ 0 on S . By a certificate ofpositivity for f on S we mean an algebraic expression for f ,usually involving elements in

∑R[X ]2, from which one can deduce

the positivity condition immediately.

For example, I claim f is psd, i.e. f ≥ 0 on R2:

f (x , y) := x4 + 4y4 − 4xy2 + 2x2y − x2 + y2 − 2y + 1

The following certificate of positivity for f proves the claim:

f = (x2 + y − 1)2 + (2y2 − x)2



This talk is about the theory and practice of certificates ofpositivity.

Theory: Results concerning the existence of certificates ofpositivity.

Practice: Results on computational and algorithmic issues, e.g.,finding and counting certificates of positivity.

We will give some history and background and take a random walkthrough results and applications.



Our story begins in 1888, when the 26-year-oldDavid Hilbert published his seminal paper onsums of squares.

It was well-known at the time that forpolynomials in R[x ] (one variable) andpolynomials in R[X ] of degree 2, psd impliessos.

Hilbert showed (the homogenized version of) the following:

every psd f of degree 4 in R[x , y ] is sos,

in all other cases there exist psd polynomials that are not sos.



In 1893, Hilbert proved that for n = 2 every psd polynomial inR[X ] can be written as a sum of squares of rational functions,equivalently, for such f there exists nonzero h ∈ R[X ] such thath2f ∈

∑R[X ]2.

Unable to prove that this holds for all psdpolynomials, it became the 17th problem onHilbert’s list of 23 problems he gave in hisaddress to the International Congress ofMathematicians in 1900.



E. Artin settled the question:

Theorem (Artin 1927)

Suppose f ∈ R[X ] is psd. Then there is a nonzero h ∈ R[X ] suchthat h2f is sos.

Note that an identity h2f = g21 + · · ·+ g2

k is a certificate ofpositivity for f on Rn.

Artin (along with Schreier) developed the theory of ordered fieldsin order to solve the 17th problem. And thus the subject of RealAlgebra was born (more or less).



PSD ternary quartics

The first result in Hilbert’s 1888 paper was that every psd ternaryquartic – homogenous polynomial in 3 variables of degree 4 – is asum of squares of three quadratic forms. Hilbert’s proof is not easyto understand and lacks detail in a number of key points.

In 1977, Choi and Lam gave a short elementary proof thatevery psd ternary quartic is a sum of squares of five quadraticforms.

In 2004, A. Pfister gave an elementary proof with fivereplaced by four and a constructive argument in the case thatthe t. q. has a non-trival real zero.

In 2012, Pfister and C. Scheiderer gave a complete proof ofHilberts Theorem, using only elementary tools such as thetheorems on implicit functions and symmetric functions.



Hilbert’s paper did not address constructive questions:

Given a psd ternary quartic p, how can one find q. formsf , g , h so that p = f 2 + g2 + h2?

How many “fundamentally different” ways can this be done?

In 1929, A. Coble showed that every ternary quartic over C can bewritten as a sum of three squares of quadratic forms in 63inequivalent ways.

Work on quartic curves due to D. Plaumann, B. Sturmfels, andC. Vinzant from 2012 gives a new proof of the Coble result,yielding an algorithm for computing all representations in thesmooth case.



What about the computational questions above?

In 2000, in joint work with B. Reznick, we described methodsfor counting representations of a psd ternary quartic as a sumof 3 squares of q. forms. Experimentally, the Coble result (63representations over C) occured, with 8 representations as asum of 3 squares of q. forms over R.

In 2004, in joint work with Reznick, F. Sottile, andC. Scheiderer, we showed that if p = 0 is smooth, a psdternary quartic p has exactly 8 inequivalent representations asa sum of 3 squares of quadratic forms.

Scheiderer extended this analysis to the singular case.



Hilbert did not give an explicit example of apsd, not sos, polynomial. The first publishedexamples did not appear until nearly 80 yearslater; the most famous example is due toMotzkin:

x4y2 + x2y4 + z6 − 3x2y2z2

Since then, many other examples and families of psd, not sos,polynomials have appeared, due to Robinson; Choi, Lam, andReznick; and others.

More recently, there have been attempts to understand Hilbert’sproof and make it accessible.



Reznick has isolated the underlying mechanism of Hilbert’s proofand shown that it applies to more general situations. He producednew families of psd forms that are not sos.

Hilbert’s proof uses the fact that forms ofdegree d satisfy the Cayley-Bacharachrelations, and they are not satisfied by forms offull degree 2d .

Very recently, G. Blekherman has shown that theCayley-Bacharach relations are in some sense the fundamentalreason that there are psd polynomials that are not sos. In smallcases, a complete characterization of the difference between psdand sos forms is given.



For example, forms of degree 6 in 3 variables:

Let H3,6 be the vector space of degree 6 forms in 3 variables.

Theorem (Blekherman)

Suppose p ∈ H3,6 is psd and not sos. Then there exist two realcubics q1, q2 intersecting in 9 (possible complex) projective pointsγ1, . . . , γ9 such that the values of p on γi certify that p is not asum of squares in the following sense: There is a linear functional lon H3,6, defined in terms of the γi ’s, such that l(q) ≥ 0 for all sosq and l(p) < 0.



Polya’s Theorem for forms positive on a simplex

In 1928, Polya’s proved the existence ofcertificates of positivity for forms positive on thestandard simplex∆n := {(x1, . . . , xn) ∈ Rn | xi ≥ 0,

∑i xi = 1}.

Theorem (Polya’s Theorem)

Suppose f ∈ R[X ] is homogeneous and is strictlypositive on ∆n, then for sufficiently large N, all ofthe coefficients of (X1 + · · ·+ Xn)N f are strictlypositive.



Bounds for Polya’s Theorem and generalizations

In 2000, in joint work with Reznick, we gave an explicit boundon the exponent N in terms of the degree, the coefficients andthe minimum of the form on the simplex.

In 2006, in joint work with Reznick, we generalized Polya’sTheorem with bound to forms which are allowed to have aspecial type of zero at the corners.

In 2012, in joint work with M. Castle and Reznick, we gave acomplete characterization of forms p such that p is Polyasemi-positive, i.e., for large enough N ∈ N,(X1 + · · ·+ Xn)Np ∈ R+[X ], along with a bound on the Nneeded.

If p has zeros on ∆n, then (X1 + · · ·+ Xn)Np will always havesome zero coefficients.



For α = (α1, . . . , αn) ∈ Nn, let |α| = α1 + · · ·+ αn. For α 6= 0,‖α‖ denotes the unit vector α

|α| ∈ ∆n.

Polya’s proof of his theorem is “coefficient by coefficient”:

For p of degree d such that p > 0 on ∆n and N = 0, 1, . . . , letε = 1

N+d ; then a sequence {pε} in R[X ] is constructed s.t. pε → p

uniformly on ∆n as N →∞ and the coefficient of Xα in (∑

Xi )Np

is a positive multiple (depending only on n and d) of pε(‖α‖). Forlarge enough N, pε(‖α‖) > 0.

We obtained“local” versions of the theorem, i.e., results forcoefficients which correspond to exponents α such that ‖α‖ lies ina given closed subset of ∆n.

To prove our main theorem, we write ∆n as a union of closedsubsets so that we can apply one of the local versions to each ofthe subsets.



Application: Copositive programming

Copositive programming is linear programming over the cone ofcopositive matrices:

Cn = {n×n symmetric matrices M | x ·M ·xT ≥ 0, for all x ∈ Rn+}.

For r ∈ N, let Crn denote the set of symmetric matrices M such that(∑Xi

)rX ·M · XT has nonnegative coefficients,

where X = (X1, . . . ,Xn).

By Polya’s Theorem, Crn converges to Cn as r →∞.



In 2001, De Klerk & Pasechnik showed thatthe stability number of a graph can be writtenas the solution to a copositive programmingproblem.

They approximated the copositive programming problem using thecones Crn and, using the degree bounds for Polya’s Theorem, gaveresults for approximating the stability number of a graph.



Finding SOS decompositions

For a given sos f ∈ R[X ], thanks to the magicof linear algebra, there are effective algorithmsfor writing f as a sum of squares.

Suppose deg f = 2d and

f =k∑

i=1

gi (x)2 (1)

The linear algebra version of (1) is

f (X ) = V · BBT · V T ,

V = monomials of deg ≤ d , and B is the matrix with i-th columnthe coefficients of gi .

A = B · BT is psd symmetric matrix, called a Gram matrix for fcorresponding to (1).



For example, p = x4 + y4 + 1 = (x2)2 + (y2)2 + 12.With V =

[x2 y2 xy x y 1

],

p = V ·

1 0 0 0 0 00 1 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 1

· VT

But also p = (x2 − y2)2 + 2(xy)2 + (1)2 =

V ·

1 −1 0 0 0 0−1 1 0 0 0 00 0 2 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 1

· VT



“f is sos” is equivalent to the existence of a Gram matrix for f ,i.e., a psd symmetric matrix A such that

f (X ) = V · A · V T , (2)

This was proven by Choi, Lam, and Reznick who also noted that:

Inequivalent sos representations of f correspond to differentGram matrices

If A is a Gram matrix for f then rank A is the number ofsquares in the sos representation of f corresponding to A.

Further, there is an easy algorithm for producing an explicitdecomposition f =

∑g2i from a Gram matrix for f .



Suppose f ∈ R[X ] is sos with deg f = 2d , and A is a Gram matrixfor f . Then A is an N × N matrix, where N =

(n+dd

).

The set of all symmetric N × N matrices A such that

f (X ) = V T · A · V (3)

is an affine subset L of the space of N × N symmetric matrices.

f is sos iff L ∩ PN 6= ∅, where PN is the convex cone of psdsymmetric N × N matrices over R.

Finding this intersection is a semidefinite program (SDP). Thereare good numerical algorithms – and software! – for solvingsemidefinite programs. There is even software for deciding if f issos: SOSTools (P. Parrilo et. al.)



Certificates of positivity on semialgebraic sets

A semialgebraic set in Rn is a subset defined by finitely manypolynomial inequalities.

For G = {g1, . . . , gr} ⊆ R[X ], S(G ) denotes the basic closedsemialgebraic set generated by G :

S(G ) = {α ∈ Rn | g1(α) ≥ 0, . . . , gr (α) ≥ 0}.

Question: Suppose f ≥ 0 or f > 0 on S(G ), does there exist acertificate of positivity for f on S(G )?

Answer: If S(G ) is compact and f > 0 on S(G ), yes!



Fix S = S(G ), there are two algebraic objects associated to G :

The quadratic module generated by G ,

Q(G ) := {s0 + s1g1 + · · ·+ srgr | each si ∈∑

R[X ]2}.

The preorder generated by G ,

P(G ) := {∑

ε∈{0,1}nsε g

ε11 . . . g εrr | sε ∈

∑R[X ]2}.

A representation of f in either Q(G ) or P(G ) yields a certificate ofpositivity for f on S .



Fix a basic closed semialgebraic set S = S(G ), the correspondingquadratic module Q = Q(G ), and the corresponding preorderP = P(G ).

Q is archimedean if there is N ∈ N such that N −∑

X 2i ∈ Q. A

similar definition holds for P.

If Q is archimedean, then S must be compact, and N −∑

X 2i ∈ Q

is a “certificate of compactness” for S . The converse is not true ingeneral.

However, S compact implies that P is archimedean. This followsfrom work of K. Schmudgen; there is a nice algebraic proof due toT. Wormann.



We have the following (beautiful!) theorems:

Theorem (Putinar)

If Q is archimedean and f > 0 on S , then f ∈ Q.

Theorem (Schmudgen)

If S is compact, then f > 0 on S implies f ∈ P.

This means that if S is compact, then there exists a certificate ofpositivity for every f > 0 on S .



Application to Optimization

Problem: Find inf{f (α) | α ∈ Rn} (if it exists).

Equivalent to finding the inf of {λ ∈ R | f − λ is psd }.

We can look for the smallest λ so that f − λ is sos:

fsos := min{λ | f − λ ∈∑

R2}. (4)

Then fsos is a lower bound for the minimum of f .

Of course, fsos = −∞ is possible: psd 6⇒ sos!

If fsos 6= −∞, then there is an a priori bound on the degree of thesquares that occur and finding fsos can be implemented as an SDP.



Optimizing on compact semialgebraic sets

Suppose S = S(g1, . . . , gk) is a compact semialgebraic set and Qthe quadratic module. Assume Q is archimedean, so Putinar’sTheorem applies.

For f ∈ R[X ], let f∗ denote the minimum of f on S (which alwaysexists, since S is compact). Let

fsos := min{λ | f − λ ∈ Q}.

As in the sos case, the problem of finding a representation of f − λin Q with a fixed upper bound on the degree of the sums ofsquares that occur can be implemented as an SDP.

Unfortunately, there cannot be any such bound in terms of deg fonly – any bound must depend on some measure of how close f isto having a zero on Q.



The Lasserre Method

Unlike the global optimization case, fsos always exists.

But we cannot implement finding fsos as a SDP directly.

Let Qa := {s0 + g1s1 + · · ·+ skgk with si ∈∑

R[X ]2,deg si ≤ 2a}.

J.-P. Lasserre developed a method for approximating f∗ byimplementing a series of SDP’s corresponding to finding

f asos := min{λ | f − λ ∈ Qa}.

By Putinar’s Theorem, the values of f asos converge to f∗.

We can ensure that Q is archimedean by adding a generator of theform N −

∑X 2i .



Rational sums of squares

Consider the following example, due to C. Hillar:

Is f = 3− 12x − 6x3 + 18y2 + 3x6 + 12x3y − 6xy3 + 6x2y4 sos?

Our SDP program might say yes and return a certificate:

f = (x3 + 3.53y + .347xy2 − 1)2 + (x3 + .12y + 1.53xy2 − 1)2+

(x3 + 2.35y − 1.88xy2 − 1)2

Note that f− RHS has terms such as −.006xy2 which are nonzero.Is f really a sum of squares?

It turns out that our SDP program has approximated an sosdecomposition for f of the form

(x3+a2y+bxy2−1)2+(x3+b2y+cxy2−1)2+(x3+c2y+axy2−1)2,

where a, b, c are real roots of the cubic x3 − 3x + 1.Vicki Powers Certificates of Positivity: Theory, Practice, and Applications


For many applications of certificates of positivity, we need exactcertificates, but SDP’s are numerical.

In theory, SDP problems can be solved purely algebraically, e.g.,using quantifier elimination. In practice, this is impossible for allbut trivial problems.

Work by J. Nie, K. Ranestand, and B. Sturmfels in 2010 showsthat optimal solutions of relatively small SDP’s can have minimumdefining polynomials of huge degree.

Much more promising are hybrid symbolic-numeric approaches.



Problem: Given a numerical (approximate) certificate f =∑

g2i

(via SDP software, say) find an exact certificate.

Peyrl and Parrilo gave an algorithm for solving this problem, insome cases. The idea: We want to find a symmetric psd matrix Awith rational entries so that

f = V · A · V T (5)

The SDP software will produce a psd matrix A which onlyapproximately satisfies (5). The idea is to project A onto the affinespace of solutions to (5) in such a way that the projection remainsin the cone of psd symmetric matrices.



The Peyrl-Parrilo method is (in theory!)guaranteed to work IF there exists a rationalsolution and the underlying SDP is strictlyfeasible, i.e., there is a solution with full rank.

In 2009, E. Kaltofen, B. Li, Z. Yang, and L. Zhi generalized thetechnique of Peyrl and Parrilo and found sos certificates certifyingrational lower bounds for some approximate polynomial g.c.d. andirreducibility problems.



Rational sums of squares decompositions

The above algorithms assume that there is a rational sos certificatefor f . But maybe one doesn’t exist, even if f has rationalcoefficients.

Suppose f ∈ Q[X ] is in∑

R[X ]2.

Sturmfels asked: Is f ∈∑

Q[X ]2?

A trivial, but illustrative example:

2x2 = (√

2x)2 ∈∑

R[x ]2

2x2 = x2 + x2 ∈∑

Q[x ]2



Recall the Hillar example

f = 3− 12x − 6x3 + 18y2 + 3x6 + 12x3y − 6xy3 + 6x2y4,

which is in∑

R[X ]2.

It turns out that f ∈∑

Q[X ]2:

f = (x3 + xy2 +3

2y − 1)2 + (x3 + 2y − 1)2+

(x3 − xy2 +5

2y − 1)2 + (2y − xy2)2 +

3

2y2 + 3x2y4

In both of these examples, the sos decompositions in∑

Q[X ]2 usemore squares than the ones in

∑R[X ]2.



Sturmfels question: If f ∈ Q[X ] and f ∈∑

R[X ]2, isf ∈

∑Q[X ]2?

In the univariate case, the answer is “yes” (Landau, Porchet,Schweighofer) and at most 5 squares are needed (Porchet).

Hillar showed that the answer to Sturmfel’s question is “yes”if f ∈

∑K 2, where K is a totally real extension of Q. He

gave bounds for the number of squares needed.

There is a simple proof of a slightly more general result with abetter bound given (independently) by Kaltofen, Scheiderer,and Quarez.

Recently, Scheiderer showed that the answer is “no”, ingeneral.



Scheiderer showed that Q[X ] ∩∑

R[X ]2 6=∑

Q[X ]2. Even iff ∈

∑R[X ]2, we might not be able to find a certificate using

software.

But there is still a problem: As Hilbert proved, f ≥ 0 on Rn doesnot imply that f is sos. So there might not exist any certificate ofpositivity for f if we insist on sums of squares of polynomials.

But recall Artin’s Theorem: Every psd f can be written as aquotient g/h where g , h ∈

∑R[X ]2.

Even better, Artin’s proof shows that if f ∈ Q[X ] is psd, thenthere always exist g , h ∈

∑Q[X ]2 such that f = g/h. The

rationality question is not an issue!



Recent work of Kaltofen, Li, Yang, and Zhi turns Artin’s theoreminto a symbolic-numeric algorithm for for finding certificates ofpositivity for any psd f ∈ Q[X ].

The algorithm finds a numerical representation of f as a quotientg/h, where g and h are sos, and then converts this to an exactrational identity.

There is even software! ArtinProver.

The authors have used their techniques and the software to settlethe dimension 4 case of the Monotone Column PermanentConjecture.



Back to the compact semialgebraic sets

What can we say about rational certificates of positivity in thecompact semialgebraic set case?

Fix G = {g1, . . . , gk} ⊆ R[X ] and let S = S(G ), the basic closedsemialgebraic set generated by G , Q = Q(G ), the quadraticmodule generated by G , and P = P(G ), the preorder generated byG .

Question: Suppose G ⊆ Q[X ], f ∈ Q[X ], and f > 0 on S . Doesthere exist a rational certificate of positivity for f on S?



Theorem (P,2011)

Suppose S = S(G ) is compact. Then for any f ∈ Q[X ] such thatf > 0 on S , there is a representation of f in the preordering P(G ),

f =∑

e∈{0,1}sσeg

e11 . . . g es

s ,

withall σe ∈

∑Q[X ]2.

The proof follows from an algebraic proof of Schmudgen’sTheorem, due to T. Wormann.



Theorem (P,2011)

Suppose G = {g1, . . . , gs} ⊆ Q[X ] and N −∑

X 2i ∈ Q for some

N ∈ N. Then given any f ∈ Q[X ] such that f > 0 on S , thereexist σ0 . . . σs , σ ∈

∑Q[X ]2 so that

f = σ0 + σ1g1 + · · ·+ σsgs + σ(N −∑

X 2i ).

This follows from an algorithmic proof of Putinar’s Theorem dueto Schweighofer, which involves reducing to Polya’s Theorem. Wefollow the proof, making sure each step preserves rationality.

Open question: If G ⊆ Q[X ] and N −∑

X 2i ∈ Q, can we find a

certificate of positivity for N −∑

X 2i in Q with so that the sos’s

are in∑

Q[X ]2?



Other topics

There are many topics that weren’t discussed.

Certificates of positivity for noncompact semialgebraic sets.

Algorithmic and complexity issues

Noncommutative sums of squares

Applications:

Max CutGeometric theorem provingLyapunov functions

...



Thanks for listening!


certificates of positivity: theory, practice, and...

Documents