geometric analysis and integral geometry: ams special session on radon transforms and geometric...

598

Geometric Analysisand Integral Geometry

AMS Special Sessionon Radon Transforms and Geometric Analysisin Honor of Sigurdur Helgason’s 85th Birthday

January 4–7, 2012Boston, MA

Tufts University Workshop on Geometric Analysison Euclidean and Homogeneous Spaces

January 8–9, 2012Medford, MA

Eric Todd QuintoFulton Gonzalez

Jens Gerlach ChristensenEditors

American Mathematical Society

598








American Mathematical SocietyProvidence, Rhode Island

EDITORIAL COMMITTEE

Dennis DeTurck, Managing Editor

Michael Loss Kailash Misra Martin J. Strauss

2010 Mathematics Subject Classification. Primary 22E30, 43A85, 44A12, 45Q05, 92C55;Secondary 22E46, 32L25, 35S30, 65R32.

Library of Congress Cataloging-in-Publication Data

AMS Special Session on Radon Transforms and Geometric Analysis (2012 : Boston, Mass.)Geometric analysis and integral geometry : AMS special session in honor of Sigurdur Helgason’s

85th birthday, radon transforms and geometric analysis, January 4-7, 2012, Boston, MA ; TuftsUniversity Workshop on Geometric Analysis on Euclidean and Homogeneous Spaces, January 8-9,2012, Medford, MA / Eric Todd Quinto, Fulton Gonzalez, Jens Gerlach Christensen, editors.

pages cm. – (Contemporary mathematics ; volume 598)Includes bibliographical references.ISBN 978-0-8218-8738-7 (alk. paper)1. Radon transforms–Congresses. 2. Integral geometry–Congresses. 3. Geometric analysis–

Congresses. I. Quinto, Eric Todd, 1951- editor of compilation. II. Gonzalez, Fulton, 1956- editorof compilation. III. Christensen, Jens Gerlach, 1975- editor of compilation. IV. Tufts University.Workshop on Geometric Analysis on Euclidean and Homogeneous Spaces (2012 : Medford, Mass.)V. Title.

QA672.A4726 2012515′.1–dc23 2013013624

Contemporary Mathematics ISSN: 0271-4132 (print); ISSN: 1098-3627 (online)

DOI: http://dx.doi.org/10.1090/conm/598

Copying and reprinting. Material in this book may be reproduced by any means for edu-cational and scientific purposes without fee or permission with the exception of reproduction byservices that collect fees for delivery of documents and provided that the customary acknowledg-ment of the source is given. This consent does not extend to other kinds of copying for generaldistribution, for advertising or promotional purposes, or for resale. Requests for permission forcommercial use of material should be addressed to the Acquisitions Department, American Math-ematical Society, 201 Charles Street, Providence, Rhode Island 02904-2294, USA. Requests canalso be made by e-mail to [email protected].

Excluded from these provisions is material in articles for which the author holds copyright. Insuch cases, requests for permission to use or reprint should be addressed directly to the author(s).(Copyright ownership is indicated in the notice in the lower right-hand corner of the first page ofeach article.)

c© 2013 by the American Mathematical Society. All rights reserved.The American Mathematical Society retains all rightsexcept those granted to the United States Government.

Copyright of individual articles may revert to the public domain 28 yearsafter publication. Contact the AMS for copyright status of individual articles.

Printed in the United States of America.

©∞ The paper used in this book is acid-free and falls within the guidelinesestablished to ensure permanence and durability.

Visit the AMS home page at http://www.ams.org/

10 9 8 7 6 5 4 3 2 1 13 12 11 10 09 08

This volume is dedicated to Sigurdur Helgasonwhose mathematics has inspired many.

v

Contents

Preface ix

List of Presenters xi

Historical Articles

Some Personal Remarks on the Radon TransformSigurdur Helgason 3

On the Life and Work of S. HelgasonG. Olafsson and R. J. Stanton 21

Research and Expository Articles

Microlocal Analysis of an Ultrasound Transform with Circular Source andReceiver Trajectories

G. Ambartsoumian, J. Boman, V. P. Krishnan,

and E. T. Quinto 45

Cuspidal discrete series for projective hyperbolic spacesNils Byrial Andersen and Mogens Flensted-Jensen 59

The Radon transform on SO(3): motivations, generalizations, discretizationSwanhild Bernstein and Isaac Z. Pesenson 77

Atomic decompositions of Besov spaces related to symmetric conesJens Gerlach Christensen 97

A double fibration transform for complex projective spaceMichael Eastwood 111

Magnetic Schrodinger equation on compact symmetric spaces and the geodesicRadon transform of one forms

Tomoyuki Kakehi 129

F -method for constructing equivariant differential operatorsToshiyuki Kobayashi 139

Schiffer’s Conjecture, Interior Transmission Eigenvalues and InvisibilityCloaking: Singular Problem vs. Nonsingular Problem

Hongyu Liu 147

vii

viii CONTENTS

Approximate Reconstruction from Circular and Spherical Mean RadonTransform Data

W. R. Madych 155

Analytic and Group-Theoretic Aspects of the Cosine TransformG. Olafsson, A. Pasquale, and B. Rubin 167

Quantization of linear algebra and its application to integral geometryHiroshi Oda and Toshio Oshima 189

Mean value theorems on symmetric spacesFrancois Rouviere 209

Semyanistyi fractional integrals and Radon transformsB. Rubin 221

Radon-Penrose transform between symmetric spacesHideko Sekiguchi 239

Principal series representations of infinite dimensional Lie groups, II:Construction of induced representations

Joseph A. Wolf 257

Preface

Geometric analysis on Euclidean and homogeneous spaces encompasses partsof representation theory, integral geometry, harmonic analysis, microlocal analysis,and partial differential equations. It is used in a wide array of applications in fieldsas diverse as inverse problems, tomography, and signal and data analysis. Thisvolume provides articles giving historical perspectives, overviews of current researchin these interrelated areas, and new results. We hope this motivates beginningresearchers in these fields, and we wish that readers will be left with a good senseof important past work as well as current research in these exciting and active fieldsof mathematics.

One theme of the volume is the geometric analysis motivated by the work ofSigurdur Helgason. An historical perspective is provided in the first article by Prof.Helgason himself and in the second article by Profs. Olafsson and Stanton. Thisemphasis is natural, since the volume is based, in part, on the AMS Special Sessionon Radon Transforms and Geometric Analysis in honor of Sigurdur Helgason’s85th birthday held in Boston during the 2012 AMS Annual Meeting. Researchpapers related to this viewpoint, in particular, on Radon transforms and relatedmathematics are presented by Bernstein & Pesenson, Kakehi, Madych, Olafsson,Pasquale & Rubin, Rouviere, Rubin, and others.

The workshop on Geometric analysis on Euclidean and homogeneous spaces,held at Tufts University immediately following the AMS annual meeting, sought toexpand on the topics presented at the special session. It was broader in scope, asevidenced by the contributions to this volume.

Among contributions in pure mathematics are articles on representation the-ory and equivariant differential operators (Kobayashi and Oda & Oshima), Pen-rose transforms (Eastwood and Sekiguchi), wavelets related to symmetric cones(Christensen), representation theory and inductive limits of Lie groups (Wolf), andnoncommutative harmonic analysis (Andersen & Flensted-Jensen).

The interplay between integral geometry and applications is explored in themore applied articles. These include developing an elliptical Radon transform forultrasound (Ambartsoumian, Boman, Krishnan & Quinto), using Schiffer’s conjec-ture to understand partial cloaking (Liu), and Radon transforms in crystallography(Bernstein & Pesenson) and thermoacoustic tomography (Madych).

We thank the U.S. National Science Foundation and the Tufts University Dean’sDiscretionary Fund for their support of the Tufts workshop. We are grateful to TuftsUniversity Staff Assistant Megan Monaghan for the work she did behind the scenesto make the workshop successful.

ix

x PREFACE

We thank the American Mathematical Society for its support of the SpecialSession honoring Sigurdur Helgason, and finally, we are indebted to Christine M.Thivierge, Associate Editor for Proceedings, for her indispensable role in makingthese proceedings a success.

List of Presenters

Radon Transforms and Geometric Analysisin Honor of Sigurdur Helgason’s 85th Birthday

American Mathematical Society National Meeting

January 4-7 2012, Boston, MA

Speakers

Mark Agranovsky (Bar-Ilan University): Abel-Radon transform and CRfunctions.

Nils Byrial Andersen, (Aarhus University, Denmark): Cusp Forms onhyperbolic spaces.

Jan Boman (Stockholm University): Local injectivity of weighted Radontransforms.

Jens Gerlach Christensen (Tufts University): Decomposition of spaces ofdistributions using Garding vectors.

Susanna Dann (University of Missouri): Paley-Wiener Theorems on Rn withrespect to the spectral parameter.

Victor Guillemin (MIT): Characters of group representations and semi-classicalanalysis.

Sigurdur Helgason (MIT): Orbital Integrals, applications and problems.

Tomoyuki Kakehi (Okayama University): Schroedinger equation on certaincompact symmetric spaces.

Adam Koranyi (H. H. Lehman College, CUNY): Twisted Poisson integralson bounded symmetric domains.

Job J. Kuit (University of Copenhagen): Radon transformation on reductivesymmetric spaces: support theorems.

Gestur Olafsson (Louisiana State University): The cosλ-transform andintertwining operators for SL(n,F).Bent Ørsted (Aarhus University): Segal-Bargmann transforms: Old and new.

Eyvindur Ari Palsson (University of Rochester): On multilinear generalizedRadon transforms.

Angela Pasquale (Universite Paul Verlaine - Metz): The bounded hyperge-ometric functions associated with root systems.

xi

xii LIST OF PRESENTERS

Isaac Z. Pesenson (Temple University): Splines for Radon transform oncompact Lie groups with application to SO(3).

Francois Rouviere (Universite de Nice): Mean value theorems on symmetricspaces.

Boris Rubin (Louisiana State University): A Generalization of the Mader-Helgason Inversion Formulas for Radon Transforms.

Henrik Schlichtkrull (University of Copenhagen): Counting lattice pointson homogeneous spaces.

Hideko Sekiguchi (The University of Tokyo): Penrose transforms betweensymmetric spaces.

Robert J. Stanton (Ohio State University): Special geometries arising fromsome special symmetric spaces.

Erik P. van den Ban (Utrecht University): Cusp forms for semisimplesymmetric spaces.

Joseph A. Wolf (University of California at Berkeley): Range of the DoubleFibration Transform.

Workshop on Geometric Analysis on Euclidean and HomogeneousSpaces

Tufts University

January 8-9 2012

Speakers

Gaik Ambartsoumian (The University of Texas at Arlington): Exact In-version of Ultrasound Operators in the Spherical Geometry.

Michael Eastwood (Australian National University): The Penrose trans-form for complex projective space.

Suresh Eswarathasan (University of Rochester): Eigenfunction supremumbounds for deformations of Schrodinger operators.

Fulton Gonzalez (Tufts University): Multitemporal Wave equations: MeanValue Solutions.

Eric Grinberg (University of Massachusetts, Boston): Admissible and in-admissible complexes in integral geometry.

Yulia Hristova (IMA University of Minnesota): Detection of low emissionradiating sources using direction sensitive detectors.

Alexander Iosevich (University of Rochester): Multi-linear generalized Radontransforms and applications to geometric measure theory and related areas.

Hiroshi Isozaki (University of Tsukuba): Inverse scattering on a generalizedarithmetic surface.

Toshiyuki Kobayashi (IPMU and University of Tokyo): Conformally Equi-variant Differential Operators and Branching Problems of Verma Modules.

LIST OF PRESENTERS xiii

Alexander Koldobsky (University of Missouri): Stability in volume compar-ison problems.

Peter Kuchment (Texas A&M): Integral geometry and microlocal analysis inthe hybrid imaging.

Venkateswaran Krishnan (Tata Institute of Fundamental Research): Aclass of singular Fourier integral operators in synthetic aperture radar imaging.

Hongyu Liu (University of California, Irvine): Enhanced Near-cloak by FSHLining.

Wolodymyr Madych (University of Connecticut): Approximate reconstruc-tion from circular and spherical mean Radon transform data.

Yutaka Matsui (Kinki University): Topological Radon transforms and theirapplications.

Tai Melcher (University of Virginia): A quasi-invariance result for heatkernel measures on infinite-dimensional Heisenberg groups.

Linh Nguyen (University of Idaho): Range description for a spherical meantransform on spaces of constant curvatures.

Hiroyuki Ochiai (Kyushu University): Positivity of an alpha determinant.

Gestur Olafsson (Louisiana State University): Spherical functions on limitsof compact symmetric spaces.

Toshio Oshima (University of Tokyo): Generalizations of Radon transformson compact homogeneous spaces.

Angela Pasquale (University of Metz): Uncertainty principles for the Schro-dinger equation on Riemannian symmetric spaces of the noncompact type.

Isaac Z. Pesenson (Temple University): Band-limited Localized tight frameson Compact Homogeneous Manifolds.

Mark A. Pinsky (Northwestern University): Can you feel the shape of amanifold with Brownian motion.

Todd Quinto (Tufts University): Algorithms in bistatic ultrasound.

Boris Rubin (Louisiana State University): Inversion Formulas for SphericalMeans in Constant Curvature Spaces.

Henrik Schlichtkrull (University of Copenhagen): A uniform bound on thematrix elements of the irreducible representations of SU(2).

Plamen Stefanov (Purdue University): The Identification problem in SPECT:uniqueness, non-uniqueness and stability.

Dustin Steinhauer (Texas A&M): Inverse Problems in Medical Imaging withInternal Information.

Gunther Uhlmann (University of Washington): Travel Time Tomographyand Tensor Tomography.

Jim Vargo (Texas A&M): The Spherical Mean Problem.

Ting Zhou (MIT): On approximate cloaking by nonsingular transformation me-dia.

xiv LIST OF PRESENTERS

Graduate Student Posters

Matthew Dawson (Louisiana State University): Direct Systems of SphericalRepresentations and Compact Riemannian Symmetric Spaces.

Daniel Fresen (University of Missouri): Concentration inequalities for ran-dom polytopes.

Vivian Ho (Louisiana State University): Paley-Wiener Theorem for LineBundles over Compact Symmetric Spaces.

Koichi Kaizuka (University of Tsukuba): Uniform resolvent estimates onsymmetric spaces of noncompact type.

Toshihisa Kubo (Oklahoma State University): Systems of second-orderinvariant differential operators of non-Heisenberg parabolic type.

Kyung-Taek Lim (Tufts University): Surjectivity and range description ofthe single radius spherical mean on Euclidean space.

Carlos Montalto (Purdue University): Stable determination of generic simplemetrics, vector field and potential from the hyperbolic Dirichlet-to-Neumann map.

Vignon Oussa (Saint Louis University): Bandlimited Spaces on Some 2-stepNilpotent Lie Groups With One Parseval Frame Generator.

Patrick Spencer (University of Missouri): Lorentz Balls Are Not IntersectionBodies.

Abstracts and coauthors, if any, can be found at the following URLs

The AMS special session:http://jointmathematicsmeetings.org/

meetings/national/jmm2012/2138 program ss17.html#title

The Tufts University workshop:http://equinto.math.tufts.edu/workshop2012/at.pdf

or from the proceedings editors.

Historical Articles

Contemporary MathematicsVolume 598, 2013http://dx.doi.org/10.1090/conm/598/12000

Some personal remarks on the Radon transform

Sigurdur Helgason

1. Introduction.

The editors have kindly asked me to write here a personal account of some ofmy work concerning the Radon transform. My interest in the subject was actuallyevoked during a train trip from New York to Boston once during the Spring 1955.

2. Some old times.

Back in 1955, I worked on extending the mean value theorem of Leifur Asgeirs-son [1937] for the ultrahyperbolic equation onRn×Rn to Riemannian homogeneousspaces G/K × G/K. I was motivated by Godement’s generalization [1952] of themean value theorem for Laplace’s equation Lu = 0 to the system Du = 0 for allG-invariant differential operators D (annihilating the constants) on G/K. At thetime (Spring 1955) I visited Leifur in New Rochelle where he was living in the houseof Fritz John (then on leave from NYU). They had both been students of Courant

in Gottingen in the 1930’s. Since John’s book [1955] treats Asgeirsson’s theoremin some detail, Leifur lent me a copy of it (in page proofs) to look through on thetrain to Boston.

I was quickly enticed by Radon’s formulas (in John’s formulation) for a func-tion f on Rn in terms of its integrals over hyperplanes. In John’s notation, letJ(ω, p) denote the integral of f over the hyperplane 〈ω, x〉 = p (p ∈ R, ω a unitvector), dω the area element on Sn−1 and L the Laplacian. Then

f(x) = 12 (2πi)

1−n(Lx)(n−1)/2

∫Sn−1

J(ω, 〈ω, x〉) dw , n odd.(2.1)

f(x) = (2πi)−n(Lx)(n−2)/2

∫Sn−1

dω

∫R

dJ(ω, p)

p− 〈ω, x〉 , n even.(2.2)

I was surprised at never having seen such formulas before. Radon’s paper[1917] was very little known, being published in a journal that was hard to find.The paper includes some suggestions by Herglotz in Leipzig and John learned of itfrom lectures by Herglotz in Gottingen. I did not see Radon’s paper until severalyears after the appearance of John’s book but it has now been reproduced in severalbooks about the Radon transform (terminology introduced by F. John). Actually,the paper is closely related to earlier papers by P. Funk [1913, 1916] (quoted in

2010 Mathematics Subject Classification. Primary 43A85, 53 C35, 22E46, 44A12; Secondary53 C65, 14 M17, 22 F30.

c©2013 American Mathematical Society

3

4 SIGURDUR HELGASON

Radon [1917]) which deal with functions on S2 in terms of their integrals over greatcircles. Funk’s papers are in turn related to a paper by Minkowski [1911] aboutsurfaces of constant width.

3. General viewpoint. Double fibration transform.

Considering formula (2.1) for R3,

(3.1) f(x) = − 1

8π2Lx

(∫S2

J(ω, 〈ω, x〉 dω))

it struck me that the formula involves two dual integrations, J the integral over theset of points in a plane and then dω, the integral over the set of planes through a

point. This suggested the operators f → f , ϕ→ ϕ∨defined as follows:

For functions f on R3, ϕ on P3 (the space of 2-planes in R3) put

f(ξ) =

∫ξ

f(x) dm(x) , ξ ∈ P3 ,(3.2)

ϕ∨(x) =

∫ξ�x

ϕ(ξ) dμ(ξ) , x ∈ R3 ,(3.3)

where dm is the Lebesgue measure and dμ the average over all hyperplanes con-taining x. Then (3.1) can be rewritten

(3.4) f = − 12L((f)

∨) .

The spaces R3 and P3 are homogeneous spaces of the same group M(3), the groupof isometries of R3, in fact

R3 = M(3)/O(3) , P3 = M(3)/Z2M(2) .

The operators f → f , ϕ → ϕ∨

in (3.2)–(3.3) now generalize ([1965a], [1966]) tohomogeneous spaces

(3.5) X = G/K , Ξ = G/H ,

f and ϕ being functions on X and Ξ, respectively, by

(3.6) f(γH) =

∫H/L

f(γhK) dhL , ϕ∨(gK) =

∫K/L

ϕ(gkH) dkL .

Here G is an arbitrary locally compact group, K and H closed subgroups, L =K ∩ H, and dhL and dkL the essentially unique invariant measure on H/L andK/L, respectively. This is the abstract Radon transform for the double fibration:

(3.7) X = G/K Ξ = G/H

G/L

��

��

The operators f → f , ϕ→ ϕ∨map functions on X to functions on Ξ and vice-

versa. These geometrically dual operators also turned out to be adjoint operatorsrelative to the invariant measures dx and dξ,

(3.8)

∫X

f(x)ϕ∨(x) dx =

∫Ξ

f(ξ)ϕ(ξ) dξ .

PERSONAL REMARKS 5

This suggests natural extensions to suitable distributions T on X, Σ on Ξ, asfollows:

T (ϕ) = T (ϕ∨) , Σ

∨(f) = Σ(f) .

Formulas (2.1) and (2.2) have another interesting feature. As functions of x theintegrands are plane waves, i.e. constant on each hyperplane L perpendicular to ω.Such a function is really just a function of a single variable so (2.1) and (2.2) canbe viewed as a decomposition of an n-variable function into one-variable functions.This feature enters into the work of Herglotz and John [1955]. I have found someapplications of an analog of this principle for invariant differential equations onsymmetric spaces ([1963], §7, [2008],Ch. V§1, No. 4), where parallel planes arereplaced by parallel horocycles.

The setup (3.5) and (3.6) above has of course an unlimited supply of examples.Funk’s example

(3.9) X = S2 , Ξ = {great circles on S2}

fits in, both X and Ξ being homogeneous spaces of O(3).Note that with given X and Ξ there are several choices for K and H. For

example, if X = Rn we can take K and H, respectively, as the isotropy groups ofthe origin O and a k-plane ξ at distance p from O. Then the second transform in(3.6) becomes

(3.10) ϕ∨p (x) =

∫d(x,ξ)=p

ϕ(ξ) dμ(ξ)

and we get another inversion formula (cf. [1990], [2011]) of f → f involving (f)∨p (x),different from (3.4).

Similarly, for X and Ξ in (3.9) we can take K as the isotropy group of theNorth Pole N and H as the isotropy group of a great circle at distance p from N .

Then (f)∨p (x) is the average of the integrals of f over the geodesics at distance pfrom x.

The principal problems for the operators f → f , ϕ→ ϕ∨would be

A. Injectivity.B. Inversion formulas.C. Ranges and kernels for specific function spaces of X and on Ξ.

D. Support problems (does compact support of f imply compact supportof f?)

These problems are treated for a number of old and new examples in [2011].Some unexpected analogies emerge, for example a complete parallel between thePoisson integral in the unit disk and the X-ray transform in R3, see pp. 86–89, loc.cit..

My first example for (3.5) and (3.6) was for X a Riemannian manifold ofconstant sectional curvature c and dimension n and Ξ the set of k-dimensionaltotally geodesic submanifolds of X. The solution to Problem B is then given [1959]by the following result, analogous to (3.4):

For k even let Qk denote the polynomial

Qk(x) = [x− c(k − 1)(n− k)][x− c(k − 3)(n− k + 2)] · · · [x− c(1)(n− 2)] .

6 SIGURDUR HELGASON

Then for a constant γ

Qk(L)((f)∨) = γf if X is noncompact.(3.11)

Qk(L)((f)∨) = γ(f + f ◦A) if X is compact.(3.12)

In the latter case X = Sn and A the antipodal mapping. The constant γ is givenby

Γ(n2

)/Γ

(n− k

2

)(−4π)k/2 .

The proof used the generalization of Asgeirsson’s theorem. At that time I had alsoproved an inversion formula for k odd and the method used in the proof of thesupport theorem for Hn ([1964, Theorem 5.2]) but not published until [1990] sincethe formula seemed unreasonably complicated in comparison to (3.11), (3.12) and

since the case k = 1 when f → f is the ”X-ray transform”, had not reached itsdistinction through tomography.

For k odd the inversion formula is a combination of f and the analog of (3.10).For X = Rn, Ξ = Pn, problems C–D are dealt with in [1965a]. This paper

also solves problem B for X any compact two-point homogeneous space and Ξ thefamily of antipodal submanifolds.

4. Horocycle duality.

In the search of a Plancherel formula for simple Lie groups, Gelfand–Naimark[1957], Gelfand–Graev [1955] and Harish-Chandra [1954], [1957] showed that a func-tion f on G is explicitly determined by the integrals of f over translates of conjugacyclasses in G. This did not fit into the framework (3.5)(3.6) so using the Iwasawadecomposition G = KAN (K compact, A abelian, N nilpotent) I replaced the con-jugacy classes by their “projections” in the symmetric space G/K, and this leadsto the orbits of the conjugates gNg−1 in G/K. These orbits are the horocyclesin G/K. They occur in classical non-Euclidean geometry (where they carry a flatmetric) and for G complex are extensively discussed in Gelfand–Graev [1964].

For a general semisimple G, the action of G on the symmetric space G/Kturned out to permute the horocycle transitively with isotropy group MN , whereM is the centralizer of A in K [1963]. This leads (3.5) and (3.6) to the doublefibration

(4.1) X = G/K Ξ = G/MN

G/M

��

��

and for functions f on X, ϕ on Ξ, f(ξ) is the integral of f over a horocycle ξ

and ϕ∨(x) is the average of ϕ over the set of horocycles through x. My papers

[1963], [1964a], [1970] are devoted to a geometric examination of this duality andits implications for analysis, differential equations and representation theory. Thuswe have double coset space representations

(4.2) K\G/K ≈ A/W , MN\G/MN ≈W ×A

based on the Cartan and Bruhat decomposition of G , W denoting the Weyl group.

PERSONAL REMARKS 7

The finite-dimensional irreducible representations with a K-fixed vector turnout to be the same as those with an MN -fixed vector. This leads to simultaneousimbeddings of X and Ξ into the same vector space and the horocycles are certainplane sections with X in analogy with their flatness for Hn [2008,II,§4]. The set ofhighest restricted weights of these representations is the dual of the lattice ΣiZ

+βi

where β1, . . . , β� is the basis of the unmultipliable positive restricted roots.For the algebras D(X),D(Ξ), respectively D(A), of G-invariant (resp. A-

invariant) differential operators on X, Ξ and A we have the isomorphisms

(4.3) D(X) ≈ D(A)/W , D(Ξ) ≈ D(A) .

The first is a reformulation of Harish-Chandra’s homomorphism, the second comesfrom the fact that the G-action on the fibration of Ξ over K/M is fiber preserving

and generates a translation on each (vector) fiber. The transforms f → f , ϕ→ ϕ∨

intertwine the members of D(X) and D(Ξ). In particular, when the operator

f → f is specialized to K-invariant functions on X it furnishes a simultaneoustransmutation operator between D(X) and the set of W -invariants in D(A) [1964a,§2]. This property, combined with a surjectivity result of Hormander [1958] andLojasiewicz [1958] for tempered distributions on Rn, yields the result that eachD ∈ D(X) has a fundamental solution [1964a]. A more technical support theorem

in [1973] for f → f on G/K then implies the existence theorem that eachD ∈ D(X)is surjective on E(X) i.e. (C∞(X)). It is also surjective on the space D′

0(X) of K-finite distributions on X ([1976]) and on the space S′(X) of tempered distributionson X ([1973a]). The surjectivity on all of D′(X) however seems as yet unproved.

The method of [1973] also leads to a Paley–Wiener type Theorem for the horo-

cycle transform f → f , that is an internal description of the range D(X) . Theformulation is quite different from the analogous result for D(Rn) . Having provedthe latter result in the summer of 1963, I always remember when I presented it ina Fall class, because immediately afterwards I heard about John Kennedy’s assas-sination.

In analogy with (3.4), (3.11), the horocycle transform has an inversion formula.The parity difference in (2.1), (2.2) and (3.11), (3.12) now takes another form([1964], [1965b]):

If G has all its Cartan subgroups conjugate then

(4.4) f = �((f)∨) ,

where � is an explicit operator in D(X). Although this remains “formally valid”for general G with � replaced by a certain pseudo-differential operator, a betterform is

(4.5) f = (Λf)∨ ,

with Λ a certain pseudo-differential operator on Ξ. These operators are constructedfrom Harish-Chandra’s c-function for G. For G complex a formula related to (4.5)is stated in Gelfand and Graev [1964], §5.

As mentioned, f → f is injective and the range D(X)is explicitly determined.

On the other hand ϕ → ϕ∨

is surjective from C∞(Ξ) onto C∞(X) but has a bigdescribable kernel.

By definition, the spherical functions on X are the K-invariant eigenfunctionsof the operators in D(X). By analogy we define conical distribution on Ξ to be the

8 SIGURDUR HELGASON

MN -invariant eigendistributions of the operators inD(Ξ). While Harish-Chandra’sformula for the spherical functions parametrizes the set F of spherical functions by

(4.6) F ≈ a∗c/W

(where a∗c is the complex dual of the Lie algebra of A) the space Φ of conicaldistribution is “essentially” parametrized by

(4.7) Φ ≈ a∗c ×W .

Note the analogy of (4.6), (4.7), with (4.2). In more detail, the spherical functionsare given by

ϕλ(gK) =

∫K

e(iλ−ρ)(H(gk)) dk ,

where g = k expH(g)n in the Iwasawa decomposition, ρ and λ as in (5.3) and λunique mod W . On the other hand, the action of the group MNA on Ξ dividesit into |W | orbits Ξs and for λ ∈ a∗c a conical distribution is constructed withsupport in the closure of Ξs. The construction is done by a specific holomorphiccontinuaton. The identification in (4.7) from [1970] is complete except for certainsingular eigenvalues. For the case of G/K of rank one the full identification of(4.7) was completed by Men-Cheng Hu in his MIT thesis [1973]. Operating asconvolutions on K/M the conical distributions in (4.7) furnish the intertwiningoperators in the spherical principal series [1970, Ch. III, Theorem 6.1]. See alsoSchiffmann [1971], Theoreme 2.4 and Knapp-Stein [1971].

5. A Fourier transform on X.

Writing f(ω, p) for John’s J(ω, p) in (2.1) the Fourier transform f on Rn canbe written

(5.1) f(rω) =

∫Rn

f(x)e−ir〈x,ω〉 dx =

∫R

f(ω, p)e−irp dp ,

which is the one-dimensional Fourier transform of the Radon transform. The horo-cycle duality would call for an analogous Fourier transform on X.

The standard representation–theoretic Fourier transform on G,

F (π) =

∫G

F (x)π(x) dx

is unsuitable here because it assigns to F a family of operators in different Hilbertspaces. However, the inner product 〈x, ω〉 in (5.1) has a certain vector–valuedanalog for G/K, namely

(5.2) A(gK, kM) = A(k−1g) ,

where expA(g) is the A-component in the Iwasawa decomposition G = NAK.Writing for x ∈ X, b ∈ B = K/M ,

(5.3) eλ,b(x) = e(iλ+ρ)(A(x,b)) , λ ∈ a∗c , ρ(H) = 1

2 Tr (ad H|n)we define in [1965b] a Fourier transform,

(5.4) f(λ, b) =

∫X

f(x)e−λ,b(x) dx .

The analog of (5.1) is then

f(λ, kM) =

∫A

f(kaMN)e(−iλ+ρ)(log a) da .

PERSONAL REMARKS 9

The main theorems of the Fourier transform on Rn, the inversion formula, thePlancherel theorem (with range), the Paley–Wiener theorem, the Riemann–Lebesguelemma, have analogs for this transform ([1965b], [1973], [2005], [2008]). The inver-sion formula is based on the new identity

(5.5) ϕλ(g−1h) =

∫K

e(iλ+ρ)(A(kh))e(−iλ+ρ)(A(kg)) dk .

Some results have richer variations, like the range theorems for the variousSchwartz spaces Sp(X) ⊂ Lp(X) (Eguchi [1979]).

The analog of (5.4) for the compact dual symmetric space U/K was developedby Sherman [1977], [1990] on the basis of (5.5).

6. Joint Eigenspaces.

The Harish-Chandra formula for spherical functions can be written in the form

(6.1) ϕλ(x) =

∫B

e(iλ+ρ)(A(x,b)) db

with λ ∈ a∗c given mod W -invariance. These are the K-invariant joint eigenfunc-tions of the algebra D(X). The spaces

(6.2) Eλ(X) = {f ∈ E(X) :

∫K

f(gk · x) dk = f(g · o)ϕλ(x)}

were in [1962] characterized as the joint eigenspaces of the algebra D(X). Let Tλ

denote the natural representation of G on Eλ(X). Similarly, for λ ∈ a∗c the spaceD′

λ(Ξ) of distributions Ψ on Ξ given by

(6.3) Ψ(ϕ) =

∫K/M

(∫A

ϕ(kaMN)e(iλ+ρ)(log a) da)dS(kM)

is the general joint distribution eigenspace for the algebra D(Ξ). Here S runs

through all of D′(B). The dual map Ψ→ Ψ∨maps D′

λ(Ξ) into Eλ(X). In terms ofS this dual mapping amounts to the Poisson transform Pλ given by

(6.4) PλS(x) =

∫B

e(iλ+ρ)(A(x,b)) dS(b) .

By definition the Gamma function of X, ΓX(λ), is the denominator in the for-mula for c(λ)c(−λ) where c(λ) is the c-function of Harish-Chandra, Gindikin andKarpelevic. While ΓX(λ) is a product over all indivisible roots, Γ+

X(λ) is the prod-uct over just the positive ones. See [2008], p. 284. In [1976] it is proved that forλ ∈ a∗c ,

(i) Tλ is irreducible if and only if 1/ΓX(λ) �= 0.(ii) Pλ is injective if and only if 1/Γ+

X(λ) �= 0.(iii) Each K-finite joint eigenfunction of D(X) has the form

(6.5)

∫B

e(iλ+ρ)(A(x,b))F (b) db

for some λ ∈ a∗c and some K-finite function F on B.For X = Hn it was shown [1970, p.139, 1973b] that all eigenfunctions of the

Laplacian have the form (6.5) with F (b) db replaced by an analytic functional (hy-perfunction). This was a bit of a surprise since this concept was in very little use

10 SIGURDUR HELGASON

at the time. The proof yielded the same result for all X of rank one providedeigenvalue is ≥ −〈ρ, ρ〉. In particular, all harmonic functions on X have the form

(6.6) u(x) =

∫B

e2ρ(A(x,b)) dS(b) ,

where S is a hyperfunction on B.For X of arbitrary rank it was proved by Kashiwara, Kowata, Minemura,

Okamoto, Oshima and Tanaka that Pλ is surjective for 1/Γ+X(λ) �= 0 [1978]. In

particular, every joint eigenfunction has the form (6.4) for a suitable hyperfunctionS on B. The image under Pλ of various other spaces on B has been widely investi-gated, we just mention Furstenberg [1963], Karpelevic [1963], Lewis [1978], Oshima–Sekiguchi [1980], Wallach [1983], Ban–Schlichtkrull [1987], Okamoto [1971], Yang[1998].

For the compact dual symmetric space U/K the eigenspace representations areall irreducible and each joint eigenfunction is of the form (6.5) (cf. Helgason [1984]Ch. V, §4, in particular p. 542). Again this relies on (5.5).

7. The X-ray transform.

The X-ray transform f → f on a complete Riemannian manifold X is given by

(7.1) f(γ) =

∫γ

f(x) dm(x) , γ a geodesic,

f being a function on X. For the symmetric space X = G/K from §4, I showed in[1980] the injectivity and support theorem for (7.1) (problems A and D in §3). In[2006], Rouviere proved an explicit inversion formula for (7.1).

For a compact symmetric space X = U/K we assume X irreducible and simplyconnected. Here we modify (7.1) by restricting γ to be a closed geodesic of minimallength, and call the transform the Funk transform. All such geodesics are conjugateunder U (Helgason [1966a] so the family Ξ = {γ} has the form U/H and the Funktransform falls in the framework (3.7). The injectivity (for X �= Sn) was provedby Klein, Thorbergsson and Verhoczki [2009]; an inversion formula and a supporttheorem by the author [2007]. To each x ∈ X is associated the midpoint locus Ax

(the set of midpoints of minimal geodesics through x) as well as a corresponding“equator” Ex. Both of these are acted on transitively by the isotropy group of x.The inversion formula involves integrals over both Ax and Ex.

For a closed subgroup H ⊂ G, invariant under the Cartan involution θ of G(with fixed group K) Ishikawa [2003] investigated the double fibration (3.7). Theorbit HK is a totally geodesic submanifold of X so this generalizes the X-raytransform. For many cases of H, this new transform was found to be injective andto satisfy a support theorem.

For one variation of these questions see Frigyik, Stefanov and Uhlmann [2008].

8. Concluding remarks.

For the sake of unity and coherence, the account in the sections above has beenrather narrow and group-theory oriented. A satisfactory account of progress onProblems A, B, C, D in §3 would be rather overwhelming. My book [2011] with itsbibliographic notes and references is a modest attempt in this direction.

Here I restrict myself to the listing of topics in the field — followed by abibliography, hoping the titles will serve as a suggestive guide to the literature.

PERSONAL REMARKS 11

Some representative samples are mentioned. These samples are just meant to besuggestive, but I must apologize for the limited exhaustiveness.

(i) Topological properties of the Radon transform. Quinto [1981], Hertle[1984a].

(ii) Range questions for a variety of examples of X and Ξ. The first paperin this category is John [1938] treating the X-ray transform in R3. For Xthe set of k-planes in Rn the final version, following intermediary resultsby Helgason [1980b], Gelfand, Gindikin and Graev [1982], Richter [1986],[1990], Kurusa [1991], is in Gonzalez [1990b] where the range is the nullspace of an explicit 4th degree differential operator. Enormous progresshas been made for many examples. Remarkable analogies have emerged,Berenstein, Kurusa, Casadio Tarabusi [1997]. For Grassmann manifoldsand spheres see e.g. Kakehi [1993], Gonzalez and Kakehi [2004]. AlsoOshima [1996], Ishikawa [1997].

(iii) Inversion formulas. Here a great variety exists even within a singlepair (X,Ξ). Examples are Grassmann manifolds, compact and noncom-pact, X-ray transform on symmetric spaces (compact and noncompact).Antipodal manifolds on compact symmetric spaces. See e.g. Gonzalez[1984], Grinberg and Rubin [2004], Rouviere [2006], Helgason [1965a],[2007], Ishikawa [2003]. Techniques of fractional integrals. Injectivity sets.Admissible families. Goncharov [1989]. The kappa operator. See Ru-bin [1998], Gelfand, Gindikin and Graev [2003], Agranovsky and Quinto[1996], Grinberg [1994] and Rouviere [2008b].

(iv) Spherical transforms (spherical integrals with centers restricted to spe-cific sets). Range and support theorems. Use of microlocal analysis, Bo-man and Quinto[1987], Greenleaf and Uhlmann [1989], Frigyik, Stefanovand Uhlmann [2008]. Mean Value operator. Agranovsky, Kuchment andQuinto [2007], Agranovsky, Finch and Kuchment [2009], Rouviere [2012],Lim [2012].

(v) Attenuated X-ray transform. Hertle [1984b], Palamodov [1996], Nat-terer [2001], Novikov [1992].

(vi) Extensions to forms and vector bundles. Okamoto [1971], Gold-schmidt [1990].

(vii) Discrete Integral Geometry and Radon transforms. Selfridge andStraus [1958], Bolker [1987], Abouelaz and Ihsane [2008].

Hopefully the titles in the following bibliography will furnish helpful contactwith topics listed above.

Bibliography

Abouelaz, A. and Fourchi, O.E.

2001 Horocyclic Radon Transform on Damek-Ricci spaces, Bull. Polish Acad.Sci. 49 (2001), 107-140. MR1829783 (2003b:42023)

Abouelaz, A. and Ihsane, A.

2008 Diophantine Integral Geometry, Mediterr. J. Math. 5 (2008), 77–99.MR2406442 (2009b:44004)

Abouelaz, A. and Rouviere, F.

2011 Radon transform on the torus, Mediterr. J. Math. 8 (2011), 463–471.MR2860679 (2012j:53103)


Agranovsky, M.L., Finch, D. and Kuchment, P.

2009 Range conditions for a spherical mean transform, Invrse Probl. Imaging 3(2009), 373–382. MR2557910 (2010k:44005)

Agranovsky, M.L., Kuchment, P. and Quinto, E.T.

2007 Range descriptions for the spherical mean Radon transform, J. Funct. Anal.248 (2007), 344–386. MR2335579 (2009f:47070)

Agranovsky, M.L. and Quinto, E.T.

1996 Injectivity sets for the Radon transform over circles and complete systemsof radial functions, J. Funct. Anal. 139 (1996), 383–414. MR1402770(98g:58171)

Aguilar, V., Ehrenpreis, L., and Kuchment, P.

1996 Range conditions for the exponential Radon transform, J. d’Analyse Math.68 (1996), 1–13. MR1403248 (97g:44006)

Ambartsoumian, G. and Kuchment, P.

2006 A range description for the planar circular Radon transform, SIAM J.Math. Anal. 38 (2006), 681–692. MR2237167 (2007e:44005)

Andersson, L.-E.

1988 On the determination of a function from spherical averages, SIAM J. Math.Anal., 19 (1988), 214–232. MR924556 (89a:44005)

Antipov, Y.A. and Rudin, B.

2012 The generalized Mader’s inversion formula for the Radon transform, Trans.Amer. Math. Soc., (to appear).

Asgeirsson, L.

1937 Uber eine Mittelwertseigenschaft von Losungen homogener linearer par-tieller Differentialgleichungen 2. Ordnung mit konstanten Koefficienten,Math. Ann. 113 (1937), 321–346. MR1513094

Ban, van den, e.p. and Schlichtkrull, H.

1987 Asymptotic expansions and boundary values of eigenfunctions on a Rie-mannian symmetric space, J. Reine Angew. Math. 380 (1987), 108–165.MR916202 (89g:43010)

Berenstein, C.A., Kurusa, A., and Casadio Tarabusi, E.

1997 Radon transform on spaces of constant curvature, Proc. Amer. Math. Soc.125 (1997), 455–461. MR1350933 (97d:53074)

Berenstein, C.A. and Casadio Tarabusi, E.

1991 Inversion formulas for the k-dimensional Radon transform in real hyperbolicspaces, Duke Math. J. 62 (1991), 613–631. MR1104811 (93b:53056)

1992 Radon- and Riesz transform in real hyperbolic spaces, Contemp. Math. 140(1992), 1–18. MR1197583 (93j:44003)

1993 Range of the k-dimensional Radon transform in real hyperbolic spaces, Fo-rum Math. 5 (1993), 603–616. MR1242891 (94k:53087)

1994 An inversion formula for the horocyclic Radon transform on the real hy-perbolic space, Lectures in Appl. Math. 30 (1994), 1–6. MR1297561(96b:53090)

Bolker, E.

1987 The finite Radon transform, Contemp. Math. 63 (1987), 27–50. MR876312(88b:51009)

Boman, J.

1991 “Helgason’s support theorem for Radon transforms: A new proof and ageneralization,” in: Mathematical Methods in Tomography, Lecture Notesin Math. No. 1497, Springer-Verlag, Berlin and New York, 1991, 1–5.MR1178765

1992 Holmgren’s uniqueness theorem and support theorems for real analyticRadon transforms, Contemp. Math. 140 (1992), 23–30.

1993 An example of non-uniqueness for a generalized Radon transform, J. Anal-yse Math. 61 (1993), 395–401. MR1253450 (94j:44004)

Boman, J. and Lindskog, F.

2009 Support theorems for the Radon transform and Cramer-Wold theorems, J.of Theoretical Probability 22 (2009), 683–710. MR2530109 (2010m:60055)

Boman, J. and Quinto, E.T.

1987 Support theorems for real-analytic Radon transforms, Duke Math. J. 55(1987), 943–948. MR916130 (89m:44004)

Branson, T.P., Olafsson, G., and Schlichtkrull, H.

1994 A bundle-valued Radon transform with applications to invariant wave equa-tions, Quart. J. Math. Oxford 45 (1994), 429–461. MR1315457 (95k:22020)

Chen, B.-Y.

2001 Helgason spheres of compact symmetric spaces of finite type, Bull. Austr.Math. Soc.63 (2001), 243–255. MR1823711 (2002e:53097)

Chern, S.S.

1942 On integral geometry in Klein spaces, Ann. of Math. 43 (1942), 178–189.MR0006075 (3,253h)

PERSONAL REMARKS 13

Cormack, A.M.

1963–64 Representation of a function by its line integrals, with some radiologicalapplications I, II, J. Appl. Phys. 34 (1963), 2722–2727; 35 (1964), 2908–2912.

Cormack, A.M. and Quinto, E.T.

1980 A Radon transform on spheres through the origin in Rn and applicationsto the Darboux equation, Trans. Amer. Math. Soc. 260 (1980), 575–581.MR574800 (81i:44001)

Debiard, A. and Gaveau, B.

1983 Formule d’inversion en geometrie integrale Lagrangienne, C. R. Acad. Sci.Paris Ser. I Math. 296 (1983), 423–425. MR703912 (84k:53064)

Droste, B.

1983 A new proof of the support theorem and the range characterization ofthe Radon transform, Manuscripta Math. 42 (1983), 289–296. MR701211(85b:44003)

Eguchi, M.

1979 Asymptotic expansions of Eisenstein integrals and Fourier transform of sym-metric spaces, J. Funct. Anal. 34 (1979), 167–216. MR552702 (81e:43022)

Felix, R.

1992 Radon Transformation auf nilpotenten Lie Gruppen, Invent. Math. 112(1992), 413–443. MR1213109 (94f:22012)

Finch, D.V. Haltmeier, M., and Rakesh

2007 Inversion and spherical means and the wave equation in even dimension,SIAM J. Appl. Math. 68 (2007), 392–412. MR2366991 (2008k:35494)

Frigyik, B., Stefanov, P., and Uhlmann, G.

2008 The X-ray transform for a generic family of curves and weights, J. Geom.Anal. 18 (2008), 81–97. MR2365669 (2008j:53128)

Fuglede, B.

1958 An integral formula, Math. Scand. 6 (1958), 207–212. MR0105724(21 #4460)

Funk, P.

1913 Uber Flachen mit lauter geschlossenen geodatischen Linien, Math. Ann. 74(1913), 278–300. MR1511763

1916 Uber eine geometrische Anwendung der Abelschen Integral-gleichung,Math. Ann. 77 (1916), 129–135.

Furstenberg, H.

1963 A Poisson formula for semisimple Lie groups, Ann. of Math. 77 (1963),335-386. MR0146298 (26 #3820)

Gasqui, J. and Goldschmidt, H.

2004 Radon Transforms and the Rigidity of the Grassmannians, Ann. Math.Studies, Princeton Univ. Press, 2004. MR2034221 (2005d:53081)

Gelfand, I.M. and Graev, M.I.

1964 The geometry of homogeneous spaces, group representations in homoge-neous spaces and questions in integral geometry related to them, Amer.Math. Soc. Transl. 37 (1964).

Gelfand, I.M., Gindikin, S.G., and Graev, M.I.

1982 Integral geometry in affine and projective spaces, J. Soviet Math. 18 (1982),39–164.

2003 Selected Topics in Integral Geometry, Amer. Math. Soc. Transl. Vol. 220,Providence, RI, 2003. MR2000133 (2004f:53092)

Gelfand, I.M. and Graev, M.I.

1955 Analogue of the Plancherel formula for the classical groups, Trudy Moscov.Mat. Obshch. 4 (1955), 375–404. MR0071714 (17,173e)

1968b Admissible complexes of lines in CPn, Funct. Anal. Appl. 3 (1968), 39–52.

Gelfand, I.M., Graev, M.I., and Vilenkin, N.

1966 Generalized Functions, Vol. 5 : Integral Geometry and RepresentationTheory, Academic Press, New York, 1966. MR0207913 (34 #7726)

Gelfand, I.M. and Naimark, M.I..

1957 Unitare Darstellungen der Klassischen Gruppen, Akademie Verlag, Berlin,(1957). MR0085262 (19,13g)

Gindikin, S.G.

1995 Integral geometry on quadrics, Amer. Math. Soc. Transl. Ser. 2 169 (1995),23–31. MR1364451 (97c:53113)

Godement, R.

1952 Une generalisation du theoreme de la moyenne pour les fonctions har-moniques C.R. Acad. Sci. Paris 234 (1952), 2137–2139. MR0047056(13,821a)

Goldschmidt, H.

1990 The Radon transform for symmetric forms on real symmetric spaces, Con-temp. Math. 113 (1990), 81–96. MR1108646 (92e:53100)


Goncharov, A.B.

1989 Integral geometry on families of k-dimensional submanifolds, Funct. Anal.Appl. 23 1989, 11–23.

Gonzalez, F.

1984 Radon transforms on Grassmann manifolds, thesis, MIT, Cambridge, MA,1984. MR2941053

1988 Bi-invariant differential operators on the Euclidean motion group and ap-plications to generalized Radon transforms, Ark. Mat. 26 (1988), 191–204.MR1050104 (92c:58144)

1990a Bi-invariant differential operators on the complex motion group and therange of the d-plane transform on Cn, Contemp. Math. 113 (1990), 97–110. MR1108647 (93c:44007)

1990b Invariant differential operators and the range of the Radon d-plane trans-form, Math. Ann. 287 (1990), 627–635. MR1066819 (92a:58141)

1991 On the range of the Radon transform and its dual, Trans. Amer. Math.Soc. 327 (1991), 601–619. MR1025754 (92a:44002)

2001 “John’s equation and the plane to line transform on R3”, in HarmonicAnalysis and Integral Geometry Safi (1998), 1–7. Chapman and Hall/RCRResearch Notes Math., Boca Raton, FL, 2001.

Gonzalez, F. and Kakehi, T.

2003 Pfaffian systems and Radon transforms on affine Grassmann manifoldsMath. Ann. 326 (2003), 237–273. MR1990910 (2004f:53093)

2004 Dual Radon transforms on affine Grassmann manifolds, Trans. Amer.Math.Soc. 356 (2004), 4161–4180. MR2058842 (2005m:44001)

2006 Invariant differential operators and the range of the matrix Radon trans-form, J. Funct. Anal. 241 (2006), 232–267. MR2264251 (2007k:53127)

Gonzalez, F. and Quinto, E.T.

1994 Support theorems for Radon transforms on higher rank symmetric spaces,Proc. Amer. Math. Soc. 122 (1994), 1045–1052. MR1205492 (95b:44002)

Greenleaf, A. and Uhlmannn, G.

1989 Non-local inversion formulas for the X-ray transform, Duke Math. J. 58(1989), 205–240. MR1016420 (91b:58251)

Grinberg, E.

1985 On images of Radon transforms, Duke Math. J. 52 (1985), 939–972.MR816395 (87e:22020)

1992 Aspects of flat Radon transform, Contemp. Math. 140 (1992), 73–85.MR1197589 (94a:53104)

1994 “That kappa operator”, in Lectures in Appl. Math. 30, 1994, 93–104.

Grinberg, E. and Rubin, B..

2004 Radon inversion on Grassmannians via Garding–Gindikin fractional inte-grals, Ann. of Math. 159 (2004), 809–843. MR2081440 (2005f:58042)

Guillemin, V.

1976 Radon transform on Zoll surfaces, Adv. in Math. 22 (1976), 85–99.MR0426063 (54 #14009)

1985 The integral geometry of line complexes and a theorem of Gelfand-Graev,Asterisque No. Hors Serie (1985), 135-149. MR837199 (87i:53102)

1987 Perspectives in integral geometry, Contemp. Math. 63 (1987), 135–150.MR876317 (88i:53108)

Guillemin, V. and Sternberg, S.

1979 Some problems in integral geometry and some related problems in microlo-cal analysis, Amer. J. Math. 101 (1979), 915–955. MR536046 (82b:58087)

Gunther, P.

1966 Spharische Mittelwerte in kompakten harmonischen Riemannschen Mannig-faltigkeiten, Math. Ann. 165 (1966), 281–296. MR0200874 (34 #760)

1994 L∞-decay estimations of the spherical mean value on symmetric spaces,Ann. Global Anal. Geom. 12 (1994), 219–236. MR1295101 (95m:58122)

Harish-Chandra

1954 The Plancherel formula for complex semisimple Lie groups. Trans. Amer.Math. Soc. 76 (1954), 485–528. MR0063376 (16,111f)

1957 A formula for semisimple Lie groups, Amer. J. Math. 79 (1957), 733–760.MR0096138 (20 #2633)

1958 Spherical functions on a semisimple Lie group I, Amer. J. Math. 80 (1958),241–310. MR0094407 (20 #925)

Helgason, S.

1959 Differential Operators on homogeneous spaces, Acta Math. 102 (1959), 239–299. MR0117681 (22 #8457)

1962 Differential Geometry and Symmetric Spaces, Academic Press, New York,1962. MR0145455 (26 #2986)

1963 Duality and Radon transforms for symmetric spaces, Amer. J. Math. 85(1963), 667–692. MR0158409 (28 #1632)

PERSONAL REMARKS 15

1964 A duality in integral geometry: some generalizations of the Radon trans-form, Bull. Amer. Math. Soc. 70 (1964), 435–446. MR0166795 (29 #4068)

1964a Fundamental solutions of invariant diffferential operators on symmetricspaces, Amer. J. Math. 86 (1964), 565–601. MR0165032 (29 #2323)

1965a The Radon transform on Euclidean spaces, compact two-point homoge-neous spaces and Grassmann manifolds, Acta Math. 113 (1965), 153–180.MR0172311 (30 #2530)

1965b Radon-Fourier transforms on symmetric spaces and related group rep-resentation, Bull. Amer. Math. Soc. 71 (1965), 757–763. MR0179295(31 #3543)

1966 “A duality in integral geometry on symmetric spaces,” in: Proc. U.S.-JapanSeminar in Differential Geometry, Kyoto, 1965, Nippon Hyronsha, Tokyo,1966, 37–56. MR0229191 (37 #4765)

1966a Totally geodesic spheres in compact symmetric spaces, Math. Ann. 165(1966), 309–317. MR0210043 (35 #938)

1970 A duality for symmetric spaces with applications to group representations,Adv. in Math. 5 (1970), 1–154. MR0263988 (41 #8587)

1973 The surjectivity of invariant differential operators on symmetric spaces,Ann. of Math. 98 (1973), 451–480.

1973a Paley–Wiener theorems and surjectivity of invariant differential operatorson symmetric spaces and Lie groups, Bull. Amer. Math. Soc., 79 (1973),129–132. MR0312158 (47 #720)

1973b The eigenfunctions of the Laplacian on a two-point homogeneous space; in-tegral representations and irreducibility, AMS Symposium Stanford (1973).Proc. XXVII, (2) 1975, 357–360. MR0376961 (51 #13136)

1976 A duality for symmetric spaces with applications to group representations,II. Differential equations and eigenspace representations, Adv. Math., 22(1976), 187–219. MR0430162 (55 #3169)

1980 “The X-ray transform on a symmetric space”, in: Proc. Conf. on Differen-tial Geometry and Global Analysis, Berlin, 1979, Lecture Notes in Math.,No. 838, Springer–Verlag, New York, 1980.

1980b The Radon Transform, Birkhasuer 1980, 2nd Ed. 1999.

1983 “The range of the Radon transform on symmetric spaces,” in: Proc. Conf.on Representation Theory of Reductive Lie Groups, Park City, Utah, 1982,P. Trombi, ed., Birkhauser, Basel and Boston, 1983, 145–151.

1984 Groups and Geometric Analysis. Integral Geometry, Invariant Differ-ential Operators and Spherical functions, Academic Press, N.Y., 1984.(Now published by American Mathematical Society, 2000.) MR1790156(2001h:22001)

1990 The totally geodesic Radon transform on constant curvature spaces, Con-temp. Math. 113 (1990), 141–149. MR1108651 (92j:53036)

1992 The flat horocycle transform for a symmetric space, Adv. in Math. 91(1992), 232–251. MR1149624 (93i:43008)

1994a “Radon transforms for double fibrations: Examples and viewpoints,” in:Proc. Conf. 75 Years of Radon Transform, Vienna, 1992, InternationalPress, Hong Kong, 1994, 163–179. MR1313933 (95m:32048)

2005 The Abel, Fourier and Radon transforms on symmetric spaces. Indag. Math.NS. 16 (2005), 531–551. MR2313637 (2008i:43008)

2007 The inversion of the X-ray transform on a compact symmmetric space, J.Lie Theory 17 (2007), 307–315. MR2325701 (2008d:43011)

2008 Geometric Analysis on Symmetric Spaces, Math. Surveys and MonographsNo. 39, American Mathematical Society, Providence, RI. Second Edition,2008. MR2463854 (2010h:22021)

2011 Integral Geometry and Radon Transforms, Springer 2011. MR2743116(2011m:53144)

2012 Support theorem for horocycles in hyperbolic spaces, Pure and Appl. Math.Quarterly 8 (2012), 921–928.

Hertle, A.

1983 Continuity of the Radon transform and its inverse on Euclidean space,Math. Z. 184 (1983), 165–192. MR716270 (86e:44004a)

1984a On the range of the Radon transform and its dual, Math. Ann. 267 (1984),91–99. MR737337 (86e:44004b)

1984b On the injectivity of the attenuated Radon transform, Proc. Amer. Math.Soc. 92 (1984) 201–205. MR754703 (86d:44007)

Hormander, L.

1958 On the division of distributions by polynomials, Arkiv for Matematik 3(1958) 555-568. MR0124734 (23 #A2044)


Hu, M.-C.

1973 Determination of the conical distributions for rank one symmetric spaces,Thesis, MIT, Cambridge, MA, 1973. MR2940491

1975 Conical distributions for rank one symmetric spaces, Bull. Amer. Math.Soc. 81 (1975), 98–100. MR0370068 (51 #6297)

Ishikawa, S.

1997 The range characterization of the totally geodesic Radon transform on thereal hyperbolic space, Duke Math. J. 90 (1997), 149–203. MR1478547(99m:43011)

2003 Symmetric subvarieties in compactifications and the Radon transform onRiemannian symmetric spaces of the noncompact type, J. Funct. Anal.204 (2003), 50–100. MR2004745 (2004m:43014)

John, F.

1938 The ultrahyperbolic differential equation with 4 independent variables,Duke Math. J. 4 (1938), 300–322. MR1546052

1955 Plane Waves and Spherical Means, Wiley–Interscience, New York, 1955.

Kakehi, T.

1992 Range characterization of Radon transforms on complex projective spaces,J. Math. Kyoto Univ. 32 (1992), 387–399. MR1173971 (93e:53084)

1993 Range characterization of Radon transforms on Sn and PnR, J. Math.Kyoto Univ. 33 (1993), 315–328. MR1231746 (94g:58221)

1999 Integral geometry on Grassmann manifolds and calculus of invariant dif-ferential operators, J. Funct. Anal. 168 (1999), 1-45. MR1717855(2000k:53069)

Kakehi, T. and Tsukamoto, C.

1993 Characterization of images of Radon transforms, Adv. Stud. Pure Math.22 (1993), 101–116. MR1274942 (95b:58148)

Karpelevich, F.I. ,1965 The geometry of geodesics and the eigenfunctions of the Beltrami–Laplace

operator on symmetric spaces, Trans. Moscow Math. Soc. 14 (1965), 51–199. MR0231321 (37 #6876)

Kashiwara, M., Kowata, A., Minemura, K., Okamoto, K.,Oshima, T. and Tanaka, M.

1978 Eigenfunctions of invariant differential operators on a symmetric space.Ann. of Math. 107 (1978), 1–39. MR485861 (81f:43013)

Klein, S. Thorbergsson, G. and Verhoczki, L.

2009 On the Funk transform on compact symmetric spaces, Publ. Math. Debre-cen 75 (2009), 485–493. MR2588219 (2011c:53107)

Knapp, A.W. and Stein, E.M.

1971 Intertwining operators on semisimple groups, Ann. of Math., 93 (1971),489–578. MR0460543 (57 #536)

Koranyi, A.

1995 On a mean value property for hyperbolic spaces, Contemp. Math., Amer.Math. Soc., 191 (1995), 107–116. MR1365538 (96k:43014)

2009 Cartan–Helgason theorem, Poisson transform and Furstenberg–Satakecompactifications, J. Lie Theory, 19 (2009), 537–542. MR2583919(2010k:22017)

Kumahara, K. and Wakayama, M.

1993 On Radon transform for Minkowski space, J. Fac. Gen. Ed. Tattori Univ.27 (1993), 139–157.

Kurusa, A.

1991 A characterization of the Radon transform’s range by a system of PDE’s,J. Math. Anal. Appl. 161 (1991), 218–226. MR1127559 (92k:44002)

1994 Support theorems for the totally geodesic Radon transform on constant cur-vature spaces, Proc. Amer. Math. Soc. 122 (1994), 429–435. MR1198457(95a:53111)

Lax, P. and Phillips, R.S.

1982 A local Paley-Wiener theorem for the Radon transform of L2 functions ina non-Euclidean setting, Comm. Pure Appl. Math. 35 (1982), 531–554.MR657826 (83i:43016)

Lewis, J.B.

1978 Eigenfunctions on symmetric spaces with distribution-valued boundaryforms, J. Funct. Anal. 29 (1978), 287–307. MR512246 (80f:43020)

Lim, K-T.

2012 A study of spherical mean-value operators, Thesis, Tufts University, 2012.

Lojasiewicz, S.

1958 Division d’une distribution par une fonction analytique, C.R. Acad. Sci.Paris 246 (1958), 271–307. MR0096120 (20 #2616)

PERSONAL REMARKS 17

Madych, W.R. and Solmon, D.C.

1988 A range theorem for the Radon transform, Proc. Amer. Math. Soc. 104(1988), 79–85. MR958047 (90i:44003)

Minkowski, H.

1911 Uber die Korper kostanter Breite, Collected Works, II, pp. 277–279, Teub-ner, Leipzig, 1911.

Natterer, F.

2001 Inversion of the attenuated transform, Inverse Problems 17 (2001), 113–119. MR1818495

Novikov, R.G.

2002 An inversion formula for the attenuated X-ray transformation, Ark. Mat.40 (2002), 145–167. MR1948891 (2003k:44004)

Okamoto, K.

1971 Harmonic analysis on homogeneous vector bundles, Lecture Notes in Math.266 (1971), 255–271. MR0486323 (58 #6079)

Olafsson, G. and E.T. Quinto

2006 The Radon Transform, Inverse Problems and Tomography, Proc. Symp.Appl. Math. Am. Math. Soc. 2006. MR2207138 (2006i:44001)

Olafsson, G. and Rubin, B.

2008 Invariant functions on Grassmannians, Contemp. Math. Amer. Math. Soc.464, 2008. MR2440137 (2009k:43014)

Orloff, J.

1987 “Orbital integrals on symmetric spaces,” in: Non-Commutative HarmonicAnalysis and Lie Groups, Lecture Notes in Math. No. 1243, Springer-Verlag, Berlin and New York, 1987, 198–219. MR897543 (88k:43013)

1990 Invariant Radon transforms on a symmetric space, Trans. Amer. Math.Soc. 318 (1990), 581–600. MR958898 (90g:44003)

Oshima, T.

1996 Generalized Capelli identities and boundary values for GL(n). Structureof Solutions of Differential Equations, Katata/Kyoto, 1995, 307–355. WorldScientific 1996. MR1445347 (98f:22021)

Oshima, T. and Sekiguchi

1980 Eigenspaces of invariant differential operators on an affine symmetric space,Invent. Math. 87, 1980, 1–81. MR564184 (81k:43014)

Palamodov, V.

1996 An inversion formula for an attenuated X-ray transform, Inverse Problems12 (1996), 717–729 MR1413429 (97j:44005)

2004 Reconstructive Integral Geometry. Birkhauser, Boston, (2004).MR2091001 (2005i:53094)

Palamodov, V. and Denisjuk, A.

1988 Inversion de la transformation de Radon d’apres des donnees incompletes,C. R. Acad. Sci. Paris Ser. I Math. 307 (1988), 181–183. MR955548(90i:44004)

Pati, V., Shahshahani, M. and Sitaram, A.

1995 The spherical mean value operator for compact symmetric spaces, PacificJ. Math. 168 (1995), 335–343. MR1339956 (96f:58163)

Quinto, E.T.

1981 Topological restrictions on double fibrations and Radon transforms, Proc.Amer. Math. Soc. 81 (1981), 570–574. MR601732 (82g:58084)

1982 Null spaces and ranges for the classical and spherical Radon transforms, J.Math. Ann. Appl. 90 (1982), 405–420. MR680167 (85e:44004)

1983 The invertibility of rotation invariant Radon transforms, J. Math. Anal.Appl. 91 (1983), 510–521; erratum, J. Math. Anal. Appl. 94 (1983), 602–603. MR706385 (84j:44007b)

1987 Injectivity of rotation invariant Radon transforms on complex hyperplanesin Cn, Contemp. Math. 63 (1987), 245–260. MR876322 (88j:53075)

1992 A note on flat Radon transforms, Contemp. Math. 140 (1992), 115–121.MR1197593 (93j:44005)

1993a Real analytic Radon transforms on rank one symmetric spaces, Proc. Amer.Math. Soc. 117 (1993), 179–186. MR1135080 (93e:44005)

2006 Support theorems for the spherical Radon transform on manifolds, Intl.Math. Research Notes, 2006, 1–17, ID 67205. MR2219205 (2007e:53096)

2008 Helgason’s support theorem and spherical Radon transforms, Contemp.Math., 2008.

Radon, J.

1917 Uber die Bestimmung von Funktionen durch ihre Integralwerte langs gewis-serMannigfaltigkeiten, Ber. Verh. Sachs. Akad. Wiss. Leipzig. Math. Nat.Kl. 69 (1917), 262–277.


Richter, F.

1986 On the k-Dimensional Radon Transform of Rapidly Decreasing Functions,in Lecture Notes in Math. No. 1209, Springer-Verlag, Berlin and New York,1986. MR863761 (88a:53071)

1990 On fundamental differential operators and the p-plane transform, Ann.Global Anal. Geom. 8 (1990), 61–75. MR1075239 (92e:58211)

Rouviere, F.

2001 Inverting Radon transforms: the group-theoretic approach, Enseign. Math.47 (2001), 205–252. MR1876927 (2002k:53149)

2006 Transformation aux rayons X sur un espace symetrique. C.R. Acad. Sci.Paris Ser. I 342 (2006), 1–6. MR2193386 (2006j:53109)

2008a X-ray transform on Damek-Ricci spaces, preprint (2008), Inverse Problemsand Imaging 4 (2010), 713–720 MR2726427 (2011k:53107)

2008b On Radon transforms and the Kappa operator, preprint (2008).2012 The Mean-Value theorems on symmetric spaces (this volume).

Rubin, B.

1998 Inversion of fractional integrals related to the spherical Radon transform,J. Funct. Anal. 157 (1998), 470–487. MR1638340 (2000a:42019)

2002 Helgason–Marchand inversion formulas for Radon transforms, Proc. Amer.Math. Soc. 130 (2002), 3017–3023. MR1908925 (2003f:44003)

2004 Radon transforms on affine Grassmannians, Trans. Amer. Math. Soc. 356(2004), 5045–5070. MR2084410 (2005e:44004)

2008 Inversion formulas for the spherical mean in odd dimension and the Euler-Poisson Darboux equation, Inverse Problems 24 (2008) No. 2. MR2408558(2009f:44001)

Sarkar, R.P. and Sitaram, A.

2003 The Helgason Fourier transform on symmetric spaces. In Perspectives inGeometry and Representation Theory. Hundustan Book Agency (2003),467–473. MR2017597 (2005b:43017)

Schiffmann, G.

1971 Integrales d’entrelacement et fonctions de Whittaker, Bull. Soc. Math.France 99 (1971), 3–72. MR0311838 (47 #400)

Sekerin, A.

1993 A theorem on the support for the Radon transform in a complex space,Math. Notes 54 (1993), 975–976. MR1248292 (94k:44003)

Selfridge, J. L. and Straus, E. G.

1958 On the determination of numbers by their sums of a fixed order, Pacific J.Math. 8 (1958), 847–856. MR0113825 (22 #4657)

Semyanisty, V.I.

1961 Homogeneous functions and some problems of integral geometry in spacesof constant curvature, Soviet Math. Dokl. 2 (1961), 59–62. MR0133006(24 #A2842)

Sherman, T.

1977 Fourier analysis on compact symmetric spaces., Bull. Amer. Math. Soc. 83(1977), 378–380. MR0445236 (56 #3580)

1990 The Helgason Fourier transform for compact Riemannian symmetric spacesof rank one, Acta Math. 164 (1990), 73–144. MR1037598 (91g:43009)

Solmon, D.C.

1976 The X-ray transform, J. Math. Anal. Appl. 56 (1976), 61–83. MR0481961(58 #2051)

1987 Asymptotic formulas for the dual Radon transform, Math. Z. 195 (1987),321–343. MR895305 (88i:44006)

Strichartz, R.S.

1981 Lp Estimates for Radon transforms in Euclidean and non-Euclidean spaces,Duke Math. J. 48 (1981), 699–727. MR782573 (86k:43008)

Volchkov, V.V.

2001 Spherical means on symmetric spaces, Mat. Sb. 192 (2001), 17–38.MR1867008 (2003m:43009)

2003 Integral Geometry and Convolution Equations, Kluwer, Dordrecht, 2003.

Wallach, N.

1983 Asymptotic expansions of generalized matrix entries of representation ofreal reductive groups, Lecture Notes in Math. 1024, Springer (1983), 287–369. MR727854 (85g:22029)

Yang, A.

1998 Poisson transform on vector bundles, Trans. Amer. Math. Soc. 350 (1998),857–887. MR1370656 (98k:22065)

Zalcman, L.

1982 Uniqueness and nonuniqueness for the Radon transforms, Bull. LondonMath. Soc. 14 (1982), 241–245. MR656606 (83h:42020)

PERSONAL REMARKS 19

Zhang, G.

2009 Radon transform on symmetric matrix domains, Trans. Amer. Math. Soc.361 (2009), 351-369. MR2457402 (2010b:22016)

Zhou, Y. and Quinto, E.T.

2000 Two-radius support theorems for spherical Radon transforms on manifolds.Contemp. Math. 251 (2000), 501–508. MR1771290 (2001k:58036)

Department of Mathematics, Massachusetts Institute of Technology, Cambridge,

Massachusetts

E-mail address: [email protected]


On the Life and Work of S. Helgason

G. Olafsson and R. J. Stanton

Abstract. This article is a contribution to a Festschrift for S. Helgason. Aftera biographical sketch, we survey some of his research on several topics ingeometric and harmonic analysis during his long and influential career. While

not an exhaustive presentation of all facets of his research, for those topicscovered we include reference to the current status of these areas.

Preface

Sigurður Helgason is known worldwide for his first book Differential Geometryand Symmetric Spaces. With this book he provided an entrance to the opus of ElieCartan and Harish-Chandra to generations of mathematicians. On the occasion ofhis 85th birthday we choose to reflect on the impact of Sigurður Helgason’s sixtyyears of mathematical research. He was among the first to investigate systemat-ically the analysis of differential operators on reductive homogeneous spaces. Hisresearch on Radon-like transforms for homogeneous spaces presaged the resurgenceof activity on this topic and continues to this day. Likewise he gave a geomet-rically motivated approach to harmonic analysis of symmetric spaces. Of coursethere is much more - eigenfunctions of invariant differential operators, propagationproperties of differential operators, differential geometry of homogeneous spaces,historical profiles of mathematicians. Here we shall present a survey of some ofthese contributions, but first a brief look at the man.

1. Short Biography

Sigurður Helgason was born on September 30, 1927 in Akureyri, in northernIceland. His parents were Helgi Skulason (1892-1983) and Kara SigurðardottirBriem (1900-1982), and he had a brother Skuli Helgason (1926-1973) and a sisterSigriður Helgadottir (1933-2003). Akureyri was then the second largest city inIceland with about 3,000 people living there, whereas the population of Icelandwas about 103,000. As with other cities in northern Iceland, Akureyri was isolated,having only a few roads so that horses or boats were the transportation of choice.Its schools, based on Danish traditions, were good. The Gymnasium in Akureyriwas established in 1930 and was the second Gymnasium in Iceland. There Helgason

2010 Mathematics Subject Classification. Primary 43A85.The first author acknowledges the support of NSF Grant DMS-1101337 during the prepara-

tion of this article.


21

22 G. OLAFSSON AND R. J. STANTON

studied mathematics, physics, languages, amongst other subjects during the years1939-1945. He then went to the University of Iceland in Reykjavık where he enrolledin the school of engineering, at that time the only way there to study mathematics.In 1946 he began studies at the University of Copenhagen from which he receivedthe Gold Medal in 1951 for his work on Nevanlinna-type value distribution theoryfor analytic almost-periodic functions. His paper on the subject became his master’sthesis in 1952. Much later a summary appeared in [H89].

Leaving Denmark in 1952 he went to Princeton University to complete hisgraduate studies. He received a Ph.D. in 1954 with the thesis, Banach Algebrasand Almost Periodic Functions, under the supervision of Salomon Bochner.

He began his professional career as a C.L.E. Moore Instructor at M.I.T. 1954-56. After leaving Princeton his interests had started to move towards two areasthat remain the main focus of his research. The first, inspired by Harish-Chandra’sground breaking work on the representation theory of semisimple Lie groups, wasLie groups and harmonic analysis on symmetric spaces; the second was the Radontransform, the motivation having come from reading the page proofs of Fritz John’sfamous 1955 book Plane Waves and Spherical Means. He returned to Princetonfor 1956-57 where his interest in Lie groups and symmetric spaces led to his firstwork on applications of Lie theory to differential equations, [H59]. He moved to theUniversity of Chicago for 1957-59, where he started work on his first book [H62]. Hethen went to Columbia University for the fruitful period 1959-60, where he sharedan office with Harish-Chandra. In 1959 he joined the faculty at M.I.T. where he hasremained these many years, being full professor since 1965. The periods 1964-66,1974-75, 1983 (fall) and 1998 (spring) he spent at the Institute for Advanced Study,Princeton, and the periods 1970-71 and 1995 (fall) at the Mittag-Leffler Institute,Stockholm.

He has been awarded a degree Doctoris Honoris Causa by several universities,notably the University of Iceland, the University of Copenhagen and the Universityof Uppsala. In 1988 the American Mathematical Society awarded him the SteelePrize for expository writing citing his book Differential Geometry and SymmetricSpaces and its sequel. Since 1991 he carries the Major Knights Cross of the IcelandicFalcon.

2. Mathematical Research

In the Introduction to his selected works, [Sel], Helgason himself gave a personaldescription of his work and how it relates to his published articles. We recommendthis for the clarity of exposition we have come to expect from him as well as theinsight it provides to his motivation. An interesting interview with him also maybe found in [S09]. Here we will discuss parts of this work, mostly those familiar tous. We start with his work on invariant differential operators, continuing with hiswork on Radon transforms, his work related to symmetric spaces and representationtheory, then a sketch of his work on wave equations.

2.1. Invariant Differential Operators. Invariant differential operators havealways been a central subject of investigation by Helgason. We find it very infor-mative to read his first paper on the subject [H59]. In retrospect, this shines abeacon to follow through much of his later work on this subject. Here we find alucid introduction to differential operators on manifolds and the geometry of ho-mogeneous spaces, reminiscient of the style to appear in his famous book [H62].

ON THE LIFE AND WORK OF S. HELGASON 23

Specializing to a reductive homogeneous space, he begins the study of D(G/H),those differential operators that commute with the action of the group of isome-tries. The investigation of this algebra of operators will occupy him through manyyears. What is the relationship of D(G/H) to D(G) and what is the relationshipof D(G/H) to the center of the universal enveloping algebra? Harish-Chandra hadjust described his isomorphism of the center of the universal enveloping algebrawith the Weyl invariants in the symmetric algebra of a Cartan subalgebra, so Hel-gason introduces this to give an alternative description of D(G/H). But the goal isalways to understand analysis on the objects, so he investigates several problems,variations of which will weave throughout his research.

For symmetric spaces X = G/K the algebra D(X) was known to be commu-tative, and Godement had formulated the notion of harmonic function in this caseobtaining a mean value characterization. Harmonic functions being joint eigenfunc-tions of D(X) for the eigenvalue zero, one could consider eigenfunctions for othereigenvalues. Indeed, Helgason shows that the zonal spherical functions are alsoeigenfunctions for the mean value operator. When X is a two-point homogeneousspace, and with Asgeirsson’s result on mean value properties for solutions of theultrahyperbolic Laplacian in Euclidean space in mind, Helgason formulates andproves an extension of it to these spaces. Here D(X) has a single generator, theLaplacian, for which he constructs geometrically a fundamental solution, therebyallowing him to study the inhomogeneous problem for the Laplacian. This papercontains still more. In many ways the two-point homogeneous spaces are ideal gen-eralizations of Euclidean spaces so following F. John [J55] he is able to define aRadon like transform on the constant curvature ones and identify an inversion op-erator. Leaving the Riemannian case, Helgason considers harmonic Lorentz spaceG/H. He shows D(G/H) is generated by the natural second order operator; he ob-tains a mean value theorem for suitable solutions of the generator and an explicitinverse for the mean value operator. Finally, he examines the wave equation onharmonic Lorentz spaces and shows the failure of Huygens principle in the non-flatcase.

Building on these results he subsequently examines the question of existence offundamental solutions more generally. He solves this problem for symmetric spacesas he shows that every D ∈ D(X) has a fundamental solution, [H64, Thm. 4.2].Thus, there exists a distribution T ∈ C∞

c (X)′ such that DT = δxo. Convolution

then provides a method to solve the inhomogeneous problem, namely, if f ∈ C∞c (X)

then there exists u ∈ C∞(X) such that Du = f . Those results had been announcedin [H63c]. The existence of the fundamental solution uses the deep results ofHarish-Chandra on the aforementioned isomorphism as well as classic results ofHormander on constant coefficient operators. It is an excellent example of thecombination of the classical theory with the semisimple theory. Here is a sketch ofhis approach.

In his classic paper [HC58] on zonal spherical functions, Harish-Chandra in-troduced several important concepts to handle harmonic analysis. One was theappropriate notion of a Schwartz-type space of K bi-invariant functions, there de-noted I(G). I(G) with the appropriate topology is a Frechet space, and havingC∞

c (X)K as a dense convolution subalgebra.


Another notion from [HC58] is the Abel transform

Ff (a) = aρ∫N

f(an) dn .

Today this is also called the ρ-twisted Radon transform and denotedRρ. EventuallyHarish-Chandra showed that this gives a topological isomorphism of I(G) ontoS(A)W , the Weyl group invariants in the Schwartz space on the Euclidean spaceA. Furthermore, the Harish-Chandra isomorphism γ : D(X) → D(A) interactscompatibly in that

Rρ(Df) = γ(D)Rρ(f) .

One can restate this by saying that the Abel transform turns invariant differentialequations on X into constant coefficient differential equations on A a RrankX .It follows then that Rt

ρ : S ′(A)W → I(G)′ is also an isomorphism. This can thenbe used to pull back the fundamental solution for γ(D) to a fundamental solutionfor D.

The article [H64] continues the line of investigation from [H59] into the struc-ture of D(X). If we denote by U(g) the universal enveloping algebra of gC, thenU(g) is isomorphic to D(G). Let Z(G) be the center of D(G). This is the algebraof bi-invariant differential operators on G. The algebra of invariant differential op-erators on X is isomorphic to D(G)K/D(G)K ∩D(G)k and therefore contains Z(G)as an Abelian subalgebra.

Let h be a Cartan subalgebra in g extending a and denote byWh its Weyl group.The subgroup Wh(a) = {w ∈ Wh | w(a) = a} induces the little Weyl group W byrestriction. It follows that the restriction p �→ p|a maps S(h)Wh into S(a)W . NowZ(G) S(gC)G S(h)Wh , and D(X) S(a)W S(s)K , s a Cartan complementof k. The structure of these various incarnations is given in cf. [H64, Prop. 7.4]and [H92, Prop 3.1]. See also the announcements in [H62a,H63c]:

Theorem 2.1. The following are equivalent:

(1) D(X) = Z(G).(2) S(hC)Wh |a = S(aC)W .(3) S(g)G|s = S(s)K.

A detailed inspection showed that (2) was always true for the classical sym-metric spaces but fails for some of the exceptional symmetric spaces. Those ideasplayed an important role in [OW11] as similar restriction questions were consideredfor sequences of symmetric spaces of increasing dimension.

The final answer, prompted by a question from G. Shimura, is [H92]:

Theorem 2.2. Assume that X is irreducible. Then Z(G) = D(X) if and only ifX is not one of the following spaces E6/SO(10)T, E6/F4, E7/E6T or E8/E7SU(2).Moreover, for any irreducible X any D ∈ D(X) is a quotient of elements of Z(G).

2.2. The Radon Transform on Rn. The Radon transform as introduced byJ. Radon in 1917 [R17] [RaGes] associates to a suitable function f : R2 → C itsintegrals over affine lines L ⊂ R2

R(f)(L) = f(L) :=

∫x∈L

f(x) dx

for which he derived an inversion formula. This ground breaking article appearedin a not easily available journal (one can find the reprinted article in [H80]), and


consequently was not well known. Nevertheless, its true worth is easily determinedby the many generalizations of it that have been made in geometric analysis andrepresentation theory, some already pointed out in Radon’s original article.

An important milestone in the development of the theory was F. John’s book[J55]. Later, the application of integration over affine lines in three dimensionsplayed an important role in the three dimensional X-ray transform. We refer to[E03,GGG00,H80,H11,N01] for information about the history and the manyapplications of the Radon transform and its descendants.

Helgason first displayed his interest in the Radon transform in that basic paper[H59]. There he considers a transform associated to totally geodesic submanifoldsin a space of constant curvature and produces an inversion formula. To use it asa tool for analysis one needs to determine if there is injectivity on some space ofrapidly decreasing functions and compatibility with invariant differential operators,just as Harish-Chandra had done for the map Ff . In [H65] Helgason starts on hislong road to answering such questions, and, in the process recognizing the under-lying structure as incidence geometry, he is able to describe a vast generalization.

As he had previously considered two-point homogeneous spaces he starts there,but to this he extends Radon’s case to affine p-planes in Euclidean space. Wesummarize the results in the important article [H65].

Denote by H(p, n) the space of p-dimensional affine subspaces of Rn. Let f ∈C∞

c (Rn) and ξ ∈ H(p, n). Define

R(f)(ξ) = f(ξ) :=

∫x∈ξ

f(x) dξx

where the measure dξx is determined in the following way. The connected Euclideanmotion group E(n) = SO(n)� Rn acts transitively on both Rn and H(p, n). Takebasepoints xo = 0 ∈ Rn and ξo = {(x1, . . . , xp, 0, . . . , 0)} ∈ H(p, n) and take dξoxLebesgue measure on ξo. For ξ ∈ H(p, n) choose g ∈ E(n) such that ξ = g · ξo.Then dξx = g∗dξox or ∫

ξ

f(x)dξx =

∫ξo

f(g · x) dx .

For x ∈ Rn the set x∨ := {ξ ∈ H(p, n) | x ∈ ξ} is compact, in fact isomorphic tothe Grassmanian G(p, n) = SO(n)/S(O(p)×O(n−p)) of all p-dimensional subspacesof Rn. Therefore each of these carries a unique SO(n)-invariant probability measuredxξ which provides the dual Radon transform. Let ϕ ∈ Cc(Ξ) and define

ϕ∨(x) =

∫x∨

ϕ(ξ) dxξ .

We have the Parseval type relationship∫Ξ

f(ξ)ϕ(ξ) dξ =

∫Rn

f(x)ϕ∨(x) dx

and both the Radon transform and its dual are E(n) intertwining operators.If p = n− 1 every hyperplane is of the form ξ = ξ(u, p) = {x ∈ Rn | 〈x, u〉 = p}

and ξ(u, p) = ξ(v, q) if and only if (u, p) = ±(v, q). Thus H(p, n) Sn−1×Z2R. We

now we have the hyperplane Radon transform considered in [H65]. This case hadbeen considered by F. John [J55] and he proved the following inversion formulas


for suitable functions f :

f(x) =1

2

1

(2πi)n−1Δ

n−12

x

∫Sn−1

f(u, 〈u, x〉) du , n odd

f(x) =1

(2πi)nΔ

n−22

x

∫Sn−1

∫R

∂pf(u, p)

p− 〈u, x〉 dpdu , n even .

The difference between the even and odd dimensions is significant, for in odddimensions inversion is given by a local operator, but not in even dimension. Thisis fundamental in Huygens’ principle for the wave equation to be discussed subse-quently.

For Helgason the problem is to show the existence of suitable function spaces onwhich these transforms are injective and to show they are compatible with the E(n)invariant differential operators. One shows that the Radon transform extends to theSchwartz space S(Rn) of rapidly decreasing functions on Rn and it maps that spaceinto a suitably defined Schwartz space S(Ξ) on Ξ. Denote by D(Rn), respectivelyD(Ξ), the algebra of E(n)-invariant differential operators on Rn, respectively Ξ .Furthermore, define a differential operator � on Ξ by �f(u, r) = ∂2

rf(u, r).However a new feature arises whose existence suggests future difficulties in

generalizations. Let

S∗(Rn) = {f ∈ S(Rn) |∫Rn

f(x)p(x) dx = 0 for all polynomials p(x)}

and

S∗(Ξ) = {ϕ ∈ S(Ξ) |∫R

ϕ(u, r)q(r) dr = 0 for all polynomials q(r)} .

Finally, let SH(Ξ) be the space of rapidly decreasing function on Ξ such that foreach k ∈ Z+ the integral

∫ϕ(u, r)rk dr can be written as a homogeneous polynomial

in u of degree k. Then we have the basic theorem for this transform and its dual:

Theorem 2.3. [H65] The following hold:

(1) D(Rn) = C[Δ] and D(Ξ) = C[�].

(2) Δf = �f .(3) The Radon transform is a bijection of S(Rn) onto SH(Ξ) and the dual

transform is a bijection SH(Ξ) onto S(Rn).(4) The Radon transform is a bijection of S∗(Rn) onto S∗(Ξ) and the dual

transform is a bijection S∗(Ξ) onto S∗(Rn).(5) Let f ∈ S(Rn) and ϕ ∈ S∗(Ξ). If n is odd then

f = cΔ(n−1)/2(f)∨ and ϕ = c�(n−1)/2(ϕ∨)∧

for some constant independent of f and ϕ.(6) Let f ∈ S(Rn) and ϕ ∈ S∗(Ξ). If n is even then

f = c1 J1(f)∨ and ϕ = c2 J2(ϕ

∨)∧

where the operators J1 and J2 are given by analytic continuation

J1 : f(x) �→ an.cont|α=1−2n

∫Rn

f(y)‖x− y‖α dy

and

J2 : ϕ �→ an.cont|β=−n

∫R

ϕ(u, r)‖s− r‖β dr


and c1 and c2 are constants independent of f and g.

In [H80] it was shown that the map

f �→ �(n−1)/4 f

extends to an isometry of L2(Rn) onto L2(Ξ) .Needed for the proof of the theorem is one of his fundamental contributions to

the subject in the following support theorem in [H65]. An important generalizationof this theorem will be crucial for his later work on solvability of invariant differentialoperators on symmetric spaces.

Theorem 2.4 (Thm 2.1 in [H65]). Let f ∈ C∞(Rn) satisfy the followingconditions:

(1) For each integer x �→ ‖x‖k|f(x)| is bounded.

(2) There exists a constant A > 0 such that f(ξ) = 0 for d(0, ξ) > A.

Then f(x) = 0 for ‖x‖ > A.

An important technique in the theory of the Radon transform, which also playsan important role in the proof of Theorem 2.3, uses the Fourier slice formula: Letr > 0 and u ∈ Sn−1 then

(2.1) F(f)(ru) = c

∫R

f(u, s)e−isr ds .

So that if f is supported in a closed ball Bnr (0) in Rn of radius r centered at

the origin, then by the classical Paley-Wiener theorem for Rn the function

r �→ F(f)(ru)extends to a holomorphic function on C such that

supz∈C

(1 + |z|)ne−r|Imz||F(f)(zu) <∞

Let C∞r,H(Ξ) be the space of ϕ ∈ SH(Ξ) such that p �→ ϕ(u, p) vanishes for p >

r. Then the Classical Paley-Wiener theorem combined with (2.1) shows that theRadon transform is a bijection C∞

r (Rn) C∞r,H(Ξ), [H65, Cor. 4.3]. (2.1) also

played an important role in Helgason’s introduction of the Fourier transform onRiemannian symmetric spaces of the noncompact type.

2.3. The Double Fibration Transform. The Radon transform on Rn andthe dual transform are examples of the double fibration transform introduced in[H66b,H70]. Recall that both Rn and H(p, n) are homogeneous spaces for thegroup G = E(n). Let K = SO(n), L = S(O(p)×O(n−p)) and N = {(x1, . . . , xp, 0,. . . 0) | xj ∈ R} Rp and H = L � N . Then Rn G/K, H(p, n) G/H andL = K ∩H. Hence we have the double fibration

(2.2) G/L

π

�� p

��

��

�

X = G/K Ξ = G/H

where π and p are the natural projections. If ξ = a · ξo ∈ Ξ and x = b ·xo ∈ X then

(2.3) f(ξ) =

∫H/L

f(ah · xo) dH/L(hL)


and

(2.4) ϕ∨(x) =

∫K/L

ϕ(bk · ξo) dK/L(kL)

for suitable normalized invariant measures on H/L N and K/L.More generally, using Chern’s formulation of integral geometry on homogeneous

spaces as incidence geometry [C42], Helgason introduced the following double fibra-tion transform. Let G be a locally compact Hausdorff topological group and K,Htwo closed subgroups giving the double fibration in 2.2. We will assume that G, K,H and L := K ∩ H are all unimodular. Therefore each of the spaces X = G/K,Ξ = G/H, G/L, K/L and H/L carry an invariant measure.

We set xo = eK and ξo = eH. Let x = aK ∈ X and ξ = bH ∈ Ξ. We say thatx and ξ are incident if aK ∩ bH �= ∅. For x ∈ X and ξ ∈ Ξ we set

x = {η ∈ Ξ | x and ξ are incident }

and similarly

ξ∨ = {x ∈ X | ξ and x are incident } .Assume that if a ∈ K and aH ⊂ HK then a ∈ H and similarly, if b ∈ H and

bK ⊂ KH then b ∈ K. Thus we can view the points in Ξ as subsets of X, andsimilarly points in X are subsets of Ξ. Then x∨ is the set of all ξ such that x ∈ ξ

and ξ is the set of points x ∈ X such that x ∈ ξ. We also have

x = p(π−1(x)) = aK · ξ0 H/L and ξ∨ = π(p−1(ξ)) = bH · xo K/L.

Under these conditions the Radon transform (2.3) and its dual (2.4) are welldefined at least for compactly supported functions. Moreover, for a suitable nor-malization of the measures we have∫

Ξ

f(ξ)ϕ(ξ) dξ =

∫X

f(x)ϕ∨(x) dx .

Helgason [H66b, p.39] and [GGA, p.147] proposed the following problems for

these transforms f → f , ϕ→ ϕ∨:

(1) Identify function spaces on X and Ξ related by the integral transforms

f �→ f and ϕ �→ ϕ∨.

(2) Relate the functions f and f∨ on X, and similarly ϕ and (ϕ∨)∧ on Ξ,including an inversion formula, if possible.

(3) Injectivity of the transforms and description of the image.(4) Support theorems.(5) For G a Lie group, with D(X), resp. D(Ξ), the algebra of invariant differ-

ential operators on X, resp. Ξ. Do there exist maps D �→ D and E �→ E∨

such that

(Df)∧ = Df and (Eϕ)∨ = E∨ϕ∨ .

There are several examples where the double fibration transform serves as aguide, e.g. the Funk transform on the sphere Sn, see [F16] for the case n = 2, andmore generally [R02]; and the geodesic X-ray transform on compact symmetricspaces, see [H07,R04]. Other uses of the approach can be found in [K11]. Werefer the reader to [E03] and [H11] for more examples.


2.4. Fourier analysis on X = G/K. From now on G will stand for a non-compact connected semisimple Lie group with finite center and K a maximal com-pact subgroup. We take an Iwasawa decomposition G = KAN and use standardnotation for projections on to the K and A component. Set X = G/K as beforeand denote by xo the base point eK. Given Helgason’s classic presentation of thestructure of symmetric spaces [H62] there is no good reason for us to repeat ithere, so we use it freely and we encourage those readers new to the subject to learnit there.

In this section we introduce Helgason’s version of the Fourier transform on X,see [H65a,H68,H70]. At first we follow the exposition in [OS08] which is basedmore on representation theory, i.e. a la von Neumann and Harish-Chandra, ratherthan geometry as did Helgason. For additional information see the more modernrepresentation theory approach of [OS08], although we caution the reader that insome places notation and definitions differ.

The regular action of G on L2(X) is �gf(y) = f(g−1 · y), g ∈ G and y ∈ X.For an irreducible unitary representation (π, Vπ) of G and f ∈ L1(X) set

π(f) =

∫G

f(g)π(g) dg .

Here we have pulled back f to a right K-invariant function on G. If π(f) �= 0 thenV Kπ = {v ∈ Vπ | (∀k ∈ K) π(k)v = v} is nonzero. Furthermore, as (G,K) is a

Gelfand pair we have dimV Kπ = 1, in which case (π, Vπ) is called spherical.

Fix a unit vector eπ ∈ V Kπ . Then Tr(π(f)) = (π(f)eπ, eπ) and ‖π(f)‖HS =

‖π(f)eπ‖. Note that both (π(f)eπ, eπ) and ‖π(f)eπ‖ are independent of the choiceof eπ. Let GK be the set of equivalence classes of irreducible unitary sphericalrepresentations of G. Then as G is a type one group, there exists a measure μ on

GK such that

(2.5) f(g · xo) =

∫GK

(π(f)eπ, π(g)eπ)dμ(π) and ‖f‖22 =

∫GK

‖π(f)eπ‖2HS dμ(π) .

Harish-Chandra, see [HC54,HC57,HC58,HC66], determined the represen-tations that occur in the support of the measure in the decomposition (2.5), as wellas an explicit formula for the Plancherel measure for the spherical Fourier transformdefined by him.

Helgason’s formulation is motivated by “plane waves”. First we fix parameters.Let (λ, b) ∈ a∗

C×K/M and define an “exponential function” eλ,b : X → C by

eλ,b(x) = eb(x)λ−ρ ,

where eb(x) = a(x−1b) from the Iwasawa decomposition. Let Hλ = L2(K/M) withaction

πλ(g)f(b) = eλ,b(g · xo)f(g−1 · b) .

It is easy to see that πλ is a representation with a K-fixed vector eλ(b) = 1 for allb ∈ K/M ; there is a G-invariant pairing

(2.6) Hλ ×H−λ → C , 〈f, g〉 :=∫K/M

f(b)g(b) db ;


and it is unitary if and only if λ ∈ ia∗ and irreducible for almost all λ [K75,H76].

With fλ := πλ(f)eλ we have

(2.7) fλ(b) = πλ(f)eλ(b) =

∫G

f(g)πλ(g)eλ(b) dg =

∫X

f(x)eλ,b(x) dx .

Then f(λ, b) := fiλ(b) is the Helgason Fourier transform on X, see [H65a, Thm2.2].

Recall the little Weyl group W . The representation πwλ is known to be equiv-alent with πλ for almost all λ ∈ a∗

C. Hence for such λ there exists an intertwining

operator

A(w, λ) : Hλ → Hwλ .

The operator is unique, up to scalar multiples, by Schur’s lemma. We normalizeit so that A(w, λ)eλ = ewλ. The family {A(w, λ)} depends meromorphically on λand A(w, λ) is unitary for λ ∈ ia∗. Our normalization implies that

(2.8) A(w, λ)fλ = fwλ .

We can now formulate the Plancherel Theorem for the Fourier transform in thefollowing way, see [H65a, Thm 2.2] and also [H70, p. 118].

First let

c(λ) =

∫N

a(n)−λ−ρ dn

be the Harish-Chandra c-function, λ in a positive chamber. The Gindikin- Karpele-vich formula for the c-function [GK62] gives a meromorphic extension of c to allof a∗

C. Moreover c is regular and of polynomial growth on ia∗.To simplify the notation let dμ(λ, kM) be the measure (#W |c(λ)|)−1 dλd(kM)

on ia∗ ×K/M .:

Theorem 2.5 ([H65a]). The Fourier transform establishes a unitary isomor-phism

L2(X) ∫ ⊕

ia∗/W

(πλ,Hλ)dλ

|c(λ)|2 .

Furthermore, for f ∈ C∞c (X) we have

f(x) =

∫ia∗×K/M

fλ(b)e−λ,b(x) dμ(λ, b) .

Said more explicitly, the Fourier transform extends to a unitary isomorphism

L2(X) → L2(ia∗, dμ, L2(K/M)

)W=

{ϕ ∈ L2

(ia∗, dμ, L2(K/M)

) ∣∣ (∀w ∈W )A(w, λ)ϕ(λ) = ϕ(wλ)}.

To connect it with Harish-Chandra’s spherical transform notice that if f is left

K-invariant, then b �→ fλ(b) = f(λ) is independent of b and the integral 2.7 can bewritten as

(2.9) f(λ) =

∫X

∫K/M

eλ,b(x) db dx =

∫X

f(x)ϕλ(x) dx

where ϕλ is the spherical function

ϕλ(x) =

∫K

a(g−1k)λ−ρ dk .


Then (2.9) is exactly the Harish-Chandra spherical Fourier transform [HC58] andthe proof of Theorem 2.5 can be reduced to that formulation.

Since ϕλ = ϕμ if and only if that there exists w ∈ W such that wλ = μ, the

spherical Fourier transform f(λ) is W invariant. The Plancherel Theorem reducesto

Theorem 2.6. The spherical Fourier transform sets up an unitary isomorphism

L2(X)K L2

(ia∗/W,

dλ

|c(λ)|2

).

If f ∈ Cc(X)K then

f(x) =1

#W

∫ia∗

f(λ)ϕ−λ(x)dλ

|c(λ)|2 .

A very related result is the Paley-Wiener theorem which describes the imageof the smooth compactly supported functions by the Helgason Fourier Transform.For K-invariant functions in [H66] Helgason formulated the problem and solvedit modulo an interchange of a specific integral and sum. The justification for theinterchange was provided in [G71]; a new proof was given in [H70, Ch.II Thm. 2.4].The Paley-Wiener theorem for functions in C∞

c (X) was announced in [H73a] anda complete proof was given in [H73b, Thm. 8.3]. Later, Torasso [T77] producedanother proof, and Dadok [D79] generalized it to distributions on X.

There are many applications of the Paley-Wiener Theorem and the ingredientsof its proof. For example an alternative approach to the inversion formula canbe obtained [R77]. The Paley-Wiener theorem was used in [H73b] in the proofof surjectivity discussed in the next section, and in [H76] to prove the necessaryand sufficient condition for the bijectivity of the Poisson transform for K-finitefunctions on K/M to be discussed subsequently. The Paley-Wiener theorem playsan important role in the study of the wave equation on X as will be discussed later.

For the group G, an analogous theorem, although much more complicatedin statement and proof, was finally obtained by Arthur [A83], see also [CD84,CD90,vBS05]. In [D05] the result was extended to non K-finite functions. Theequivalence of the apparently different formulations of the characterization can befound in [vBSo12]. For semisimple symmetric spaces G/H it was done by vanden Ban and Schlichtkrull [vBS06]. The local Paley-Wiener theorem for compactgroups was derived by Helgason’s former student F. Gonzalez in [G01] and then

for all compact symmetric spaces in [BOP05,C06, OS08, OS10, OS11].

2.5. Solvability for D ∈ D(X). We come to one of Helgason’s major results:a resolution of the solvability problem for D ∈ D(X). We have seen the existenceof a fundamental solution allows one to solve the inhomogeneous equation: givenf ∈ C∞

c (X) does there exists u ∈ C∞(X) with Du = f? But what if f ∈ C∞(X)?This is much more difficult. Given Helgason’s approach outlined earlier it is naturalthat once again he needs a Radon-type transform but more general than for K bi-invariant functions.

The Radon transform on symmetric spaces of the noncompact type is, as men-tioned in the earlier section, an example of the double fibration transform andprobably one of the motivating examples for S. Helgason to introduce this general


framework. Here the double fibration is given by

(2.10) G/M

π

��

��

p

��

��

X = G/K Ξ = G/MN

and the corresponding transforms are for compactly supported functions:

f(g · ξo) =∫N

f(gn · xo) dn and ϕ∨(g · xo) =

∫K

ϕ(gk · ξo) dk .

As mentioned before, in theK bi-iinvariant setting this type of Radon transformhad already appeared (with an extra factor aρ) in the work of Harish-Chandra[HC58] via the map f �→ Ff . It also appeared in the fundamental work by Gelfandand Graev [GG59,GG62] where they introduced the “horospherical method”.

In this section we introduce the Radon transform on X and discuss some of itsproperties. It should be noted that Helgason introduced the Radon transform in[H63a,H63b] but the Fourier transform only appeared later in [H65a], see also[H66b].

We have seen that the Fourier transform on X gives a unitary isomorphism

L2(X) ∫ ⊕

a+

(πλ,Hλ)dλ

|c(λ)|2

whereas the Fourier transform in the A-variable gives a unitary isomorphism

L2(Ξ) ∫ ⊕

ia

(πλ,Hλ) dλ .

As the representations πλ and πwλ, w ∈ W , are equivalent this has the equiv-alent formulation

L2(Ξ) (#W )L2(X) .

In hindsight we could construct an intertwining operator from the followingsequence of maps

L2(X)→ L2

(K/M × ia∗,

dλ

|c(λ)|2

)→ L2(K/M × ia∗, dλ)→ L2(Ξ)

obtained with b = k · bo from the sequence:

f �→ fλ(b) �→1

c(λ)f(λ, b) �→ F−1

A

(1

c(·) f(·, b))(a) =: Λ(f)(ka · ξo) .


This idea plays a role in the inversion of the Radon transform, but instead westart with the Fourier transform on X given by (2.7). Then using b = k · bo we have

f(λ, b) =

∫X

f(x)eλ,b(x) dx

=

∫X

f(g · xo)a(g−1l)λ−ρ dg

=

∫X

f(lg · xo)a(g−1)λ−ρ, dg

=

∫A

∫N

f(lan · xo)a−λ+ρ dnda

= FA((·)ρR(f)(l(·))(λ) .Here R(f) = f is the Radon Transform from before. Thus we obtain that the

factorization of the unitary G map discussed above, namely the Fourier transformon L2(X) is followed by the Radon transform, which is then followed by the AbelianFourier transform on A, all this modulo the application of the pseudo-differentialoperator J corresponding to the Fourier multiplier 1/c(λ). Following [H65a] and[H70, p. 41 and p. 42] we therefore define the operator Λ by

Λ(f)(ka · ξo) = a−ρJa(aρf(ka · ξo)) .

We then get [H65a, Thm. 2.1] and [H70]:

Theorem 2.7. Let f ∈ C∞c (X). Then

#W

∫X

|f(x)|2 dx =

∫Ξ

|ΛR(f)(ξ)|2 dξ

and f �→ 1#W ΛR(f) extends to an isometry into L2(X). Moreover, for f ∈ C∞

c (X)

f(x) =1

#W(ΛΛ∗f)∨(x) .

With inversion in hand, in [H63b] and [H73b] Helgason obtains the key prop-erties of the Radon transform needed for the analysis of invariant differential op-erators on X. First we have the compatibility with a type of Harish-Chandraisomorphism:

Theorem 2.8. There exists a homorphism Γ : D(X) → D(Ξ) such that forf ∈ Cc(X) we have R(Df) = Γ(D)R(f).

Then using the Paley-Wiener Theorem for the symmetric space X Helgasongeneralizes his earlier support theorem.

Theorem 2.9 ( [H73b]). Let f ∈ C∞c (X) satisfy the following conditions:

(1) There is a closed ball V in X.

(2) The Radon transform f(ξ) = 0 whenever the horocycle ξ is disjoint fromV .

Then f(x) = 0 for x /∈ V .

He now has all the pieces of the proof of his surjectivity result.

Theorem 2.10. [H73b, Thm. 8.2] Let D ∈ D(X). Then

DC∞(X) = C∞(X).


The support theorem has now been extended to noncompact reductive sym-metric spaces by Kuit [K11].

2.6. The Poisson Transform. On a symmetric space X the use of the Pois-son transform has a long and rich history. But into this story fits a very preciseand important contribution - the “Helgason Conjecture”. In this section we recallbriefly the background from Helgason’s work leading to this major result.

Let g ∈ L2(K/M) and f ∈ C∞c (X). Recall from Theorem 2.5 that the Fourier

transform can be viewed as having values in L2(ia∗, dλ#W |c(λ)|2 , L

2(K/M))W . Denote

the Fourier transform on X by FX(f)(λ) = fλ and by F∗X its adjoint. Then we

evaluate F∗X as follows

〈FX(f), g〉 = 〈f,F∗X(g)〉

=

∫X

f(x)

∫ia∗

(∫K/M

e−λ,b(x)g(b) db

)dλ

|c(λ)|2 dx .

The function inside the parenthesis is the Poisson transform

(2.11) Pλ(g)(x) :=

∫K/M

e−λ,b(x)g(b) db.

Helgason had made the basic observation that the functions eλ,b are eigenfunctionsfor D(X), i.e., there exists a character χλ : D(X)→ C such that

Deλ,b = χλ(D)eλ,b .

Indeed, they are fundamental to the construction of the Helgason Fourier transform.Here they form the kernel of the construction of eigenfunctions.

Let

(2.12) Eλ(X) := {f ∈ C∞(X) | (∀D ∈ D(X))Df = χλ(D)f} .Since D ∈ D(X) is invariant the group G acts on Eλ. This defines a continuous

representation of G where Eλ carries the topology inherited from C∞(X). We havePλg ∈ Eλ and Pλ : H∞

λ = C∞(K/M)→ Eλ is an intertwining operator.In the basic paper [H59] we have seen that various properties of joint solutions

of operators in D(X) are obtained. In hindsight, one might speculate about eigen-values different than 0 for operators in D(X), and what properties the eigenspacesmight have. In fact, such a question is first formulated precisely in [H70] whereseveral results are obtained. Are the eigenspaces irreducible? Do the eigenspaceshave boundary values? What is the image of the Poisson transform on variousfunction spaces?

In [H70] Helgason observed that, as b �→ e−λ,b(x) is analytic, the Poissontransform extends to the dual A′(K/M) of the space A(K/M) of analytic functionson K/M .

Recall the Harish-Chandra c-function c(λ) and denote by ΓX(λ) the denomi-nator of c(λ)c(−λ). The Gindikin- Karpelevich formula for the c-function gives anexplicit formula for ΓX(λ) as a product of Γ-functions. An element λ ∈ a∗

Cis simple

if the Poisson transform Pλ : C∞(K/M, )→ Eλ(X) is injective.

Theorem 2.11 (Thm. 6.1 [H76]). λ is simple if and only if the denominatorof the Harish-Chandra c-function is non-singular at λ.

This result was used by Helgason for the following criterion for irreducibility:


Theorem 2.12 (Thm 9.1, Thm. 12.1, [H76]). The following are equivalent:

(1) The representation of G on Eλ(X) is irreducible.(2) The principal series representation πλ is irreducible.(3) ΓX(λ)−1 �= 0.

In [H76] p.217 he explains in detail the relationship of this result to [K75].With irreducibility under control, Helgason turns to the range question. In [H76]for all symmetric spaces of the non-compact type, generalizing [H70, Thm. 3.2]for rank one spaces, he proves

Theorem 2.13. Every K-finite function in Eλ(X) is of the form Pλ(F ) forsome K-finite function on K/M .

In [H70, Ch. IV,Thm. 1.8] he examines the critical case of the Poincaredisk. Utilizing classical function theory on the circle he shows that eigenfunctionshave boundary values in the space of analytic functionals. This, coupled with theaforementioned analytic properties of the Poisson kernel allow him to prove

Theorem 2.14. Eλ(X) = Pλ(A′(T)) for λ ∈ ia∗

Those results initiated intense research related to finding a suitable compactifi-cation of X compatible with eigenfunctions of D(X); to hyperfunctions as a suitableclass of objects on the boundary to be boundary values of eigenfunctions; to thegeneralization of the Frobenius regular singular point theory to encompass the op-erators in D(X); and finally to the analysis needed to treat the Poisson transformand eigenfunctions on X. The result culminated in the impressive proof by Kashi-wara, Kowata, Minemura, Okamoto, Oshima and Tanaka [KKMOOT78] that thePoisson transform is a surjective map from the space of hyperfunctions on K/Monto Eλ(X), referred to as the “Helgason Conjecture”.

2.7. Conical Distributions. Let X be the upper halfplane C+ = {z ∈ C |Re (z) > 0} = SL(2,R)/SO(2). A horocycle in C is a circle in X meeting the realline tangentially or, if the point of tangency is ∞, real lines parallel to the x-axis.It is easy to see that the horocycles are the orbits of conjugates of the group

N =

{(1 x0 1

) ∣∣∣∣ x ∈ R}

.

This leads to the definition for arbitrary symmetric spaces of the noncompact type:

Definition 2.15. A horocycle in X is an orbit of a conjugate of N .

Denote by Ξ the set of horocycles. Using the Iwasawa decomposition it is easyto see that the horocycles are the subsets of X of the form gN · xo. Thus G actstransitively on Ξ and Ξ = G/MN . As we saw before

(2.13) L2(Ξ) ∫ ⊕

ia∗(πλ,Hλ) dλ (#W )L2(X)

the isomorphism being given by

φλ(g) :=

∫A

[aρϕ(ga · ξo)]a−λ da = FA([(·)ρ�g−1ϕ]|A)(λ) .

The description of L2(Ξ) (#W )L2(X) suggests the question of relating Kinvariant vectors with MN invariant vectors. But, as MN is noncompact, it follows


from the theorem of Howe and Moore [HM79] that the unitary representations Hλ,λ ∈ ia∗ do not have any nontrivial MN -invariant vectors. But they have MN -fixeddistribution vectors as we will explain.

Let (π, Vπ) be a representation of G in the Frechet space Vπ. Denote by V ∞π

the space of smooth vectors with the usual Frechet topology. The space V ∞π is

invariant under G and we denote the corresponding representation of G by π∞. Theconjugate linear dual of V ∞

π is denoted by V −∞π . The dual pairing V −∞

π ×V ∞π → C,

is denoted 〈·, ·〉. The group G acts on V −∞π by

〈π−∞(a)Φ, φ〉 := 〈Φ, π∞(a−1)φ〉 .The reason to use the conjugate dual is so that for unitary representations

(π, Vπ) we have canonical G-equivariant inclusions

V ∞π ⊂ Vπ ⊂ V −∞

π .

For the principal series representations we have more generally by (2.6)G-equivariantembeddings Hλ ⊂ H−∞

−λ .

Assume that there exists a nontrivial distribution vector Φ ∈ (V −∞π )MN . Then

we define TΦ : V ∞φ → C∞(Ξ) by TΦ(v; g · ξo) = 〈π−∞(g)Φ, v〉. Similarly, if T :

V ∞π → C∞(Ξ) is a continuous intertwining operator we can define a MN -invariant

distribution vector ΦT : V ∞π → C by 〈ΦT , v〉 = T (v; ξo). Clearly those two maps

are inverse to each other. The decomposition of L2(Ξ) in (2.13) therefore suggeststhat for generic λ we should have dim(H−∞

λ )MN = #W .As second motivation for studying MN -invariant distribution vectors is the fol-

lowing. Let (π, Vπ) be an irreducible unitary representation of G (or more generallyan irreducible admissible representation) and let Φ,Ψ ∈ (V −∞

π )MN . If f ∈ C∞c (Ξ)

then π−∞(f)Φ is well defined and an element in V ∞π . Hence 〈Ψ, π−∞(f)Φ〉 is a well

defined MN -invariant distribution on Ξ and all the invariant differential differentialoperators on Ξ coming from the center of the universal enveloping algebra act onthis distribution by scalars.

A final motivation for Helgason to study MN -invariant distribution vectors isthe construction of intertwining operators between the representations (πλ,Hλ) and(πwλ,Hwλ), w ∈W . This is done in Section 6 in [H70] but we will not discuss thishere but refer to [H70] as well as [S68,KS71,KS80,VW90] for more information.

We now recall Helgason’s construction for the principal series represenations(πλ,Hλ). For that it is needed that Hλ = L2(K/M) is independent of λ andH∞

λ = C∞(K/M). Let m∗ ∈ NK(a) be such that m∗M ∈ W is the longestelement. Then the Bruhat big cell, Nm∗AMN , is open and dense. Define

(2.14) ψλ(g) =

{aλ−ρ if g = n1m

∗aman2 ∈ Nm∗MAN0 if otherwise.

If Reλ > 0 then ψλ ∈ H−∞−λ

is an MN -invariant distribution vector. Helgason

then shows in Theorem 2.7 that λ �→ ψλ ∈ H−∞−λ

extends to a meromorphic family of

distribution vectors on all of a∗C. Similar construction works for the other N -orbits

NwMAN , w ∈W , leading to distribution vectors ψw,λ.Denote by D(Ξ) the algebra of G-invariant differential operators on Ξ. Then

H �→ DH extends to an isomorphisms of algebras S(a) D(Ξ), see [H70, Thm.2.2].

Definition 2.16. A distribution Ψ (conjugate linear) on G is conical if


(1) Ψ is MN -biinvariant.(2) Ψ is an eigendistribution of D(Ξ).

The distribution vectors ψw,λ then leads to conical distributions Ψw,λ and it isshown in [H70,H76] that those distributions generate the space of conical distri-butions for generic λ.

For λ ∈ a∗Clet C∞

c (Ξ)′λ (with the relative strong topology) denote the jointdistribution eigenspaces of D(Ξ) containing the function g · ξo �→ a(x)λ−ρ. Then Gacts on C∞

c (Ξ)′λ and according to [H70, Ch. III, Prop. 5.2] we have:

Theorem 2.17. The representation on C∞c (Ξ)′λ is irreducible if and only if πλ

is irreducible.

2.8. The Wave Equation. Of the many invariant differential equations onX the wave equation frequently was the focus of Helgason’s attention. We shalldiscuss some of this work, but will omit his later work on the multitemporal waveequation [H98a,HS99].

Let ΔRn =∑n

j=1

∂2

∂x2j

denote the Laplace operator on Rn. The wave-equation

on Rn is the Cauchy problem

(2.15) ΔRnu(x, t) =∂2

∂t2u(x, t) u(x, 0) = f(x),

∂

∂tu(x, 0) = g(x)

where the initial values f and g can be from C∞c (X) or another “natural” function

space. Assume that f, g ∈ C∞c (Rn) with support contained in a closed ball BR(0)

of radius R > 0 and centered at zero. The solution has a finite propagation speedin the sense that u(x, t) = 0 if ‖x‖ − R ≥ |t|. The Huygens’ principle asserts thatu(x, t) = 0 for |t| ≥ ‖x‖ + R. It always holds for n > 1 and odd but fails in evendimensions. It holds for n = 1 if g ∈ C∞

c (R) with mean zero.This equation can be considered for any Riemannian or pseudo-Riemannian

manifold. In particular it is natural to consider the wave equation for Riemanniansymmetric spaces of the compact or noncompact type. Helgason was interested inthe wave equation and the Huygens’ principle from early on in his mathematicalcareer, see [H64,H77,H84a,H86,H92a,H98]. One can probably trace that

interest to his friendship with L. Asgeirsson, an Icelandic mathematican who studiedwith Courant in Gottingen and had worked on the Huygens’ principle on Rn.

One can assume that in (2.15) we have f = 0 and for simplicity assume thatg is K-invariant. Then u can also be taken K-invariant. It is also more natural toconsider the shifted wave equation

(2.16) (ΔX + ‖ρ‖2)u(x, t) = ∂2

∂t2u(x, t) u(x, 0) = 0,

∂

∂tu(x, 0) = g(x)

There are three main approaches to the problem. The first is to use the HelgasonFourier transform to reduce (2.16) to the differential equation

(2.17)d2

dt2u(iλ, t) = −‖λ‖2u(λ, t) , u(λ, 0) = 0 and

d

dtu(λ, 0) = g(λ)

for λ ∈ ia∗. From the inversion formula we then get

u(x, t) =1

#W

∫ia∗

g(λ)ϕλ(x)sin ‖λ‖t‖λ‖

dλ

|c(λ)|2 .


One can then use the Paley-Wiener Theorem to shift the path of integration. Doingthat one might hit the singularity of the c(λ) function. If all the root multiplicitiesare even, then 1/c(λ)c(−λ) is a W -invariant polynomial and hence corresponds toan invariant differential operator on X.

Another possibility is to use the Radon transform and its compatibility withinvariant operators

R((Δ + ‖ρ‖2)f)|A = ΔAR(f)|Athen use the Helgason Fourier transform, and finally the Euclidean result on theHuygens’ principle. This was the method used in [OS92].

Finally, in [H92a] Helgason showed that

sin ‖λ‖t‖λ‖ =

∫X

eiλ,b(x) dτt(x) =

∫X

ϕ−λ(x) dτt(x)

for certain distribution τt and then proving a support theorem for τt.The result is [OS92,H92a]:

Theorem 2.18. Assume that all multiplicities are even. Then Huygens’s prin-ciple holds if rankX is odd.

It was later shown in [BOS95] that in general the solution has a specific ex-

ponential decay. In [BO97] it was shown, using symmetric space duality, that theHuygens’ principle holds locally for a compact symmetric spaces if and only it holdsfor the noncompact dual. The compact symmetric spaces were then treated moredirectly in [BOP05].

Acknowledgements. The authors want to acknowledge the work that thereferee did for this paper. His thorough and conscientious report was of great valueto us for the useful corrections he made and the helpful suggestions he offered.

References

[AF-JS12] N. B. Andersen, M. Flensted-Jensen, and H. Schlichtkrull, Cuspidal discrete seriesfor semisimple symmetric spaces, J. Funct. Anal. 263 (2012), no. 8, 2384–2408,DOI 10.1016/j.jfa.2012.07.009. MR2964687

[A83] J. Arthur, A Paley-Wiener theorem for real reductive groups, Acta Math. 150(1983), no. 1-2, 1–89, DOI 10.1007/BF02392967. MR697608 (84k:22021)

[vBS05] E. P. van den Ban and Henrik Schlichtkrull, Paley-Wiener spaces for real reductiveLie groups, Indag. Math. (N.S.) 16 (2005), no. 3-4, 321–349, DOI 10.1016/S0019-3577(05)80031-X. MR2313629 (2008c:22006)

[vBS06] E. P. van den Ban and H. Schlichtkrull, A Paley-Wiener theorem for reductivesymmetric spaces, Ann. of Math. (2) 164 (2006), no. 3, 879–909, DOI 10.4007/an-nals.2006.164.879. MR2259247 (2007k:22004)

[vBSo12] E. van den Ban and S. Souaifi, A comparison of Paley-Wiener theorems for realreductive Lie Groups Journal fur die reine und angewandte Mathematik, to appear.

[BO97] T. Branson and G. Olafsson, Helmholtz operators and symmetric space duality,Invent. Math. 129 (1997), no. 1, 63–74, DOI 10.1007/s002220050158. MR1464866(98f:58188)

[BOP05] T. Branson, G. Olafsson, and A. Pasquale, The Paley-Wiener theorem andthe local Huygens’ principle for compact symmetric spaces: the even multiplic-ity case, Indag. Math. (N.S.) 16 (2005), no. 3-4, 393–428, DOI 10.1016/S0019-3577(05)80033-3. MR2313631 (2008k:43021)

[BOS95] T. Branson, G. Olafsson, and H. Schlichtkrull, Huyghens’ principle in Rie-mannian symmetric spaces, Math. Ann. 301 (1995), no. 3, 445–462, DOI10.1007/BF01446638. MR1324519 (97f:58128)


[C06] R. Camporesi, The spherical Paley-Wiener theorem on the complex Grass-mann manifolds SU(p + q)/S(Up × Uq), Proc. Amer. Math. Soc. 134 (2006),no. 9, 2649–2659 (electronic), DOI 10.1090/S0002-9939-06-08408-5. MR2213744(2007a:43008)

[C42] S-S. Chern, On integral geometry in Klein spaces, Ann. of Math. (2) 43 (1942),178–189. MR0006075 (3,253h)

[D79] J. Dadok, Paley-Wiener theorem for singular support of K-finite distributions on

symmetric spaces, J. Funct. Anal. 31 (1979), no. 3, 341–354, DOI 10.1016/0022-1236(79)90008-9. MR531136 (80k:43011)

[CD84] L. Clozel and P. Delorme, Le theoreme de Paley-Wiener invariant pour lesgroupes de Lie reductifs, Invent. Math. 77 (1984), no. 3, 427–453, DOI10.1007/BF01388832 (French). MR759263 (86b:22015)

[CD90] L. Clozel and P. Delorme, Le theoreme de Paley-Wiener invariant pour les groupes

de Lie reductifs. II, Ann. Sci. Ecole Norm. Sup. (4) 23 (1990), no. 2, 193–228(French). MR1046496 (91g:22013)

[D05] P. Delorme, Sur le theoreme de Paley-Wiener d’Arthur, Ann. of Math. (2) 162

(2005), no. 2, 987–1029, DOI 10.4007/annals.2005.162.987 (French, with Englishsummary). MR2183287 (2006g:22009)

[E03] L. Ehrenpreis, The universality of the Radon transform, Oxford MathematicalMonographs, The Clarendon Press Oxford University Press, New York, 2003.With an appendix by Peter Kuchment and Eric Todd Quinto. MR2019604(2007d:58047)

[F16] P. Funk, Uber eine geometrische Anwendung der Abelschen Integralgleichung,Math. Ann. 77 (1915), no. 1, 129–135, DOI 10.1007/BF01456824 (German).MR1511851

[G71] R. Gangolli, On the Plancherel formula and the Paley-Wiener theorem for spher-ical functions on semisimple Lie groups, Ann. of Math. (2) 93 (1971), 150–165.MR0289724 (44 #6912)

[GGG00] I. M. Gelfand, S. G. Gindikin, and M. I. Graev, Selected topics in integral geome-

try, Translations of Mathematical Monographs, vol. 220, American MathematicalSociety, Providence, RI, 2003. Translated from the 2000 Russian original by A.Shtern. MR2000133 (2004f:53092)

[GG59] I. M. Gel′fand and M. I. Graev, Geometry of homogeneous spaces, representationsof groups in homogeneous spaces and related questions of integral geometry. I,Trudy Moskov. Mat. Obsc. 8 (1959), 321–390; addendum 9 (1959), 562 (Russian).MR0126719 (23 #A4013)

[GG62] I. M. Gel′fand and M. I. Graev, An application of the horisphere method to thespectral analysis of functions in real and imaginary Lobatchevsky spaces, TrudyMoskov. Mat. Obsc. 11 (1962), 243–308 (Russian). MR0148802 (26 #6306)

[G06] S. Gindikin, The horospherical Cauchy-Radon transform on compact symmetricspaces, Mosc. Math. J. 6 (2006), no. 2, 299–305, 406 (English, with English andRussian summaries). MR2270615 (2007j:43014)

[GK62] S. G. Gindikin and F. I. Karpelevic, Plancherel measure for symmetric Riemann-ian spaces of non-positive curvature, Dokl. Akad. Nauk SSSR 145 (1962), 252–255(Russian). MR0150239 (27 #240)

[GKO06] S. Gindikin, B. Krotz, and G. Olafsson, Horospherical model for holomorphicdiscrete series and horospherical Cauchy transform, Compos. Math. 142 (2006),no. 4, 983–1008, DOI 10.1112/S0010437X06001965. MR2249538 (2008a:22014)

[G01] F. B. Gonzalez, A Paley-Wiener theorem for central functions on compact Liegroups, Radon transforms and tomography (South Hadley, MA, 2000), Contemp.Math., vol. 278, Amer. Math. Soc., Providence, RI, 2001, pp. 131–136, DOI10.1090/conm/278/04601. MR1851484 (2002f:43005)

[HC54] Harish-Chandra, On the Plancherel formula for the right-invariant functions ona semisimple Lie group, Proc. Nat. Acad. Sci. U. S. A. 40 (1954), 200–204.MR0062748 (16,11f)

[HC57] Harish-Chandra, Spherical functions on a semisimple Lie group, Proc. Nat. Acad.Sci. U.S.A. 43 (1957), 408–409. MR0087043 (19,292f)


[HC58] Harish-Chandra, Spherical functions on a semisimple Lie group. I, Amer. J. Math.80 (1958), 241–310. MR0094407 (20 #925)

[HC66] Harish-Chandra, Discrete series for semisimple Lie groups. II. Explicit determi-nation of the characters, Acta Math. 116 (1966), 1–111. MR0219666 (36 #2745)

[H57] S. Helgason, Partial differential equations on Lie groups, Treizieme congres desmathematiciens scandinaves, tenu a Helsinki 18-23 aout 1957, Mercators Tryckeri,Helsinki, 1958, pp. 110–115. MR0106357 (21 #5091)

[H59] S. Helgason,Differential operators on homogeneous spaces, Acta Math. 102 (l959),239–299.

[H62] S. Helgason, Differential geometry and symmetric spaces, Pure and Applied Math-ematics, Vol. XII, Academic Press, New York, 1962. MR0145455 (26 #2986)

[H62a] S. Helgason, Some results on invariant theory, Bull. Amer. Math. Soc. 68 (1962),367–371. MR0166303 (29 #3580)

[H63a] S. Helgason, Duality and Radon transform for symmetric spaces, Bull. Amer.Math. Soc. 69 (1963), 782–788. MR0158408 (28 #1631)

[H63b] S. Helgason, Duality and Radon transform for symmetric spaces, Amer. J. Math.85 (1963), 667–692. MR0158409 (28 #1632)

[H63c] S. Helgason, Fundamental solutions of invariant differential operators on symmet-ric spaces, Bull. Amer. Math. Soc. 69 (1963), 778–781. MR0156919 (28 #162)

[H64] S. Helgason, Fundamental solutions of invariant differential operators on sym-metric spaces, Amer. J. Math. 86 (1964), 565–601. MR0165032 (29 #2323)

[H65] S. Helgason, The Radon transform on Euclidean spaces, compact two-point ho-mogeneous spaces and Grassmann manifolds, Acta Math. 113 (1965), 153–180.MR0172311 (30 #2530)

[H65a] S. Helgason, Radon-Fourier transforms on symmetric spaces and related grouprepresentations, Bull. Amer. Math. Soc. 71 (1965), 757–763. MR0179295(31 #3543)

[H66] S. Helgason, An analogue of the Paley-Wiener theorem for the Fourier trans-form on certain symmetric spaces, Math. Ann. 165 (1966), 297–308. MR0223497(36 #6545)

[H66b] S. Helgason, A duality in integral geometry on symmetric spaces, Proc. U.S.-JapanSeminar in Differential Geometry (Kyoto, 1965), Nippon Hyoronsha, Tokyo, 1966,pp. 37–56. MR0229191 (37 #4765)

[H68] S. Helgason, Lie groups and symmetric spaces, Battelle Rencontres. 1967 Lecturesin Mathematics and Physics, Benjamin, New York, 1968, pp. 1–71. MR0236325(38 #4622)

[H69] S. Helgason, Applications of the Radon transform to representations of semisim-ple Lie groups, Proc. Nat. Acad. Sci. U.S.A. 63 (1969), 643–647. MR0263987(41 #8586)

[H70] S. Helgason, A duality for symmetric spaces with applications to group represen-tations, Advances in Math. 5 (1970), 1–154 (1970). MR0263988 (41 #8587)

[H73a] S. Helgason, Paley-Wiener theorems and surjectivity of invariant differential op-erators on symmetric spaces and Lie groups, Bull. Amer. Math. Soc. 79 (1973),129–132. MR0312158 (47 #720)

[H73b] S. Helgason, The surjectivity of invariant differential operators on symmetricspaces. I, Ann. of Math. (2) 98 (1973), 451–479. MR0367562 (51 #3804)

[H76] S. Helgason, A duality for symmetric spaces with applications to group repre-sentations. II. Differential equations and eigenspace representations, Advances inMath. 22 (1976), no. 2, 187–219. MR0430162 (55 #3169)

[H77] S. Helgason, Solvability questions for invariant differential operators, Group the-oretical methods in physics (Proc. Fifth Internat. Colloq., Univ. Montreal, Mon-treal, Que., 1976), Academic Press, New York, 1977, pp. 517–527. MR0492067

(58 #11222)[H78] S. Helgason, Differential geometry, Lie groups, and symmetric spaces, Pure and

Applied Mathematics, vol. 80, Academic Press Inc. [Harcourt Brace JovanovichPublishers], New York, 1978. MR514561 (80k:53081)

[H80] S. Helgason, The Radon Transform, Birkhauser, Boston 1980, Russian translation,MIR, Moscow, 1983. 2nd edition, 1999.


[GGA] S. Helgason, Groups and geometric analysis, Pure and Applied Mathematics,vol. 113, Academic Press Inc., Orlando, FL, 1984. Integral geometry, invariantdifferential operators, and spherical functions. MR754767 (86c:22017)

[H84a] S. Helgason, Wave equations on homogeneous spaces, Lie group representations,III (College Park, Md., 1982/1983), Lecture Notes in Math., vol. 1077, Springer,Berlin, 1984, pp. 254–287, DOI 10.1007/BFb0072341. MR765556 (86c:58141)

[H86] S. Helgason, Some results on Radon transforms, Huygens’ principle and

X-ray transforms, Integral geometry (Brunswick, Maine, 1984), Contemp.Math., vol. 63, Amer. Math. Soc., Providence, RI, 1987, pp. 151–177, DOI10.1090/conm/063/876318. MR876318 (88d:22017)

[H89] S. Helgason, Value-distribution theory for analytic almost periodic functions, TheHarald Bohr Centenary. Proc. Symp. Copenhagen 1987, Munksgaard , Copen-hagen 1989, 93-102.

[H92] S. Helgason, Some results on invariant differential operators on symmetric spaces,Amer. J. Math. 114 (1992), no. 4, 789–811, DOI 10.2307/2374798. MR1175692(94a:22020)

[H92a] S. Helgason, Huygens’ principle for wave equations on symmetric spaces, J.Funct. Anal. 107 (1992), no. 2, 279–288, DOI 10.1016/0022-1236(92)90108-U.MR1172025 (93i:58151)

[H98] S. Helgason, Radon transforms and wave equations, Integral geometry, Radontransforms and complex analysis (Venice, 1996), Lecture Notes in Math., vol. 1684,Springer, Berlin, 1998, pp. 99–121, DOI 10.1007/BFb0096092. MR1635613(99j:58206)

[H98a] S. Helgason, Integral geometry and multitemporal wave equations, Comm.Pure Appl. Math. 51 (1998), no. 9-10, 1035–1071, DOI 10.1002/(SICI)1097-0312(199809/10)51:9/10¡1035::AID-CPA5¿3.3.CO;2-H. Dedicated to the memoryof Fritz John. MR1632583 (99j:58207)

[H00] S. Helgason, Groups and Geometric Analysis, A.M.S., Providence, RI, 2000.[H07] S. Helgason, The inversion of the X-ray transform on a compact symmetric space,

J. Lie Theory 17 (2007), no. 2, 307–315. MR2325701 (2008d:43011)

[H11] S. Helgason, Integral geometry and Radon transforms, Springer, New York, 2011.MR2743116 (2011m:53144)

[HS99] S. Helgason and H. Schlichtkrull, The Paley-Wiener space for the multitem-poral wave equation, Comm. Pure Appl. Math. 52 (1999), no. 1, 49–52, DOI10.1002/(SICI)1097-0312(199901)52:1¡49::AID-CPA2¿3.0.CO;2-S. MR1648417(99j:58208)

[HM79] R. E. Howe and C. C. Moore, Asymptotic properties of unitary representations,J. Funct. Anal. 32 (1979), no. 1, 72–96, DOI 10.1016/0022-1236(79)90078-8.MR533220 (80g:22017)

[J55] F. John, Plane waves and spherical means applied to partial differential equations,Interscience Publishers, New York-London, 1955. MR0075429 (17,746d)

[KKMOOT78] M. Kashiwara, A. Kowata, K. Minemura, K. Okamoto, T. Oshima, and M. Tanaka,Eigenfunctions of invariant differential operators on a symmetric space, Ann. ofMath. (2) 107 (1978), no. 1, 1–39, DOI 10.2307/1971253. MR485861 (81f:43013)

[KS71] A. W. Knapp and E. M. Stein, Intertwining operators for semisimple groups, Ann.of Math. (2) 93 (1971), 489–578. MR0460543 (57 #536)

[KS80] A. W. Knapp and E. M. Stein, Intertwining operators for semisimple groups.II, Invent. Math. 60 (1980), no. 1, 9–84, DOI 10.1007/BF01389898. MR582703(82a:22018)

[K75] B. Kostant, On the existence and irreducibility of certain series of representations,Bull. Amer. Math. Soc. 75 (1969), 627–642. MR0245725 (39 #7031)

[K09] B. Krotz, The horospherical transform on real symmetric spaces: kernel and cok-

ernel, Funktsional. Anal. i Prilozhen. 43 (2009), no. 1, 37–54, DOI 10.1007/s10688-009-0004-3 (Russian, with Russian summary); English transl., Funct. Anal. Appl.43 (2009), no. 1, 30–43. MR2503864 (2010h:43010)

[K11] J. J. Kuit, Radon transformation on reductive symmetric spaces: support theorems.Preprint, arXiv:1011.5780.


[N01] F. Natterer, The mathematics of computerized tomography, Classics in AppliedMathematics, vol. 32, Society for Industrial and Applied Mathematics (SIAM),Philadelphia, PA, 2001. Reprint of the 1986 original. MR1847845 (2002e:00008)

[OS92] G. Olafsson and H. Schlichtkrull, Wave propagation on Riemannian symmet-ric spaces, J. Funct. Anal. 107 (1992), no. 2, 270–278, DOI 10.1016/0022-1236(92)90107-T. MR1172024 (93i:58150)

[OS08] G. Olafsson and H. Schlichtkrull, Representation theory, Radon transform andthe heat equation on a Riemannian symmetric space, Group representations, er-godic theory, and mathematical physics: a tribute to George W. Mackey, Con-temp. Math., vol. 449, Amer. Math. Soc., Providence, RI, 2008, pp. 315–344, DOI10.1090/conm/449/08718. MR2391810 (2009g:22021)

[OS08a] G. Olafsson and H. Schlichtkrull, A local Paley-Wiener theorem for com-

pact symmetric spaces, Adv. Math. 218 (2008), no. 1, 202–215, DOI10.1016/j.aim.2007.11.021. MR2409413 (2010c:43019)

[OS10] G. Olafsson and H. Schlichtkrull, Fourier series on compact symmetric spaces: K-finite functions of small support, J. Fourier Anal. Appl. 16 (2010), no. 4, 609–628,DOI 10.1007/s00041-010-9122-9. MR2671174 (2012d:43014)

[OS11] G. Olafsson and H. Schlichtkrull, Fourier Transform of Spherical Distributions onCompact Symmetric Spaces, Mathematica Scandinavica 190 (2011), 93–113.

[OW11] G. Olafsson and J. A. Wolf, Extension of symmetric spaces and restriction ofWeyl groups and invariant polynomials, New developments in Lie theory and itsapplications, Contemp. Math., vol. 544, Amer. Math. Soc., Providence, RI, 2011,pp. 85–100, DOI 10.1090/conm/544/10749. MR2849714

[Sel] G. Olafsson and H. Schlichtkrull (Ed.), The selected works of Sigurður Helgason.AMS, Providence, RI, 2009.

[R17] J. Radon, Uber die Bestimmung von Funktionen durch ihre Integralwerte langs

gewisser Mannigfaltigkeiten, Ber. Verth. Sachs. Akad. Wiss. Leipzig. Math. Nat.kl. 69 (1917) 262–277.

[RaGes] J. Radon, Gesammelte Abhandlungen. Band 1, Verlag der OsterreichischenAkademie der Wissenschaften, Vienna, 1987 (German). With a foreword by OttoHittmair; Edited and with a preface by Peter Manfred Gruber, Edmund Hlawka,Wilfried Nobauer and Leopold Schmetterer. MR925205 (89i:01142a)

[R77] J. Rosenberg, A quick proof of Harish-Chandra’s Plancherel theorem for sphericalfunctions on a semisimple Lie group, Proc. Amer. Math. Soc. 63 (1977), no. 1,143–149. MR0507231 (58 #22391)

[R04] F. Rouviere, Geodesic Radon transforms on symmetric spaces, C. R. Math. Acad.Sci. Paris 342 (2006), 1–6.

[R02] B. Rubin, Inversion formulas for the spherical Radon transform and the gen-eralized cosine transform, Adv. in Appl. Math. 29 (2002), no. 3, 471–497, DOI

10.1016/S0196-8858(02)00028-3. MR1942635 (2004c:44006)[S68] G. Schiffmann, Integrales d’entrelacement. C. R. Acad. Sci. Paris Ser. A–B 266

(1968) A47–A49.[S09] Recountings, A K Peters Ltd., Wellesley, MA, 2009. Conversations with MIT math-

ematicians; Edited by J. Segel. MR2516491 (2010h:01010)[T77] P. Torasso, Le theoreme de Paley-Wiener pour l’espace des fonctions indefiniment

differentiables et a support compact sur un espace symetrique de type non-compact,J. Functional Analysis 26 (1977), no. 2, 201–213 (French). MR0463811 (57 #3750)

[VW90] D. A. Vogan Jr. and N. R. Wallach, Intertwining operators for real reductivegroups, Adv. Math. 82 (1990), no. 2, 203–243, DOI 10.1016/0001-8708(90)90089-6. MR1063958 (91h:22022)

Department of Mathematics, Louisiana State University, Baton Rouge, Louisiana

70803


Department of Mathematics, Ohio State University, Columbus, Ohio 43210


Research and Expository Articles


Microlocal analysis of an ultrasound transform with circularsource and receiver trajectories

G. Ambartsoumian, J. Boman, V. P. Krishnan, and E. T. Quinto

This article is dedicated to Sigurdur Helgason on the occasion of his eighty fifth birthday. Wethank him for creating so much beautiful mathematics and for being a friend and mentor to so

many people in the field.

Abstract. We consider a generalized Radon transform that is used in ul-trasound reflection tomography. In our model, the ultrasound emitter andreceiver move at a constant distance apart along a circle. We analyze themicrolocal properties of the transform R that arises from this model. As aconsequence, we show that, for distributions with support contained in a discDb sufficiently inside the circle, R∗R is an elliptic pseudodifferential operator.We provide a local filtered back projection algorithm, L = R∗DR where D isa well-chosen differential operator. We prove that L is an elliptic pseudodiffer-ential operator of order 1 and so for f ∈ E′(Db), Lf shows all singularities off , and we provide reconstructions illustrating this point. Finally, we discussan extension with some modifications of our result outside of Db.

1. Introduction

Ultrasound reflection tomography (URT) is one of the safest and most costeffective modern medical imaging modalities (e.g., see [13–16] and the referencesthere). During its scanning process, acoustic waves emitted from a source reflectfrom inhomogeneities inside the body, and their echoes are measured by a receiver.This measured data is then used to recover the unknown ultrasonic reflectivityfunction, which is used to generate cross-sectional images of the body.

2010 Mathematics Subject Classification. Primary 44A12, 92C55, 35S30, 35S05 Secondary:

58J40, 35A27.The authors thank the American Mathematical Society for organizing the Mathematical Re-

search Communities Conference on Inverse Problems that encouraged our research collaboration.

The first and fourth author thank MSRI at Berkeley for their hospitality while they discussedthese results. The first, third, and fourth author appreciate the support of the American Instituteof Mathematics where they worked on this article as part of their SQuaREs program. The thirdauthor thanks Tufts University and TIFR CAM for providing an excellent research environment.All authors thank the anonymous referee for thoughtful comments.The first author was supported in part by DOD CDMRP Synergistic Idea AwardBC063989/W81XWH-07-1-0640, by Norman Hackerman Advanced Research Program (NHARP)Consortium Grant 003656-0109-2009 and by NSF grant DMS-1109417. The third author wassupported in part by NSF Grants DMS-1028096 and DMS-1129154 (supplements to the fourthauthor’s NSF Grant DMS-0908015) and DMS-1109417. The fourth author was supported in partby NSF Grant DMS-0908015.


45

46 G. AMBARTSOUMIAN, J. BOMAN, V. P. KRISHNAN, AND E. T. QUINTO

In a typical setup of ultrasound tomography, the emitter and receiver are com-bined into one device (transducer). The transducer emits a short acoustic pulse intothe medium, and then switches to receiving mode, recording echoes as a functionof time. Assuming that the medium is weakly reflecting (i.e., neglecting multiplereflections), and that the speed of sound propagation c is constant1, the echoesmeasured at time t uniquely determine the integrals of the reflectivity functionover concentric spheres centered at the transducer location and radii r = ct/2 (seeFig. 1 (a) below, [16] and the references there). By focusing the transducer one canconsider echoes coming only from a certain plane, hence measuring the integralsof the reflectivity function in that plane along circles centered at the transducerlocation [15]. Moving the transducer along a curve on the edge of the body, andrepeating the measurements one obtains a two-dimensional family of integrals ofthe unknown function along circles. Hence the problem of image reconstruction inURT can be mathematically reduced to the problem of inverting a circular Radontransform, which integrates an unknown function of two variables along a two-dimensional family of circles.

In the case when the emitter and receiver are separated, the echoes recorded bya transducer correspond to the integrals of the reflectivity function along confocalellipses. The foci of these ellipses correspond to the locations of the emitter andreceiver moving along a fixed curve. While this more general setup has been gainingpopularity in recent years (e.g., see [13,14,20]), the mathematical theory relatedto elliptical Radon transforms is relatively undeveloped.

In this paper we consider a setup where the separated emitter and receiver movealong a circle at a fixed distance apart (see Fig. 1 (b)). The circular trajectory oftheir motion is both the simplest case mathematically and the one most often usedin practice. By using a dilation and translation, we can assume the circle hasradius r = 1 centered at 0. We study the microlocal properties of transform Rwhich integrates an unknown function along this family of ellipses.

We prove that R is an elliptic Fourier integral operator (FIO) of order −1/2using the microlocal framework of Guillemin and Guillemin-Sternberg [6,8] for gen-eralized Radon transforms. We use this to understand when the imaging operatorR∗R is a pseudodifferential operator. Specifically, we show that for distributionssupported in a smaller disc (the disc Db of (2.2)), a microlocal condition introducedby Guillemin [6], the so called Bolker assumption, is satisfied and, consequently,for such distributions R∗R is an elliptic pseudodifferential operator. We constructa differential operator D such that L = R∗DR is elliptic of order 1. From thetomographic point of view this means that using the measured data one can stablyrecover all singularities of objects supported inside that disc. We provide recon-structions that illustrate this. We note that, even when appropriately defined (seeRemark 5.1), L does not recover all singularities in the complement of Db andL can mask or add singularities. So, in this sense Db is optimal. Stefanov andUhlmann [18] show for a related problem in monostatic radar that singularities canbe masked or added, even with arbitrary flight paths.

In Section 2, we introduce the basic notation and microlocal analysis as wellas Guillemin’s framework for understanding Radon transforms. In Section 3, we

1This assumption is reasonable in ultrasound mammography, since the speed of sound isalmost constant in soft tissue.

MICROLOCAL ANALYSIS OF AN ULTRASOUND TRANSFORM 47

transducer

Collocated emitter and receiver

r=ct/2

(a)

Separated emitted and receiver

emitter

receiver

r1

r2

r1+r2=ct

(b)

Figure 1. A sketch of integrating curves in URT

present the microlocal regularity theorem, and in Section 4 we present reconstruc-tions from a local filtered backprojection algorithm (see equation (4.1)) that illus-trates the conclusion of the main theorem. The proof of the microlocal regularitytheorem is in Section 5.

2. Definitions and Preliminaries

We will first define the elliptical Radon transform we consider, provide thegeneral framework for the microlocal analysis of this transform, and show that ourtransform fits within this framework.

2.1. The Elliptical Transform. Recall that in the URT model we considerin this paper, the emitter and receiver move along the circle of radius 1 centeredat 0 and are at a fixed distance apart. We denote the fixed difference between thepolar angles of emitter and receiver by 2α, where α ∈ (0, π/2) (see Fig. 2) anddefine

(2.1) a = sinα, b = cosα.

As we will see later our main result relies on the assumption that the supportof the function is small enough. More precisely, we will assume our function issupported in the ball

(2.2) Db = {x ∈ R2∣∣ |x| < b}.

We parameterize the trajectories of the transmitter (emitter) and receiver, re-spectively, as

γT (s) = (cos(s− α), sin(s− α))

γR(s) = (cos(s+ α), sin(s+ α)) for s ∈ [0, 2π].

Thus, the emitter and receiver rotate around the unit circle and are always 2aunits apart. For s ∈ [0, 2π] and L > 2a, let

E(s, L) = {x ∈ R2∣∣|x− γT (s)|+ |x− γR(s)| = L}.

Note that the center of the ellipse E(s, L) is (b cos s, b sin s) and L is the diameterof the major axis of E(s, L), the so called major diameter. This is why we require Lto be greater than the distance between the foci, 2a. As a function of s, the ellipse


1

b

bs

aa

L

Figure 2. A sketch of the domain and the notations

E(s, L) is 2π-periodic, and so we will identify s ∈ [0, 2π] with the point (cos s, sin s)on the unit circle when convenient.

Let

Y = {(s, L)∣∣s ∈ [0, 2π], L > 2a},

then Y is the set of parameters for the ellipses.Let (s, L) ∈ Y . The elliptical Radon transform of a locally integrable function

f : R2 → R is defined as

Rf(s, L) =

∫x∈E(s,L)

f(x)dt(x)

where dt is the arc length measure on the ellipse E(s, L). The backprojectiontransform is defined for g ∈ C(Y ) and x ∈ Db as

(2.3) R∗g(x) =

∫s∈[0,2π]

g(s, |x− γR(s)|+ |x− γT (s)|)w(s, x)ds

where the positive smooth weight w(s, x) is chosen so that R∗ is the L2 adjoint ofR with measure dx on Db and ds dL on Y (see equation (2.13) in Example 2.2).Using the parameterization of ellipses (s, L) one sees that R∗g(x) integrates with asmooth measure over the set of all ellipses in our complex passing through x ∈ Db.These transforms can be defined for distributions with support larger than Db, butthe definition of R∗ is more complicated for x /∈ Db as will be discussed in Remark5.1.

We can compose R and R∗ on domain E ′(Db) for the following reasons. Iff ∈ D(Db) then Rf has compact support in Y since Rf(s, L) is zero for L near 2a.Clearly, R : D(Db) → D(Y ) is continuous so R∗ : D′(Y ) → D′(Db) is continuous.Since x ∈ Db in the definition of R∗, (2.3), R∗ integrates over a compact set [0, 2π].Therefore, R∗ : E(Y )→ E(Db) is continuous, so R : E ′(Db)→ E ′(Y ) is continuous.Therefore, R∗ can be composed with R on domain E ′(Db).

2.2. Microlocal Definitions. We now introduce some notation so we candescribe our operators microlocally. Let X and Y be smooth manifolds and let

C ⊂ T ∗(Y )× T ∗(X),

then we let

C′ = {(y, η, x, ξ)∣∣(y, η, x,−ξ) ∈ C}.


The transpose relation is Ct ⊂ T ∗(X)× T ∗(Y ):

Ct = {(x, ξ, y, η)∣∣(y, η, x, ξ) ∈ C}

If D ⊂ T ∗(X)× T ∗(Y ), then the composition D ◦ C is defined

D ◦ C = {(x′, ξ′, x, ξ)∣∣∃(y, η) ∈ T ∗(Y )

with (x′, ξ′, y, η) ∈ D, (y, η, x, ξ) ∈ C}.

2.3. The Radon Transform and Double Fibrations in General. Guille-min first put the Radon transform into a microlocal framework, and we now describethis approach and explain how our transform R fits into this framework. We willuse this approach to prove Theorem 3.1.

Guillemin used the ideas of pushforwards and pullbacks to define Radon trans-forms and show they are Fourier integral operators (FIOs) in the technical report[5], and these ideas were outlined in [8, pp. 336-337, 364-365] and summarized in[6]. He used these ideas to define FIOs in general in [7,8]. The dependence on themeasures and details of the proofs for the case of equal dimensions were given in[17].

Given smooth connected manifolds X and Y of the same dimension, let Z ⊂Y ×X be a smooth connected submanifold of codimension k < dim(X). We assumethat the natural projections

(2.4)Z

πL↙ ↘πR

Y X

are both fiber maps. In this case, we call (2.4) a double fibration. This frameworkwas used by Helgason [9] to define Radon transforms in a group setting, and it waslater generalized to manifolds without a group structure [4].

Following Guillemin and Sternberg, we assume that πR is a proper map; thatis, the fibers of πR : Z → X are compact.

The double fibration allows us to define sets of integration for the Radon trans-form and its dual as follows. For each y ∈ Y let

E(y) = πR

(π−1L ({y})

),

then E(y) is a subset of X that is diffeomorphic to the fiber of πL : Z → Y . Foreach x ∈ X let

F (x) = πL

(π−1R ({x})

),

then F (x) ⊂ Y is diffeomorphic to the fiber of πR : Z → X. Since πR is proper,F (x) is compact.

Guillemin defined the Radon transform and its dual using pushforwards andpullbacks. The pullback of the function f ∈ C∞(X) is

πR∗f(z) = f(πR(z))

and the pushforward to X of a measure ν on Z is the measure satisfying∫X

f(x)dπR∗(ν) =

∫Z

(πR∗f)dν.

The pushforward and pullback for πL are defined similarly.


By choosing smooth nowhere zero measures μ on Z, m on X, and n on Y ,one defines the generalized Radon transform of f ∈ C∞

c (X) as the function Rf forwhich

(2.5) (Rf)n = πL∗ ((πR∗f)μ) .

The dual transform for g ∈ C∞(Y ) is the function R∗g for which

(R∗g)m = πR∗ ((πL∗g)μ) .

This definition is natural because R∗ is automatically the dual to R by the dualitybetween pushforwards and pullbacks.

The measures μ, n and m give the measures of integration for R and R∗ asfollows. Since πL : Z → Y is a fiber map locally above y ∈ Y , the measure μ can bewritten as a product of the measure n and a smooth measure on the fiber. This fiberis diffeomorphic to E(y), and the measure on the fiber can be pushed forward usingthis diffeomorphism to a measure μy on E(y): the measure μy satisfies μ = μy × n(under the identification of E(y) with the fiber of Z above y), and the generalizedRadon transform defined by (2.5) can be written

Rf(y) =

∫x∈E(y)

f(x)dμy(x)

In a similar way, the measure μx on each set F (x) satisfies μ = μx ×m (under theidentification of the fiber of πR with F (x)), and the dual transform can be written

R∗g(x) =

∫y∈F (x)

g(y)dμx(y)

[8] (see also [17, p. 333]).Since the sets F (x) are compact, one can compose R∗ and R for f ∈ Cc(X).

We include the uniqueness assumptions E(y1) = E(y2) if and only if y1 = y2 andF (x1) = F (x2) if and only if x1 = x2.

Guillemin showed ([5, 6] and with Sternberg [8]) that R is a Fourier inte-gral distribution associated with integration over Z and canonical relation C =(N∗(Z)\{0})′. To understand the properties of R∗R, one must investigate themapping properties of C. Let ΠL : C → T ∗(Y ) and ΠR : C → T ∗(X) be theprojections. Then we have the following diagram:

(2.6)

CΠL↙ ↘ΠR

T ∗(Y ) T ∗(X)

This diagram is the microlocal version of (2.4).

Definition 2.1 ([5, 6]). Let X and Y be manifolds with dim(Y ) = dim(X)and let C ⊂ (T ∗(Y ) × T ∗(X))\{0} be a canonical relation. Then, C satisfies theBolker Assumption if

ΠY : C → T ∗(Y )

is an injective immersion.

This definition was originally proposed by Guillemin [5],[6, p. 152], [8, p. 364-365] because Ethan Bolker proved R∗R is injective under a similar assumption fora finite Radon transform. Guillemin proved that if the measures that define theRadon transform are smooth and nowhere zero, and if the Bolker Assumption holds


(and R is defined by a double fibration for which πR is proper), then R∗R is anelliptic pseudodifferential operator.

Since we assume dim(Y ) = dim(X), if ΠY : C → T ∗(Y ) is an injective immer-sion, then ΠY maps to T ∗(Y )\{0} and ΠX is also an immersion [10]. Therefore,ΠX maps to T ∗(X)\{0}. So, under the Bolker Assumption, C ⊂ (T ∗(Y )\{0}) ×(T ∗(X)\{0}) and so R is a Fourier integral operator according to the definition in[19].

We now put our elliptical transform into this framework.

Example 2.2. For our transform R, the incidence relation is

(2.7) Z = {(s, L, x) ⊂ Y ×Db

∣∣x ∈ E(s, L)}.The double fibration is

(2.8)Z

πL↙ ↘πR

Y Db

and both projections are fiber maps. These projections define the sets we integrateover: the ellipse E(s, L) = πR(π

−1L ({(s, L)})) and the closed curve in Y

F (x) = πL(πR−1({x})) = {(s, �(s, x))

∣∣s ∈ [0, 2π]}where

(2.9) �(s, x) = |x− γR(s)|+ |x− γT (s)|.Note that πR is proper and F (x) is diffeomorphic to the circle.

One chooses measure m = dx on Db and measure n = ds dL on Y . For each(s, L) ∈ Y one parameterizes the ellipse E(s, L)∩Db by arc length with coordinatet so that

(2.10) x = x(s, L, t) ∈ E(s, L)

is a smooth function of (s, L, t). Then, Z can be parameterized by (s, L, t) and thisgives the measure we use on Z, μ = ds dL dt. Since the measure on Y is ds dL andμ = (ds dL) dt, the measure on the fiber of πL is dt. This gives measure μ(s,L) = dtwhich is the arc length measure on the ellipse E(s, L).

To find the measure on F (x) note that the factor, w(s, x), giving this measuresatisfies

(2.11) ds dL dt = w(s, x)ds dx, or dL dt = w(s, x) dx.

For fixed s, (L, t) �→ x(s, L, t) give coordinates on Db. The Jacobian factor w(s, x)in equation (2.11) must be

(2.12) w(s, x) = |∂x�||∂xt|where L = �(s, x) and t are considered as functions of x and where ∂x is thegradient in x and ∂t is the derivative in t. This expression is valid since the vectorsin (2.12) are perpendicular because the first vector is normal to the ellipse E(s, L)at x(s, L, t) and the second vector is tangent to the ellipse. Since t parameterizesarc length, the second factor on the right-hand side of (2.12) is 1. This meansw(s, x) = |∂x�(s, x)|. To calculate this expression for w(s, x) we note that

∂x�(s, x) =x− γR(s)

|x− γR(s)|+

x− γT (s)

|x− γT (s)|.


Since this expression is the sum of two unit vectors, its length is 2 cos(ϕ/2) whereϕ is the angle between these two vectors. A calculation shows that this is

(2.13) w(s, x) = 2 cos(ϕ/2) =

√2 + 2

((x− γR(s)) · (x− γT (s))

|x− γR(s)||x− γT (s)|

)where cosϕ is the expression in parentheses in the square root. Note that ϕ < πsince x is not on the segment between the two foci. Therefore, the weight w(s, x) �=0. The second expression (found using the law of cosines) gives w explicitly in termsof s and x.

This discussion shows that R and R∗ satisfy the conditions outlined in the firstpart of this section so that Guillemin and Sternberg’s framework can be applied.

3. The Main Result

We now state the main result of this article. Proofs are in Section 5.

Theorem 3.1. Let α ∈ (0, π/2) be a constant and let

γT (s) = (cos(s− α), sin(s− α)) and

γR(s) = (cos(s+ α), sin(s+ α)) for s ∈ [0, 2π]

be the trajectories of the ultrasound emitter and receiver respectively. Denote byE ′(Db) the space of distributions supported in the open disc, Db, of radius b centeredat 0, where b = cosα.

The elliptical Radon transform R when restricted to the domain E ′(Db) is anelliptic Fourier integral operator (FIO) of order −1/2. Let C ⊂ T ∗(Y ) × T ∗(Db)be the canonical relation associated to R. Then, C satisfies the Bolker Assumption(Definition 2.1).

As a consequence of this result, we have the following corollary.

Corollary 3.2. The composition of R with its L2 adjoint R∗ when restrictedas a transformation from E ′(Db) to D′(Db) is an elliptic pseudo-differential operatorof order −1.

This corollary shows that, for supp f ⊂ Db, the singularities of R∗Rf (as adistribution on Db) are at the same locations and co-directions as the singularitiesof f , that is, the wavefront sets are the same. In other words, R∗R reconstructsall the singularities of f . In the next section, we will show reconstructions from analgorithm.

Remark 3.3. Theorem 3.1 is valid for any elliptic FIO that has the canonicalrelation C given by (5.1) because the composition calculus of FIO is determined bythe canonical relation. If the forward operator is properly supported (as R is), thenCorollary 3.2 would also be valid. This means that our theorems would be truefor any other model of this bistatic ultrasound problem having the same canonicalrelation C.

4. Reconstructions from a Local Backprojection Algorithm

In this section, we describe a local backprojection type algorithm and show re-constructions from simulated data. The reconstructions and algorithm development


−1.5 −1 −.5 0 .5 1 1.5 2

1.5

1

.5

0

−.5

−1

−1.5

−2

−3000

−2000

−1000

0

1000

2000

3000

4000

(a) Reconstruction of the charac-teristic function of two disks.

−1.5 −1 −.5 0 .5 1 1.5 2

1.5

1

.5

0

−.5

−1

−1.5

−2−3000

−2000

−1000

0

1000

2000

3000

4000

(b) Reconstruction of the charac-teristic function of a rectangle.

Figure 3. Reconstructions using the operator L in (4.1) by TuftsSenior Honors Thesis student Howard Levinson [12]. The recon-structions were done with 300 values of L and 360 values of s, andα = π/32.

were a part of an REU project and senior honors thesis [12] of Tufts Universityundergraduate Howard Levinson. Prof. Quinto’s algorithm

(4.1) Lf = R∗(−∂2/∂L2)Rf

is a generalization of Lambda Tomography [2,3], which is a filtered backprojectiontype algorithm with a derivative filter. Note that the algorithm is local in the sensethat one needs only data over ellipses near a point x ∈ Db to reconstruct L(f)(x).We infer from our next theorem that L detects all singularities inside Db.

Theorem 4.1. The operator L : E ′(Db) → D′(Db) is an elliptic pseudodiffer-ential operator of order 1.

Proof. The order of L is one because R and R∗ are both of order −1/2 and−∂2/∂L2 is of order 2. L is elliptic for the following reasons. From Theorem 3.1, weknow that R is elliptic for distributions in E ′(Db). Then, −∂2/∂L2 is elliptic on dis-tributions with wavefront in ΠL(C) because the dL component of such distributionsis never zero as can be seen from (5.1). Finally, by the Bolker Assumption on C,one can compose R∗ and −∂2/∂L2R to get an elliptic pseudodifferential operatorfor distributions supported on Db. �

Mr. Levinson also tried replacing −∂2/∂L2 by −∂2/∂s2 in (4.1) but someboundaries were not as well-defined in the reconstructions. This reflects the factthat the analogous operator corresponding to L is not elliptic since the symbol of−∂2/∂s2 is zero on a subset of ΠL(C). For instance, the ds component is 0 on cov-ectors corresponding to points on the minor axis of the ellipse E(s, L) determinedby s and L.

Remark 4.2. These reconstructions are consistent with Theorem 4.1 since allsingularities of the objects are visible in the reconstructions and no singularities areadded inside Db. Notice that there are added singularities in the reconstructions inFigure 3, but they are outside Db. Added singularities are to be expected becauseof the left-right ambiguity: an object on one side of the major axis of an ellipse hasthe same integral over that ellipse as its mirror image in the major axis. This is


most pronounced in the common-offset case in which the foci γT and γR travel ona line [11].

5. Proofs of Theorem 3.1 and Corollary 3.2

If g is a function of (s, x) ∈ [0, 2π] × Db, then we will let ∂sg denote the firstderivative of g with respect to s, and ∂xg will denote the derivative of g with respectto x. When x = (x1, x2) ∈ R2, we define ∂xgdx = ∂g

∂x1dx1 +

∂gx2dx2. Here we use

boldface for covectors, such as dx, ds, and dL to distinguish them from measures,such as dx, ds, and dL.

Proof of Theorem 3.1. First, we will calculate C = (N∗(Z)\{0})′ where Zis given by (2.7) and then show that C satisfies the Bolker Assumption. The set Zis defined by L− �(s, x) = 0 where � is defined by (2.9) and the differential of thisfunction is a basis for N∗(Z). Therefore, C = N∗(Z)\{0}′ is given by

(5.1) C = {(s, L,−ω∂s�ds+ ωdL, x, ω∂x�dx)∣∣(s, L) ∈ Y, x ∈ E(s, L), ω �= 0} .

The Schwartz kernel of R is integration on Z (e.g., [17, Proposition 1.1]) and so Ris a Fourier integral distribution associated to C [6].

We now show that the projection

(5.2)ΠL (s, L,−ω∂s�(s, x)ds+ ωdL, x, ω∂x�(s, x)dx)

= (s, L,−ω∂s�(s, x)ds+ ωdL)

is an injective immersion. Let (s, L, ηs, ηL) be coordinates on T ∗(Y ). Note thats, L and ω = ηL are determined by ΠL, so we just need to determine x ∈ Db from(5.2). From the value of L in (5.2) we know that x ∈ E(s, L)∩Db, so we fix L. Byrotation invariance, we can assume s = 0. Now, we let

(5.3) Eb = E(0, L) ∩Db.

Let m be the length of the curve Eb and let x(t) be a parameterization of Eb byarc length for t ∈ (0,m) so that x(t) moves up Eb (x2 increases) as t increases.

The ηs coordinate in (5.2) with x = x(t) and ω = −1 is

(5.4) ηs(t) = ∂s�(s, x) =x(t)− γR(0)

|x(t)− γR(0)|· γ′

R(0) +x(t)− γT (0)

|x(t)− γT (0)|· γ′

T (0).

To show ΠL is an injective immersion, we show that ηs(t) has a positive derivativeeverywhere on (0,m). To do this, we consider the terms in (5.4) separately.

The first term

(5.5) T1(t) =(x(t)− γR(0))

|x(t)− γR(0)|· γ′

R(0)

is the cosine of the angle, β1(t), between the vector (x(t)− γR(0)) and the tangentvector γ′

R(0):T1(t) = cos(β1(t)).

The vector x(t) − γR(0) is transversal to the ellipse E(s, L) at x(t) since γR(0) isinside the ellipse and x(t) is on the ellipse. Therefore, β′

1(t) �= 0 for all t ∈ (0,m).Since x(t) is inside the unit disk, β1(t) is neither 0 nor π so T1(t) = cosβ1(t) isneither maximum or minimum. This implies that T ′

1(t) �= 0 for all t ∈ (0,m). Bythe Intermediate Value Theorem T ′

1 must be either positive or negative everywhereon (0,m). Since x(t) travels up Eb as t increases, T1(t) = cos(β1(t)) increases, andso T ′

1(t) > 0 for all t ∈ (0,m). A similar argument shows that the second term in


(5.4) has positive derivative for t ∈ (0,m). Therefore, ∂tηs(t) > 0 for all t ∈ (0,m)and the Inverse Function Theorem shows that the function ηs(t) is invertible by asmooth function. This proves that ΠL is an injective immersion.

As mentioned after Definition 2.1, the projections ΠL and ΠR map away fromthe 0 section. Therefore,R is a Fourier integral operator [19]. Since the measures μ,dx and ds dL are nowhere zero, R is elliptic. The order of R is given by (dim(Y )−dim(Z))/2 (see e.g., [6, Theorem 1] which gives the order of R∗R). In our case, Zhas dimension 3 and Y has dimension 2, hence R has order −1/2. This concludesthe proof of Theorem 3.1. �

Proof of Corollary 3.2. The proof that R∗R is an elliptic pseudodiffer-ential operator follows from Guillemin’s result [6, Theorem 1] as a consequence ofTheorem 3.1 and the fact πR : Z → R2 is proper. We will outline the proof sincethe proof for our transform is simple and instructive. As discussed previously, wecan compose R∗ and R for distributions in E ′(Db).

By Theorem 3.1, R is an elliptic Fourier integral operator associated with C.By the standard calculus of FIO, R∗ is an elliptic FIO associated to Ct. Becausethe Bolker Assumption holds above Db, C is a local canonical graph and so thecomposition R∗R is a FIO for functions supported in Db. Now, because of theinjectivity of ΠY , Ct ◦C ⊂ Δ where Δ is the diagonal in (T ∗(Db)\{0})2 by the cleancomposition of Fourier integral operators [1].

To show Ct ◦ C = Δ, we need to show ΠR : C → T ∗(Db)\{0} is surjective. Thiswill follow from (5.1) and a geometric argument. Let (x, ξ) ∈ T ∗(Db)\{0}. We nowprove there is a (s, L) ∈ Y such that (x, ξ) is conormal the ellipse E(s, L). First notethat any ellipse E(s, L) that contains x must have L = |x−γR(s)|+|x−γT (s)|. As sranges from 0 to 2π the normal line at x to the ellipse E(s, |x−γR(s)|+|x−γT (s)|) ats rotates completely around 2π radians and therefore for some value of s0 ∈ [0, 2π],(x, ξ) must be conormal E(s0, |x−γR(s0)|+ |x−γT (s0)|). Since the ellipse is givenby the equation L = |x− γR(s)|+ |x− γT (s)|, its gradient is normal to the ellipseat x; conormals co-parallel this gradient are exactly of the form

ξ = ω

(x− γR(s0)

|x− γR(s0)|+

x− γT (s0)

|x− γT (s0)|

)dx

for some ω �= 0. Using (5.1), we see that for this s0, x, ω and L = |x − γR(s0)| +|x− γT (s0)|, there is a λ ∈ C with ΠR(λ) = (x, ξ). This finishes the proof that ΠR

is surjective. Note that one can also prove this using the fact that πR is a fibration(and so a submersion) and a proper map, but our proof is elementary. This showsthat R∗R is an elliptic pseudodifferential operator viewed as an operator fromE ′(Db)→ D′(Db). Because the R∗ and R have order −1/2, R∗R has order −1. �

Remark 5.1. In this remark, we investigate the extent to which our resultscan be extended to the open unit disk, D1.

The Guillemin framework discussed in Section 2.3 breaks down outside Db

because πR : Z → D1 is no longer a proper map. If x ∈ D1 \ cl(Db), then there aretwo degenerate ellipses through x: there are two values of s such that �(s, x) = 2a,so, for these values of s, the “ellipse” E(s, �(s, x)) is the segment between the fociγR(s) and γT (s), and such points (s, 2a) are not in Y . This means that the fibersof πR are not compact above such points. This is more than a formal problem sinceit implies we cannot evaluate R∗ on arbitrary distributions in D′(Y ). Basically,one cannot integrate arbitrary distributions on Y over this noncompact fiber of


πR. More formally, because R : D(D1) → E(Y ), R∗ : E ′(Y ) → D′(D1). Similarly,R : E ′(D1) → D′(Y ). Therefore, one cannot use standard arguments to composeR and R∗ for f ∈ E ′(D1) without using a cutoff function, ϕ, on Y that is zero nearL = 2a. With such a cutoff function R∗ϕR can be defined on domain E ′(D1).

Now we consider the operator R∗ϕR on domain E ′(D1). It is straightforwardto see that ΠL is not injective if points outside of Db are included. For each ellipseE(s, L), covectors in C above the two vertices of E(s, L) on its minor axis projectto the same covector under ΠL. This is even true for thin ellipses for which bothhalves meet D1. This means that, even using a cutoff function ϕ on Y , R∗ϕRwill not be a pseudodifferential operator (unless ϕ is zero for all L for which bothhalves of E(s, L) intersect D1). Let C now denote the canonical relation of R overD1. Because ΠL is not injective, Ct ◦ C contains covectors not on the diagonal and,therefore, R∗ϕR is not a pseudodifferential operator.

For these reasons, we now introduce a half-ellipse transform. Let the curve,Eh(s, L) be the half of the ellipse E(s, L) that is on the one side of the line betweenthe foci γR(s) and γT (s) closer to the origin. This half of the ellipse E(s, L) meetsDb and the other half does not. We denote the transform that integrates functionson D1 over these half-ellipses by Rh and its dual by R∗

h. The incidence relation ofRh will be denoted Zh and its canonical relation will be Ch.

We now use arguments from the proof of Theorem 3.1 to show ΠL : Ch → T ∗(Y )is an injective immersion. The parameterization x(t) of Eb below equation (5.3) canbe extended for t in a larger interval (α, β) ⊃ (0,m) to become a parameterizationof Eh(0, L)∩D1. The function T1 in equation (5.5) is defined for t ∈ (α, β), and theproof we gave that T ′

1(t) > 0 is valid for such points since they are inside the unitdisk (see the last paragraph of the proof of Theorem 3.1). For a similar reason, thesecond term in (5.4) has a positive derivative for t ∈ (α, β). As with our proof forDb, this shows that ΠL : Ch → Y is an injective immersion.

We now investigate generalizations of Corollary 3.2. For the same reasons asfor R, we cannot compose R∗

h and Rh for distributions on D1. We choose a smoothcutoff function ϕ(L) that is zero near L = 2a and equal to 1 for L > 2a + ε forsome small ε > 0. Then, R∗

hϕRh is well defined on E ′(D1). Because ΠL satisfiesthe Bolker Assumption, R∗

hϕRh is a pseudodifferential operator.We will now outline a proof that R∗

hϕRh is elliptic on D1 if ε is small enough.Let x ∈ D1. Then, one can use the same normal line argument as in the lastparagraph of the proof of Corollary 3.2 to show that ΠR is surjective. Namely, foreach x ∈ D1 and each conormal, ξ above x, there are two ellipses containing x andconormal to ξ, and at least one of them will meet x in the inner half. That is, forsome s ∈ [0, 2π), Eh(s, �(s, x)) is conormal to (x, ξ). Furthermore, if ε is chosensmall enough (independent of (x, ξ)), �(s, x) will be greater than 2a+ε. So R∗

hϕRh

is an elliptic pseudodifferential operator.Finally, note that ΠR : Ch → T ∗(Db) is a double cover because each covector

in T ∗(Db) is conormal to two ellipses in Y . However, this is not true for Rh andCh and for points in D1 \ Db, and that is why we need the more subtle argumentson D1.

References

[1] J. J. Duistermaat and V. W. Guillemin, The spectrum of positive elliptic operators andperiodic bicharacteristics, Invent. Math. 29 (1975), no. 1, 39–79. MR0405514 (53 #9307)


[2] A. Faridani, D. V. Finch, E. L. Ritman, and K. T. Smith, Local tomography. II, SIAM J.Appl. Math. 57 (1997), no. 4, 1095–1127, DOI 10.1137/S0036139995286357. MR1462053(98h:92016)

[3] A. Faridani, E. L. Ritman, and K. T. Smith, Local tomography, SIAM J. Appl. Math. 52(1992), no. 2, 459–484, DOI 10.1137/0152026. MR1154783 (93b:92008)

[4] I. M. Gel′fand, M. I. Graev, and Z. Ja. Sapiro, Differential forms and integral geometry,Funkcional. Anal. i Prilozen. 3 (1969), no. 2, 24–40 (Russian). MR0244919 (39 #6232)

[5] V. Guillemin. Some remarks on integral geometry. Technical report, MIT, 1975.[6] V. Guillemin, On some results of Gel′fand in integral geometry, Pseudodifferential operators

and applications (Notre Dame, Ind., 1984), Proc. Sympos. Pure Math., vol. 43, Amer. Math.Soc., Providence, RI, 1985, pp. 149–155. MR812288 (87d:58137)

[7] V. Guillemin and D. Schaeffer, Fourier integral operators from the Radon transform point ofview, Differential geometry (Proc. Sympos. Pure Math., Vol. XXVII, Stanford Univ., Stan-ford, Calif., 1973), Part 2, Amer. Math. Soc., Providence, R.I., 1975, pp. 297–300. MR0380520(52 #1420)

[8] V. Guillemin and S. Sternberg, Geometric asymptotics, American Mathematical Society,Providence, R.I., 1977. Mathematical Surveys, No. 14. MR0516965 (58 #24404)

[9] S. Helgason, A duality in integral geometry on symmetric spaces, Proc. U.S.-Japan Seminar inDifferential Geometry (Kyoto, 1965), Nippon Hyoronsha, Tokyo, 1966, pp. 37–56. MR0229191(37 #4765)

[10] L. Hormander, Fourier integral operators. I, Acta Math. 127 (1971), no. 1-2, 79–183.MR0388463 (52 #9299)

[11] V. P. Krishnan, H. Levinson, and E. T. Quinto. Microlocal Analysis of Elliptical RadonTransforms with Foci on a Line. In I. Sabadini and D. C. Struppa, editors, The MathematicalLegacy of Leon Ehrenpreis, volume 16 of Springer Proceedings in Mathematics, pages 163–182, Berlin, New York, 2012. Springer Verlag.

[12] H. Levinson. Algorithms for Bistatic Radar and Ultrasound Imaging. Senior Honors Thesis(Highest Thesis Honors), Tufts University, pages 1–48, 2011.

[13] S. Mensah and E. Franceschini. Near-field ultrasound tomography. The Journal of the Acous-tical Society of America, 121(3):1423–1433, 2007.

[14] S. Mensah, E. Franceschini, and M.-C. Pauzin. Ultrasound mammography. Nuclear Instru-ments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectorsand Associated Equipment, 571(1-2):52 – 55, 2007. Proceedings of the 1st International Con-

ference on Molecular Imaging Technology - EuroMedIm 2006.[15] S. J. Norton, Reconstruction of a two-dimensional reflecting medium over a circular domain:

exact solution, J. Acoust. Soc. Amer. 67 (1980), no. 4, 1266–1273, DOI 10.1121/1.384168.MR565125 (81a:76040)

[16] S. J. Norton and M. Linzer. Ultrasonic reflectivity imaging in three dimensions: Exact inversescattering solutions for plane, cylindrical, and spherical apertures. Biomedical Engineering,IEEE Transactions on, BME-28(2):202 –220, feb. 1981.

[17] E. T. Quinto, The dependence of the generalized Radon transform on defining measures,Trans. Amer. Math. Soc. 257 (1980), no. 2, 331–346, DOI 10.2307/1998299. MR552261(81a:58048)

[18] P. Stefanov and G. Uhlmann. Is a curved flight path in SAR better than a straight one?SIAM J. Appl. Math., 2013. to appear.

[19] F. Treves, Introduction to pseudodifferential and Fourier integral operators. Vol. 2, PlenumPress, New York, 1980. Fourier integral operators; The University Series in Mathematics.MR597145 (82i:58068)

[20] R. S. Vaidyanathan, M. A. Lewis, G. Ambartsoumian, and T. Aktosun. Reconstruction algo-rithms for interior and exterior spherical Radon transform-based ultrasound imaging. Proc.of SPIE, 7265:72651 I 1–8, 2009.


Department of Mathematics, University of Texas, Arlington, Texas


Department of Mathematics, Stockholm University, Stockholm, Sweden


Tata Institute of Fundamental Research Centre for Applicable Mathematics, Ban-

galore, India.


Department of Mathematics, Tufts University, Medford, Massachusetts 02155



Cuspidal discrete seriesfor projective hyperbolic spaces

Nils Byrial Andersen and Mogens Flensted–Jensen

Dedicated to Sigurdur Helgason on the occasion of his 85th birthday

Abstract. We have in [1] proposed a definition of cusp forms on semisimplesymmetric spaces G/H, involving the notion of a Radon transform and arelated Abel transform. For the real non-Riemannian hyperbolic spaces, weshowed that there exists an infinite number of cuspidal discrete series, andat most finitely many non-cuspidal discrete series, including in particular thespherical discrete series. For the projective spaces, the spherical discrete seriesare the only non-cuspidal discrete series. Below, we extend these results tothe other hyperbolic spaces, and we also study the question of when the Abeltransform of a Schwartz function is again a Schwartz function.

1. Introduction

We initiated, in joint work with Henrik Schlichtkrull, in [1] a generalization ofHarish-Chandra’s notion of cusp forms for real semisimple Lie groups G to semisim-ple symmetric spaces G/H. In the group case, all the discrete series are cuspidal,and this plays an important role in Harish-Chandra’s work on the Plancherel for-mula. However, in the established generalizations to G/H, cuspidality plays no roleand, in fact, was hitherto not defined at all.

The notion of cuspidality relates to the integral geometry on the symmetricspace by using integration over a certain unipotent subgroup N∗ ⊂ G, whose defi-nition is given in [1]. The map f �→

∫N∗ f(·nH) dn, which maps functions on G/H

to functions on G/N∗, is a kind of a Radon transform for G/H. A discrete seriesis said to be cuspidal if it is annihilated by this transform.

Let p, q denote positive integers. The Radon transform, and the question ofcuspidality, on the real hyperbolic spaces SO(p, q + 1)e/SO(p, q)e, was treated indetail in [1]. We showed that there is at most a finite number of non-cuspidaldiscrete series, including in particular all the spherical discrete series, but alsosome non-spherical discrete series. The non-spherical non-cuspidal discrete seriesare given by odd functions on the real hyperbolic space, which means that they donot descend to functions on the real projective hyperbolic space.

In the present paper, we consider the projective hyperbolic spaces over theclassical fields F = R,C,H,

G/H = O(p+ 1, q + 1)/(O(p+ 1, q)×O(1)), U(p+ 1, q + 1)/(U(p+ 1, q)×U(1)),

2010 Mathematics Subject Classification. Primary 43A85; Secondary 22E30.


59

60 NILS BYRIAL ANDERSEN AND MOGENS FLENSTED–JENSEN

Sp(p+ 1, q + 1)/(Sp(p+ 1, q)× Sp(1)),

for p ≥ 0, q ≥ 1. Notice the change of indices from p to p+ 1, to simplify formulaeand calculations.

Our main result, Theorem 6.1, states that the non-cuspidal discrete series forthe projective hyperbolic spaces precisely consist of the spherical discrete series.The Radon transform of the generating functions is also given explicitly. Finally,we show that the Abel transform maps (a dense subspace of) the Schwartz functionson G/H perpendicular to the non-cuspidal discrete series into Schwartz functions.The latter result also holds for the non-projective real case, and is a new result forall cases.

Our calculations and main results are also valid, with p = 0, q = 1 and d =8, for the Cayley numbers O, corresponding to the exceptional symmetric spaceF4(−20)/Spin(1, 8). Although the model for this space, and the group action on it,is more complicated, this space can for our purposes be viewed as

F4,(−20)/Spin(1, 8) = ”U(1, 2;O)/U(1, 1;O)×U(1;O)”.

We state our results in full generality, but only give complete proofs for the non-exceptional projective spaces, with some remarks on the other cases in the lastsection.

We would like to thank Henrik Schlichtkrull for input and fruitful discussions,which in the real case lead to the explicit formulae involving the Hypergeometricfunction. We also want to thank Job Kuit for discussions of part (vi) of Theorem 6.1,explaining how to prove a similar result in split rank one, using general theory.

Part of this work was outlined by the first author at the Special Session ‘RadonTransforms and Geometric Analysis in Honor of Sigurdur Helgason’s 85th Birth-day’, at the 2012 AMS National Meeting in Boston. He is grateful to the organizersJens Christensen, Fulton Gonzalez, and Todd Quinto, for their invitation to speak,and the hospitality at the meeting, and the subsequent Workshop on GeometricAnalysis on Euclidean and Homogeneous Spaces.

2. Model and structure

Let F be one of the classical fields R, C or H, and let x �→ x be the standard(anti-) involution of F. We make the standard identifications between C and R2, andbetween H and R4. Let p ≥ 0, q ≥ 1 be two integers, and consider the Hermitianform [·, ·] on Fp+q+2 given by

[x, y] = x1y1+· · ·+xp+1yp+1−xp+2yp+2−· · ·−xp+1+q+1yp+1+q+1, (x, y ∈ Fp+q+2).

Let G = U(p+1, q+1;F) denote the group of (p+ q+2)× (p+ q+2) matrices overF preserving [·, ·]. Thus U(p + 1, q + 1;R) = O(p + 1, q + 1), U(p + 1, q + 1;C) =U(p+ 1, q + 1) and U(p+ 1, q + 1;H) = Sp(p+ 1, q + 1) in standard notation. PutU(p;F) = U(p, 0;F).

Let x0 = (0, . . . , 0, 1)T , where superscript T indicates transpose. Let H =U(p+ 1, q;F) × U(1;F) be the subgroup of G stabilizing the line F · x0 in Fp+q+2.An involution σ of G fixing H is given by σ(g) = JgJ , where J is the diagonalmatrix with entries (1, . . . , 1,−1). The reductive symmetric space G/H (of rank 1)can be identified with the projective hyperbolic space X = X(p+ 1, q + 1;F):

X = {z ∈ Fp+q+2 : [z, z] = −1}/ ∼,where ∼ is the equivalence relation z ∼ zu, u ∈ F∗.

CUSPIDAL DISCRETE SERIES 61

The Lie algebra g of G consists of (p+ q + 2)× (p+ q + 2) matrices

g =

(A BB∗ C

),

where A is a skew Hermitian (p + 1) × (p + 1) matrix, C is a skew Hermitian(q + 1) × (q + 1) matrix, and B is an arbitrary (p+ 1) × (q + 1) matrix. Here B∗

denotes the conjugated transpose of B.Let K = K1 ×K2 = U(p + 1;F) × U(q + 1;F) be the maximal compact sub-

group of G consisting of elements fixed by the classical Cartan involution on G,θ(g) = (g∗)−1, g ∈ G. Here g∗ denotes the conjugated transpose of g. The Cartaninvolution on g is given by: θ(X) = −X∗. Let g = k ⊕ p be the decomposi-tion of g into the ±1-eigenspaces of θ, where k = {X ∈ g : θ(X) = X} andp = {X ∈ g : θ(X) = −X}. Similarly, let g = h ⊕ q be the decomposition of ginto the ±1-eigenspaces of σ(X) = JXJ , where h = {X ∈ g : σ(X) = X} andq = {X ∈ g : σ(X) = −X}.

We choose a maximal abelian subalgebra aq ⊂ p ∩ q as

aq =

⎧⎨⎩Xt1 =

⎛⎝ 0 0 t10 0p,q 0t1 0 0

⎞⎠ : t1 ∈ R

⎫⎬⎭ ,

where 0p,q is the (p+q)×(p+q) null matrix. The exponential ofXt1 , at1 = exp(Xt1),is given by

at1 = exp(Xt1) =

⎛⎝ cosh t1 0 sinh t10 Ip,q 0

sinh t1 0 cosh t1

⎞⎠ ,

where Ip,q is the (p+ q)× (p+ q) identity matrix. Also define Aq = exp(aq).Let A+

q = {at1 : t1 > 0}. Let a(x) = a(kah) = a denote the projection onto

the A+q component in the Cartan decomposition G = KA+

q H of G. Let M be thecentralizer of X1 ∈ aq (i.e., when t1 = 1) in K ∩H. Then M is the stabilizer of theline F(1, 0, . . . , 0, 1), and the homogeneous space K/M can be identified with theprojective image Y = Yp+1,q+1 of the product of unit spheres Sp × Sq:

Y = {y ∈ Fp+q+2 : |y1|2 + · · ·+ |yp+1|2 = |yp+2|2 + · · ·+ |yp+q+2|2 = 1}/ ∼ .

The image of the set {z ∈ Fp+q+2 : [z, z] = −1, (z1, . . . , zp+1) �= 0} in X is an opendense subset, which we will denote by X′. The map

K/M × R+ → X, (kM, t1) �→ kat1H,

is a diffeomorphism onto X′.We introduce spherical coordinates on X as the pull back of the map:

x(t1, y) = (u sinh t1; v cosh t1), t1 ∈ R+, y = (u; v) ∈ Sp × Sq.

We define a (K-invariant) ‘distance’ from x ∈ X to the origin as |x| = |x(t1, y)| =|t1|. Then X′ = {x ∈ X| |x| > 0}. We note that cosh2(|x|) = |xp+2|2+· · ·+|xp+q+2|2.For g ∈ G, we define |g| = |gH|.

Let r = min{p, q}, and let Xt be the (r+1)× (r+1) anti-diagonal matrix withentries t = (t1, . . . , tr+1) ∈ Rr+1, starting from the upper right corner. We extend


aq (viz. as t2 = · · · = tr+1 = 0) to a maximal subalgebra a ⊂ p as

a =

⎧⎨⎩Xt =

⎛⎝ 0 0 Xt

0 0 0X∗

t 0 0

⎞⎠⎫⎬⎭ .

We will also consider the sub-algebra ah = a ∩ h = {Xt ∈ a : t1 = 0}.Let (considered as row vectors)

u = (u1, . . . , up) ∈ Fp and v = (vq, . . . , v1) ∈ Fq.

It turns out to be convenient to number the entries of v from right to left asindicated. Let furthermore w ∈ ImF (i.e.,w = 0 for F = R). Now define Nu,v,w ∈ g

as the matrix given by

Nu,v,w =

⎛⎜⎜⎝−w u v w−uT 0 0 uT

vT 0 0 −vT−w u v w

⎞⎟⎟⎠ .

Then exp(Nu,v,w) = I +Nu,v,w + 1/2N2u,v,w, and

(2.1) exp(Nu,v,w) · x0 = (1/2(|u|2 − |v|2) +w, uT ;−vT , 1 + 1/2(|u|2 − |v|2) +w)T .

A small calculation also yields that

(2.2) at1 exp(Nu,v,w) · x0 =

(sinh t1 + 1/2et1(|u|2 − |v|2) + et1w, uT ;−vT , cosh t1 + 1/2et1(|u|2 − |v|2) + et1w)T ,

for any t1 ∈ R.We note that [Xt1 , Nu,v,0] = t1Nu,v,0, and [Xt1 , N0,0,w] = 2t1N0,0,w. Let

γ(Xt1) = t1. Then the root system Σq for aq is given by Σq = {±γ}, forF = R, and Σq = {±γ} ∪ {±2γ}, for F = C, H. The associated nilpotent sub-algebra nq is given by nq = gγ = {Nu,v,0 : u ∈ Fp, v ∈ Fq}, when F = R, andnq = gγ + g2γ = {Nu,v,w : u ∈ Fp, v ∈ Fq, w ∈ ImF}, when F = C, H. Half thesum of the positive roots, ρq = 1

2

∑α∈Σ+

qmαα, where mα is the multiplicity of the

root α, is thus

〈ρq, Xt1〉 =1

2(dp+ dq + 2(d− 1))t1,

where d = dimR F. Using the identification Aq ∼ R, we will also sometimes use thedefinition ρq = 1

2 (dp+ dq + 2(d− 1)) ∈ R.The (restricted) Σ for a is given by {±ti ± tj}, i �= j, i, j ∈ {1, . . . , r + 1},

{±ti}, i ∈ {1, . . . , r + 1}, if p �= q, and {±2ti}, i ∈ {1, . . . , r + 1}, if d ≥ 2. Letαi,j(Xt) = ti + tj , i < j, βi,j(Xt) = ti − tj , i < j, and γi(Xt) = ti.

We choose two sets of positive roots

Σ+ = {αi,j , βi,j , γi, 2γi},

which corresponds to the (standard) ordering t1 > t2 > · · · > tr+1, and

Σ+1 = {αi,j , γi, 2γi} ∪ {βi,j : i �= 1} ∪ {−βi,j : i = 1},

which corresponds to the ordering t2 > t3 > · · · > tr+1 > t1. The double roots{±2γi} are not present for F = R, the single roots {±γi} are not present when


p = q. The associated nilpotent subalgebras are denoted by n and n1 respectively.The half sum of positive roots ρ1 with regards to Σ+

1 is given by (restricted to Aq)

〈ρ1, Xt1〉 =1

2((|dp− dq|+ 2(d− 1))t1.

As before, we will sometimes use the definition ρ1 = 12 ((|dp− dq|+ 2(d− 1)) ∈ R.

We note that γ ∈ Σ+q is the restriction of the roots {α1,1+j , γ1, β1,1+j}, with

j ∈ {1, . . . , r}, wheregα1,j+1 = {Nu,v,0 : uj = −vj , ui = vi = 0, i �= j},

gβ1,j+1 = {Nu,v,0 : uj = vj , ui = vi = 0, i �= j},and, for p > q,

gγ1 = {Nu,v,0 : u = (0, . . . , 0, uq+1, . . . , up), v = 0},which for p < q becomes u = 0, v = (vq, vq−1, . . . , vp+1, 0, . . . , 0).

Define n∗ = n1 ∩nq as the subalgebra associated to the roots {α1,1+j , γ1, 2γ1}.Then, for p ≥ q

(2.3) n∗ = {Nu,v,w : u = (−vr, u′), v ∈ Fq, u′ ∈ Fp−q},and, for p < q,

(2.4) n∗ = {Nu,v,w : v = (−ur, v′), u ∈ Fp, v′ ∈ Fq−p},where ur, vr means that the order of the indices is reversed. In the following weshall by abuse of notation leave out the r.

Remark 2.1. We have the identity Σ+q = {α ∈ Σ+ : α|aq

> 0}|aq. We then

have the disjoint union Σ+q = Σ++ ∪ Σ+0 ∪ Σ+−, where the second sign refers

to α|ah. The choice of the nilpotent subalgebra n∗ can thus be described by the

correspondence n∗ ∼ Σ++ +Σ+0.

3. The discrete series

From [3, Section 8] and [4, Table 2], we have the following parametrizationof the discrete series for the projective hyperbolic spaces, with an exception forq = d = 1:

{Tλ |λ =1

2(dq − dp)− 1 + μλ > 0, μλ ∈ 2Z}.

The spherical discrete series are given by the parameters λ for which μλ ≤ 0, includ-ing the ’exceptional’ discrete series corresponding to the (finitely many) parametersλ > 0 for which μλ < 0. We notice that spherical discrete series exists if, and onlyif, d(q − p) > 2. For q = d = 1, the discrete series is parameterized by λ ∈ R\{0}such that |λ|+ ρq ∈ 2Z, and there are no spherical discrete series.

The parameter λ is, via the formula Δf = (λ2−ρ2q)f , related to the eigenvalueof the Laplace-Beltrami operator Δ of G/H on functions f in the correspondingrepresentation space in L2(G/H) (with suitable normalization of Δ). Using [4,Theorem 5.1] (see [1, Proposition 3.2] for more details), we can explicitly describethe discrete series by generating functions ψλ as follows. Let s = s1 ∈ R describethe elements as = as1 ∈ Aq. Let λ be a discrete series parameter. For μλ ≥ 0, wehave

ψλ(kasH) = ψλ(x(s, y)) = φμλ(k)(cosh s)−λ−ρq ,


where φμλis a K ∩H-invariant zonal spherical function, in particular φ0 = 1. For

μλ = −2m ≤ 0, we have

ψλ(kasH) = Pλ(cosh2 s)(cosh s)−λ−ρq−2m,

where Pλ is a polynomial of degree m. For q = d = 1, consider the one-parametersubgroup T = {kθ} ⊂ K2 defined by

kθ =

⎛⎝ Ip+1 0 00 cos θ sin θ0 − sin θ cos θ

⎞⎠ ,

where Ij denotes the identity matrix of size j, then ψλ(kθasH) = eimθ(cosh s)−|λ|−ρq ,with m = λ ± ρq, and the sign determined by the sign of λ. See [1, Section 3] forfurther details.

4. Schwartz functions

In this section we recall some results from [2, Chapter 17] regarding L2-Schwartzfunctions on G/H. Let Ξ denote Harish-Chandra’s bi-K-invariant elementaryspherical function ϕ0 on G, and define the real analytic function Θ : G/H → R+

by

Θ(x) =√Ξ(xσ(x)−1) (x ∈ G).

We notice that there exists a positive constant C, and a positive integer m, suchthat

(4.1) a−ρq ≤ Θ(a) ≤ Ca−ρq(1 + |a|)m, (a ∈ A+q ).

Here we use the definition aλ = e〈λ,log a〉, for a ∈ A+q , λ ∈ aq

∗C.

The space C2(G/H) of L2-Schwartz functions on G/H can be defined as thespace of all smooth functions on G/H satisfying

μ2n,D(f) = sup

x∈G/H

Θ−1(x)(1 + |x|)n|f(D, x)| <∞,

for all n ∈ N ∪ {0} and D ∈ U(g).Let f ∈ C2(G/H). Let S ⊂ G be a compact set. Then, for any n ∈ N ∪ {0},

there exists a positive constant C, such that

(4.2) |f(g · x)| ≤ C Θ(a(x))(1 + |x|)−n, (g ∈ S, x ∈ G/H).

5. A Radon transform and an Abel transform

Let N∗ = exp(n∗) and N1 = exp(n1) denote the two nilpotent subgroupsgenerated by n∗ and n1 respectively. For functions on G/H we define, assumingconvergence,

(5.1) Rf(g) =

∫N∗

f(gn∗H) dn∗ (g ∈ G).

Let HA denote the centralizer of A in H. Then Rf(gm) = Rf(g), m ∈ HA, and

Theorem 5.1. Let f ∈ C2(G/H).

(i) The integral defining the Radon transform R converges uniformly on com-pact sets.

(ii) Rf ∈ C∞(G/HAN1).(iii) The Radon transform is G- and g-equivariant.


Proof. We first assume p ≥ q. Let f ∈ C2(G/H), and fix a compact setS ⊂ G. Let n ∈ N. Then

(5.2)

∫N∗|f(gn∗H)|dn∗ ≤ C

∫N∗

a(n∗)−ρq(1 + |n∗|)−n+mdn∗ (g ∈ S),

for the constants C and m given by (4.1) and (4.2). From (2.1) and (2.3), we have

cosh2(| exp(N(v,u′),v,w)|) = (1 + 1/2|u′|2)2 + |v|2 + |w|2.

Using that log s ≤ arccosh s ≤ log s+log 2, when s ≥ 1, we see that the last integralin (5.2) is bounded by

C

∫Rdp−dq×Rdq×Rd−1

((1 + 1/2|u′|2)2 + |v|2 + |w|2)−dp+dq+2(d−1)

4

× (1 + log((1 + 1/2|u′|2)2 + |v|2 + |w|2))−n+mdu′dvdw,

where C is a positive constant.Consider the integral (x ∈ Rk, y ∈ Rl), with n > 2,∫

Rk×Rl

(1 + |x|4 + |y|2)−a(1 + log(1 + |x|4 + |y|2))−ndxdy.

With the substitution y =√1 + |x|4z ∈ Rl, we get∫

Rk×Rl

(1 + |x|4)−a+ l2 (1 + |z|2)−a(1 + log(1 + |x|4) + log(1 + |z|2))−ndxdz ≤∫

Rk

(1 + |x|4)−a+ l2 (1 + log(1 + |x|4))−n

2 dx

∫Rl

(1 + |z|2)−a(1 + log(1 + |z|2))−n2 dz,

which is finite if, and only if, k ≤ 4a− 2l and l ≤ 2a.We have k = dp − dq, l = dq + d − 1 and a = (dp + dq + 2(d − 1))/4, whence

k = 4a− 2l and l ≤ 2a, and the integral (5.1) converges uniformly on compact sets.In the p < q case, we see from (2.1) and (2.4), that cosh2(| exp(Nu,(u,v′),w)|) =

|v′|2 + |u2|+ (1− 1/2|v′|2)2 + |w|2 = 1 + |u|2 + 1/4|v′|4 + |w|2, and we proceed asbefore, reversing the roles of u and v. �

We define the Abel transform A by Af(a) = aρ1Rf(a), for a ∈ Aq.

Theorem 5.2. Let g ∈ G and f ∈ C2(G/H). Let Δ denote the Laplace–Beltrami operator on G/H and let ΔAq

denote the Euclidean Laplacian on Aq.Then

(5.3) A(Δf) = (ΔAq− ρ2q)Af (a ∈ Aq).

Proof. See [1, Lemma 2.4], and the discussion before and after this lemma.�

Let ψλ belong to the discrete series with parameter λ. Since Δψλ = (λ2−ρ2q)ψλ,we see that Aψλ is an eigenfunction for the Euclidean Laplacian ΔAq

on Aq with the

eigenvalue λ2. This implies in particular that s �→ Rψλ(as) is a linear combinationof e(λ−ρ1)s and e(−λ−ρ1)s.


6. The main result

Here we state the main theorem, to be proven in the following sections. Wewill in particularly be interested in the values of Rf on the elements as ∈ Aq, sofor simplicity we write Rf(s) = Rf(as), and, similarly, Af(s) = Af(as).

Let R > 0, and let C∞R (G/H) denote the subspace of smooth functions on

G/H with support inside the (K-invariant) ‘ball’ of radius R. Let similarly C∞R (R)

denote the subspace of smooth functions on R with support inside [−R,R]. Finally,let S(R) denote the Schwartz functions on R.

Theorem 6.1. Let G/H be a projective hyperbolic space over R, C, H, withp ≥ 0, q ≥ 1, or over O, with p = 0, q = 1.

(i) If d(q − p) ≤ 2, then all discrete series are cuspidal.(ii) If d(q − p) > 2, then non-cuspidal discrete series exists, given by the

parameters λ > 0 with μλ ≤ 0. More precisely, if 0 �= f ∈ C2(G/H)belongs to Tλ, then Af(s) = Ceλs, with C �= 0.

(iii) Tλ is non-cuspidal if and only if Tλ is spherical.(iv) If p ≥ q, and f ∈ C∞

R (G/H), for R > 0, then Af ∈ C∞R (R).

(v) If d(q − p) ≤ 1, and f ∈ C2(G/H), then Af ∈ S(R).(vi) Assume d(q − p) > 1. Let D be the G-invariant differential operator

Δρ(Δρ − λ21) . . . (Δρ − λ2

r), where λ1, . . . , λr are the parameters of thenon-cuspidal discrete series, and Δρ = Δ+ ρ2q. Then A(Df) ∈ S(R), forf in a dense subspace of C2(G/H).

Remark 6.2. The theorem also holds for the non-projective spaces SO(p+1, q+1)e/SO(p + 1, q)e, except for item (iii), due to the existence of non-cuspidal non-spherical discrete series, corresponding to the parameters λ > 0, with μλ ∈ 2Z+ 1and μλ < 0.

Remark 6.3. The conditions in item (vi) essentially state thatAf is a Schwartzfunction if f is perpendicular to all non-cuspidal discrete series. The factor Δρ,however, cannot be avoided, except in the cases d = 1 and q − p odd.

Remark 6.4. For the exceptional case, only (ii), (iii) and (vi) are relevant.The spherical discrete series corresponds to λ = 3 (μλ = 0) and λ = 1 (μλ = −2).

7. Proof of the main theorem for p ≥ q

Proposition 7.1. Let p ≥ q.

(i) Let f ∈ C∞R (G/H), for R > 0. Then Af ∈ C∞

R (R).(ii) Let f ∈ C2(G/H). Then Af ∈ S(R).

Proof. Let f ∈ C∞R (G/H), for R > 0. By (2.2) and (2.3), we have

cosh2(|as exp(N(v,u1),v,w)|) = (cosh s+ 1/2es|u′|2)2 + |v|2 + |esw|2 ≥ cosh2 s,

and thus Rf(s) = 0, for |s| > R, which shows (i).For (ii), let f ∈ C2(G/H). As before we have, for n ∈ N,

(7.1)

∫N∗|f(asn∗H)|dn∗ ≤ C

∫N∗

a(asn∗)−ρq(1 + |asn∗|)−ndn∗,

where C is a positive constant.


The integral in (7.1) is bounded by∫Rdp−qd×Rdq×Rd−1

((cosh s+ 1/2es|u′|2)2 + |v|2 + |esw|2)−dp+dq+2(d−1)

4

× (1 + log((cosh s+ 1/2es|u′|2)2 + |v|2 + |esw|2)1/2))−ndu′dvdw

≤ (cosh s)−dp+dq+2(d−1)

2


(1 + (1/√2(cosh s)−

12 es/2|u′|)2)2

+ |(cosh s)−1v|2 + |(cosh s)−1esw|2)−dp+dq+2(d−1)

4

× (1 + log((1 + (1/√2(cosh s)−

12 es/2|u′|)2)2

+ |(cosh s)−1v|2 + |(cosh s)−1esw|2)−ndu′dvdw,

since log cosh s ≥ 0.Consider the substitutions u = 1/

√2(cosh s)−

12 es/2u′, v = (cosh s)−1v and

w = (cosh s)−1esw. Then du′ = (√2(cosh s)

12 e−s/2)dp−dqdu, dv = (cosh s)dqdv,

and dw = ((cosh s)e−s)d−1dw, and the above integral becomes

=√2dp−dq

e−dp−dq+2(d−1)

2 s


((1 + |u|2)2 + |v|2 + |w|2)−dp+dq+2(d−1)

4

× (1 + log((1 + |u|2)2 + |v|2 + |w|2))−ndudvdw

≤ Cp,qa−ρ1s ,

where Cp,q is a constant only depending on p and q. The proposition follows usingthe U(g)-equivariance of the Radon transform from Theorem 5.1 (iii). �

Let C2(G/H)d = L2(G/H)d ∩ C2(G/H) denote the span of the discrete seriesin C2(G/H).

Proposition 7.2. Let p ≥ q. Then Rf = 0, for f ∈ C2(G/H)d.

Proof. Let f ∈ C2(G/H)d. Then Af belongs to S(Aq) by Theorem 7.1, butat the same time Af is also an eigenfunction of ΔAq

on Aq. We conclude thatAf = 0, and thus Rf = 0. �

8. Proof of (i) - (v) of the main theorem for q > p

Let ψλ be a generating function for the discrete series with parameter λ. Wenotice that μλ = λ− 1

2 (dq − dp) + 1 = λ+ ρq − dq − (d− 1) + 1.

Proposition 8.1. Let p < q. Then Rψλ = 0, for μλ > 0. For μλ ≤ 0, we have

(8.1) Rψλ(as) =

∫N∗

ψλ(asn∗)dn∗ = Ce(μλ−d)s, (s ∈ R),

where C �= 0 is a constant depending on p, q.

Proof. We define a K-invariant function ψλ as ψλ(kasH) = (cosh s)−λ−ρq .

Then by (2.2) and (2.4), the Radon transform Rψλ(as) is

(8.2)

∫Rdq−dp×Rdp×Rd−1

((cosh s−1/2es|v′|2)2+ |v′|2+ |u|2+ |esw|2)−λ+ρq

2 dv′dudw.


Substituting v′ = (es/(2 cosh s))1/2v′, u = (1/ cosh s)u and w = (es/ cosh s)w, thisbecomes

=

(es

2 cosh s

)− dq−dp2(

1

cosh s

)−dp(es

cosh s

)−(d−1)

(cosh s)−λ−ρq

×∫Rdq−dp×Rdp×Rd−1

((1− |v′|2)2 +

(2

es cosh s

)|v′|2 + |u|2 + |w|2

)−λ+ρq2

dv′dudw

= 2dq−dp

2 e−dq−dp+2(d−1)

2 s(cosh s)−λ

×∫Rdq−dp×Rdp×Rd−1

(1 + |v′|4 − 2(tanh s)|v′|2 + |u|2 + |w|2

)−λ+ρq2 dv′dudw.

Using the substitution ˜u = (1+ |v1|4− 2(tanh s)|v′|2)−1/2u, and likewise for w,the integral becomes∫

Rdq−dp×Rdp×Rd−1

(1 + |v1|4 − 2(tanh s)|v′|2

)−λ+ρq2 + dp

2 + d−12

× (1 + |˜u|2 + | ˜w|2)−λ+ρq

2 dv′d˜ud ˜w

= C

∫ ∞

0

(1 + ξ4 − 2(tanh s)ξ2

)−λ+ρq−dp−(d−1)

2 ξdq−dp−1dξ

=1

2C

∫ ∞

0

(1 + x2 − 2(tanh s)x

)−λ+ρq−dp−(d−1)

2 xdq−dp

2 −1dx

using polar coordinates, where C is the positive constant given by

C =

∫ ∞

0

∫ ∞

0

(1 + η2 + σ2)−λ+ρq

2 ηdp−1dησd−2dσ <∞.

From [5, 3.252(10)], we get∫ ∞

0

(1 + x2 + 2(cos t)x

)−νxμ−1dx =

2ν−12 (sin t)

12−νΓ(ν +

1

2)B(μ, 2ν − μ)P

12−ν

μ−ν− 12

(cos t).

We also have

P12−ν

μ−ν− 12

(y) =

1/Γ(ν +1

2)

(1 + y

1− y

) 12 (

12−ν)

2F1

(−μ+ ν +

1

2, μ− ν +

1

2; ν +

1

2;1

2− 1

2y

).

With y = cos t = − tanh s, for 0 < t < π, we get sin t = 1/ cosh s, 1− y = es/ cosh sand 1 + y = e−s/ cosh s. Putting this together, we get∫ ∞

0

(1 + x2 − 2(tanh s)x

)−νxμ−1dx = B(μ, 2ν − μ)(2(cosh s)es)ν−

12

× 2F1

(−μ+ ν +

1

2, μ− ν +

1

2; ν +

1

2;

1

1 + e−2s

).

With μ = dq−dp2 and ν =

λ+ρq−dp−(d−1)2 , we get

Rψλ(as) = Cλe−ds(1 + e−2s)−

μλ2 2F1

(μλ

2, 1− μλ

2;μλ + dq − dp

2;

1

1 + e−2s

),


where Cλ is a positive constant depending on p, q and λ.The hypergeometric function z �→ 2F1(μλ/2, 1− μλ/2; (μλ + dq − dp)/2; z) is a

polynomial of degree −μλ/2 for μλ ≤ 0, and degree μλ/2− 1 for μλ > 0. We thusimmediately get (8.1) for μλ = 0.

Now let μλ = −2m < 0. We can write ψλ(as) as the sum

ψλ(as) = (cosh s)−λ−ρq +m∑j=1

Cj(cosh s)−(λ+2j)−ρq ;

or

ψλ = ψλ +

m∑j=1

Cjψλ+2j .

It follows that Rψλ(as) can be written as a sum

Rψλ(as) = C0e−ds(1 + e−2s)m +

m−1∑j=0

Cje−ds(1 + e−2s)j ,

where C0 is a non-zero constant corresponding to the factor ψλ(as) = (cosh s)−λ−ρq .Thus, since we know that Rψλ(as) is a linear combination of e(λ−ρ1)s = e(μλ−d)s

and e(−λ−ρ1)s, we get Rψλ(as) = Ce(μλ−d)s, for a non-zero constant C.

Let finally μλ > 0. Then |ψλ| ≤ ‖φμλ‖∞ψλ, and |Rψλ| ≤ ‖φμλ

‖∞Rψλ. Wehave

(8.3) |Rψλ(as)| ≤ C1Rψλ(as) ≤ C2e−ds for s→∞,

and

(8.4) |Rψλ(as)| ≤ C1Rψλ(as) ≤ C2e(μλ−d)s for s→ −∞,

for positive constants Ci. Since s �→ Rψλ(as) again is a linear combination ofe(μλ−d)s and e(−λ−ρ1)s, we see from (8.3) and (8.4) that Rψλ = 0. �

Consider the cases where p < q and d(q − p) ≤ 2, i.e., the cases (d, p, q) =(1, q − 1, q), (d, p, q) = (1, q − 2, q) and (d, p, q) = (2, q − 1, q). In the first caseμλ = λ + 1/2, and μλ = λ in the last two cases. This means that μλ > 0 and (i)follows from Proposition 8.1.

For the proof of (v), we need to consider the cases where p < q and d(q−p) ≤ 1,i.e., the cases (d, p, q) = (1, q − 1, q). From [1, Theorem 5.1 (iii)(a)] (recall that inthat paper p := p − 1), we see that the Schwartz condition in the real case is alsosatisfied for p = q − 1.

9. Reduction to the real case (d = 1)

Some of our results above for the projective hyperbolic spaces could also beestablished from [1, Theorem 5.2] via the remark below. However, we feel that thenew and different presentation, and in particular the new proof of Proposition 8.1,merits the space given.

Let F = C, H, with p ≥ 0, q ≥ 1, and d = dimR F. There is a natural projection

X(dp, dq,R)→ X(p, q,F),

with a natural action of U(1;F) on X(dp, dq;R). Let

Eλ(p, q,F) = {f ∈ C∞(X(p, q,F)) | Δf = (λ2 − ρ2q)f},


then there is a G-homomorphism,

Eλ(p, q,F)→ Eλ(dp, dq,R),which is an isomorphism onto the U(1;F) invariant functions in Eλ(dp, dq,R). Werefer to [7] for more details.

Let p′+1 = d(p+1) and q′+1 = d(q+1). We note that ρq = 12 (dp+dq+2(d−

1)) = 12 (p

′ + q′) = ρ′q. Let ψλ be as in the proof of Theorem 8.1, and let Rdp,q and

R1p′,q′ denote the Radon transforms corresponding to the spaces X(p + 1, q + 1,F)

and X(p′ + 1, q′ + 1,R) respectively. Using the substitution w′ = esw in (8.2), weget the identity:

(9.1) Rdp,qψλ(as) = e−(d−1)sR1

p′,q′ ψλ(as), (s ∈ R),

which shows that some of our results for the projective hyperbolic spaces followfrom the real (d = 1) case. Notice though, that the elements as ∈ Aq on theleft-hand and the right-hand side of (9.1) belong to different groups, and that thereduction only works for the Abelian part. Similarly, we get

Adp,qψλ = A1

p′,q′ ψλ.

10. Proof of (vi) of the main theorem - A Closer study of Rf near +∞We want to prove that A(Df) ∈ S(R), for f ∈ C2(G/H) and d(q − p) > 1.

Although we believe this to be true in general, our proof, near +∞, is only validfor the dense G-invariant subspace generated by the K-finite and (K∩H)-invariantfunctions.

For d = 1, [1, Theorem 5.1 (iii)(b)] yields that the Schwartz decay conditionsare satisfied near −∞ for A(f), and thus also for A(Df). For d > 1, the proof from[1] is easily adapted in the same way as formula (9.1), which leaves us to study Rfnear +∞. We will concentrate on the proof for d = 2, 4 below, with some commentson the d = 1 case, and further remarks in Section 11.

Consider the subgroup T given by

kθ =

⎛⎜⎜⎝Ip+1 0 0 00 cos θ 0 sin θ0 0 Iq−1 00 − sin θ 0 cos θ

⎞⎟⎟⎠ .

Thenkθat · x0 = (sinh t, 0, . . . , 0; sin θ cosh t, 0, . . . , 0, cos θ cosh t).

We see that H ⊃ K1, with K1 normalizing T , and K2 = (K2 ∩H)T (K2 ∩H),where K2 ∩ H = U(q,F) × U(1,F). Furthermore U(q,F) centralizes A, and as iseasily seen, (K ∩ H)kθwatH = (K ∩ H)kθatH, for w ∈ U(1,F). From this wededuce that

(10.1) K ∩H = (K ∩H)T (K ∩H)Aq and G = (K ∩H)TAH,

where (K ∩ H)T and (K ∩ H)Aq denote the centralizers of T and Aq in K ∩ Hrespectively. It follows that a K ∩H-invariant function is uniquely determined bythe values f(kθatH), for (θ, t) ∈ [0, π]× R+.

From the equation (K ∩H)kθatH = (K ∩H)asnu,v′,wH, we get

(10.2)(cosh t)2 = (cosh s− 1/2es|v′|2)2 + |v′|2 + |u|2 + |esw|2, and

(cos θ cosh t)2 = (cosh s− 1/2es|v′|2)2 + |esw|2.


Let x = |u|, y = |v′| and z = es|w|. Let v = − sinh s + 1/2esy2, then y2 =1 + 2e−sv − e−2s, and

(10.3)(cosh t)2 = 1 + x2 + v2 + z2, and

(cos θ cosh t)2 = (v − e−s)2 + z2.

For p = 0, the variable x = 0 and the integration over x disappears, and for d = 1,the integration over z disappears, Furthermore, the equations (10.2) and (10.3) areslightly different in these cases, see Section 11.

Consider a K∩H-invariant function f of irreducible K-type. Then the functionk �→ f(kat) is a zonal spherical function on K/(K ∩ H), a Jacobi polynomial incos θ, of even order. We can thus decompose f as a finite sum of functions of theform h(kθat) = (cos θ)mh(at), where m is even and h is an even function.

Define an auxiliary function H by H(cosh2 t) = (cosh t)−mh(at). Then, usingthe change of coordinates from before, we have:

h(kθat) = (cos θ cosh t)mH(cosh2 t) = ((v − e−s)2 + z2)m2 H(1 + x2 + v2 + z2),

where for each N ∈ N,

|H(1 + x2 + v2 + z2)| < C(1 + x2 + v2 + z2)−ρq+m

2 (1 + log(1 + x2 + v2 + z2))−N .

With the above substitutions, we find

Rh(s) = e−ds

∫ ∞

0

∫ ∞

0

∫ ∞

− sinh s

H(1 + x2 + v2 + z2)((v − e−s)2 + z2)m2

× (1 + 2e−sv − e−2s)β−12 xαzd−2 dv dx dz,

where α = dp− 1 , β = d(q − p)− 1 > 0, i.e.,β is a positive integer.We have the following upper bound, for s ≥ 0, since β ≥ 1:

|Rh(s)| ≤ Ce−ds

∫ ∞

0

∫ ∞

0

∫ ∞

−∞

(1 + x2 + v2 + z2)−ρq+m

2

(1 + log(1 + x2 + v2 + z2))N(1 + v2 + z2)

m2

× (1 + v2)β−12 xαzd−2 dv dx dz < +∞.

Applying Lebesgue’s theorem, we get

lims→∞

edsRh(s) =

∫ ∞

0

∫ ∞

0

∫ ∞

−∞H(1 + x2 + v2 + z2)(v2 + z2)

m2 xαzd−2 dv dx dz.

For convenience, we replace z by u. We can define Rh(s) as a function of thevariable z = e−s near z = 0, for z > 0. Let F (z) = edsRh(s), then

F (z) =

∫ ∞

0

∫ ∞

0

∫ ∞

12 (z−z−1)

H(1 + x2 + v2 + u2)((v − z)2 + u2)m2

× (1 + 2zv − z2)β−12 xαud−2 dv dx du.

Let k0 be the largest integer such that k0 < (β−1)/2+1, and 0 ≤ k < k0. Thederivatives dk/dzk of the integrand are zero at v = − sinh s = 1

2 (z − z−1), whencethe integrand is at least k0 times differentiable near z = 0, and we can compute thederivatives dk/dzkF (z). For k0 > 0, we will use Taylor’s formula to express F (z)as a polynomial of degree k0 − 1, plus a remainder term involving dk0/dzk0F (ξ),for some 0 < ξ(z) < z.


Lemma 10.1. Fix v, u ∈ R, m ∈ 2Z+ and δ ∈ 12Z

+, and define

S(z) = Sv,u,m,δ(z) = ((v − z)2 + u2)m2 (1 + 2zv − z2)δ.

For 0 ≤ j < δ + 1, dj/dzjS(z) is a polynomial in ((v − z)2 + u2), (v − z) and(1+2zv−z2), of degree at most m+δ in v, m in u and m+2δ− j in z. For z = 0,the degree is at most m + j in v, and m in u. When j is odd, dj/dzjS is an oddfunction of v at z = 0.

Proof. Straightforward, using that d/dz(1+2zv−z2) = −d/dz((v−z)2+u2) =2(v − z). �

Note, that for d(q− p) odd, that is, d = 1 and q− p odd, the term (β − 1)/2 =((q − p) − 2)/2 is a half-integer, and the statements in Lemma 10.1 have to bechanged accordingly.

Using Taylors formula, we get

F (z) = c0 + c1z + c2z2 + · · ·+ ck0−1z

k0−1 +Rk0(ξ)zk0 ,

where 0 < ξ < z, and

cj =1

j!

∫ ∞

0

∫ ∞

0

∫ ∞

−∞H(1 + x2 + v2 + u2)

dj

dzjSv,u,m,(β−1)/2(0) x

αud−2 dv dx du,

for j ∈ {0, . . . , k0 − 1}. By Lemma 10.1, cj = 0, for j odd. The remainder termRk0

(ξ) is given by:

1

k0!

∫ ∞

0

∫ ∞

0

∫ ∞

12 (ξ−ξ−1)

H(1 + x2 + v2 + u2)dk0

dzk0Sv,u,m,(β−1)/2(ξ) x

αud−2 dv dx du.

Consider Ah(s) = eρ1sRh(s) = z−(ρ1−d)F (z), which is equal to

c0z−(ρ1−d) + c2z

−(ρ1−d−2) + ...+ ck0−2z−2 + ck0−1z

−1 +Rk0(ξ).

The exponents ρ1−d− 2j = d(q−p)/2− 1− 2j, for j ∈ {0, . . . , k0− 1}, correspondto the parameters λ1, . . . , λr of the non-cuspidal discrete series. From (5.3), andthe definition of the differential operator D in Theorem 6.1 (vi), A(Dh) thus onlyhas a possible contribution from the remainder term, and, due to the term d2/ds2,no constant term at ∞.

Note, that for d(q − p) odd, the last two terms are: ck0−1z− 1

2 + z12Rk0

(ξ),where the last term is rapidly decreasing. For the other cases, the constant termCRk0

= lims→∞ Rk0(e−s) could be non-zero, but we will prove that Rk0

(ξ)−CRk0

is rapidly decreasing at +∞, where ξ = ξ(s), with 0 < ξ < e−s. We also considerthe case k0 = 0, with ξ = e−s.

Let G(v, u, z) = 1/k0! dk0/dzk0Sv,u,m,(β−1)/2(z). Then G(v, u, z)−G(v, u, 0) =

zP (v, u, z), where P is a polynomial of degree less than m + (β − 1)/2 in v, andless than m in u. Let |P | and |G| denote the polynomials defined from P and G bytaking absolute values in all coefficients.


Let in the following C denote (possibly different) positive constants. With0 < ξ < e−s, we get the following estimates

|Rk0(ξ)−CRk0

| ≤

e−s

∫ ∞

0

∫ ∞

0

∫ ∞

−∞|H(1 + x2 + v2 + u2)||P |(v, u, 1) xαud−2 dv dx du

+

∫ ∞

0

∫ ∞

0

∫ − sinh s

−∞|H(1 + x2 + v2 + u2)||G|(v, u, 0) xαud−2 dv dx du.

The first integral is bounded by Ce−s, since the double integral is convergent.The second integral is bounded near infinity by Cs−N , for all N , which is seen asfollows. For s large, the integrand is for every N ∈ N bounded by C(x2 + v2 +u2)−(ρ+m)/2|v|m+k0 log(x2+v2+u2)−N . Substituting v = −v, x = x′v and u = u′v,we have the estimates

≤ C

∫ ∞

0

∫ ∞

0

∫ ∞

sinh s

(1 + x′2 + u′2)−ρq+m

2 v−ρq−m+m+k0+α+1+d−1

× (log(v2) + log(1 + x′2 + u′2))−N x′αu′d−2 dv dx′ du′

≤ C

∫ ∞

sinh s

v−ρq+k0+α+d(log(v))−N dv.

Inserting the values ρq = d(p+ q)/2+ d− 1, k0 = (d(q− p)− 2)/2, and α = dp− 1,we end up with

C

∫ ∞

sinh s

v−1(log(v))−N dv = C(N − 1)−1(log(sinh s))−N+1 ≤ Cs−N+1.

It follows that Rk0(ξ)−CRk0

is rapidly decreasing at +∞, whence A(Dh) is rapidlydecreasing at +∞, since the constant term is not present, which finishes the proofof Theorem 6.1 (vi) for K-irreducible (K ∩H)-invariant functions.

Finally, consider the G-invariant subspace V of C2(G/H) generated by the K-irreducible (K ∩H)-invariant functions. The conclusion in (vi) is clearly satisfiedfor f ∈ V . We need to show that V is dense in C2(G/H). Let 0 �= f ∈ L2(G/H)be perpendicular to V . Let U be the closed G-invariant subspace of L2(G/H)generated by f . Then U contains a non-zero C∞-vector f1 ∈ C2(G/H), and aftera translation, we may assume that f1(eH) �= 0. The function f2 defined by 0 �=f2(gH) =

∫K∩H

f1(kgH) dk is then a (K ∩H)-invariant element in U , belonging tothe closure of V , which is a contradiction.

11. Final Remarks - the remaining cases

Theorem 6.1 also holds for the real non-projective space G/H = SO(p+ 1, q+1)e/SO(p + 1, q)e, except for item (iii). The statements (i), (ii), (iv) and (v) areproved in [1]. For the proof of (vi), the last equations in (10.2) and (10.3) shouldbe replaced by

cos θ cosh t = cosh s− 1/2es|v′|2, andcos θ cosh t = (v − e−s).


Then (θ, t) ∈ [0, 2π] × R+, and m could be odd. For p = 0, the first equations in(10.2) and (10.3) should be replaced by:

sinh t = sinh s− 1/2esv′2, and

sinh t = −v,

with (θ, t) ∈ [0, 2π]× R, H defined by H(− sinh t) = (cosh t)−mh(at), and

|H(v)| < C(1 + v2)−ρq+m

2 (1 + log(1 + v2))−N .

With these remarks it is not difficult to modify Lemma 10.1, and complete the proof.Notice, that a priori all constants cj in the Taylor expansion could be non-zero.

Finally, we consider the exceptional case, with F = O (and p = 0, q = 1, d = 8).We will show that the formulas (10.1) and (10.2) are meaningful and true for thiscase as well. The formula (10.1) was already shown to be true and used in [4]. Wegive a brief outline of the proof of (10.2).

According to [6] and [8], the exceptional group G can be defined by the auto-morphisms of a 27 dimensional Jordan Algebra J1,2 parameterized by ξ, u ∈ R3×O3,with basis E1, E2, E3, F1, F2, F3. We denote an element in J1,2 by X(ξ, u).

The subgroups H and K are in fact equal to the stabilizers of E3, respec-tively of E1. The subgroup N , which in this case equals N∗, is defined in [6] asu(y, z), y ∈Im(O) and z ∈ O. The subgroups A = {at} and T = {kθ} are also de-fined there. In [6] the expression asu(y, z)E3 is calculated, and in [8] the expressionkθX(ξ, u) is calculated; combining these two calculations kθatE3 = kθatu(0, 0)E3

can be calculated.Recall that ξ1, ξ3 are invariant under K ∩ H. To derive the first formula in

(10.2), we only need to compare the first coordinates of kθatE3 and kθatu(y, z)E3,ξ1(s, y, z) = ξ1(t, θ); to derive the second formula in (10.2), we only need to comparethe third coordinates of kθatE3 and kθatu(y, z)E3, ξ3(s, y, z) = ξ3(t, θ). We have

ξ1(t, θ) = −(cosh (2t)− 1)/2,

ξ1(s, y, z) = − cosh (2s)((1− z)/2 + |z|4/4 + |y|2)+ sinh (2s)(1/2|z|2(1− |z|2/2)− |y|2) + (1− |z|2)/2,

ξ3(t, θ) = (cosh (2t) + 1)/4(1 + cos (2θ)),

ξ3(s, y, z) = cosh (2s)((1− z)/2 + |z|4/4 + |y|2)− sinh (2s)(1/2|z|2(1− |z|2/2)− |y|2) + (1− |z|2)/2.

A tedious, but straightforward calculation, leads to the formulas (10.2), with v′

replaced by z, and w replaced by y.

References

[1] N. B. Andersen, M. Flensted-Jensen, and H. Schlichtkrull, Cuspidal discrete seriesfor semisimple symmetric spaces, J. Funct. Anal. 263 (2012), no. 8, 2384–2408, DOI10.1016/j.jfa.2012.07.009. MR2964687

[2] E. P. van den Ban, The principal series for a reductive symmetric space. II. Eisensteinintegrals, J. Funct. Anal. 109 (1992), no. 2, 331–441, DOI 10.1016/0022-1236(92)90021-A.MR1186325 (93j:22025)

[3] Mogens Flensted-Jensen, Discrete series for semisimple symmetric spaces, Ann. of Math. (2)111 (1980), no. 2, 253–311, DOI 10.2307/1971201. MR569073 (81h:22015)


[4] Mogens Flensted-Jensen and Kiyosato Okamoto, An explicit construction of the K-finite vec-tors in the discrete series for an isotropic semisimple symmetric space, Mem. Soc. Math.France (N.S.) 15 (1984), 157–199. Harmonic analysis on Lie groups and symmetric spaces(Kleebach, 1983). MR789084 (87c:22025)

[5] I. S. Gradshteyn and I. M. Ryzhik, Table of integrals, series, and products, 6th ed., AcademicPress Inc., San Diego, CA, 2000. Translated from the Russian; Translation edited and with apreface by Alan Jeffrey and Daniel Zwillinger. MR1773820 (2001c:00002)

[6] M. T. Kosters, Spherical Distributions on Rank One Symmetric Spaces Thesis, Leiden, 1983.[7] Henrik Schlichtkrull, Eigenspaces of the Laplacian on hyperbolic spaces: composition se-

ries and integral transforms, J. Funct. Anal. 70 (1987), no. 1, 194–219, DOI 10.1016/0022-1236(87)90130-3. MR870761 (88f:22040)

[8] Reiji Takahashi, Quelques resultats sur l’analyse harmonique dans l’espace symetrique noncompact de rang 1 du type exceptionnel, Analyse harmonique sur les groupes de Lie (Sem.,Nancy-Strasbourg 1976–1978), II, Lecture Notes in Math., vol. 739, Springer, Berlin, 1979,pp. 511–567 (French). MR560851 (81i:22012)

Department of Mathematics, Aarhus University, Ny Munkegade 118, Building 1530,

DK-8000 Aarhus C, Denmark


Department of Mathematical Sciences, University of Copenhagen, Universitetspar-

ken 5, DK-2100 Copenhagen Ø, Denmark



The Radon transform on SO(3): motivations, generalizations,discretization

Swanhild Bernstein and Isaac Z. Pesenson

Dedicated to S. Helgason on his 85-th Birthday

Abstract. In this paper we consider a version of the Radon transform R onthe group of rotations SO(3) and closely related crystallographic X-ray trans-form P on SO(3). We compare the Radon transform R on SO(3) and thetotally geodesic 1-dimensional Radon transform on S3. An exact reconstruc-tion formula for bandlimited function f on SO(3) is introduced, which usesonly a finite number of samples of the Radon transform Rf .

1. Introduction

In this paper we consider a version of the Radon transform R on the group ofrotations SO(3) and closely related crystallographic X-ray transform P on SO(3)1

We show that both of these transforms naturally appear in texture analysis, i.e.the analysis of preferred crystallographic orientation. Although we discuss onlyapplications to texture analysis both transforms have other applications as well.

The structure of the paper is as follows. In section 2 we start with motivationsand applications. In section 3 we collect some basic facts about Fourier analysison compact Lie groups. In section 4 we introduce and analyze an analog of R forgeneral compact Lie groups. In the case of the group SO(n+1) we compute imageR(W) whereW is the span of Wigner polynomials in SO(n+1). In section 5 we givea detailed analysis of the Radon transform R on SO(3). In section 6 we describerelations between S3, SO(3) and S2 × S2 and we compare the Radon transformR on SO(3) and the totally geodesic 1-dimensional Radon transform on S3. Insection 7 we show non-invertibility of the crystallographic X-ray transform P . Insection 8 we describe an exact reconstruction formula for bandlimited function fon SO(3), which uses only a finite number of samples of the Radon transform Rf .Some auxiliary results for this section are collected in Appendix.

The Radon transform on SO(3) has recently attracted attention of many math-ematicians. In addition to articles, which will be mentioned in our paper later wealso refer to [6], [16], [17], [18], [22], [25].

2000 Mathematics Subject Classification. Primary 44A12, 43A85, 58E30, 41A99 .The author was supported in part by the National Geospatial-Intelligence Agency University

Research Initiative (NURI), grant HM1582-08-1-0019.1In [25] the same transform R was termed as the Funk transform.


77

78 SWANHILD BERNSTEIN AND ISAAC Z. PESENSON

2. Texture goniometry

A first mathematical description of the inversion problem in texture analysiswas given in [7] and [8]. Let us recall the basics of texture analysis and texturegoniometry (see [4] and [5]). Texture analysis is the analysis of the statistical dis-tribution of orientations of crystals within a specimen of a polycrystalline material,which could be metals or rocks. A crystallographic orientation is a set of crystalsymmetrically equivalent rotations between an individual crystal and the specimen.

The main objective is to determine orientation probability density function f(ODF) representing the probability law of random orientations of crystal grains byvolume.

In X-ray diffraction experiments, the orientation density function f (ODF)that represents the probability law of random orientations of crystal grains cannotbe measured directly. Instead, by using a texture goniometer the pole densityfunction (PDF) Pf(x, y) can be sampled. Pf(x, y) represents probability that afixed crystal direction x ∈ S2 or its antipodal −x statistically coincides with thespecimen direction y ∈ S2 due to Friedel’s law in crystallography [11].

To define the pole density function Pf(x, y) some preliminaries are necessary.The group of rotations SO(3) of R3 consists of 3 × 3 real matrices U such thatUTU = I, detU = 1. It is known that any g ∈ SO(3) has a unique representationof the form

g = Z(γ)X(β)Z(α), 0 ≤ β ≤ π, 0 ≤ α, γ < 2π,

where

Z(θ) =

⎛⎝ cos θ − sin θ 0sin θ cos θ 00 0 1

⎞⎠ , X(θ) =

⎛⎝ 1 0 00 cos θ − sin θ0 sin θ cos θ

⎞⎠are rotations about the Z- and X-axes, respectively. In the coordinates α, β, γ,which are known as Euler angles, the Haar measure of the group SO(3) is given as(see [24])

dg =1

8π2sinβdα dβ dγ.

In other words the following formula holds:∫SO(3)

f(g) dg =

∫ 2π

0

∫ π

0

∫ 2π

0

f(g(α, β, γ))1

8π2sinβdα dβ dγ.

First, we introduce Radon transform Rf of a smooth function f defined on SO(3).If S2 is the standard unit sphere in R3 , then for a pair (x, y) ∈ S2 × S2 the valueof the Radon transform Rf at (x, y) is defined by the formula

(Rf)(x, y) =1

2π

∫{g∈SO(3):x=gy}

f(g)dνg =

(2.1) 4π

∫SO(3)

f(g)δy(g−1x)dg = (f ∗ δy)(x), (x, y) ∈ S2 × S2,

where dνg = 8π2dg, and δy is the measure concentrated on the set of all g ∈ SO(3)such that x = gy.

The pole density function Pf or crystallographic X–ray transform of an orien-tation density function f is an even function on S2 × S2, which is defined by the

THE RADON TRANSFORM ON SO(3) 79

following formula

(2.2) Pf(x, y) =1

2(Rf(x, y) +Rf(−x, y)), (x, y) ∈ S2 × S2.

Note, that since ODF f is a probability density it has to have the following prop-erties:

(a) f(g) ≥ 0,(b)∫SO(3)

f(g)dg = 1.

In what follows we will discuss inversion of the crystallographicX-ray transformPf and the Radon transform Rf .

First we formulate what can be called analytic reconstruction problem.

Problem 1. Reconstruct the ODF f(g), g ∈ SO(3), from PDF Pf(x, y), x, y ∈S2.

It will be shown in section 7 that this problem is unsolvable in general sincethe mapping f → Pf has a non-trivial kernel.

Problem 2. Reconstruct f(g), g ∈ SO(3), from all Rf(x, y), x, y ∈ S2.

An explicit solution to this problem will be given in section 5.In practice only a finite number of pole figures P (x, y), x, y ∈ S2, can be mea-

sured. Therefore the real life reconstruction problem is the following.

Problem 3. Using a finite number of pole figures P (xi, yj), xi, yj ∈ S2, i =1, . . . , n, j = 1, . . . , m, find a function f on SO(3), which would satisfy (in somesense) equations (2.2) and conditions (a) and (b).

An approximate solution to this problem in terms of Gabor frames was foundin [9].

The corresponding discrete problem for Rf can be formulated as follows.

Problem 4. Reconstruct f(g), g ∈ SO(3), from a finite number of samplesRf(xj , yj), xj , yj ∈ S2, j = 1, ...,m.

This problem will be solved in section 8 for bandlimited functions on SO(3).We were able to obtain an exact reconstruction formula for bandlimited functions,which uses only a finite number of samples of their Radon transform. Anotherapproach to this problem which uses the so-called generalized splines on SO(3) andS2 × S2 was developed in our paper [3].

In section 4 we suggest a new type of Radon transform associated with a pair(G, H) where G is a compact Lie group and H its closed subgroup. This definitionappeared for the first time in our paper [3]. Namely, for every continuous functionf on G the corresponding Radon transform is defined by the formula

(2.3) Rf(x, y) =

∫Hf(xhy−1) dh, x, y ∈ G.

Problem 5. Determine domain and range for the Radon transform R

Some partial solutions to this problem are given in section 4. In section 3 werecall basic facts about Fourier analysis on compact Lie groups. In section 6 wecompare crystallographic X-ray transform on SO(3) and Funk transform on S3. InAppendix 9 we briefly explain the major ingredients of the proof of our DiscreteInversion Formula which is obtained in section 8.


3. Fourier Analysis on compact groups

Let G be a compact Lie group. A unitary representation of G is a continuousgroup homomorphism π: G → U(dπ) of G into the group of unitary matrices ofa certain dimension dπ. Such representation is irreducible if π(g)M = Mπ(g) forall g ∈ G and some M ∈ Cdπ×dπ implies M = cI, where I is the identity matrix.Equivalently, Cdπ does not have non-trivial π-invariant subspaces V ⊂ Cdπ withπ(g)V ⊂ V for all g ∈ G. Two representations π1 and π2 are equivalent, if thereexists an invertible matrix M such that π1(g)M = Mπ2(g) for all g ∈ G.

Let G denote the set of all equivalence classes of irreducible representations.This set parameterizes an orthogonal decomposition of the Hilbert space L2(G)constructed with respect to the normalized Haar measure. Let {ej} be an orthonor-mal basis for the unitary matrices U(dπ) of dimension dπ. Then for any unitaryrepresentation of G the πij(g) = 〈π(g)ej , ei〉 are called matrix elements of π. Wedenote the linear span of the matrix elements of π by Hπ.

Theorem 3.1 (Peter-Weyl, [31]). Let G be a compact Lie group. Then thefollowing statements are true.

a: The Hilbert space L2(G) decomposes into the orthogonal direct sum

L2(G) =⊕π∈G

Hπ(3.1)

b: For each irreducible representation π ∈ G the orthogonal projectionL2(G)→ Hπ is given by

f �→ dπ

∫Gf(h)χπ(h

−1g) dh = dπ f ∗ χπ,(3.2)

in terms of the character χπ(g) = trace(π(g)) of the representation anddh is the normalized Haar measure.

We will denote the matrix M in the equation f ∗ χπ = trace(π(g)M) as the

Fourier coefficient f(π) of f at the irreducible representation π. The Fourier coef-ficient can be calculated as

f(π) =

∫Gf(g)π∗(g) dg, π ∈ G.

The inversion formula (the Fourier expansion) is then given by

f(g) =∑π∈G

dπ trace(π(g)f(π)).

If we denote by ||M ||2HS = trace(M∗M) the Frobenius or Hilbert-Schmidt norm ofa matrix M, then the following Parseval identity is true.

Theorem 3.2 (Parseval identity). Let f ∈ L2(G). Then the matrix-valued

Fourier coefficients f ∈ Cdπ×dπ satisfy

||f ||2 =∑π∈G

dπ ||f(π)||2HS.(3.3)

On the group G one defines the convolution of two integrable functions f, r ∈L1(G) as

f ∗ r(g) =∫Gf(h)r(h−1g) dh.


Since f ∗ r ∈ L1(G), the Fourier coefficients are well-defined and they satisfy

Theorem 3.3 (Convolution theorem on G). Let f, r ∈ L1(G) then f ∗r ∈ L1(G)and

f ∗ r(π) = f(π)r(π).

The group structure gives rise to the left and right translations Tgf �→ f(g−1·)and T gf �→ f(·g) of functions on the group. A simple computation shows

Tgf(π) = f(π)π∗(g) and T gf(π) = π(g)f(π).

These formulas are direct consequences of the definition of the Fourier transform.The Laplace-Beltrami operator ΔG of an invariant metric on the group G is

bi-invariant, i.e. commutes with all Tg and T g. Therefore, all its eigenspaces arebi-invariant subspaces of L2(G). As Hπ are minimal bi-invariant subspaces, eachof them has to be the eigenspace of ΔG with the corresponding eigenvalue −λ2

π.Hence, we obtain

ΔGf = −∑π∈G

dπ λ2π trace(π(g)f(π)).

4. Problem 5: Radon transform on compact groups

4.1. Radon transform. In this section we discuss some basic properties onthe Radon transform Rf which was defined in (2.3).

Theorem 4.1 ([3]). The Radon transform ( 2.3) is invariant under right shiftsof x and y, hence it maps functions on G to functions on G/H× G/H.

Proof. First, we take the Fourier transform of Rf with respect to the x andlet y be fixed and regard Rf(x, y) as a function of x ∈ G only. Then

Rf(·, y)(π) = πHπ∗(y)f(π), π ∈ G.

It is easily seen that Rf(x, y) is invariant under the projection PH and we obtain

Rf(x · h, y) = Rf(x, y) ∀h ∈ H.

If we look at the Radon transform as a function in y while the first argument x isfixed, we find

PH(Rf)(x, y) =

∫HRf(x, yh) dh =

∫H

∑π∈G

dπtrace (f(π)π(x))πHπ(h−1y−1) dh

(4.1) =∑π∈G

dπtrace (f(π)π(x))πHπ∗(y) = (R)f(x, y).

Consequently, Rf(x, y) is constant over fibers of the form yH and

Rf(x, ·)(π) = πHπ∗(x)f(π), π ∈ G.

�

The next Theorem is a refinement of the previous result.


Theorem 4.2 ([3]). Let H be a subgroup of G which determines the Radon

transform on G and let G1 ⊂ G be the set of irreducible representations with respectto H. Then for f ∈ C∞(G) we have

||Rf ||2L2(G/H×G/H) =∑π∈G1

rank (πH)||f ||2HS .

Proof. We expand Rf(x, y) for fixed y into a series with respect to x andapply Parseval’s theorem

||Rf ||2L2(G/H×G/H) =∑π∈G

dπ

∫G||πHπ∗(y)f(π)||2HSdy =

∑π∈G

dπ

∫Gtrace (f∗(π)π(y)πHπ∗(y)f(π)) dy =

∑π∈G

dπtrace (f∗(π)

(∫Gπ(y)πHπ∗(y) dy

)f(π)) =

∑π∈G

dπtrace (f∗(π)

(rank πH∑k=1

∫Gπik(y)πkj(y) dy

)dπ

i,j=1

f(π)) =

∑π∈G

dπtrace (f∗(π)rankπH

dπIdf(π)) =

(4.2)∑π∈G

rank πHtrace (f∗(π)f(π)) =∑π∈G1

rank (πH)||f ||2HS .

�4.2. The case G = SO(n+ 1), H = SO(n). We start with the orthonormal

system of spherical harmonics Yik ∈ C∞(Sn), k ∈ N0, i = 1, . . . , dk(n) normalized

with respect to the Lebesgue measure on Sn. Obviously Hk = span {Yik}

dk(n)i=1 . Then

the Wigner polynomials on SO(n+ 1) T ijk (g), g ∈ SO(n+ 1) are given by

T ijk (g) =

∫Sn

Yik(g

−1x)Yjk(x) dx

and due to the orthogonality of the spherical harmonics

Yik(g

−1x) =

dk(n)∑j=1

T ijk (g)Yj

k(x).

From these properties and the orthonormality of the spherical harmonics it easy tosee that the Wigner polynomials build an orthonormal system in L2(SO(n + 1)).Unfortunately, Wigner polynomials do not give all irreducible unitary representa-tions of SO(n+ 1) if n > 2.

Definition 4.3. A unitary representation of a group G in a liner space L issaid to be of class-1 relative subgroup H if L contains non-trivial vectors that areinvariant with respect to H.

Definition 4.4. If in the space L of any representation of class-1 relative Hthere is only one normalized invariant vector, then H is called a massive subgroup.


Lemma 4.5 ([30], Chapter IX.2). SO(n) is a massive subgroup of SO(n+ 1).Furthermore, the family Tk, k ∈ N0, gives all class-1 representations of SO(n+ 1)with respect to SO(n) up to equivalence.

For the following let x0 be the base point of SO(n+1)/SO(n) ∼ Sn (x0 is usu-ally chosen to be the ”north pole”.) In this case the set of zonal spherical harmonics

is one-dimensional and spanned by the Gegenbauer polynomials C(n−1)/2k (xT

0 x). Werecall some helpful and well known results.

Lemma 4.6 (Addition theorem). For all x, y ∈ Sn, k ∈ N0 and i = 1, . . . , dk(n)

C(n−1)/2k (xT y)

C(n−1)/2k (1)

=|Sn|dk(n)

dk(n)∑i=1

Yik(x)Y

jk(y).

Lemma 4.7 (Zonal averaging).∫SO(n)

Yik(gx) dg =

Yik(x0)

C(n−1)/2k (1)

C(n−1)/2k (xT

0 x).

Lemma 4.8 (Funk-Hecke formula). Let f : [−1, 1] → C be continuous. Thenfor all i = 1, . . . , dk(n)∫

Sn

f(xT y)Yik(x) dx = Yi

k(y)|Sn−1|

C(n−1)/2k (1)

∫ 1

−1

f(t)C(n−1)/2k (t)(1− t2)n/2−1 dt.

Since we are interested in functions on Sn, which we obtain by the projectionfrom SO(n + 1), we have to consider all irreducible representations of SO(n +1) which do not have vanishing matrix coefficients under the projection PSO(n).These irreducible representations form the class-1 representations of SO(n + 1)with respect to SO(n) and the projections are given by

PSO(n)T ijk =

∫SO(n)

T ijk (g) dg =

∫Sn

∫SO(n)

Yik(g

−1x) dg Yjk(x) dx =

Y ik(x0)

C(n−1)/2k

∫Sn

C(n−1)/2k (xT

0 x)Yik(x) dx =

Y ik(x0)Yj

k(x0)

(C(n−1)/2k (1))2

|Sn|∫ 1

−1

(C(n−1)/2k (t))2(1− t2)n/2−1 dt =

(4.3)|Sn|dk(n)

Yik(gx0)Yj

k(x0),

due to the Funk-Hecke formula and the normalization of Gegenbauer polynomials.We assume that the basis of spherical harmonics Y i

k(x) is chosen in such a way that

Y1k(x0) =

√dk(n)|Sn| and Yi

k(x0) = 0 for all i > 0, then√|Sn|dk(n)

Yik(x) = (PSO(n)T i1

k )(x) =

∫SO(n)

T i1k (gh) dh = T i1

k (g), x = gx0.

Theorem 4.9. If f belongs toW=span{Tk}, i.e. f(g)=∑∞

k=0

∑dk(n)i,j=1 f(k)ijT

ijk

then

Rf(x, y) = |Sn|∞∑k=0

dk(n)∑i,j=1

f(k)ijYik(x)Y

jk(y).


Proof. One has

Rf(x, y) =∞∑k=0

dk(n)trace (f(k)Tk(x)πSO(n)T ∗k (y)) =

∞∑k=0

dk(n)

dk(n)∑i,j=1

f(k)ijT i1k (x)T 1j

k (y) =∞∑k=0

|Sn|dk(n)

dk(n)

dk(n)∑i,j=1

f(k)ijYik(x)Y

jk(y) =

(4.4) |Sn|∞∑k=0

dk(n)∑i,j=1

f(k)ijYik(x)Y

jk(y).

�

5. Problem 2: Radon transform on SO(3)

In this section we concentrate on the case G = SO(3), H = SO(2) and thusG/H = SO(3)/SO(2) = S2. An orthonormal system in L2(S2) is provided bythe spherical harmonics {Yi

k, k ∈ N0, i = 1, . . . , 2k + 1}. The subspaces Hk :=span {Y i

k, i = 1, . . . , 2k + 1} spanned by the spherical harmonics of degree k arethe invariant subspaces of the quasi-regular representation T (g) : f(x) �→ f(g−1 ·x),(where · denotes the canonical action of SO(3) on S2). Representation T decom-poses into (2k + 1)-dimensional irreducible representation Tk in Hk. The corre-sponding matrix coefficients are the Wigner-polynomials

T ijk (g) = 〈Tk(g)Yi

k,Yjk〉.

If ΔSO(3) and ΔS2 are Laplace-Beltrami operators of invariant metrics on SO(3)

and S2 respectively, then

ΔSO(3)T ijk = −k(k + 1)T ij

k and ΔS2Yik = −k(k + 1)Y i

k.

Using the fact that ΔSO(3) is equal to −k(k + 1) on the eigenspace Hk we obtain

||f ||2L2(SO(3)) =

∞∑k=1

(2k + 1)||f(k)||2HS =

∞∑k=1

(2k + 1)||(4π)−1f(k)||2L2(S2×S2) = ||(4π)−1(I − 2ΔS2×S2)1/4Rf ||2L2(S2×S2),

where ΔS2×S2 = Δ1 + Δ2 is the Laplace-Beltrami operator of the natural metricon S2 × S2. We define the following norm on the space C∞(S2 × S2)

|||u|||2 = ((I − 2ΔS2×S2)1/2u, u)L2(S2×S2).

Because R is essentially an isometry between L2(SO(3)) with the natural norm andL2(S2 × S2) with the norm ||| · ||| the inverse of R is given by its adjoint operator.To calculate the adjoint operator we express the Radon transform R in anotherway. Going back to our problem in crystallography we first state that the greatcircle Cx,y = {g ∈ SO(3) : g ·x = y} in SO(3) can also be described by the followingformula

Cx,y = x′SO(2)(y′)−1 := {x′h(y′)−1, h ∈ SO(2)}, x′, y′ ∈ SO(3),


where x′ · x0 = x, y′ · x0 = y and SO(2) is the stabilizer of x0 ∈ S2. Hence,

Rf(x, y) =

∫SO(2)

f(x′h(y′)−1) dh = 4π

∫Cx,y

f(g) dg

= 4π

∫SO(3)

f(g)δy(g−1 · x) dg, f ∈ L2(SO(3)).

To calculate the adjoint operator we use the last representation of R. We have

(R∗u, f)L2(SO(3)) = ((I − 2ΔS2×S2)1/2u, Rf)L2(S2×S2) =

(4π)

∫S2×S2

(I − 2ΔS2×S2)u(x, y)

∫SO(3)

f(g)δy(g−1 · x) dg dx dy =

(4π)

∫SO(3)

∫S2

(I − 2ΔS2×S2)1/2u(g · y, y) dy f(g) dg,

i.e. the L2-adjoint operator is given by

R∗u = (4π)

∫S2

(I − 2ΔS2×S2)1/2u(g · y, y) dy.(5.1)

Definition 5.1 (Sobolev spaces on S2 × S2). The Sobolev space Ht(S2 ×

S2), t ∈ R, is defined as the domain of the operator (I − 2ΔS2×S2)t2 with graph

norm

||f ||t = ||(I − 2ΔS2×S2)t2 f ||L2(S2×S2),

and the Sobolev spaceHΔt (S2×S2), t ∈ R, is defined as the subspace of all functions

f ∈ Ht(S2 × S2) such Δ1f = Δ2f.

Definition 5.2 (Sobolev spaces on SO(3)). The Sobolev space Ht(SO(3)), t ∈R, is defined as the domain of the operator (I − 4ΔSO(3))

t2 with graph norm

|||f |||t = ||(I − 4ΔSO(3))t2 f ||L2(SO(3)), f ∈ L2(SO(3)).

Theorem 5.3. For any t ≥ 0 the Radon transform on SO(3) is an invertiblemapping

R : Ht(SO(3))→ HΔt+ 1

2(S2 × S2).(5.2)

and

f(g) =

∫S2

(I − 2ΔS2×S2)12 (Rf)(gy, y)dy =

1

4π(R∗Rf)(g).(5.3)

Proof. For the mapping properties it is sufficient to consider case t = 0.Because the Radon transform is an isometry up to the factor 4π, we obtain (5.3). �

SinceR(T k)(x, y) = T k(x)πSO(2)

(T k(y)

)∗we have

RT kij (x, y) = T k

i1(x)T kj1(y) =

4π

2k + 1Y ik(x)Y

jk(y).

One can also verify that the following relations hold

(5.4) ΔS2×S2Rf = 2RΔSO(3)f, f ∈ H2(SO(3)),

(5.5) (1− 2ΔS2×S2)t/2Rf = R(1− 4ΔSO(3)

)t/2f, f ∈ Ht(SO(3)), t ≥ 0,


(5.6) R−1 (1− 2ΔS2×S2)t/2 g =(1− 4ΔSO(3)

)t/2R−1g,

where g ∈ HΔt+1/2(S

2 × S2), t ≥ 0.

Theorem 5.4 (Reconstruction formula). Let

G(x, y) = Rf(x, y) =

∞∑k=0

2k+1∑i,j=1

G(k)ijYik(x)Y

jk(y) ∈ HΔ

12+t(S

2 × S2), t ≥ 0,

be a result of the Radon transform. Then the pre-image f ∈ Ht(SO(3)), t ≥ 0, isgiven by

f =

∞∑k=0

2k+1∑i,j=1

(2k + 1)

4πG(k)ijT k

ij =

∞∑k=0

2k+1∑i,j=1

(2k + 1)f(k)ijT kij

=∞∑k=0

(2k + 1)trace (f(k)T k).

6. Radon transforms on the group SO(3) and the sphere S3

At the beginning of this section we show that S3 is a double cover of SO(3).This fact allows us to identify every function f on SO(3) with an even functionon S3. After this identification the crystallographic Radon transform on SO(3)becomes the geodesic Radon transform on S3 in the sense of Helgason [13], [14],[15]. In Theorem 6.7 we show how Helgason’s inversion formula for this transformcan be interpreted in crystallographic terms.

6.1. Quaternions and rotations. To understand the crystallographic Radontransform one has to understand relations between SO(3), S3, S2×S2. One of theways to describe these relations is by using the algebra of quaternions (see [19], [4],[25]).

Definition 6.1. Quaternions H are hypercomplex numbers of the form

q = a0 + a1i+ a2j + a3k,

where a0, a1, a2, a3 are real numbers and the generalized imaginary units i, j, ksatisfy the following multiplication rules:

i2 = j2 = k2 = −1,ij = k = −ji, jk = i = −kj, ki = j = −ik.

Definition 6.2. A quaternion q = a0 + a1i + a2j + a3k = q0 + q is the sumof the real part q0 = a0 and the pure part q = a1i + a2j + a3k. A quaternion q iscalled pure if its real part vanishes. The conjugate q of a quaternion q = a0 + q isobtained by changing the sign of the pure part:

q = a0 − q.

The norm ||q|| of a quaternion q is given by ||q||2 = qq = a20 + a21 + a22 + a23 andcoincises with the Euclidean norm of the associated element in R4.

All non-zero quaternions are invertible with inverse q−1 = q||q||2 . Next, we

connect quaternions and rotations in R3. Take a pure quaternion or a vector

a = a1i+ a2j + a3k ∈ R3


with norm ||a|| =√a21 + a22 + a23. For a non-zero quaternion q ∈ H the element

qaq−1 is again a pure quaternion with same length, i.e. ||qaq−1|| = ||a||. Thatmeans that the mapping R3 → R3

a �→ qaq−1

is a rotation with the natural identification of R3 with the set of pure quaternions.Each rotation in SO(3) = {U ∈ Mat(3,R) : UTU = I, detU = 1} can berepresented in such form and there are two unit quaterions q and −q representingthe same rotation qaq−1 = (−q)a(−q−1). That means that

S3 = {q ∈ H : ||q|| = 1}is a two-fold covering group of SO(3), i.e. SO(3) S3/{±1}.

Definition 6.3 ([4]). Let q1, q2 be two unit orthogonal quaternions, i.e. thescaler part of q1q2 which is equal to Euclidean scalar product of the vectors q1 andq2 is zero. The set of quaternions

q(t) = q1 cos t+ q2 sin t, t ∈ [0, 2π)

is called a circle in the space of unit quaternions and denoted as Cq1,q2 .

Obviously, the circle Cq1,q2 is the intersection of the unit sphere S3 with theplane E(q1, q2) spanned by q1, q2 and passing though the origin O.

Theorem 6.4 ([4]). Given a pair of unit vectors (x, y) ∈ S2×S2 with x �= −y,the ”great circle” Cx,y ∈ SO(3) of all rotations with gy = x in SO(3) may berepresented as a great circle Cq1,q2 of unit quaternions such that

Cq1,q2 := E(q1, q2) ∩ S3

with

(6.1) q1 := cosη

2+

y × x

||y × x|| sinη

2, q2 :=

y + x

||y + x|| ,

where η denotes the angle between x and y, i.e. cos η = y · x. Rotation gy = x inSO(3) corresponds to rotation x = qyq in H.

For an arbitrary quaternion q we define the linear map τ (q) of the algebra ofquaternions H into itself which is given by the formula

(6.2) τ (q)h = qhq, h ∈ H.

One can check that if q ∈ S3 then τ (q) ∈ SO(3).Let us summarize the following important facts (see [22], [25] for more details).

(1) The map τ : q → τ (q) has the property τ (q) = τ (−q) which shows that τis a double cover of S3 onto SO(3).

(2) τ maps

(6.3) τ : Cq1,q2 → Cx,y,

where Cq1,q2 = E(q1, q2) ∩ S3 is a great circle in S3 and Cx,y is a greatcircle in SO(3) of all rotations g with gy = x, (x, y) ∈ S2 × S2 (relationsbetween (q1, q2) and (x, y) are given in (6.1)). Conversely, pre-image ofCx,y is Cq1,q2 .

(3) Great circles Cq1,q2 are geodesics in S3 in the natural metric.


(4) The variety of all great circles Cx,y ∈ SO(3), (x, y) ∈ S2 × S2, (whichare sets of all rotations g with gy = x) can be identified with the productS2 × S2. For any (x, y) ∈ S2 × S2 the circles Cx,y and C−x,y, x �= −y,are contained in orthogonal 2-planes in H.

6.2. Radon transforms on S3 and on SO(3). Let Ξ denote the set of all1–dimensional geodesic submanifolds ξ ⊂ S3. According to the previous subsectioneach ξ ∈ Ξ is a great circle of S3, i.e. a circle with centre O. The manifold Ξ canbe identified with the manifold S2 × S2.

Following Helgason (see [13], [14], [15]), we introduce the next definition.

Definition 6.5. For a continuous function F defined on S3 its 1–dimensionalspherical (geodesic) Radon transform F is a function, which is defined on any 1-dimensional geodesic submanifold ξ ⊂ S3 by the following formula

(6.4) F (ξ) =1

2π

∫ξ

F (q) dω1(q) =

∫ξ

F (q) dm(q),

with the normalized measurem = 12πω1 where ω1 denotes the usual one–dimensional

circular Riemannian measure.

To invert transformation (6.4) Helgason introduces dual transformation

(6.5) φ(q) =

∫q∈ξ

φ(ξ) dμ(ξ), q ∈ S3,

which represents the average of a continuous function φ over all ξ ∈ Ξ passingthrough q ∈ S3. Further,

φρ(q) =

∫{d(q,ξ)=ρ}

φ(ξ)dμ(ξ), ρ ≥ 0, q ∈ S3,

where dμ is the average over the set of great circles ξ at distance ρ from q. We usethe inversion formula of S. Helgason [15], which was obtained for the general casetwo-point homogeneous spaces. For two dimensional sphere the totally geodesicRadon transform is also known as the Funk transform. The inversion formula canbe written as

(6.6) F (q) =1

π

[d

du2

∫ u

0

(F )ˇcos−1(v)(q)v(u2 − v2)−1/2dv

]∣∣∣∣u=1

, q ∈ S3.

Let us describe relations between geodesic Radon transform of functions definedon S3 and the Radon transform R of functions defined on SO(3). Given a functionf on SO(3) one can consider its Radon transform Rf which is defined on the setof all great circles Cx,y ⊂ SO(3). On the other hand one can construct an evenfunction F on S3 by using the formula

(6.7) F (q) = f(τ (q)), q ∈ S3,

where the mapping τ : S3 → SO(3) was defined in (6.2). For the function F one

can consider its geodesic Radon transform F which is defined on the set of all greatcircles Cq1,q2 ⊂ S3. One can check that the following formula holds

(6.8) Rf(Cx,y) =1

2π

∫Cx,y

f(g)dω(g) =1

π

∫Cq1,q2

F (q) dq = 2F (Cq1,q2),

where relations between circles Cx,y and Cq1,q2 where described in Proposition 6.4.Since varieties of great circles on S3 and on SO(3) can be parametrized by points


(x, y) ∈ S2 × S2 both transforms Rf and F can be considered as functions onS2 × S2.

To describe connection between different transforms and functions it is usefulto introduce the angle density function

(AF )(x, y; ρ) =1

2π

∫c(y;ρ)

F (x, y′)dω1(y′),

where F is a function on S3 and where c(y; ρ) is a small circle of radius ρ centeredat y. Note, that (AF )(x, y; ρ) was introduced in [7] and [8].

The following properties hold

(AF )(x, y; 0) = F (x, y),(6.9)

(AF )(x, y;π) = F (x,−y).According to its definition the quantity (AF )(x, y;π) is the mean value of the spher-ical pole probability density function over any small circle centered at y. Thus, itis the probability density that the crystallographic direction x statistically enclosesthe angle ρ, 0 ≤ ρ ≤ π, with the specimen direction y given the orientation prob-ability density function F . Its central role for the inverse Radon transform wasrecognized in [23] [20].

Our objective is to present two other inversion formulas.

Lemma 6.6. Let F be an even continuous function on S3. Then the geodesicRadon transform F can be inverted by the following formula

(6.10) F (q) =1

2π

[(F )ˇπ

2(q) + 2

∫ π

0

(d

d cos θ(F )ˇθ

2

(q)

)cos θ

2dθ

], q ∈ S3.

Proof. We start with t = v2 to obtain

F (q) =1

2π

[d

du2

∫ u2

0

(F )ˇcos−1(

√t)(q)

1√u2 − t

dt

]∣∣∣∣∣u=1

,

and s = u2

F (q) =1

2π

[d

ds

∫ s

0

(F )ˇcos−1(

√t)(q)

1√s− t

dt

]∣∣∣∣s=1

,

to shift the singularity inside the integral we set γ = s− t which leads to

F (q) =1

2π

[d

ds

∫ s

0

(F )ˇcos−1(

√t)(q)

1√γdγ

]∣∣∣∣s=1

,

now we take the derivative

F (q) =1

2π

[((F )ˇcos−1(0)(q)

1√s+

∫ s

0

d

ds(F )ˇcos−1(

√s−γ)(q)

1√γdγ

]∣∣∣∣s=1

.

Usingd

ds(F )ˇcos−1(

√s−γ)(q) = −

d

dγ(F )ˇcos−1(

√s−γ)(q)

and incorrporate s = 1 we get

F (q) =1

2π

[(F )ˇcos−1(0)(q)−

∫ 1

0

d

dγ(F )ˇcos−1(

√1−γ)(q)

1√γdγ

].

Substitution2γ = 1− cos θ = 2 sin2 θ

2 ,√1− γ = cos θ

2


gives the formula (6.10). Lemma is proved. �

The formula in the next Theorem coincides with an inversion formula whichwas reported by S. Matthies in [20] without any proof. The practical importanceof this formula is that AF is easily experimentally accessible and might yield animproved inversion algorithm.

Theorem 6.7. Suppose that f is a continuous function on SO(3) and functionF on S3 is defined according to ( 6.7). Then the following reconstruction formulaholds

f(g) =1

4π

∫S2

F (x,−gx)dω2(x)+

(6.11)1

2π

∫ π

0

∫S2

d

d cos θ(AF )(x, gx; θ)dω2(x) cos

θ2dθ, g ∈ SO(3),

where ω2 is the usual two–dimensional spherical Riemann measure.

Proof. According to Lemma 6.6 we need to show that∫S2

F (x,−qxq)dω2(x) = 2(F )ˇπ2(q),(6.12) ∫

S2

(AF )(x, qxq; θ)dω2(x) = 2(F )ˇθ2

(q),(6.13)

are fulfilled. Because (6.12) is a special case of (6.13) it is enough to verify the lastequation. For g = τ (q) we have∫

S2

(AF )(x, qxq, θ)dω2(x) = 2

∫{d(g,ξ)= θ

2}F (ξ)dμ(ξ) = 2(F )ˇθ

2

(q),

where dμ is the average over the set of ξ at distance θ2 from g = τ (q). Since

τ (q)x = qxq = gq, g = τ (q), we obtain the second formula. Theorem is proved. �

7. Problem 1: Inversion of crystallographic X-ray transform

Unfortunately, neither the Radon transform Rf over SO(3) nor the Radon

transform f over S3 allows us to solve the crystallographic problem. The point isthat since

Yik(−x) = (−1)kYi

k(x),

one has for Φ(x, y) = Rf(x, y):

Pf(x, y) =1

2(Rf(x, y) +Rf(−x, y)) = 1

2(Φ(x, y) + Φ(−x, y))

=1

2

⎛⎝ ∞∑k=0

2k+1∑i,j=1

Φ(k)ijYik(x)Y

jk(y) +

∞∑k=0

2k+1∑i,j=1

Φ(k)ijYik(−x)Y

jk(y)

⎞⎠=

1

2

⎛⎝ ∞∑k=0

2k+1∑i,j=1

Φ(k)ijYik(x)Y

jk(y) +

∞∑k=0

(−1)k2k+1∑i,j=1

Φ(k)ijYik(x)Y

jk(y)

⎞⎠=

∞∑l=0

4l+1∑i,j=1

Φ(2l)ijYi2l(x)Y

j2l(y).


In other words we loose half of the data needed for the reconstruction, because the

experiment (which is measuring PDF Pf) only gives the even coefficients Φ(2l)ij .

Since the odd Fourier coefficients Φ(2l+1)ij of the function Φ(x, y) = Rf(x, y)disappear one cannot reconstruct the function f(g), g ∈ SO(3), from Pf(x, y).Note that we have two additional conditions stemming from the fact that f is aprobability distribution function:

(1) f(g) ≥ 0,(2)

∫SO(3)

f(g)dg = 1.

The second condition is just a normalization, the first condition is less trivial.We obviously can reconstruct the even part fe(g) from the even coefficients

Φ(2l)ij . In our future work we are planning to utilize properties (1) and (2) toobtain some information about the odd component of f .

8. Problem 4: Exact reconstruction of a bandlimited function f onSO(3) from a finite number of samples of Rf

It is clear that in practice one has to face situations described in the Problems3 and 4. Concerning the Problem 3 we refer to [9] where an approximate inversewas found using the language of Gabor frames. A solution to the Problem 4 willbe described in the present section.

Let B((x, y), r) be a metric ball on S2×S2 whose center is (x, y) and radius isr. As it is explained in Appendix there exists a natural number NS2×S2 , such thatfor any sufficiently small ρ > 0 there exists a set of points {(xν , yν)} ⊂ S2 × S2

such that:

(1) the balls B((xν , yν), ρ/4) are disjoint,(2) the balls B((xν , yν), ρ/2) form a cover of S2 × S2,(3) the multiplicity of the cover by balls B((xν , yν), ρ) is not greater than

NS2×S2 .

Any set of points, which has properties (1)-(3) will be called a metric ρ-lattice.For an ω > 0 let us consider the space Eω(SO(3)) of ω-bandlimited functions

on SO(3) i.e. the span of all Wigner functions T kij with k(k + 1) ≤ ω.

In what follows Eω(S2 × S2) will denote the span in the space L2(S2 × S2) of

all Y ik(ξ)Y

jk(η) with k(k + 1) ≤ ω .

The goal of this section is to prove the following discrete reconstruction formula(8.2) for functions f in Eω(SO(3)), which uses only a finite number of samples ofRf .

Theorem 8.1. (Discrete Inversion Formula) There exists a C > 0 such thatfor any ω > 0, if

ρ = C(ω + 1)−1/2,

then for any ρ-lattice {(xν , yν)}mων=1 of S2 × S2 , there exist positive weights

μν � ω−2,

such that for every function f in Eω(SO(3)) the Fourier coefficients cki,j (Rf) of itsRadon transform, i.e.

Rf(x, y) =∑i,j,k

cki,j (Rf)× Yik(x)Y

jk(y), k(k + 1) ≤ ω, (x, y) ∈ S2 × S2,


are given by the formulas

(8.1) cki,j (Rf) =

mω∑ν=1

μν × (Rf) (xν , yν)× Y ik(xν)Yj

k(yν).

The function f can be reconstructed by means of the formula

(8.2) f(g) =∑k

2k+1∑i,j

(2k + 1)

4π× cki,j (Rf)× T i,j

k (g), g ∈ SO(3),

in which k runs over all natural numbers such that k(k + 1) ≤ ω.

Proof. As the formulas

ΔSO(3)T ijk = −k(k + 1)T ij

k , ΔS2Yik = −k(k + 1)Yi

k.(8.3)

and

RT ijk (x, y) =

4π

2k + 1Yik(x)Y

jk(y)(8.4)

show the Radon transform of a function f ∈ Eω(SO(3)) is ω-bandlimited on S2×S2

in the sense that its Fourier expansion involves only functions YikY

jk which are

eigenfunctions of ΔS2×S2 with eigenvalue −2k(k + 1) ≥ −2ω. Let Eω(S2 × S2) be

the span of YikY

jk with k(k + 1) ≤ ω. Thus

R : Eω(SO(3))→ Eω(S2 × S2).

Let {(x1, y1), ..., (xm, ym)} be a set of pairs of points in SO(3) andMν = xνSO(2)y−1ν

are corresponding submanifolds of SO(3), ν = 1, ...,m. For a function f ∈Eω(SO(3)) and a vector of samples v = (vν)

m1 where

vν =

∫Mν

f,

one has

Rf(xν , yν) = vν .

We are going to find exact formulas for all Fourier coefficients of Rf ∈ Eω(S2×S2)in terms of a finite set of samples of Rf . According to Theorem 9.3 (see Appendix)

every product YikY

jk, where k(k + 1) ≤ ω belongs to E2ω(S2 × S2).

By Theorem 9.1 (see Appendix) there exists a positive constant C, such that ifρ = C(ω+1)−1/2, then for any ρ-lattice {(x1, y1), ..., (xmω

, ymω, )} in S2×S2 there

exist a set of positive weights μν � (2ω)−2 such that

cki,j (Rf) =

∫S2×S2

(Rf) (x, y)× Y ik(x)Y

jk(y)dxdy =

(8.5)N∑

ν=1

μν × (Rf) (xν , yν)× Y ik(xν)Yj

k(yν).

Thus,

(Rf) (x, y) =∑ν

cki,j (Rf)× Y ik(x)Y

jk(y),


where

(8.6) cki,j (Rf) =N∑

ν=1

μν × (Rf) (xν , yν)× Y ik(yν)Y

jk(xν).

Now the reconstruction formula of Theorem 5.4 gives our result (8.2).�

9. Appendix

We explain Theorems 9.1 and 9.2, which played the key role in the proof ofTheorem 8.1.

9.1. Positive cubature formulas on compact manifolds. We consider acompact connected Riemannian manifold M. Let B(ξ, r) be a metric ball on Mwhose center is ξ and radius is r.

It was shown in [26], [27], that if M is compact then there exists a naturalnumber NM, such that for any sufficiently small ρ > 0 there exists a set of points{ξk} such that: (1) the balls B(ξk, ρ/4) are disjoint; (2) the balls B(ξk, ρ/2) forma cover of M; (3) the multiplicity of the cover by balls B(ξk, ρ) is not greater thanNM.

Any set of points Mρ = {ξk} which has properties (1)-(3) will be called a metricρ-lattice.

Let L be an elliptic second order differential operator on M, which is self-adjoint and positive semi-definite in the space L2(M) constructed with respect toRiemannian measure. Such operator has a discrete spectrum 0 < λ1 ≤ λ2 ≤ ....which goes to infinity and does not have accumulation points. Let {uj} be anorthonormal system of eigenvectors of L, which is complete in L2(M).

For a given ω > 0 the notationEω(L) will be used for the span of all eigenvectorsuj that correspond to eigenvalues not greater than ω.

Now we are going to prove existence of cubature formulas which are exact onEω(L), and have positive coefficients of the ”right” size.

The following exact cubature formula was established in [12], [28].

Theorem 9.1. There exists a positive constant C, such that if

(9.1) ρ = C(ω + 1)−1/2,

then for any ρ-lattice Mρ = {ξk}, there exist strictly positive coefficients μξk >0, ξk ∈Mρ, for which the following equality holds for all functions in Eω(L):

(9.2)

∫M

fdx =∑

ξk∈Mρ

μξkf(ξk).

Moreover, there exists constants c1, c2, such that the following inequalities hold:

c1ρn ≤ μξk ≤ c2ρ

n, n = dim M.

It is worth to noting that this result is essentially optimal in the sense that (9.1)and Weyl’s asymptotic formula Nω(L) � CMωn/2, for the number of eigenvaluesof L imply that cardinality of Mρ has the same order as dimension of the spaceEω(L).


9.2. On the product of eigenfunctions of the Casimir operator L oncompact homogeneous manifolds. A homogeneous compact manifold M is aC∞-compact manifold on which a compact Lie group G acts transitively. In thiscase M is necessarily of the form G/H, where H is a closed subgroup of G. Thenotation L2(M), is used for the usual Hilbert spaces L2(M) = L2(M, dξ), wheredξ is an invariant measure.

If g is the Lie algebra of a compact Lie group G then ([13], Ch. II,) it is adirect sum g = a + [g,g], where a is the center of g, and [g,g] is a semi-simplealgebra. LetQ be a positive-definite quadratic form on g which, on [g,g], is oppositeto the Killing form. Let X1, ..., Xd be a basis of g, which is orthonormal withrespect to Q. By using differential of the quasi-regular representation of G in thespace L2(M) one can identify every Xj , j = 1, ..., d, with a first-order differentialoperator Dj , j = 1, ..., d, in the space L2(M). Since the form Q is Ad(G)-invariant,the operator L = −D2

1 −D22 − ... −D2

d, d = dim G, commutes with all operatorsDj , j = 1, ..., d.

This elliptic second order differential operator L is usually called the Laplaceoperator. In the case of a compact semi-simple Lie group, or a compact symmetricspace of rank one, the operator L is proportional to the Laplace-Beltrami operatorof an invariant metric on M.

The following theorem was proved in [12], [28].

Theorem 9.2. If M = G/H is a compact homogeneous manifold and L isdefined as in ( 9.2), then for any f and g belonging to Eω(L), their product fgbelongs to E4dω(L), where d is the dimension of the group G.

In the case when M is the rank one compact symmetric space one can show abetter results.

Theorem 9.3. If M = G/H is a compact symmetric space of rank one thenfor any f and g belonging to Eω(L), their product fg belongs to E2ω(L).

Acknowledgment

The authors thank the anonymous referee for encouraging them to improve theoriginal manuscript, and Meyer Pesenson for helping them to address some of thereferee’s concerns.

References

[1] Asgeirsson, L., Uber eine Mittelwerteigenschaft von Losungen homogener linearer partiellerDifferentialgleichungen zweiter Ordnung mit konstanten Koeffizienten, Annals of Mathemat-ics 1937; 113, 312-346.

[2] H. Berens, P. L. Butzer, and S. Pawelke, Limitierungsverfahren von Reihen mehrdimension-aler Kugelfunktionen und deren Saturationsverhalten, Publ. Res. Inst. Math. Sci. Ser. A 4(1968/1969), 201–268 (German). MR0243266 (39 #4588)

[3] Bernstein, S., Ebert, S., Pesenson, I., Generalized Splines for Radon Transform on CompactLie Groups with Applications to Crystallography, J. Fourier Anal. Appl., doi 10.1007/s00041-012-9241-6,

[4] Swanhild Bernstein and Helmut Schaeben, A one-dimensional Radon transform on SO(3)and its application to texture goniometry, Math. Methods Appl. Sci. 28 (2005), no. 11, 1269–1289, DOI 10.1002/mma.612. MR2150156 (2006m:43008)

[5] Swanhild Bernstein, Ralf Hielscher, and Helmut Schaeben, The generalized totally geodesicRadon transform and its application to texture analysis, Math. Methods Appl. Sci. 32 (2009),no. 4, 379–394, DOI 10.1002/mma.1042. MR2492914 (2010a:53163)


[6] K. G. van den Boogaart, R. Hielscher, J. Prestin, and H. Schaeben, Kernel-based methods forinversion of the Radon transform on SO(3) and their applications to texture analysis, J. Com-put. Appl. Math. 199 (2007), no. 1, 122–140, DOI 10.1016/j.cam.2005.12.003. MR2267537(2008b:44008)

[7] Bunge, H.-J., Mathematische Methoden der Texturanalyse: Akademie Verlag, Berlin (1969),[8] Bunge, H.-J., Morris, P.R., Texture Analysis in Materials Science – Mathematical Methods:

Butterworths (1982),

[9] Paula Cerejeiras, Milton Ferreira, Uwe Kahler, and Gerd Teschke, Inversion of the noisyRadon transform on SO(3) by Gabor frames and sparse recovery principles, Appl. Com-put. Harmon. Anal. 31 (2011), no. 3, 325–345, DOI 10.1016/j.acha.2011.01.005. MR2836027(2012j:44002)

[10] W. Freeden, T. Gervens, and M. Schreiner, Constructive approximation on the sphere, Nu-merical Mathematics and Scientific Computation, The Clarendon Press Oxford UniversityPress, New York, 1998. With applications to geomathematics. MR1694466 (2000e:41001)

[11] Friedel G., Sur les symetries cristallines que peut reveler la diffraction des rayons X, C.R.Acad. Sci. Paris, 157, 1533-1536 (1913),

[12] Daryl Geller and Isaac Z. Pesenson, Band-limited localized Parseval frames and Besovspaces on compact homogeneous manifolds, J. Geom. Anal. 21 (2011), no. 2, 334–371, DOI10.1007/s12220-010-9150-3. MR2772076 (2012c:43013)

[13] Sigurdur Helgason, Differential geometry and symmetric spaces, Pure and Applied Mathe-matics, Vol. XII, Academic Press, New York, 1962. MR0145455 (26 #2986)

[14] Sigurdur Helgason, Geometric analysis on symmetric spaces, 2nd ed., Mathematical Surveysand Monographs, vol. 39, American Mathematical Society, Providence, RI, 2008. MR2463854(2010h:22021)

[15] Sigurdur Helgason, The Radon transform, 2nd ed., Progress in Mathematics, vol. 5,Birkhauser Boston Inc., Boston, MA, 1999. MR1723736 (2000m:44003)

[16] Hielscher, R., Die Radontransformation auf der Drehgruppe – Inversion und Anwendung inder Texturanalyse. PhD thesis, University of Mining and Technology Freiberg, 2007,

[17] R. Hielscher, D. Potts, J. Prestin, H. Schaeben, and M. Schmalz, The Radon transform onSO(3): a Fourier slice theorem and numerical inversion, Inverse Problems 24 (2008), no. 2,

025011, 21, DOI 10.1088/0266-5611/24/2/025011. MR2408548 (2008m:44003)[18] Tomoyuki Kakehi and Chiaki Tsukamoto, Characterization of images of Radon transforms,

Progress in differential geometry, Adv. Stud. Pure Math., vol. 22, Math. Soc. Japan, Tokyo,1993, pp. 101–116. MR1274942 (95b:58148)

[19] Pertti Lounesto, Clifford algebras and spinors, 2nd ed., London Mathematical Society Lec-ture Note Series, vol. 286, Cambridge University Press, Cambridge, 2001. MR1834977(2002d:15031)

[20] Matthies, S., On the reproducibility of the orientation distribution function of texture samplesfrom pole figures (ghost phenomena), Phys. Stat. Sol. (b), 92, K135–K138 (1979),

[21] Matthies, S., Aktuelle Probleme der quantitativen Texturanalyse, ZfK-480. Zentralinstitut furKernforschung Rossendorf bei Dresden, ISSN 0138-2950, August 1982,

[22] L. Meister and H. Schaeben, A concise quaternion geometry of rotations, Math. MethodsAppl. Sci. 28 (2005), no. 1, 101–126, DOI 10.1002/mma.560. MR2105795 (2005g:74029)

[23] J. Muller, C. Esling, and H.-J. Bunge, An inversion formula expressing the texture functionin terms of angular distribution functions, J. Physique 42 (1981), no. 2, 161–165. MR603954(82f:82044)

[24] M. A. Naimark, Linear representations of the Lorentz group, Translated by Ann Swinfenand O. J. Marstrand; translation edited by H. K. Farahat. A Pergamon Press Book, TheMacmillan Co., New York, 1964. MR0170977 (30 #1211)

[25] Victor P. Palamodov, Reconstruction from a sampling of circle integrals in SO(3), InverseProblems 26 (2010), no. 9, 095008, 10, DOI 10.1088/0266-5611/26/9/095008. MR2665426(2011j:94113)

[26] Isaac Pesenson,A sampling theorem on homogeneous manifolds, Trans. Amer. Math. Soc. 352(2000), no. 9, 4257–4269, DOI 10.1090/S0002-9947-00-02592-7. MR1707201 (2000m:41012)

[27] Isaac Pesenson, An approach to spectral problems on Riemannian manifolds, Pacific J. Math.215 (2004), no. 1, 183–199, DOI 10.2140/pjm.2004.215.183. MR2060498 (2005d:31012)


[28] Isaac Z. Pesenson and Daryl Geller, Cubature formulas and discrete Fourier transform oncompact manifolds, From Fourier analysis and number theory to radon transforms and geom-etry, Dev. Math., vol. 28, Springer, New York, 2013, pp. 431–453, DOI 10.1007/978-1-4614-4075-8 21. MR2986970

[29] Michael E. Taylor, Noncommutative harmonic analysis, Mathematical Surveys and Mono-graphs, vol. 22, American Mathematical Society, Providence, RI, 1986. MR852988(88a:22021)

[30] N. Ja. Vilenkin and A. U. Klimyk, Representation of Lie groups and special functions. Vol.2, Mathematics and its Applications (Soviet Series), vol. 74, Kluwer Academic PublishersGroup, Dordrecht, 1993. Class I representations, special functions, and integral transforms;Translated from the Russian by V. A. Groza and A. A. Groza. MR1220225 (94m:22001)

[31] N. Ja. Vilenkin, Special functions and the theory of group representations, Translated fromthe Russian by V. N. Singh. Translations of Mathematical Monographs, Vol. 22, AmericanMathematical Society, Providence, R. I., 1968. MR0229863 (37 #5429)

TU Bergakademie Freiberg, Institute of Applied Analysis, Germany


Department of Mathematics, Temple University, Philadelphia, Pennsylvania 19122



Atomic decompositions of Besov spaces related to symmetriccones

Jens Gerlach Christensen

Abstract. In this paper we extend the atomic decompositions previously ob-tained for Besov spaces related to the forward light cone to general symmetriccones. We do so via wavelet theory adapted to the cone. The wavelet trans-forms sets up an isomorphism between the Besov spaces and certain reproduc-ing kernel function spaces on the group, and sampling of the transformed datawill provide the atomic decompositions and frames for the Besov spaces.

1. Introduction

Besov spaces related to symmetric cones were introduced by Bekolle, Bonami,Garrigos and Ricci in a series of papers [1, 3] and [2]. The purpose was to useFourier-Laplace extensions for the Besov spaces in order to investigate the continu-ity of Bergman projections and boundary values for Bergman spaces on tube typedomains.

Classical homogeneous Besov spaces were introduced via local differences andmodulus of continuity. Through work of Peetre [14], Triebel [15] and Feichtingerand Grochenig [10] these spaces were given a characterization via wavelet theory.The theory of Feichtinger and Grochenig [10,13] further provided atomic decom-positions and frames for the homogeneous Besov spaces.

In the papers [6] and [4] we gave a wavelet characterization and several atomicdecompositions for the Besov spaces related to the special case of the forwardlight cone. In this paper we will show that the machinery carries over to Besovspaces related to any symmetric cone. Our approach contains some representationtheoretic simplifications compared with the work of Feichtinger and Grochenig, andwe in particular exploit smooth representations of Lie groups. The results presentedhere are also interesting in the context of recent results by Fuhr [12] dealing withcoorbits for wavelets with general dilation groups.

2. Wavelets, sampling and atomic decompositions

In this section we use representation theory to set up a correspondance betweena Banach space of distributions and a reproducing kernel Banach space on a group.For details we refer to [4–6] which generalizes work in [10].

2010 Mathematics Subject Classification. Primary 43A15,42B35; Secondary 22D12.Key words and phrases. Coorbit spaces, Gelfand triples, representation theory of Lie groups.


97

98 JENS GERLACH CHRISTENSEN

2.1. Wavelets and coorbit theory. Let S be a Frechet space and let S∗

be the conjugate linear dual equipped with the weak* topology (any reference toweak convergence in S∗ will always refer to the weak* topology). We assume thatS is continuously embedded and weakly dense in S∗. The conjugate dual pairing ofelements φ ∈ S and f ∈ S∗ will be denoted by 〈f, φ〉. Let G be a locally compactgroup with a fixed left Haar measure dg, and assume that (π, S) is a continuousrepresentation of G, i.e. g �→ π(g)φ is continuous for all φ ∈ S. A vector φ ∈ S iscalled cyclic if 〈f, π(g)φ〉 = 0 for all g ∈ G means that f = 0 in S∗. As usual, definethe contragradient representation (π∗, S∗) by

〈π∗(g)f, φ〉 = 〈f, π(g−1)φ〉 for f ∈ S∗.

Then π∗ is a continuous representation of G on S∗. For a fixed vector ψ ∈ S definethe linear map Wψ : S∗ → C(G) by

Wψ(f)(g) = 〈f, π(g)ψ〉 = 〈π∗(g−1)f, ψ〉.The map Wψ is called the voice transform or the wavelet transform. If F is afunction on G then define the left translation of F by an element g ∈ G as

�gF (h) = F (g−1h).

A Banach space of functions Y is called left invariant if F ∈ Y implies that �gF ∈ Yfor all g ∈ G and there is a constant Cg such that ‖�gF‖Y ≤ Cg‖F‖Y for all F ∈ Y .In the following we will always assume that the space Y of functions on G is a leftinvariant Banach space for which convergence implies convergence (locally) in Haarmeasure on G. Examples of such spaces are Lp(G) for 1 ≤ p ≤ ∞ and any spacecontinuously included in an Lp(G).

A non-zero cyclic vector ψ is called an analyzing vector for S if for all f ∈ S∗

the following convolution reproducing formula holds

Wψ(f) ∗Wψ(ψ) = Wψ(f).

Here convolution between two functions F and G on G is defined by

F ∗G(h) =

∫F (g)G(g−1h) dg.

For an analyzing vector ψ define the subspace Yψ of Y by

Yψ = {F ∈ Y |F = F ∗Wψ(ψ)},and let

CoψSY = {f ∈ S∗ |Wψ(f) ∈ Y }equipped with the norm ‖f‖ = ‖Wψ(f)‖Y .

A priori we do not know if the spaces Yψ and CoψSY are trivial, but the follow-ing theorem lists conditions that ensure they are isometrically isomorphic Banachspaces. The main requirements are the existence of a reproducing formula and aduality condition involving Y .

Theorem 2.1. Let π be a representation of a group G on a Frechet space Swith conjugate dual S∗ and let Y be a left invariant Banach function space on G.Assume ψ is an analyzing vector for S and that the mapping

Y × S � (F, φ) �→∫GF (g)〈π∗(g)ψ, φ〉 dg ∈ C

is continuous. Then

ATOMIC DECOMPOSITIONS OF BESOV SPACES RELATED TO SYMMETRIC CONES 99

(1) Yψ is a closed reproducing kernel subspace of Y with reproducing kernelK(g, h) = Wψ(ψ)(g

−1h).

(2) The space CoψSY is a π∗-invariant Banach space.

(3) Wψ : CoψSY → Y intertwines π∗ and left translation.

(4) If left translation is continuous on Y, then π∗ acts continuously on CoψSY.

(5) CoψSY = {π∗(F )ψ | F ∈ Yψ}.(6) Wψ : CoψSY → Yψ is an isometric isomorphism.

Note that (5) states that each member of CoψSY can be written weakly as

f =

∫GWψ(f)(g)π

∗(g)ψ dg.

In the following section we will explain when this reproducing formula can be dis-cretized and how coefficents {ci(f)} can be determined in order to obtain an ex-pression

f =∑i

ci(f)π∗(gi)ψ

for any f ∈ CoψSY .

2.2. Frames and atomic decompositions through sampling on Liegroups. In this section we will decompose the coorbit spaces constructed in theprevious section. For this we need sequence spaces arising from Banach functionspaces on G. The decomposition of coorbit spaces is aided by smooth representa-tions of Lie groups.

We assume that G is a Lie group with Lie algebra denoted g. A vector ψ ∈ Sis called π-weakly differentiable in the direction X ∈ g if there is a vector denotedπ(X)ψ ∈ S such that for all f ∈ S∗

〈f, π(X)ψ〉 = d

dt

∣∣∣t=0〈f, π(etX)ψ〉.

Fix a basis {Xi}dimGi=1 for g, then for a multi-index α we define π(Dα)ψ (when it

makes sense) by

〈f, π(Dα)ψ〉 = 〈f, π(Xα(k))π(Xα(k−1)) · · ·π(Xα(1))ψ〉.

A vector f ∈ S∗ is called π∗-weakly differentiable in the direction X ∈ g ifthere is a vector denoted π∗(X)f ∈ S∗ such that for all φ ∈ S

〈π∗(X)f, ψ〉 = d

dt

∣∣∣t=0〈π∗(etX)f, ψ〉.

For a multi-index α define π∗(Dα)ψ (when it makes sense) by

π∗(Dα)ψ = π∗(Xα(k))π∗(Xα(k−1)) · · ·π∗(Xα(1))ψ

Let U be a relatively compact set in G and let I be a countable set. A sequence{gi}i∈I ⊆ G is called U -dense if {giU} cover G, and V -separated if for some relativelycompact set V ⊆ U the giV are pairwise disjoint. Finally we say that {gi}i∈I ⊆ Gis well-spread if it is U -dense and a finite union of V -separated sequences. Forproperties of such sequences we refer to [10]. A Banach space Y of measurablefunctions is called solid, if |f | ≤ |g|, f measurable and g ∈ Y imply that f ∈ Y .


For a U -relatively separated sequence of points {gi}i∈I in G, and a solid Banachfunction space Y on G, define the space Y #(I) of sequences {λi}i∈I for which

‖{λi}‖Y # :=

∥∥∥∥∥∑i∈I

|λi|1giU

∥∥∥∥∥Y

<∞.

These sequence spaces were introduced in [9] (see also [10]), and we remark thatthey are independent on the choice of U (for a fixed well spread sequence). Fora well-spread set {gi} a U -bounded uniform partition of unity (U -BUPU) is acollection of functions ψi on G such that 0 ≤ ψi ≤ 1giU and

∑i ψi = 1.

In the sequel we will only investigate sequences which are well-spread withrespect to compact neighbourhoods of the type

Uε = {et1X1 · · · etnXn | t1, . . . , tn ∈ [−ε, ε]},where {Xi}ni=1 is the fixed basis for g.

Theorem 2.2. Let Y be a solid and left and right invariant Banach func-tion space for which right translations are continuous. Assume there is a cyclicvector ψ ∈ S satisfying the properties of Theorem 2.1. and that ψ is both π-weakly and π∗-weakly differentiable up to order dim(G). If the mappings Y � F �→F ∗ |Wπ(Dα)ψ(ψ)| ∈ Y are continuous for all |α| ≤ dim(G), then we can choose εand positive constants A1, A2 such that for any Uε-relatively separated set {gi}

A1‖f‖CoψSY ≤ ‖{〈f, π(gi)ψ〉}‖Y # ≤ A2‖f‖CoψSY .

Furthermore, there is an operator T1 such that

f = W−1ψ T−1

1

(∑i

Wψ(f)(gi)ψi ∗Wψ(ψ)

),

where {ψi} is any Uε-BUPU for which supp(ψi) ⊆ giUε

The operator T1 : Yψ �→ Yψ (first introduced in [13]) is defined by

T1F =∑i

F (gi)ψi ∗Wψ(ψ).

Theorem 2.3. Let ψ ∈ S be π∗-weakly differentiable up to order dim(G) sat-isfying the assumptions in Theorem 2.1 and let Y be a solid left and right invari-ant Banach function space for which right translation is continuous. Assume thatY � F �→ F ∗ |Wψ(π

∗(Dα)ψ)| ∈ Y is continuous for |α| ≤ dim(G). We can chooseε small enough that for any Uε-relatively separated set {gi} there is an invertible

operator T2 and functionals λi (defined below) such that for any f ∈ CoψSY

f =∑i

λi(T−12 Wψ(f))π(gi)ψ

with convergence in S∗. The convergence is in CoψSY if Cc(G) are dense in Y .

The operator T2 : Yψ �→ Yψ (also introduced in [13]) is defined by

T2F =∑i

λi(F )�giWψ(ψ),

where λi(F ) =∫F (g)ψi(g) dg.


3. Besov spaces on symmetric cones

3.1. Symmetric cones. For an introduction to symmetric cones we refer tothe book [8]. Let V be a Euclidean vector space over the real numbers of finitedimension n. A subset Ω of V is a cone if λΩ ⊆ Ω for all λ > 0. Assume Ω is openand convex, and define the open dual cone Ω∗ by

Ω∗ = {y ∈ V | (x, y) > 0 for all non-zero x ∈ Ω}.The cone Ω is called symmetric if Ω = Ω∗ and the automorphism group

G(Ω) = {g ∈ GL(V ) | gΩ = Ω}acts transitively on Ω. In this case the set of adjoints of elements in G(Ω) is G(Ω)itself, i.e. G(Ω)∗ = G(Ω). Define the characteristic function of Ω by

ϕ(x) =

∫Ω∗

e−(x,y) dy,

thenϕ(gx) = | det(g)|−1ϕ(x).

Also,

(1) f �→∫Ω

f(x)ϕ(x) dx

defines a G(Ω)-invariant measure on Ω. The connected component G0(Ω) of G(Ω)has Iwasawa decomposition

G0(Ω) = KANwhere K = G0(Ω) ∩O(V ) is compact, A is abelian and N is nilpotent. The uniquefixed point in Ω for the mapping x �→ ∇ logϕ(x) will be denoted e, and we notethat K fixes e. The connected solvable subgroup H = AN of G0(Ω) acts simplytransitively on Ω and the integral (1) thus also defines the left-Haar measure on H.Throughout this paper we will identify functions on H and Ω by right-K-invariantfunctions on G0(Ω). If F is a right-K-invariant function on G and we denote by fthe corresponding function on the cone Ω, then

F �→∫HF (h) dh :=

∫Ω

f(x)ϕ(x) dx

gives an integral formula for the left-Haar measure on H which we will denote bydh or μH.

Lemma 3.1. If F is an μH-integrable right-K-invariant function on G0(Ω), thenthere is a constant C such that∫

F (h) dh = C

∫F ((h∗)−1) dh.

Here h∗ denotes the adjoint element of h with respect to the inner product on V .

Proof. Without loss of generality we will assume that F is compactly sup-ported. Note first that the function h �→ F ((h∗)−1) is right-K-invariant and there-fore can be identified with a function on Ω. Since the measure ϕ(x) dx on Ω isG0(Ω)-invariant, the measure on H is also G0(Ω)-invariant. For g ∈ G0(Ω) we havethat �gF ((h∗)−1) = F (((g∗h)∗)−1), and therefore the mapping

F �→∫HF ((h∗)−1) dh


defines a left-invariant measure on H. By uniqueness of Haar measure we concludethat ∫

HF ((h∗)−1) dh = C

∫HF (h) dh.

�

For f ∈ L1(V ) the Fourier transform is defined by

f(w) =1

(2π)n/2

∫V

f(x)e−i(x,w) dx for w ∈ V,

and it extends to an unitary operator on L2(V ) in the usual way. Denote by S(V )the space of rapidly decreasing smooth functions with topology induced by thesemi-norms

‖f‖k = sup|α|≤k

supx∈V

|∂αf(x)|(1 + |x|)k.

Here α is a multi-index, ∂α denotes usual partial derivatives of functions, and k ≥ 0is an integer. The convolution

f ∗ g(x) =∫V

f(y)g(x− y) dy

of functions f, g ∈ S(V ) satisfies

f ∗ g(w) = f(w)g(w).

The space S ′(V ) of tempered distributions is the linear dual of S(V ). For functions

on V define τxf(y) = f(y− x), f∨(y) = f(−y) and f∗(y) = f(−y). Convolution off ∈ S ′(V ) and φ ∈ S(V ) is defined by

f ∗ φ(x) = f(τxφ∨).

The space of rapidly decreasing smooth functions with Fourier transform vanishingon Ω is denoted SΩ. It is a closed subspace of S(V ) and will be equipped with thesubspace topology.

The space V can be equipped with a Jordan algebra structure such that Ω isidentified with the set of all squares. This gives rise to the notion of a determinantΔ(x). We only need the fact that the determinant is related to the characteristicfunction ϕ by

ϕ(x) = ϕ(e)Δ(x)−n/r,

where r denotes the rank of the cone. If x = ge we have

(2) Δ(x) = Δ(ge) = |Det(g)|r/n.

The following growth estimates hold for functions in SΩ (see Lemma 3.11 in[2]):

Lemma 3.2. If φ ∈ SΩ and k, l are non-negative integers, then there is anN = N(k, l) and a constant CN such that

|φ(w)| ≤ CN‖φ‖NΔ(w)l

(1 + |w|)k .


3.2. Besov spaces on symmetric cones. The cone Ω can be identified asa Riemannian manifold Ω = G0(Ω)/K where K is the compact group fixing e. TheRiemannian metric in this case is defined by

〈u, v〉y = (g−1u, g−1v)

for u, v tangent vectors to Ω at y = ge. Denote the balls of radius δ centered at xby Bδ(x). For δ > 0 and R ≥ 2 the points {xj} are called a (δ, R)-lattice if

(1) {Bδ(xj)} are disjoint, and(2) {BRδ(xj)} cover Ω.We now fix a (δ, R)-lattice {xj} with δ = 1/2 and R = 2. Then there are

functions ψj ∈ SΩ, such that 0 ≤ ψj ≤ 1, supp(ψj) ⊆ B2(xj), ψj is one on B1/2(xj)

and∑

j ψj = 1 on Ω. Using this decomposition of the cone, the Besov space norm

for 1 ≤ p, q <∞ and s ∈ R is defined in [2] by

‖f‖Bp,qs

=

⎛⎝∑j

Δ(xj)−s‖f ∗ ψj‖qp

⎞⎠1/q

.

The Besov space Bp,qs consists of the equivalence classes of tempered distributions

f in S ′Ω {f ∈ S ′(V ) | supp(f) ⊆ Ω}/S ′

∂Ω for which ‖f‖Bp,qs

<∞.

Theorem 3.3. Let ψ be a function in SΩ for which 1B1/2(e) ≤ ψ ≤ 1B2(e).Defining ψh by

ψh(w) = ψ(h−1w),

then

‖f‖Bp,qs (∫

H‖f ∗ ψh‖qpDet(h)−sr/n dh

)1/q

for f ∈ S∗Ω.

Proof. Before we prove the theorem, let us note that

(ψh)g = ψgh.

The cover of Ω corresponds to a cover of H: if hj ∈ H is such that xj = hjethen hjU covers H with U = {h ∈ H | he ∈ B1(e)}.(∫

H‖f ∗ ψh‖qp det(h)−sr/n dh

)1/q

≤

⎛⎝∑j

∫hjU

‖f ∗ ψh‖qp det(h)−sr/n dh

⎞⎠1/q

≤ C

⎛⎝∑j

∫hjU

‖f ∗ ψh‖qp det(hj)−sr/n dh

⎞⎠1/q

.

In the last inequality we have used that, if h ∈ hjU then det(h) ∼ det(hj) uniformlyin j. This follows since for h ∈ hjU , det(h) = det(hj) det(u) for some u ∈ U , andsince U is bounded (compact) there is a γ such that 1/γ ≤ det(u) ≤ γ uniformly in

j. For h ∈ hjU all the functions ψh−1j h have compact support contained in a larger

compact set. Therefore there is an finite set I such that

ψh−1j h = ψh−1

j h

∑i∈I

ψi.


Then, for h ∈ hjU we get

ψh = ψh

∑i∈I

(ψi)hj,

and, since ψh ∈ L1(V ),

‖f ∗ ψh‖p ≤∑i∈I

‖f ∗ (ψi)hj‖p.

So we get(∫H‖f ∗ ψh‖qp det(h)−sr/n dh

)1/q

≤ C∑i∈I

⎛⎝∑j

‖f ∗ (ψi)hj‖qp det(hj)

−sr/n

⎞⎠1/q

≤ C‖f‖Bp,qs

.

In the last inequality we used that each of the collections {(ψi)hj}j partitions the

frequency plane, and the expression can thus be estimated by a Besov norm (seeLemma 3.8 in [2]).

The opposite inequality can be obtained in a similar fashion. �

4. A Wavelet Characterization of Besov spaces on symmetric cones

We will now show that the Besov spaces can be characterized as coorbits for

the group G = H � V , with isomorphism given by the mapping f �→ f from S ′Ω to

S∗Ω defined via 〈f , φ〉 = f(φ). Notice that convolution f ∗ φ can be expressed via

the conjugate linear dual pairing as

f ∗ φ(x) = 〈f , τxφ∗〉.

4.1. Wavelets and coorbits on symmetric cones. The group of interestto us is the semidirect product G = H � V with group composition

(h, x)(h1, x1) = (hh1, hx1 + x).

Here H = AN is the connected solvable subgroup of the connected component ofthe automorphism group on Ω ⊆ V . If dh denotes the left Haar measure on H anddx the Lebesgue measure on V , then the left Haar measure on G is given by dx dh

det(h) .

The quasi regular representation of this group on L2(V ) is given by

π(h, x)f(t) =1√

Det(h)f(h−1(t− x)),

and it is irreducible and square integrable on L2Ω = {f ∈ L2(V ) | supp(f) ⊆ Ω}

(see [7,11]). In frequency domain the representation becomes

π(h, x)f(w) =√

det(h)e−i(x,w)f(h∗w).

By π we will also denote the restriction of π to SΩ.

Remark 4.1. The norm equivalence we have shown in Theorem 3.3 is relatedto the unitary representation

ρ(h, x)f(t) =√Det(h∗)f(h∗(t− x))

and not the representation π. However, Lemma 3.1 allows us to make a change ofvariable h �→ (h∗)−1 in order to relate the norm equivalence to π.


Lemma 4.2. The representation π of G on SΩ is continuous, and if ψ is thefunction from from Theorem 3.3, then the function φ = ψ∗ is a cyclic vector for π.

Proof. The Fourier transform ensures that this is equivalent to showing that πis a continuous representation. The determinant is continuous, so we will investigatethe L∞-normalized representation instead. For f ∈ S(V ) with support in Ω define

fh,x(w) = f(h∗w)e−i(x,w),

for h ∈ H and x ∈ V . Since h∗w ∈ Ω if w ∈ Ω we see that fh,x is a Schwartzfunction supported in Ω, so SΩ is π-invariant.

We now check that fh,x → f in the Schwartz semi-norms as h→ I and x→ 0.Taking one partial derivative we see that

∂fh,x∂wk

(w)− ∂f

∂wk(w)

=∑l

hlk∂f

∂wl(h∗w)e−i(x,w) − iwkf(h

∗w)e−i(x,w) − ∂f

∂wk(w)

= (hkk − 1)∂f

∂wk(h∗w)e−i(x,w) +

∑l �=k

hlk∂f

∂wl(h∗w)e−i(x,w)

− iwkf(h∗w)e−i(x,w) +

∂f

∂wk(h∗w)e−i(x,w) − ∂f

∂wk(w)

=∑

|β|≤|α|cβ(h, x)∂

βf(h∗w)e−i(x,w) + (∂αf(h∗w)e−i(x,w) − ∂αf(w)),

where α = ek and cβ(h, x) → 0 as (h, x) → (I, 0). By repeating the argument weget

∂αfh,x(w)− ∂αf(w)

=∑

|β|≤|α|cβ(h, x)∂

βf(h∗w)e−i(x,w) + (∂αf(h∗w)e−i(x,w) − ∂αf(w)).

where cβ(h, x) → 0 as (h, x) → (I, 0). Using the fact that |w| = |(h∗)−1h∗w| ≤‖(h∗)−1‖|h∗w| we see that (1+|w|)N

(1+|h∗w|)N ≤ CN (h), where CN (h) depends continuously

on h. For |α| ≤ N we thus get

(1 + |w|)N |∂αfh,x(w)− ∂αf(w)|

≤ CN (h)∑

|β|≤|α|cβ(h, x)(1 + |h∗w|)N |∂βf(h∗w)|

+ (1 + |w|)N |∂αf(h∗w)e−i(x,w) − ∂αf(w)|

≤ CN (h)∑

|β|≤|α|cβ(h, x)‖f‖N + (1 + |w|)N |∂αf(h∗w)e−i(x,w) − ∂αf(w)|.

Since cβ(h, x) tend to 0 as (h, x)→ (I, 0), we investigate the remaining term

|∂αf(h∗w)e−i(x,w) − ∂αf(w)| ≤ |∂αf(h∗w)− ∂αf(w)|+ |∂αf(w)(e−i(x,w) − 1)|


First, let γ(t) = w + t(h∗w − w). For |α| = N we get

|∂αf(h∗w)− ∂αf(w)|(1 + |w|2)N

≤∫ 1

0

|∇∂αf(γh,w(t))||γ′h,w(t)|(1 + |w|2)N dt

≤ ‖h∗ − I‖∫ 1

0

|∇∂αf(γh,w(t))|(1 + |γ(t)|2)N+1

(1 + |w|21 + |γ(t)|2

)N+1

dt

≤ C‖h∗ − I‖‖f‖N+1,

where the constant C is uniformly bounded in h. Next let γ(t) = tx, then

(1 + |w|)N |∂αf(w)(e−i(x,w) − 1)| ≤ (1 + |w|)N |∂αf(w)||∫ 1

0

−iwγ′(t)e−it(x,w)) dt|

≤ (1 + |w|)N |∂αf(w)||w||x|≤ ‖f‖N+1|x|.

This shows that the representation π is continuous on SΩ.To show cyclicity, assume that f is in S∗

Ω and 〈f , π(a, x)φ〉 = 0. Notice that

〈f , π(a, x)φ〉 = f ∗ ψ(h−1)∗(x), where f is the tempered distribution in S ′Ω corre-

sponding to f . By the norm equivalence of Theorem 3.3 and Lemma 3.1, we seethat f = 0 in all Besov spaces Bp,q

s and thus also in S ′Ω (see [2] Lemma 3.11 and

3.22 and note that S ′Ω is equipped with the weak∗ topology). This proves that

f = 0 and φ is cyclic. �

For ψ ∈ SΩ define the wavelet transform of f ∈ S∗Ω by

Wψ(f)(h, x) = 〈f, π(h, x)ψ〉.Under certain assumptions on ψ we get a reproducing formula.

Lemma 4.3. If ψ ∈ SΩ is such that ψ has compact support and∫H|ψ(h∗e)|2 dh = 1,

then

Wψ(f) ∗Wψ(ψ) = Wψ(f)

for all f ∈ S∗Ω. Here the convolution is the group convolution on G = H � V .

Proof. For φ we denote by φh the function defined by

φh(w) = φ(h∗w).

Then φh1 ∗ φh

2 = | det(h)|(φ1 ∗ φ2)h, and

Wψ(f) ∗Wψ(ψ)(h1, x1) =1√

| det(h1)|

∫H〈f, τx1

ψh1 ∗ (ψ∗)h ∗ ψh〉 dh

| det(h)|2

=1√

| det(h1)|

∫H〈f, τx1

ψh1 ∗ (ψ∗ ∗ ψ)h〉 dh

| det(h)| .

The function inside the last integral is continuous, so it is enough to show thatfor φ ∈ SΩ the net

gC(x) =

∫C

φ ∗ (ψ∗ ∗ ψ)h dh

| det(h)| ,


converges to φ in SΩ for growing compact sets C → H. By the assumption on ψ we

get that gC → φ pointwise. Thus we only need to show that gC converges, whichwill happen if the integral∫

Hsupx(1 + |x|2)N |∂αφ ∗ (ψ∗ ∗ ψ)h(x)| dh

| det(h)| <∞

is finite for all N and α. Since both ∂αφ and ψ∗ ∗ ψ are in SΩ, we need only focuson showing that ∫

Hsupx(1 + |x|2)N |φ1 ∗ φh

2 (x)|dh

| det(h)| <∞

for all N and any φ1, φ2 ∈ SΩ. We can further assume that φ2 has compact support.

Note that the Parseval identity, integration by parts and the fact that φ1, φ2 vanishon the boundary of Ω, give

|φ1 ∗ φh2 (x)| =

∣∣∣∣∫Ω

φ1(w)φ2(h∗w)ei(x,w) dw

∣∣∣∣≤ 1

|xα|

∫Ω

∑|β|≤|α|

|pβ(h∗)||∂βφ1(w)∂α−βφ2(h

∗w)| dw.

Here pβ(h∗) is a polynomium in the entries of h∗. Choosing |α| large enough takes

care of the terms (1 + |x|2)N for large |x| (and for small |x| we use α = 0), so

supx|φ1 ∗ φh

2 (x)|(1 + |x|2)N ≤∑

|β|≤|α||pβ(h∗)|

∫Ω

|∂βφ1(w)∂α−βφ2(h

∗w)| dw.

Each partial derivative is again in S(V ) and with support in Ω, so we investigate

terms of the general form∫|φ1(w)φ2(h

∗w)| dw. Denote by hw the unique elementin H for which w = hwe, then∫

H

∫Ω

p(‖h‖)|φ1(w)||φ2(h∗w)| dw dh =

∫H

∫Ω

p(‖(h∗w)

−1h‖)|φ1(w)||φ2(h∗e)| dw dh.

Now φ2 is assumed to have compact support and thus the integral over H is finite,so we get

≤ C

∫Ω

p(1/‖hw‖)|φ1(w)| dw.

This will be finite, because the estimate |φ1(w)| ≤ C Δ(w)l

(1+|w|2)k . When ‖hw‖ ∼ |w|is close to zero we use l sufficiently large and for large ‖hw‖ ∼ |w| the integral isfinite for k large enough. This finishes the proof. �

For 1 ≤ p, q < ∞ and s ∈ R define the mixed norm Banach space Lp,qs (G) on

the group G to be the measurable functions for which

‖F‖Lp,qs

:=

(∫H

(∫V

|F (h, x)|p dx)q/p

|Det(h)|s dh)1/q

<∞.

Lemma 4.4. For ψ, φ ∈ SΩ the wavelet transform Wψ(φ) is in Lp,qs (G) for

1 ≤ p, q <∞ and any real s.


Proof. Since Wψ(φ)(h, x) = φ ∗ψ∗(h−1)∗(x), this follows from the norm equiv-

alence of Theorem 3.3 coupled with Lemma 3.1, as well as the fact that a functionin SΩ is in any Besov space (see Proposition 3.9 in [2]). �

This verifies that the requirements of Theorem 2.1 are satisfied. It also showsthat the representation involved has integrable matrix coefficients, which is thebasis for the investigation in [10]. We thus complete our wavelet characterization

of the Besov spaces by the following result. Remember that f ∈ S∗Ω corresponds to

f ∈ S ′Ω via 〈f , φ〉 = f(φ).

Theorem 4.5. Given 1 ≤ p, q < ∞ and s ∈ R let s′ = sr/n − q/2. If φ isthe cyclic vector from Lemma 4.2 normalized to also satisfy Lemma 4.3, then the

mapping f �→ f (restricted to Bp,qs ) is a Banach space isomorphism from the Besov

space Bp,qs to the coorbit CoφSΩ

Lp,qs′ (G) for the representation π.

Proof. We will use Theorem 3.3 to determine s′. Let φ = cψ∗, and noticethat

〈f , π(h, x)φ〉 = c√| det(h)|f ∗ ψ(h∗)−1(x).

Then by Lemma 3.1 and Theorem 3.3 we get that

‖Wφ(f)‖Lp,q

s′= C

(∫H‖f ∗ ψh‖qpDet(h)−q/2−s′ dh

)1/q

,

which is equivalent to ‖f‖Bp,qs

if −q/2− s′ = −sr/n. �4.2. Atomic decompositions. In order to obtain atomic decompositions and

frames from Theorems 2.2 and 2.3, we need to show that SΩ are smooth vectors forπ. A vector ψ ∈ SΩ is called smooth if g �→ π(g)ψ is smooth G → SΩ. For smoothvectors ψ define a representation of g by

π∞(X)ψ =d

dt

∣∣∣t=0

π(exp(tX))ψ.

This also induces a representation of the universal enveloping algebra U(g) whichwe also denote π∞.

Theorem 4.6. The space SΩ is the space of smooth vectors for the representa-tion (π,SΩ), and (π∞,SΩ) is a representation of both g and U(g).

Proof. Again, the determinant does not change the smoothness of vectorsso we work with the L∞-normalized representation. Let γ(t) = (h(t), x(t)) bea smooth curve in G with γ(0) = (I, 0) and γ′(t) = (H,X), then the pointwisederivative (on the frequency side) of functions fh,x(w) = f(h∗w)e−i(x,w) is

d

dt

∣∣∣t=0

f(h(t)∗w)e−i(x(t),w) = (H∗w) · ∇f(w)− iX · wf(w).

This is another Schwartz function and we will show it is also the limit of thederivative in SΩ.

1

t(fh(t),x(t)(w)− f(w))− (H∗w) · ∇f(w) + iX · wf(w),

=1

t

∫ t

0

((h′(s)∗w) · ∇f(h(s)∗w)e−i(x(s),w) − (H∗w) · ∇f(w) ds

+1

t

∫ T

0

iX · wf(w)− ix′(t) · wf(h(s)∗w) ds


From the proof of Lemma 4.2 it is evident that each term inside the integral ap-proaches 0 in the Schwartz topology, and the proof is complete. �

This result proves that any vector in SΩ is both π-weakly and π∗-weakly dif-ferentiable of all orders, and this ensures that Wπ(Dα)ψ(ψ) and Wψ(π(D

α)ψ) are

in L1s(G) for all s by Lemma 4.4. Thus the continuities required by Theorems 2.2

and 2.3 are satisfied and we conclude with the promised atomic decompositions.

Corollary 4.7. Let s ∈ R and 1 ≤ p, q < ∞ be given. There exists an indexset I, and a well-spread sequence of points {(hi, xi)}I ⊆ G, such that the collection

π(hi, xi)ψ forms both a Banach frame and an atomic decomposition for Bp,qs with

sequence space (Lp,qs′ (G))# when s′ = sr/n− q/2.

References

[1] David Bekolle, Aline Bonami, and Gustavo Garrigos, Littlewood-Paley decompositions relatedto symmetric cones, IMHOTEP J. Afr. Math. Pures Appl. 3 (2000), no. 1, 11–41. MR1905056(2003d:42027)

[2] D. Bekolle, A. Bonami, G. Garrigos, and F. Ricci, Littlewood-Paley decompositions relatedto symmetric cones and Bergman projections in tube domains, Proc. London Math. Soc. (3)89 (2004), no. 2, 317–360, DOI 10.1112/S0024611504014765. MR2078706 (2005e:42056)

[3] David Bekolle, Aline Bonami, Marco M. Peloso, and Fulvio Ricci, Boundedness of Bergmanprojections on tube domains over light cones, Math. Z. 237 (2001), no. 1, 31–59, DOI10.1007/PL00004861. MR1836772 (2002d:32004)

[4] Jens Gerlach Christensen, Sampling in reproducing kernel Banach spaces on Lie groups, J.Approx. Theory 164 (2012), no. 1, 179–203, DOI 10.1016/j.jat.2011.10.002. MR2855776(2012k:42061)

[5] Jens Gerlach Christensen and Gestur Olafsson, Examples of coorbit spaces for dual pairs,Acta Appl. Math. 107 (2009), no. 1-3, 25–48, DOI 10.1007/s10440-008-9390-4. MR2520008(2010h:43002)

[6] Jens Gerlach Christensen and Gestur Olafsson, Coorbit spaces for dual pairs, Appl. Com-put. Harmon. Anal. 31 (2011), no. 2, 303–324, DOI 10.1016/j.acha.2011.01.004. MR2806486

(2012e:22005)

[7] R. Fabec and G. Olafsson, The continuous wavelet transform and symmetric spaces,Acta Appl. Math. 77 (2003), no. 1, 41–69, DOI 10.1023/A:1023687917021. MR1979396(2004k:42053)

[8] Jacques Faraut and Adam Koranyi, Analysis on symmetric cones, Oxford MathematicalMonographs, The Clarendon Press Oxford University Press, New York, 1994. Oxford SciencePublications. MR1446489 (98g:17031)

[9] Hans G. Feichtinger and Peter Grobner, Banach spaces of distributions defined by decom-position methods. I, Math. Nachr. 123 (1985), 97–120, DOI 10.1002/mana.19851230110.

MR809337 (87b:46020)[10] Hans G. Feichtinger and K. H. Grochenig, Banach spaces related to integrable group repre-

sentations and their atomic decompositions. I, J. Funct. Anal. 86 (1989), no. 2, 307–340,DOI 10.1016/0022-1236(89)90055-4. MR1021139 (91g:43011)

[11] Hartmut Fuhr, Wavelet frames and admissibility in higher dimensions, J. Math. Phys. 37(1996), no. 12, 6353–6366, DOI 10.1063/1.531752. MR1419174 (97h:42014)

[12] H. Fuhr. Coorbit spaces and wavelet coefficient decay over general dilation groups. Availableunder http://arxiv.org/abs/1208.2196v3

[13] Karlheinz Grochenig, Describing functions: atomic decompositions versus frames, Monatsh.Math. 112 (1991), no. 1, 1–42, DOI 10.1007/BF01321715. MR1122103 (92m:42035)

[14] Jaak Peetre, New thoughts on Besov spaces, Mathematics Department, Duke University,Durham, N.C., 1976. Duke University Mathematics Series, No. 1. MR0461123 (57 #1108)

[15] Hans Triebel, Characterizations of Besov-Hardy-Sobolev spaces: a unified approach, J. Ap-prox. Theory 52 (1988), no. 2, 162–203, DOI 10.1016/0021-9045(88)90055-X. MR929302(89i:46040)


Department of Mathematics, Tufts University



A double fibration transform for complex projective space

Michael Eastwood

To Sigurdur Helgason on the occasion of his eighty-fifth birthday.

Abstract. We develop some theory of double fibration transforms where thecycle space is a smooth manifold and apply it to complex projective space.

1. Introduction

The classical Penrose transform is concerned with (anti)-self-dual 4-dimensionalRiemannian manifolds. If M is such a manifold then, as shown in [1], there is acanonically defined 3-dimensional complex manifold Z, known as the twistor spaceof M , that fibres over M

(1.1) τ : Z →M

in the sense that τ is a submersion with holomorphic fibres intrinsically isomorphicto CP1. In fact, this construction depends only on the conformal structure on Mand the Penrose transform then identifies the Dolbeault cohomology Hr(Z,O(V ))for the various natural holomorphic vector bundles V on Z with the cohomologyof certain conformally invariant elliptic complexes of linear differential operatorson M . Some typical examples are presented in [12,16].

The two main examples of this construction are for M = S4, the flat model of4-dimensional conformal geometry, and for M = CP2 with its Fubini-Study metric.In both cases, the twistor space is a well-known complex manifold. For S4 it is CP3

and for CP2 it is the flag manifold

F1,2(C3) ≡ {(L, P ) | L ⊂ P ⊂ C3 with dimC L = 1, dimC P = 2}.For CP2 the fibration is

(1.2) F1,2(C3) � (L, P )τ�−→ L⊥ ∩ P ∈ CP2,

where the orthogonal complement L⊥ of L is taken with respect to a fixed Hermitianinner product on C3, namely the same inner product that induces the Fubini-Study metric on CP2 as a homogeneous space SU(3)/S(U(1)×U(2)). The Penrosetransform in this setting is carried out in detail in [7,9].

There are several options for generalising this twistor geometry of CP2 to higherdimensions. Perhaps the most obvious is to take as twistor space the flag manifoldF1,2(Cn+1) and define τ : F1,2(Cn+1) → CPn by (L, P ) �→ L⊥ ∩ P . This is the

2010 Mathematics Subject Classification. Primary 32L25; Secondary 53C28.The author is supported by the Australian Research Council.


111

112 MICHAEL EASTWOOD

option adopted in [10]. Perhaps a more balanced option is to take as twistor spacethe flag manifold

Z = F1,n(Cn+1) ≡ {(L,H) | L ⊂ H ⊂ Cn+1 with dimC L = 1, dimC H = n}and consider the double fibration

(1.3)

Z

X

CPn

��

��

η τ

where X ⊂ F1,n(Cn+1)× CPn is the incidence variety given by

(1.4) X = {(L,H, �) | � ⊆ L⊥ ∩H}and the fibrations η and τ are the forgetful mappings ,

F1,n(Cn+1) � (L,H)η←−� (L,H, �)

τ�−→ � ∈ CPn.

Of course, when n = 2 the dimensions force η to be an isomorphism and this doublefibration (1.3) reverts to the single fibration (1.2).

The aim of this article is to explain a transform on Dolbeault cohomology fordouble fibrations of this type and then execute the transform in this particular case.Then, since the Bott-Borel-Weil Theorem [4] computes the Dolbeault cohomologyof Z = F1,n(Cn+1) with coefficients in any homogeneous vector bundle, we maydraw conclusions concerning the cohomology of various elliptic complexes on CPn.

This work was outlined at the meeting ‘Geometric Analysis on Euclidean andHomogeneous Spaces’ held at Tufts University in January 2012. The author isgrateful to the organisers, Jens Christensen, Fulton Gonzalez, and Todd Quinto,for their invitation to speak and hospitality at that meeting and also to Joseph Wolffor many crucial conversations concerning this work.

2. The general transform

There is a better established double fibration transform defined for

Z

X

M

��

��

μ ν

in which all manifolds are complex and both μ and ν are holomorphic. Classicaltwistor theory, for example, is concerned with the holomorphic correspondence

CP3

F1,2(C4)

Gr2(C4).

��

��

μ ν

The Penrose transform in this setting is explained in [11] and generalised to arbi-trary holomorphic correspondences between complex flag manifolds in [2]. Anothervast generalisation is concerned with the holomorphic correspondences arising fromthe cycle spaces of general flag domains as in [15].

On the face of it, the double fibration (1.3) is of a different nature since CPn

is only to be considered as a smooth manifold. In fact, a link will emerge withthe complex correspondences and this will ease some of the computations involved.For the moment, however, let us develop some general machinery applicable to thissmooth setting. This machinery is a generalisation of the Penrose transform for asingle fibration (1.1), which goes as follows. The only requirements on (1.1) are

A DOUBLE FIBRATION TRANSFORM FOR COMPLEX PROJECTIVE SPACE 113

that τ should be a smooth submersion from a complex manifold Z to a smoothmanifold M and that the fibres of τ should be compact complex submanifolds of Z.

Let us denote by Λ0,qZ the bundle of (0, q)-forms on Z and by ∂Z : Λ0,q

Z → Λ0,q+1Z

the ∂-operator on Z so that

Hr(Z,O) ≡ Hr(Γ(Z,Λ0,•Z ), ∂Z)

is the Dolbeault cohomology of Z. The 1-forms along the fibres of τ , defined bythe short exact sequence

0→ τ∗Λ1M → Λ1

Z → Λ1τ → 0,

are decomposed as Λ1τ = Λ0,1

τ ⊕ Λ1,0τ by the complex structure on these fibres and

the fact that this complex structure is acquired from that on Z implies that thereis a commutative diagram

(2.1)

0 → Λ0,0 ∂τ−−→ Λ0,1τ

∂τ−−→ Λ0,2τ

∂τ−−→ · · ·‖ ↑↑ ↑↑

0 → Λ0,0 ∂Z−−→ Λ0,1Z

∂Z−−→ Λ0,2Z

∂Z−−→ · · ·

where the top row is the ∂-complex along the fibres of τ . Though the notation mayseem bizarre at first, let us define the bundle Λ1,0

μ on Z by the short exact sequence

(2.2) 0→ Λ1,0μ → Λ0,1

Z → Λ0,1τ → 0.

Regarded as a filtration of Λ0,1Z , this short exact sequence induces filtrations on Λ0,q

Z

for all q and (2.1) implies that ∂Z is compatible with this filtration. An immediateconsequence is that the bundle Λ1,0

μ acquires a holomorphic structure along thefibres of τ . To see this by hand, one notes that the composition

Λ1,0μ → Λ0,1

Z∂Z−−→ Λ0,2

Z → Λ0,2τ

vanishes by dint of the definition (2.2) of Λ1,0μ and the commutative diagram (2.1)

whence the short exact sequence

0→ Λ2,0μ → [ker : Λ0,2

Z → Λ0,2τ ]→ Λ0,1

τ ⊗ Λ1,0μ → 0

induced by (2.2) implies that ∂Z |Λ1,0μ

induces an operator

∂τ : Λ1,0μ → Λ0,1

τ ⊗ Λ1,0μ ,

as required. To see this (and much more) by machinery, one employs the spectral

sequence of the filtered complex Λ0,•Z , arriving at the E0-level

Λ0,0

Λ0,1τ

Λ0,2τ

Λ0,3τ

Λ1,0μ

Λ0,1τ ⊗ Λ1,0

μ

Λ0,2τ ⊗ Λ1,0

μ

Λ2,0μ

Λ0,1τ ⊗ Λ2,0

μ

Λ3,0μ

�

�

�

�

�

�

�

�

�

�∂τ

∂τ

∂τ

∂τ

∂τ

∂τ

p

q


and, in particular, the differential Λ1,0μ

∂τ−−→ Λ0,1τ ⊗ Λ1,0

μ . This spectral sequence for

Γ(Z,Λ0,•Z ), at the E1-level, reads

Γ(M, τ∗Λ0,0μ )

Γ(M, τ1∗Λ0,0μ )

Γ(M, τ2∗Λ0,0μ )

Γ(M, τ3∗Λ0,0μ )

Γ(M, τ∗Λ1,0μ )

Γ(M, τ1∗Λ1,0μ )

Γ(M, τ2∗Λ1,0μ )

Γ(M, τ∗Λ2,0μ )

Γ(M, τ1∗Λ2,0μ )

Γ(M, τ∗Λ3,0μ )

�

p

q

where τ q∗Λp,0μ is the qth direct image of the vector bundle Λp,0

μ with respect to itsholomorphic structure in the fibre directions. Note that, with the fibres of τ beingcompact, these direct images generically define smooth vector bundles on M andcertainly this will be the case when the fibration (1.1) is homogeneous. In any case,we have proved the following.

Theorem 2.1. Suppose that τ : Z → M is a submersion of smooth manifoldsand that Z has a complex structure such that the fibres of τ are compact complexsubmanifolds of Z. Then the bundle Λ1,0

μ ≡ ker : Λ0,1Z → Λ0,1

τ acquires a naturalholomorphic structure along the fibres of τ and there is a spectral sequence

Ep,q1 = Γ(M, τ q∗Λ

p,0μ ) =⇒ Hp+q(Z,O).

This theorem only comes to life with examples in which it is possible to computethe direct images τ q∗Λ

p,0μ . There is also a coupled version of the spectral sequence


p,0μ (V )) =⇒ Hp+q(Z,O(V ))

for any holomorphic vector bundle V on Z. The proof is easily modified but theadded scope for interesting examples is significantly increased.

For the moment, however, let us continue with generalities, firstly by extendingTheorem 2.1 to cover double fibrations of the form

(2.3)

Z

X

M

��

��

η τ

(of which (1.2) is typical) where M is smooth and the fibres of τ are identified byη as compact complex submanifolds of the complex manifold Z. To do this, let usdefine a bundle Λ0,1

X on X by means of the short exact sequence

(2.4) 0→ η∗Λ1,0Z → Λ1

X → Λ0,1X → 0,

where Λ1X is the bundle of complex-valued 1-forms on X. Geometrically, this pulls

back the complex structure from Z to an involutive structure [3] onX. In particular,there is an induced complex of differential operators

0→ Λ0,0X

∂X−−→ Λ0,1X

∂X−−→ Λ0,2X

∂X−−→ · · · .Comparing (2.4) with the bundle Λ1

η of 1-forms along the fibres of η defined by theshort exact sequence

0→ η∗Λ1Z → Λ1

X → Λ1η → 0


we see that there is a short exact sequence

0→ η∗Λ0,1Z → Λ0,1

X → Λ1η → 0.

The complex Γ(X,Λ0,•X ) thereby acquires a filtration, the spectral sequence for

which reads at the E0-level

(2.5)

Γ(X, η∗Λ0,0Z )

Γ(X,Λ1η ⊗ η∗Λ0,0

Z )

Γ(X,Λ2η ⊗ η∗Λ0,0

Z )

Γ(X,Λ3η ⊗ η∗Λ0,0

Z )

Γ(X, η∗Λ0,1Z )

Γ(X,Λ1η ⊗ η∗Λ0,1

Z )

Γ(X,Λ2η ⊗ η∗Λ0,1

Z )

Γ(X, η∗Λ0,2Z )

Γ(X,Λ1η ⊗ η∗Λ0,2

Z )

Γ(X, η∗Λ0,3Z )

�

�

�

�

�

�

�

�

�

dη

dη

dη

dη

dη

dη

p

q

where dη : Λqη ⊗ η∗Λ0,p → Λq+1

η ⊗ η∗Λ0,p is the exterior derivative along the fibres

of η coupled with the pullback bundle η∗Λ0,p. Notice that such a coupling

dη : η∗V → Λ1η ⊗ η∗V and hence dη : Λq

η ⊗ η∗V → Λq+1η ⊗ η∗V

is valid for any smooth vector bundle V on Z because the pullback η∗V may bedefined by transition functions that are constant along the fibres, hence annihilatedby dη. When the fibres of η are contractible, this is exactly the setting in whichBuchdahl’s theorem [6] applies and we deduce the following.

Proposition 2.2. Suppose that the fibres of η : X → Z are contractible. Then

0→ Γ(Z, V )→ Γ(X, η∗V )dη−−→ Γ(X,Λ1

η ⊗ η∗V )dη−−→ Γ(X,Λ2

η ⊗ η∗V )dη−−→ · · ·

is exact for any smooth vector bundle V on Z.

In this case, our spectral sequence (2.5) collapses at the E1-level and we haveproved the following.

Proposition 2.3. Suppose that the fibres of η : X → Z are contractible. Then

Hr(Z,O) ∼= Hr(Γ(X,Λ0,•X ), ∂X) for all r = 0, 1, 2, . . . .

In fact, for the double fibration (1.3), the fibres of η are not contractible andin §3 we shall have to revisit the spectral sequence (2.5) to relate the Dolbeault

cohomology Hr(Z,O) with the involutive cohomology Hr(Γ(X,Λ0,•X ), ∂X).

Nevertheless, we may deal with the fibration τ : X →M exactly as in our proofof Theorem 2.1. Specifically, we define a bundle Λ1,0

μ on X by the exact sequence

(2.6) 0→ Λ1,0μ → Λ0,1

X → Λ0,1τ → 0

and employ the spectral sequence of the corresponding filtered complex Λ0,•X to

conclude that the following theorem holds.

Theorem 2.4. Suppose that

(2.3)

Z

X

M

��

��

η τ

is a double fibration of smooth manifolds such that


• Z is a complex manifold,• the fibres of τ are embedded by η as compact complex submanifolds of Z.

Then the bundle Λ1,0μ , defined as the middle cohomology of the complex

(2.7) 0→ η∗Λ1,0Z → Λ1

X → Λ0,1τ → 0,

acquires a natural holomorphic structure along the fibres of τ and there is a spectralsequence

(2.8) Ep,q1 = Γ(M, τ q∗Λ

p,0μ ) =⇒ Hp+q(Γ(X,Λ0,•

X ), ∂X).

Corollary 2.5. If, in addition, the fibres of η are contractible, then


p,0μ ) =⇒ Hp+q(Z,O).

Proof. Immediate from Proposition 2.3. �

In §4 we shall present an example for which the fibres of η are, indeed, contractibleand to which Corollary 2.5 applies.

Before we continue, let us glance ahead to §3 in which the first thing we do is use(2.5) to deal with the topology along the fibres of η for the double fibration (1.3).Another thing we need in order to apply Theorem 2.4 is a computation of the directimages τ q∗Λ

p,0μ as homogeneous bundles on CPn. This computation is best viewed

in the light of a geometric interpretation of Λ1,0μ as follows.

Suppose that M is a totally real submanifold of a complex manifold M suchthat the double fibration (2.3) embeds as

(2.9)

Z

X

M

��

��

η τ ↪→Z

X

M ,

��

��

μ ν

where the ambient double fibration is in the holomorphic category and the fibresof ν coincide with the fibres of τ over M .

Proposition 2.6. Under these circumstances the bundle Λ1,0μ of (1, 0)-forms

along the fibres of μ coincides, when restricted to X ⊂ X, with the bundle alreadydenoted in the same way and defined as the middle cohomology of (2.7).

Proof. If we write

n = dimC Z m = dimR M s = dimC(fibres of τ ),

then dimR X = m+2s and X has real codimension 2(n− s) in Z ×M . This is thesame as the real codimension of X in Z ×M and it follows that the complexifiedconormal bundle C of X in Z × M coincides with the restriction to X of thecomplexified conormal bundle of X in Z ×M. Hence it splits as C = C0,1 ⊕ C1,0 inline with the complex structure on the ambient double fibration. For any doublefibration there is a basic commutative diagram with exact rows and columns, which


in the case of (2.3) looks as follows.

0 0↑ ↑Λ1τ = Λ1

τ

↑ ↑0 → η∗Λ1

Z → Λ1X → Λ1

η → 0↑ ↑ ‖

0 → C → τ∗Λ1M → Λ1

η → 0↑ ↑0 0

But we have just observed that the left hand column has the additional featurethat it splits

Λ1τ = Λ0,1

τ ⊕ Λ1,0τ

↑ ↑ ↑η∗Λ1

Z η∗Λ0,1Z ⊕ η∗Λ1,0

Z

↑ ↑ ↑C = C0,1 ⊕ C1,0

in line with the ambient complex structure. Hence we obtain the diagram

0 0↑ ↑

Λ0,1τ = Λ0,1

τ

↑ ↑0 → η∗Λ0,1

Z → Λ1X/η∗Λ1,0

Z → Λ1η → 0

↑ ↑ ‖0 → C0,1 → τ∗Λ1

M/C1,0 → Λ1η → 0

↑ ↑0 0

and it follows from (2.4) and (2.6) that Λ1,0μ = τ∗Λ1

M/C1,0. On the other hand,

since M ↪→M is totally real, we may identify Λ1M with Λ1,0

Malong M and therefore

τ∗Λ1M with ν∗Λ1,0

Malong X ⊂ X, at which point the basic diagram on X

(2.10)

0 0↑ ↑

Λ1,0ν = Λ1,0

ν

↑ ↑0 → μ∗Λ1,0

Z → Λ1,0X

→ Λ1,0μ → 0

↑ ↑ ‖0 → C1,0 → ν∗Λ1,0

M→ Λ1,0

μ → 0↑ ↑0 0

for the ambient double fibration in the holomorphic setting finishes the proof. �

Finally, it is left to the reader also to check that the holomorphic structure forthe bundle Λ1,0

μ on X along the fibres of τ coincides with the standard holomorphic

structure along the fibres of μ for the bundle Λ1,0μ on X when restricted to X ↪→ X.

In summary, for a double fibration of the form (2.3), firstly we have a spectralsequence (2.5) that can be used to interpret Dolbeault cohomology on Z in terms


of involutive cohomology on X, secondly another spectral sequence (2.8) that canbe used to interpret the involutive cohomology on X in terms of smooth data onM and, thirdly, in case that (2.3) complexifies as (2.9), a geometric interpretationof the bundles Λp,0

μ occurring in this spectral sequence. In the following section,we shall see that this is just what we need to operate the transform onto complexprojective space starting with the double fibration (1.3).

3. A particular transform

This section is entirely concerned with the double fibration (1.3), which will bedealt with mainly by means of Theorem 2.4. But, as foretold in §2, the first thingwe should do is deal with the topology of the fibres of η.

Proposition 3.1. For the double fibration (1.3)

• the fibres of η are isomorphic to CPn−2 as smooth manifolds,• the fibres of τ are isomorphic to F1,n−1(Cn) as complex manifolds.

Proof. It is useful to draw a picture in CPn of the incidence variety (1.4)(although, of course, this is a picture over the reals in case n = 3).

There are two points L and � and three hyperplanes L⊥, �⊥, and H. Since L⊥ ∩His the intersection of two hyperplanes in CPn it is intrinsically CPn−2 and, sincethe mapping η from this configuration to F1,n(Cn+1) forgets everything but L ∈ H,we have identified its fibres with CPn−2. On the other hand, if � is fixed, then therest of the configuration may be constructed by choosing an arbitrary point L ∈ �⊥

and an arbitrary hyperplane in �⊥ passing through L, defining H as the join of thishyperplane with �. �

Examining this configuration also shows how the double fibration (1.3) may benaturally complexified to obtain (2.9). One simply allows the point � ∈ CPn andthe hyperplane �⊥ ∈ CP∗

n to become unrelated save for retaining that � �∈ �⊥. More


precisely, let

M ≡ {(�, h) ∈ CPn × CP∗n | � �∈ h} = F1(Cn+1)×Fn(Cn+1) \ F1,n(Cn+1)

with CPn ≡ M ↪→ M given by � �→ (�, �⊥), where the orthogonal complement istaken with respect to a fixed Hermitian inner product on Cn+1. If we set

X ≡ {(L,H, �, h) | L ⊂ h and � ⊂ H}then clearly this extendsX in (1.4): the geometry is exactly the same except that �⊥

is replaced by the less constrained hyperplane h. An advantage of the complexifieddouble fibration is that it is homogeneous under the action of GL(n+ 1,C):

(3.1)

GL(n+ 1,C)

/⎧⎪⎪⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎪⎪⎩

⎡⎢⎢⎢⎢⎢⎢⎢⎣

∗ 0 ∗ · · · ∗ ∗∗ ∗ ∗ · · · ∗ ∗∗ 0 ∗ · · · ∗ ∗...

......

......

∗ 0 ∗ · · · ∗ ∗0 0 0 · · · 0 ∗

⎤⎥⎥⎥⎥⎥⎥⎥⎦

⎫⎪⎪⎪⎪⎪⎪⎪⎬⎪⎪⎪⎪⎪⎪⎪⎭

GL(n+ 1,C)

/⎧⎪⎪⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎪⎪⎩

⎡⎢⎢⎢⎢⎢⎢⎢⎣

∗ 0 0 · · · 0 00 ∗ ∗ · · · ∗ ∗0 0 ∗ · · · ∗ ∗...

......

......

0 0 ∗ · · · ∗ ∗0 0 0 · · · 0 ∗

⎤⎥⎥⎥⎥⎥⎥⎥⎦

⎫⎪⎪⎪⎪⎪⎪⎪⎬⎪⎪⎪⎪⎪⎪⎪⎭

GL(n+ 1,C)

/⎧⎪⎪⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎪⎪⎩

⎡⎢⎢⎢⎢⎢⎢⎢⎣

∗ 0 0 · · · 0 00 ∗ ∗ · · · ∗ ∗0 ∗ ∗ · · · ∗ ∗...

......

......

0 ∗ ∗ · · · ∗ ∗0 ∗ ∗ · · · ∗ ∗

⎤⎥⎥⎥⎥⎥⎥⎥⎦

⎫⎪⎪⎪⎪⎪⎪⎪⎬⎪⎪⎪⎪⎪⎪⎪⎭

��

��

��

μ ν

Before exploiting this homogeneity, however, there is an immediate consequence ofProposition 3.1, as follows.

Proposition 3.2. Concerning the double fibration (1.3), there are canonicalisomorphisms

Hr(Γ(X,Λ0,•X ), ∂X) = C for r = 0, 2, 4, 6, · · · , 2n− 4,

and the cohomology in other degrees vanishes.

Proof. From Proposition 3.1 and the well-known de Rham cohomology ofcomplex projective space [5], it follows from (2.5) that the E1-level of this spectralsequence is isomorphic to

(3.2)

Γ(Z,Λ0,0)

0

Γ(Z,Λ0,0)

0

Γ(Z,Λ0,0)

Γ(Z,Λ1,0)

0

Γ(Z,Λ1,0)

0

Γ(Z,Λ1,0)

Γ(Z,Λ2,0)

0

Γ(Z,Λ2,0)

0

· · ·

Γ(Z,Λ3,0)

0

· · ·

�

p

q

But, the fibres of η are not only isomorphic to CPn−2 as smooth manifolds but asKahler manifolds—the fixed Hermitian inner product on Cn+1 endows each fibrewith a canonical Kahler metric. In particular, the Kahler form and its exterior


powers provide an explicit basis for the de Rham cohomology and therefore thisidentification of the E1-level becomes canonical. Now, as a very special case of theBott-Borel-Weil Theorem [4], the cohomology of each row of (3.2) is concentrated inzeroth position where it is canonically identified with C. As this spectral sequenceconverges to Hp+q(Γ(X,Λ0,•

X ), ∂X), the proof is complete. �

Now we come to the task of interpreting the spectral sequence (2.8). As alreadymentioned in §2 we shall use Proposition 2.6 and the complexified double fibration

F1,n(Cn+1)

X

CPn

��

��

η τ ↪→F1,n(Cn+1)

X

M = {(�, h) ∈ CPn × CP∗n | � �∈ h}

��

��

μ ν

to identify the direct images τ q∗Λ0,pμ . This, in turn, will be facilitated by the fact

that the complexification is GL(n+ 1,C)-homogeneous as in (3.1).For simplicity, we shall now restrict to the case n = 3, the general case being

only notationally more awkward. Adapting the notation of [8], the irreduciblehomogeneous vector bundles on M may be denoted

(a ‖ b, c, d) for integers a, b, c, d with b ≤ c ≤ d.

For example, the holomorphic cotangent bundle is

(3.3) (−1 ‖ 0, 0, 1)⊕ (1 ‖−1, 0, 0),

being the analytic continuation of the bundle Λ1M = Λ0,1

M ⊕ Λ1,0M on M . Similarly,

the irreducible homogeneous vector bundles on X are necessarily line bundles andmay be denoted

(a ‖ b | c | d) for arbitrary integers a, b, c, d.

By carefully unravelling the meaning of these symbols in terms of weights, one cancheck that the bundle Λ1,0

μ is reducible and

(3.4) Λ1,0μ =

(−1 ‖ 0 | 0 | 1) + (−1 ‖ 0 | 1 | 0)⊕

(1 ‖−1 | 0 | 0) + (1 ‖ 0 | −1 | 0)

where (−1 ‖ 0 | 0 | 1)+(−1 ‖ 0 | 1 | 0), for example, means that this is a rank 2 bundlewith composition factors as indicated, equivalently that there is an exact sequence

0→ (−1 ‖ 0 | 1 | 0)→ (−1 ‖ 0 | 0 | 1) + (−1 ‖ 0 | 1 | 0)→ (−1 ‖ 0 | 0 | 1)→ 0.

The procedure for computing direct images is explained in [8] and here we find

ν∗(−1 ‖ 0 | 0 | 1) = (−1 ‖ 0, 0, 1) ν∗(1 ‖−1 | 0 | 0) = (1 ‖−1, 0, 0)

with all other direct images vanishing (e.g. (−1 ‖ 0 | 1 | 0) is singular along the fibresof ν). Bearing in mind that the fibres of ν coincide with those of τ over M , we haveproved the following.

Lemma 3.3. For the double fibration (1.3) and Λ1,0μ defined on X by the exact

sequence (2.6), we have

τ∗Λ1,0μ = Λ1

M and all higher direct images vanish.


From (3.4) and the algorithms in [8] the higher forms are

Λ2,0μ = (−2 ‖ 0 | 1 | 1)⊕

[(0 ‖−1 | 0 | 1)+

(0 ‖ 0 | −1 | 1)⊕

(0 ‖−1 | 1 | 0)+(0 ‖ 0 | 0 | 0)

]⊕(2 ‖−1 | −1 | 0)

and therefore

ν∗Λ2,0μ = (−2 ‖ 0, 1, 1)⊕ (0 ‖−1, 0, 1)⊕ (0 ‖ 0, 0, 0)⊕ (2 ‖−1,−1, 0) = Λ2

M

with all higher direct images vanishing. Next,

Λ3,0μ =

(−1 ‖−1 | 1 | 1) + (−1 ‖ 0 | 0 | 1)⊕

(1 ‖−1 | −1 | 1) + (1 ‖−1 | 0 | 0)

whence

ν∗Λ3,0μ =

(−1 ‖−1, 1, 1)⊕ (−1 ‖ 0, 0, 1)⊕

(1 ‖−1,−1, 1)⊕ (1 ‖−1, 0, 0)=

Λ1,2M

⊕Λ2,1M

with all higher direct images vanishing. Finally,

Λ4,0μ = (0 ‖−1 | 0 | 1) =⇒ ν∗Λ

4,0μ = (0 ‖−1, 0, 1) = Λ2,2

M,⊥,

where Λ2,2M,⊥ denotes the (2, 2)-forms orthogonal to κ∧κ where κ is the Kahler form

on M = CP3. Again, the higher direct images vanish.Feeding all this information into the spectral sequence of Theorem 2.4 causes

it to collapse to an identification of the involutive cohomology Hr(Γ(X,Λ0,•X ), ∂X)

as the global cohomology of the elliptic complex

(3.5) 0→ Λ0 d−→ Λ1 d−→ Λ2 →Λ1,2

⊕Λ2,1

→ Λ2,2⊥ → 0

on CP3 and from Proposition 3.2 we deduce the following.

Theorem 3.4. The complex (3.5) is exact on CP3 except at Λ0 and Λ2, whereits cohomology is canonically identified with C.

In fact, the Kahler form on CP3 generates the cohomology at Λ2. It is interestingto compare (3.5) with the complex that emerges from the Penrose transform ofHr(F1,2(C4),O) under the submersion F1,2(C4)→ CP3 as computed in [10], namely

0→ Λ0 d−→ Λ1 →Λ0,2

⊕Λ1,1⊥

→ Λ1,2⊥ → 0,

which is exact except for the constants at Λ0. The complex (3.5) is better balancedwith respect to type, as one would expect.

As a simple variation on this theme, one can consider a similar transform forthe Dolbeault cohomology of Z = F1,3(C4) but having coefficients in any complexhomogeneous line bundle or, indeed, vector bundle on Z. Following the notationof [8], let us next consider the homogeneous line bundle (1 | 0, 0 | 0) on Z. The only


additional difficulty that must be addressed is that F1,3(C4), as it appears in (3.1),is not written in standard form. Specifically, we have

GL(4,C)/⎧⎨⎩

⎡⎣ ∗ 0 ∗ ∗∗ ∗ ∗ ∗∗ 0 ∗ ∗0 0 0 ∗

⎤⎦⎫⎬⎭ rather than GL(4,C)/⎧⎨⎩

⎡⎣ ∗ ∗ ∗ ∗0 ∗ ∗ ∗0 ∗ ∗ ∗0 0 0 ∗

⎤⎦⎫⎬⎭ .

But these two realisations are equivalent under conjugation by

(3.6)

⎡⎣ 0 1 0 01 0 0 00 0 1 00 0 0 1

⎤⎦and, as explained in [10,13], the effect of this conjugation is that the formula forpulling back a homogeneous vector bundle from Z to X includes the action of theWeyl group element represented by (3.6). Specifically,

μ∗(a | b, c | d) = (b ‖ a | c | d) + · · ·and, in particular,

(3.7) μ∗(1 | 0, 0 | 0) = (0 ‖ 1 | 0 | 0).This bundle on X makes its effect felt in modifying the spectral sequence (2.8) as

Ep,q1 = Γ(M, τ q∗ (Λ

p,0μ ⊗ (0 ‖ 1 | 0 | 0)|X)) =⇒ Hp+q(Γ(X,Λ0,•

X ⊗ (0 ‖ 1 | 0 | 0)|X), ∂X)

and also the spectral sequence (2.5) as applied in proving Proposition 3.2. In fact,since Hr(F1,3(C4),O(1 | 0, 0 | 0)) = 0 for all r (as a particular instance of the Bott-Borel-Weil Theorem [4]), following the proof of Proposition 3.2 demonstrates thefollowing.

Proposition 3.5. Concerning the double fibration (1.3), we have

Hr(Γ(X,Λ0,•X ⊗ (0 ‖ 1 | 0 | 0)|X)), ∂X) = 0 ∀ r.

Therefore, the spectral sequence

Ep,q1 = Γ(M, τ q∗ (Λ

p,0μ ⊗ (0 ‖ 1 | 0 | 0)|X))

converges to zero. It remains to compute the bundles involved and for this we mayproceed as before, instead computing

νq∗(Λp,0μ ⊗ (0 ‖ 1 | 0 | 0)) for ν : X→M

and then restricting to CP3 = M ↪→ M. This is a matter of combining (3.7) with(3.4) and applying the Bott-Borel-Weil Theorem as formulated in [8].

Proposition 3.6. The direct images νq∗(Λp,0μ ⊗ (0 ‖ 1 | 0 | 0)) vanish for q ≥ 1

and for q = 0 are as follows

(3.8)p = 0 p = 1 p = 2 p = 3 p = 4

0 0 (−2 ‖ 1, 1, 1) (−1 ‖ 0, 1, 1)⊕ (1 ‖ 0, 0, 0) (0 ‖ 0, 0, 1) .

Proof. According to the Bott-Borel-Weil Theorem, some particular directimages are

ν∗(a ‖ b | c | d) = (a ‖ b, c, d) if b ≤ c ≤ dν1∗(a ‖ b | c | d) = (a ‖ b+ 1, c− 1, d) if b+ 1 ≤ c− 1 ≤ dν1∗(a ‖ b | c | d) = (a ‖ b, d+ 1, c− 1) if b ≤ d+ 1 ≤ c− 1


and, in these cases, all other direct images vanish. Furthermore, (a ‖ b | c | d) has alldirect images vanishing if any two of b, c+ 1, d+ 2 coincide. This will be sufficientfor our purposes. In particular,

νq∗(Λ0,0μ ⊗ (0 ‖ 1 | 0 | 0)) = νq∗(0 ‖ 1 | 0 | 0) = 0 ∀ q.

Next,

Λ1,0μ ⊗ (0 ‖ 1 | 0 | 0) =

(−1 ‖ 1 | 0 | 1) + (−1 ‖ 1 | 1 | 0)⊕

(1 ‖ 0 | 0 | 0) + (1 ‖ 1 | −1 | 0)so

(3.9) νq∗(Λ1,0μ ⊗ (0 ‖ 1 | 0 | 0)) = νq∗((1 ‖ 0 | 0 | 0) + (1 ‖ 1 | −1 | 0)).

This requires further work since we need to know the connecting homomorphism

(3.10)

ν∗(1 ‖ 0 | 0 | 0) → ν1∗(1 ‖ 1 | −1 | 0)‖ ‖

(1 ‖ 0, 0, 0) ?−→ (1 ‖ 0, 0, 0)induced by this extension. For this, we consult again the diagram (2.10), findingthat the bottom row in case of (3.1) with n = 3 is

0 → C1,0 → ν∗Λ1,0M

→ Λ1,0μ → 0

‖ ‖ ‖

0 →(−1 ‖ 1 | 0 | 0)

⊕(1 ‖ 0 | 0 | −1)

→ ν∗

⎡⎣ (−1‖0, 0, 1)⊕

(1‖−1, 0, 0)

⎤⎦ → (−1 ‖ 0 | 0 | 1) + (−1 ‖ 0 | 1 | 0)⊕

(1 ‖−1 | 0 | 0) + (1 ‖ 0 | −1 | 0)→ 0 ,

which, in particular, yields the short exact sequence

0→ (1 ‖ 0 | 0 | −1)→ ν∗(1‖−1, 0, 0)→ (1 ‖−1 | 0 | 0) + (1 ‖ 0 | −1 | 0)→ 0

and, therefore, when tensored with (0 ‖ 1 | 0 | 0) the short exact sequence

0→ (1 ‖ 1 | 0 | −1)→ ν∗(1‖−1, 0, 0)⊗(0 ‖ 1 | 0 | 0)→ (1 ‖ 0 | 0 | 0)+(1 ‖ 1 | −1 | 0)→ 0

from which it follows that all the direct images (3.9) vanish (equivalently, that theconnecting homomorphism (3.10) is an isomorphism, as one might expect).

Next, we should compute the direct images of Λ2,0μ ⊗ (0 ‖ 1 | 0 | 0), i.e. of

(−2 ‖ 1 | 1 | 1)⊕[(0 ‖ 0 | 0 | 1) +

(0 ‖ 1 | −1 | 1)⊕

(0 ‖ 0 | 1 | 0)+ (0 ‖ 1 | 0 | 0)

]⊕ (2 ‖ 0 | −1 | 0).

The induced connecting homomorphism ν∗(0 ‖ 0 | 0 | 1) → ν1∗(0 ‖ 1 | −1 | 1) is againan isomorphism by similar reasoning and only (−2 ‖ 1 | 1 | 1) contributes to the directimages, as claimed in (3.8).

Next,

Λ3,0μ ⊗ (0 ‖ 1 | 0 | 0) =

(−1 ‖ 0 | 1 | 1) + (−1 ‖ 1 | 0 | 1)⊕

(1 ‖ 0 | −1 | 1) + (1 ‖ 0 | 0 | 0)and, finally,

Λ4,0μ ⊗ (0 ‖ 1 | 0 | 0) = (0 ‖ 0 | 0 | 1)

from which the rest of (3.8) is immediate. �

Assembling these various computations yields the following.


Theorem 3.7. There is an elliptic and globally exact complex on CP3

(3.11)0 → (−2 ‖ 1, 1, 1) → (−1 ‖ 0, 1, 1) → (0 ‖ 0, 0, 1) → 0.

⊕ ↗(1 ‖ 0, 0, 0)

Proof. Everything is shown save for the following two observations. Firstly,there is no possible first order differential operator (−2 ‖ 1, 1, 1)→ (1 ‖ 0, 0, 0) sincethere is no possible SU(4)-invariant symbol. Indeed, from (3.3), we have

Λ1M ⊗ (−2 ‖ 1, 1, 1) = (−3 ‖ 1, 1, 2)⊕ (−1 ‖ 0, 1, 1).

Similar symbol considerations

Λ1M ⊗ (−1 ‖ 0, 1, 1) = (−2 ‖ 0, 1, 2)⊕ (−2 ‖ 1, 1, 1)⊕ (0 ‖−1, 1, 1)⊕ (0 ‖ 0, 0, 1)Λ1M ⊗ (1 ‖ 0, 0, 0) = (0 ‖ 0, 0, 1)⊕ (2 ‖−1, 0, 0)

also allow one to check that the complex is elliptic. �

Invariance under SU(4) identifies the operators explicitly. Specifically, if wedenote by L the homogeneous line bundle L = (−2 ‖ 1, 1, 1), then (3.11) becomes

0 → L∂−→ Λ1,0

M ⊗ L∂−→ Λ2,0

M ⊗ L → 0.⊕ ↓ κ∧κ∧

Λ3,0M ⊗ L

∂−→ Λ3,1M ⊗ L

As a check, Theorem 3.7 says that L has no global anti-holomorphic sections andthis is certainly true because its complex conjugate (2 ‖−1,−1,−1) has no globalholomorphic sections (it is the homogeneous holomorphic bundle (2 | −1,−1,−1) inthe notation of [8], which has singular infinitesimal character).

Other homogeneous holomorphic bundles on the twistor space Z = F1,n(Cn+1)will give rise to other invariant complexes of differential operators on CPn. Theauthor suspects that the holomorphic tangent bundle Θ will give rise to an especiallyinteresting complex (since H1(Z,Θ) parameterises the infinitesimal deformationsof Z as a complex manifold). Unfortunately, he has not yet been able to completethe calculation in this case.

4. Another particular transform

This section is concerned with an instance of the holomorphic double fibrationtransform as formulated in general in [15]. Specifically, let us consider the complexflag manifold Z = F1,n(Cn+1) under the action of SU(n, 1). There are three openorbits for this action, easily described in terms of geometry in CPn. The orbits ofSU(n, 1) acting on CPn are the open ball B, its boundary, and the complement ofits closure. As in §1 and §3, an element (L,H) in Z may be viewed as a point on ahyperplane in CPn. The three open orbits are given by the following restrictions.

• the point L lies in the ball B,• the hyperplane H lies outside the ball B,• the point L lies outside B but the hyperplane H intersects B.

The set of hyperplanes lying outside B defines an open subset in the dual projectivespace CP∗

n, which we may identify with B, i.e. the ball B with its conjugate complexstructure.

Let us consider the third of the options above for the open orbits of SU(n, 1)acting on Z and call it D. By definition it is a flag domain. Following the notation


of [15], its cycle space MD is B × B inside CPn × CP∗n and the correspondence

space XD is exactly ν−1(MD) for the complexified correspondence of §3. Thus, wehave an open inclusion

D

XD

MD

��

��

μ ν open ↪→F1,n(Cn+1)

X

M

��

��

μ ν

and, in particular, the fibres of ν over MD coincide with the fibres of ν : X → Mrestricted to MD. The spectral sequence [2,13] for the resulting double fibrationtransform starting with a holomorphic vector bundle E on D reads

(4.1) Ep,q1 = Γ(MD, νq∗(Λ

p,0μ ⊗ μ∗E)) =⇒ Hp+q(D,O(E))

under the assumptions that MD is Stein and that μ : XD → D has contractiblefibres, both of which are true for any flag domain [15] and directly seen to be thecase here.

Observe that the terms in this spectral sequence (when E is trivial) are almostthe same as in (2.8). Certainly, the direct image bundles may be obtained byworking on the homogeneous correspondence (3.1) and then restricting toMD. Asour final example, let us carry this out for n = 3 and for E being the canonicalbundle on D. This is precisely the restriction to D of the homogeneous line bundle(3 | 0, 0 | −3) on Z. Following exactly the procedures of §3 we find the followinghomogeneous bundles for Λp,0

μ ⊗ μ∗(3 | 0, 0 | −3) = Λp,0μ ⊗ (0 ‖ 3 | 0 | −3).

p = 0 (0 ‖ 3 | 0 | −3)

p = 1

(−1 ‖ 3 | 0 | −2) + (−1 ‖ 3 | 1 | −3)

⊕(1 ‖ 2 | 0 | −3) + (1 ‖ 3 | −1 | −3)

p = 2 (−2 ‖ 3 | 1 | −2)⊕[(0 ‖ 2 | 0 | −2) +

(0 ‖ 3 | −1 | −2)⊕

(0 ‖ 2 | 1 | −3)+ (0 ‖ 3 | 0 | −3)

]⊕ (2 ‖ 2 | −1 | −3)

p = 3(−1 ‖ 2 | 1 | −2) + (−1 ‖ 3 | 0 | −2)

⊕(1 ‖ 2 | −1 | −2) + (1 ‖ 2 | 0 | −3)

p = 4 (0 ‖ 2 | 0 | −2)

Using the Bott-Borel-Weil Theorem, as formulated in [8], we find that the onlynon-zero direct images νq∗

(Λp,0μ ⊗ μ∗(3 | 0, 0 | −3)

)are when q = 3 as follows.

p = 0 (0 ‖−1, 0, 1)

p = 1(−1 ‖ 0, 0, 1)⊕ (−1 ‖−1, 1, 1)

⊕(1 ‖−1, 0, 0)⊕ (1 ‖−1,−1, 1)

p = 2 (−2 ‖ 0, 1, 1)⊕[(0 ‖ 0, 0, 0)⊕ (0 ‖−1, 0, 1)

]⊕ (2 ‖−1,−1, 0)

p = 3(−1 ‖ 0, 0, 1)

⊕(1 ‖−1, 0, 0)

p = 4 (0 ‖ 0, 0, 0)

We conclude, for example, from (4.1) that H3(D,O(3 | 0, 0 | −3)) is realised on

MD = B × B ⊂ CP3 × CP∗3

as the kernel of the holomorphic differential operator

(0 ‖−1, 0, 1)ð

ð��

(−1 ‖ 0, 0, 1) ⊕ (−1 ‖−1, 1, 1)⊕

(1 ‖−1, 0, 0) ⊕ (1 ‖−1,−1, 1)


where ð (respectively ð) denotes holomorphic differentiation in the direction of CP3

(respectively CP∗3) followed by projection to the indicated bundles:

(0 ‖−1, 0, 1) −→ Λ1,0CP3⊗ (0 ‖−1, 0, 1)

= (1 ‖−1, 0, 0)⊗ (0 ‖−1, 0, 1)= (1 ‖−2, 0, 1)⊕ (1 ‖−1,−1, 1)⊕ (1 ‖−1, 0, 0)� (1 ‖−1,−1, 1)⊕ (1 ‖−1, 0, 0).

It is interesting to note that the entire complex ν3∗(Λ•,0μ ⊗ μ∗(3 | 0, 0 | −3)

)on

M is the analytic continuation of the elliptic complex

0→ Λ1,1⊥↗↘

Λ1,2

⊕Λ2,1

↗↘↗↘

Λ1,3

⊕Λ2,2

⊕Λ3,1

↘↗↘↗

Λ2,3

⊕Λ3,2

↘↗ Λ3,3 → 0

on M = CP3 and that this complex is the formal adjoint of (3.5), exactly aspredicted by duality [13, Theorem 4.1]. The transform described in this section isan example of the much more general theory developed in [14].

References

[1] M. F. Atiyah, N. J. Hitchin, and I. M. Singer, Self-duality in four-dimensional Rie-mannian geometry, Proc. Roy. Soc. London Ser. A 362 (1978), no. 1711, 425–461, DOI10.1098/rspa.1978.0143. MR506229 (80d:53023)

[2] Robert J. Baston and Michael G. Eastwood, The Penrose transform, Oxford MathematicalMonographs, The Clarendon Press Oxford University Press, New York, 1989. Its interactionwith representation theory; Oxford Science Publications. MR1038279 (92j:32112)

[3] Shiferaw Berhanu, Paulo D. Cordaro, and Jorge Hounie, An introduction to involutive struc-tures, New Mathematical Monographs, vol. 6, Cambridge University Press, Cambridge, 2008.MR2397326 (2009b:32048)

[4] Raoul Bott, Homogeneous vector bundles, Ann. of Math. (2) 66 (1957), 203–248. MR0089473(19,681d)

[5] Raoul Bott and Loring W. Tu, Differential forms in algebraic topology, Graduate Texts inMathematics, vol. 82, Springer-Verlag, New York, 1982. MR658304 (83i:57016)

[6] N. Buchdahl, On the relative de Rham sequence, Proc. Amer. Math. Soc. 87 (1983), no. 2,363–366, DOI 10.2307/2043718. MR681850 (85f:58003)

[7] N. P. Buchdahl, Instantons on CP2, J. Differential Geom. 24 (1986), no. 1, 19–52. MR857374(88b:32066)

[8] Michael G. Eastwood, The generalized Penrose-Ward transform, Math. Proc. Cambridge Phi-

los. Soc. 97 (1985), no. 1, 165–187, DOI 10.1017/S030500410006271X. MR764506 (86f:32032)[9] Michael Eastwood, Some examples of the Penrose transform, Surikaisekikenkyusho

Kokyuroku 1058 (1998), 22–28. Analysis and geometry appearing in multivariable functiontheory (Japanese) (Kyoto, 1997). MR1689417 (2000e:32031)

[10] Michael Eastwood, The Penrose transform for complex projective space, Complex Var. El-liptic Equ. 54 (2009), no. 3-4, 253–264, DOI 10.1080/17476930902760435. MR2513538(2010g:32031)

[11] Michael G. Eastwood, Roger Penrose, and R. O. Wells Jr., Cohomology and massless fields,Comm. Math. Phys. 78 (1980/81), no. 3, 305–351. MR603497 (83d:81052)

[12] Michael G. Eastwood and Michael A. Singer, The Frohlicher [Frolicher] spectral sequence ona twistor space, J. Differential Geom. 38 (1993), no. 3, 653–669. MR1243789 (94k:32050)

[13] M.G. Eastwood & J.A. Wolf, A duality for the double fibration transform, in Geometry,Analysis and Quantum Field Theory, Contemp. Math., Amer. Math. Soc., to appear.

[14] M.G. Eastwood & J.A. Wolf, The range of the double fibration transform I: Duality and theHermitian holomorphic cases, in preparation.


[15] Gregor Fels, Alan Huckleberry, and Joseph A. Wolf, Cycle spaces of flag domains, Progressin Mathematics, vol. 245, Birkhauser Boston Inc., Boston, MA, 2006. A complex geometricviewpoint. MR2188135 (2006h:32018)

[16] N. J. Hitchin, Linear field equations on self-dual spaces, Proc. Roy. Soc. London Ser. A 370(1980), no. 1741, 173–191, DOI 10.1098/rspa.1980.0028. MR563832 (81i:81057)

Mathematical Sciences Institute, Australian National University, ACT 0200



Magnetic Schrodinger equation on compact symmetricspaces and the geodesic Radon transform of one forms

Tomoyuki Kakehi

Dedicated to Professor Sigurdur Helgason on the occasion of his 85-th birthday.

Abstract. In this article, we will give a support theorem for the fundamental

solution to the magnetic Schodinger equation on certain compact symmetricspaces under the assumption that the corresponding vector potential satisfiesthe zero energy condition.

1. Introduction

Let us consider the magnetic Schrodinger equation on a compact symmetricspace M = U/K.

(1.1) (Sch)[M,ω]

{ √−1∂tψ +Hωψ = 0, t ∈ R,

ψ(0, x) = δo(x), x ∈M,

where Hω := −( d +√−1ω )∗( d +

√−1ω ) is the magnetic Schrodinger operator

with vector potential ω on M and δo denotes the delta function with singularityat the origin o = eK ∈ U/K = M . Let Eω

M (t, x) be the solution to (Sch)[M,ω].In particular, if ω = 0, then H0(= −d∗d) coincides with the Laplacian ΔM onM . Therefore, E0

M (t, x) is the fundamental solution to the Schrodinger equationcorresponding to a free particle starting from o at t = 0. Here we note that thefundamental solution Eω

M (t, ·) exists as a distribution on M . Then our main resultsare as follows.

Theorem 1.1 (See Theorem 3.4). Under some assumptions on M and on ω,we have

SuppEωM (t, ·) = SuppE0

M (t, ·).The above theorem shows that if a vector potential ω satisfies a certain condi-

tion called the zero energy condition (see (ZEC) below in Section 2), then ω doesnot affect the motion of a particle on M .

Theorem 1.2 (See Theorem 4.2). In particular, if M = P2m+1R (the odddimensional real projective space), then we have

(I) In the case t/2π ∈ Q, SuppEωM (t, ·) becomes a lower dimensional subset.

2000 Mathematics Subject Classification. Primary 33C67, 43A90; Secondary 43A85.Key words and phrases. Schrodinger equation, support, compact symmetric spaces, magnetic

field, zero energy condition.


129

130 TOMOYUKI KAKEHI

(II) In the case t/2π /∈ Q, SuppEωM (t, ·) = SingSuppEω

M (t, ·) = M .

2. Magnetic Schodinger equation on compact symmetric spaces

In this section, we start with a magnetic Schodinger equation on a general Rie-mannian manifold. Let (M, g) be an n-dimensional compact Riemannian manifoldwith metric g = (gjk).

Let Ωp(M) be the space of smooth p-forms on M and let d : Ωp(M) →Ωp+1(M) be the differential.

For a real valued 1-form ω ∈ Ω1(M), we define a magnetic Schrodinger operatorHω on M by

Hω := −( d+√−1ω )∗( d+

√−1ω ),

where ( d+√−1ω )∗ : Ω1(M) → C∞(M) is the adjoint of d +

√−1ω : C∞(M) →

Ω1(M) w.r.t. the L2-inner product of 1-forms.The above 1-form ω is called a vector potential and the 2-form dω is called a

magnetic field.Here the L2-inner product on the space of 1-forms is defined as follows. For

α =∑

αj(x)dxj , β =

∑βj(x)dx

j ∈ Ω1(M), we define the L2-inner product of αand β by

〈α, β 〉L2(M→T∗M) :=

∫M

∑j,k

αj(x)βk(x)gjk(x) dμ,

where (gjk) = (gjk)−1 and dμ =

√det(gjk)dx. Then the adjoint of d +

√−1ω is

defined by

〈 (d+√−1ω)f, β 〉L2(M→T∗M) = 〈 f, (d+

√−1ω)∗β 〉L2(M),

for f ∈ C∞(M) and β ∈ Ω1(M).Using the local coordinates x = (x1, · · · , xn), we write ω =

∑nj=1 ωj(x)dx

j .Then Hω is expressed as

(2.1) Hω ψ =1√G

n∑j,k=1

(∂xj +

√−1ωj(x)

) [√Ggjk(x)

(∂xk +

√−1ωk(x)

)ψ].

Here G = det(gjk(x)) and ∂xj = ∂/∂xj .From now on, we assume the following condition on ω.

(ZEC) :

∫γ

ω = 0, for any closed geodesic γ.

Remark: The above condition (ZEC) is called the zero energy condition. (SeeGasqui and Goldschmidt [G-G-3], and Bailey and Eastwood [B-E].)

3. Geodesic Radon transform of 1-forms

In this section, we will give a brief summary on the results by Gasqui andGoldschmidt.

Let Geod(M) be the set of all the closed geodesics in M . Then if M is acompact symmetric space of rank 1, Geod(M) becomes a manifold. For example,Geod(Sn) ∼= SO(n+ 1)/SO(2)× SO(n− 1).

MAGNETIC SCHRODINGER EQUATION ON COMPACT SYMMETRIC SPACES 131

Let us first consider the geodesic Radon transform R0 from the space of smoothfunctions on M to the space of functions on Geod(M) defined by

R0f(γ) :=

∫γ

f(x) dμγ(x), for f ∈ C∞(M),

where dμγ denotes the measure on γ induced from the canonical measure on MIt is known that R0 is injective unless M = Sn. (See, for example, Helgason

[H-5] Chap. IV, Section 1, Theorem 1.1.)Similarly let us define a geodesic Radon transform R1 from the space of smooth

1-forms on M to the space of functions on Geod(M) by

(3.1) R1ω(γ) :=

∫γ

ω, for ω ∈ Ω1(M).

However, in this case, R1 is no longer injective for any M . In fact, if ω is exact,namely if ω is written as ω = dF for some F ∈ C∞(M), then

R1ω(γ) = R1(dF )(γ) ≡∫γ

dF = 0.

In other words, any exact 1-form satisfies the zero energy condition (ZEC). So onemay think of the following question.

Question: Suppose that ω ∈ Ω1(M) satisfies (ZEC). Is such a 1-form ω anexact 1-form?

Before answering the above question, we introduce a certain rigidity on asmooth manifold.

Definition 3.1. We say that a compact smooth manifold M is rigid in thesense of Gasqui and Goldschmidt if any smooth 1-form on M satisfying (ZEC) isexact.

Gasqui and Goldschmidt gave the following answer to the above question.

Theorem 3.2 (Gasqui and Goldschmidt [G-G-3]).

(I) PnR, PnC (n ≥ 2), PnH (n ≥ 2), and P2Cay are rigid in the sense ofGasqui and Goldschmidt.

(II) Flat tori are rigid in the sense of Gasqui and Goldschmidt.(III) SO(n)/(SO(2)× SO(n− 2)) (n ≥ 5) are rigid in the sense of Gasqui

and Goldschmidt.(IV) If two compact symmetric spaces M1 and M2 are rigid in the sense of

Gasqui and Goldschmidt, then M1 ×M2 is rigid in the sense of Gasquiand Goldschmidt.

Remark 3.3. (I) This rigidity has been extensively studied by a lot of peoplein connection with the infinitesimal rigidity or the rigidity in the sense of Guillemin.In particular, Gasqui and Goldschmidt prove this rigidity for several different kindsof symmetric spaces. For the details, see their book [G-G-3]. (See also their papers[G-G-1] and [G-G-2].) (II) Bailey and Eastwood [B-E] generalizes the result byGasqui and Goldschmidt to tensor fields on real projective spaces.

Let us now state our first main theorem.

132 TOMOYUKI KAKEHI

Theorem 3.4. Suppose that a compact symmetric space M is rigid in thesense of Gasqui and Goldschmidt and that a real valued smooth 1-form ω on Msatisfies the zero energy condition (ZEC). Then the support of the solution Eω

M (t, ·)to the magnetic Schrodinger equation (Sch)[M,ω] coincides with that of the solution

E0M (t, ·) to the free particle Schrodinger equation (Sch)[M,0] for any t ∈ R. Namely

we have


M (t, ·) for t ∈ R.

Proof. By Definition 3.1, we have ω = dF for some F ∈ C∞(M). Therefore,the magnetic Schrodinger operator Hω is rewritten as

(3.2) Hω ≡ −( d+√−1ω )∗( d+

√−1ω ) = −e−

√−1FΔM ◦ e

√−1F .

Here ΔM denotes the Laplacian w.r.t. the standard metric on M . Noting thatψ = E0

M satisfies √−1∂tψ +ΔMψ = 0,

we see easily from (3.2) that EωM (t, x) = e−

√−1{F (x)−F (o)}E0

M (t, x). In particular,we obtain the assertion. �

The following corollary is a direct consequence of Theorem 3.2 and Theorem3.4.

Corollary 3.5. If M is a compact symmetric space in the list of Theorem 3.2and if a real valued 1-form ω on M satisfies the zero energy condition (ZEC), thenwe have


M (t, ·) for t ∈ R.

4. Magnetic Schrodinger equation on real projective spaces

In this section, we will deal with the magnetic Schrodinger equation on odddimensional real projective spaces.

Let us start with the Schrodinger equation on the odd dimensional sphereS2m+1 = {x = (x1, · · · , x2m+2 ) ∈ R2m+2 | ||x|| = 1 }.

(4.1) (Sch)S2m+1

{ √−1∂tψ +ΔS2m+1ψ = 0, t ∈ R,

ψ(0, x) = δo(x), x ∈ S2m+1.

S2m+1 is regarded as the symmetric space SO(2m + 2)/SO(2m + 1) in the usualmanner. So in the above equation, the origin o = e1, the 1-st unit vector. LetES2m+1(t, x) be the solution to (Sch)S2m+1 , namely, the fundamental solution tothe free particle Schrodinger equation on S2m+1.

In order to describe the support of ES2m+1(t, x), we define a finite subset Gq ofS1 for a positive integer q as follows.

(i) q ≡ 0 (mod 4)

Gq :=

{[2πk

q

]2πZ

∈ R/2πZ ∼= S1 | k = 0, 2, 4, · · · , q − 2.

}.

(ii) q ≡ 2 (mod 4)

Gq :=

{[2πk

q

]2πZ

∈ R/2πZ ∼= S1 | k = 1, 3, 5, · · · , q − 1.

}.


(iii) q ≡ 1, 3 (mod 4)

Gq :=

{[2πk

q

]2πZ

∈ R/2πZ ∼= S1 | k = 0, 1, 2, 3, · · · , q − 1.

}.

As a special case of the main theorem in Kakehi [K], we have the followingsupport theorem.

Theorem 4.1 (See [K], Section 6). (I) In the case t/2π ∈ Q. We putt = 2πp/q, where p, q ∈ Z, q > 0, and p and q are coprime. Then thesupport of ES2m+1(t, ·) is given by

SuppES2m+1

(2πp

q, ·)

={k · x(θ) ∈ S2m+1 | k ∈ SO(2m+ 1), [θ]2πZ ∈ Gq

},

where x(θ) := (cos θ)e1 + (sin θ)e2 ( θ ∈ R ).(II) In the case t/2π /∈ Q.

SuppES2m+1( t, · ) = SingSuppES2m+1( t, · ) = S2m+1.

The above theorem implies that at a rational time a free particle on S2m+1

starting from the origin o = e1 at t = 0 exists only on the lower dimensional subsetof S2m+1 given in the above theorem whereas at an irrational time such a freeparticle can exist anywhere on S2m+1.

From now on, we identify a function (resp. a distribution) on P2m+1R with aneven function (resp. an even distribution) on S2m+1. Then we see that the solutionEP2m+1R(t, x) to the Schodinger equation on P2m+1R

(4.2) (Sch)P2m+1R

{ √−1∂tψ +ΔP2m+1Rψ = 0, t ∈ R,

ψ(0, x) = δo(x), x ∈ P2m+1R,

is given by

(4.3) EP2m+1R(t, x) =1

2{ES2m+1(t, x) + ES2m+1(t,−x)} .

Similarly, we identify Ω1(P2m+1R) with Ω1even(S

2m+1) the space of smooth even1-forms on S2m+1.

Let us now consider the magnetic Schodinger equation on real projective spacesP2m+1R.

(4.4) (Sch)[P2m+1R,ω]

{ √−1∂tψ +Hωψ = 0, t ∈ R,

ψ(0, x) = δo(x), x ∈ P2m+1R.

In addition, we assume that a real valued 1-form ω ∈ Ω1(P2m+1R) satisfies the zeroenergy condition (ZEC).

Then by Theorem 3.2 (I), ω is written as ω = dF for some F ∈ C∞(P2m+1R).So as in the proof of Theorem 3.4, by making use of (3.2), we have Eω

P2m+1R(t, x) =

e−√−1{F (x)−F (o)}E0

P2m+1R(t, x).

Therefore, by Theorem 4.1, we obtain the second main theorem.

Theorem 4.2. (I) If t/2π ∈ Q, then SuppEωP2m+1R

(t, ·) becomes a lowerdimensional subset.

(II) If t/2π /∈ Q, then SuppEωP2m+1R

(t, ·) = SingSuppEωP2m+1R

(t, ·) = P2m+1R.

134 TOMOYUKI KAKEHI

Remark 4.3. It follows easily from Theorem 4.1 and (4.3) that the lowerdimensional subset in Theorem 4.2 (I) is given by{ [

k · x(θ)]∈ P2m+1R | k ∈ SO(2m+ 1), [θ]2πZ ∈ Gq

}for t = 2πp/q. In the above, [x] denotes the real one dimensional subspace in R2m+2

spanned by x.

5. Magnetic Schrodinger equation on spheres

In this section, we deal with the magnetic Schodinger equation on spheres.

(5.1) (Sch)[Sn,ω]

{ √−1∂tψ +Hωψ = 0, t ∈ R,

ψ(0, x) = δo(x), x ∈ Sn,

Obviously, Sn (n ≥ 2) is not rigid in the sense of Gasqui and Goldschmidt. Infact, any odd 1-form on Sn (n ≥ 2) satisfies the zero energy condition (ZEC). Sowe cannot expect the same result as in Theorem 4.2 for the magnetic Schodingerequation on spheres.

However, a singular support theorem holds in this case. The key is to use theWeinstein’s method.

Following Weinstein [W], we will construct an asymptotic solution to the mag-netic Schodinger equation on spheres.

The eigenvalues of the Laplacian ΔSn are given by −�(� + n − 1), (� =0, 1, 2, · · · ). Let

A :=

√−ΔSn +

(n− 1

2

)2

.

Then the operator A above satisfies exp(2π√−1A) = cI for some constant c. More-

over, as is well known in microlocal analysis, A is a pseudo-differential operator oforder 1.

Using this operator A, we can rewrite the magnetic Schodinger operator Hω as

(5.2) −Hω = A2 +B,

where B is a first order differential operator. Here we note that the principal symbolσ1(B) of B coincides with ω if we consider ω to be a section of the cotangent bundleT ∗Sn of Sn.

For the above operators A and B, we put

(5.3)

Bt := e−2π√−1tABe2π

√−1tA, B := (2π)−1

∫ 2π

0

Bt dt,

P := (2π√−1)−1

∫ 2π

0

∫ t

0

Bs dsdt, Q :=1

4(PA−1 +A−1P )

A(1) := e−QAeQ, B(1) := e−QBeQ.

In addition, we denote the bicharacteristic stripe of A starting from (x, ξ) ∈ T ∗Sn att = 0 by { (Xt(x, ξ),Ξt(x, ξ)) ∈ T ∗Sn | t ∈ R}. Then we see that the correspondingbicharacteristic curve {Xt(x, ξ)}t∈R is a closed geodesic on Sn, namely, a great


circle. Then we have(5.4)

σ1(B)(x, ξ) = (2π)−1

∫ 2π

0

σ1(Bt)(x, ξ) dt

= (2π)−1

∫ 2π

0

σ1(B)(Xt(x, ξ),Ξt(x, ξ)) dt (by Egorov’s Theorem)

= (2π)−1

∫ 2π

0

ω(Xt(x, ξ)) dt

= (2π)−1

∫γ

ω = 0, (by (ZEC)).

Here we put γ := {Xt(x, ξ)}0≤t≤2π. The above computation shows that the first

order part of B vanishes. In other words, B is a pseudo-differential operator oforder 0. (For Egorov’s Theorem and related results on Fourier integral operators,see Hormander [Hor-2].)

By applying Weinstein’s method in [W] to the above operators A and B, weobtain the following.

Proposition 5.1.(i) A(1) and B(1) commute, namely [A(1), B(1)] = 0.

(ii) A(1) ∈ Ψ1(Sn) and B(1) ∈ Ψ0(Sn).

(iii) Q, eQ ∈ Ψ0(Sn). Moreover eQ ∈ Ψ0(Sn) is a unitary operator on Sn.(iv) A2 +B − {A2

(1) +B(1) } =: C(1) ∈ Ψ0(Sn).

Here in general ΨN (Sn) denotes the space of pseudo-differential operators onSn of order N . (For the definition of pseudo-differential operators, see Hormander[Hor-1] Section 18.1.)

In the same manner, we make operators A(2), B(2) and C(2) from A(1), B(1) andC(1) and repeat this procedure. Then we get a sequence of operators {A(N), B(N),C(N) } (N = 1, 2, 3, · · · , ) with the following property.(5.5)

−Hω = A2 + B = A2(N) +B(N) + C(N), [A(N), B(N)] = 0, exp(2π

√−1A(N)) = const.I,

A(N) ∈ Ψ1(Sn), B(N) ∈ Ψ0(Sn), C(N) ∈ Ψ−(N−1)(Sn).

In addition, A(N) is written in the form A(N) = eQ(N)Ae−Q(N) for some skew

symmetric operator Q(N) ∈ Ψ0(Sn).

Here we put u(t) := exp(√−1tHω) ≡ exp(−

√−1t(A2 + B)) and H(N) :=

−A2(N) −B(N). Then by (5.5)

√−1 d

dtu(t) +H(N)u(t) = C(N)u(t).

By Duhamel’s principle,

(5.6) u(t) = exp(√−1tH(N)) +

∫ t

0

exp(√−1(t− s)H(N))C(N)u(s) ds.

Since C(N) ∈ Ψ−(N−1)(Sn), C(N) is a bounded linear operator from L2(Sn) to

HN−1(Sn) the Sobolev space on Sn of order N −1. So is the second term of R.H.S.of (5.6). Let L(L2,HN−1) be the space of bounded linear operators from L2(Sn)

136 TOMOYUKI KAKEHI

to HN−1(Sn). Thus we have(5.7)

u(t) ≡ exp(√−1tH(N)) = exp(

√−1tB(N)) exp(

√−1tA2

(N)) (mod L(L2,HN−1)).

The above equality holds for any positive integer N . Here we note thatexp(

√−1tB(N)) is an elliptic pseudo-differential operator and thus it preserves the

singular support of a distribution. Therefore, it follows easily from (5.7) that

(5.8)

SingSuppEωM (t, ·) = SingSupp u(t)δo

= SingSupp exp(−√−1tA2)δo

= SingSupp exp(√−1tΔM )δo

= SingSuppE0Sn(t, ·).

Summarizing the argument above, we obtain the following.

Theorem 5.2. We assume that a real valued 1-form ω ∈ Ω1(Sn) satisfies thezero energy condition (ZEC). Then we have

SingSuppEωSn(t, ·) = SingSuppE0

Sn(t, ·) for t ∈ R.

The above theorem implies that a particle with high energy is not so affectedby a vector potential satisfying the zero energy condition.

Finally let us state another singular support theorem without proof.

Theorem 5.3. We assume that a real valued 1-form ω ∈ Ω1(Sn) satisfies thezero energy condition (ZEC).

(I) If t/2π ∈ Q, then SingSuppEωSn(t, ·) becomes a lower dimensional subset.

(II) If t/2π /∈ Q, then SingSuppEωSn(t, ·) = Sn.

6. Some remarks

(I) As is well known, Huygens principle holds for the modified wave equationon odd dimensional symmetric spaces with even root multiplicities. See Branson-Olafsson-Pasquale [BOP], Branson-Olafsson-Schlichtkrull [BOS], Chalykh-Veselov[Cha-Ves], Gonzalez [Gonz], Helgason [H-3], [H-4], Helgason-Schlichtkrull[H-Sch], Olafsson-Schlichtkrull [Olaf-Sch], and Solomatina [Sol]. As far as theauthor knows, Huygens principle is the only known result on the support of so-lutions to differential equations on symmetric spaces. In fact, my research aboutSchrodinger equations on compact symmetric spaces was motivated by those pa-pers, especially papers by Helgason [H-3], [H-4].

(II) In this article, we stated a singular support theorem on magnetic Schro-dinger equations in the case of spheres Sn (n ≥ 2). In general it is expectedthat a singular support theorem such as Theorems 5.2 and 5.3 holds for magneticSchrodinger equations on a compact symmetric space which is not necessarily rigidin the sense of Gasqui and Goldschmidt.

(III) The meaning of the zero energy condition in physics is as follows. Takeany surface whose boundary is a closed geodesic γ and denote it by Sγ . Then byStokes theorem, ∫

Sγ

dω =

∫γ

ω = 0.


In physics, the integral in the left hand side of the above equality is called themagnetic flux. If a compact symmetric space has sufficiently many closed geodesicsand if for any closed geodesic the corresponding magnetic flux is zero, then it isexpected that such a magnetic field (or a vector potential) gives no energy to a

particle. For this reason, it makes sense to call the condition[ ∫

γ

ω = 0, ∀γ]the

zero energy condition.

Acknowledgments

I would like to thank Professor Sigurdur Helgason for giving me a lot of valuablesuggestions for my research and for encouraging me in my research. In addition,his books and papers have been inspiring me a lot since I started to study integralgeometry. So it is my great pleasure to write this article in order to celebrate his85-th birthday. I am also grateful to the organizers of the conference on RadonTransforms and Geometric Analysis, Professor Jens Christensen, Professor FultonGonzalez, and Professor Eric Todd Quinto for giving me an opportunity to talkabout my result.

References

[B-E] Toby N. Bailey and Michael G. Eastwood, Zero-energy fields on real projectivespace, Geom. Dedicata 67 (1997), no. 3, 245–258, DOI 10.1023/A:1004939917121.MR1475870 (98h:32054)

[BOP] Thomas Branson, Gestur Olafsson, and Angela Pasquale, The Paley-Wiener theo-rem and the local Huygens’ principle for compact symmetric spaces: the even mul-tiplicity case, Indag. Math. (N.S.) 16 (2005), no. 3-4, 393–428, DOI 10.1016/S0019-3577(05)80033-3. MR2313631 (2008k:43021)

[BOS] T. Branson, G. Olafsson, and H. Schlichtkrull, Huyghens’ principle in Riemanniansymmetric spaces, Math. Ann. 301 (1995), no. 3, 445–462, DOI 10.1007/BF01446638.MR1324519 (97f:58128)

[Cha-Ves] Oleg A. Chalykh and Alexander P. Veselov, Integrability and Huygens’ principle onsymmetric spaces, Comm. Math. Phys. 178 (1996), no. 2, 311–338. MR1389907(97i:43005)

[G-G-1] Jacques Gasqui and Hubert Goldschmidt, Une caracterisation des formes exactes dedegre 1 sur les espaces projectifs, Comment. Math. Helv. 60 (1985), no. 1, 46–53, DOI10.1007/BF02567399 (French). MR787661 (86g:53052)

[G-G-2] Jacques Gasqui and Hubert Goldschmidt, Infinitesimal rigidity of products of symmet-ric spaces, Illinois J. Math. 33 (1989), no. 2, 310–332. MR987827 (90d:58171)

[G-G-3] Jacques Gasqui and Hubert Goldschmidt, Radon transforms and the rigidity of the

Grassmannians, Annals of Mathematics Studies, vol. 156, Princeton University Press,Princeton, NJ, 2004. MR2034221 (2005d:53081)

[Gonz] Fulton B. Gonzalez, A Paley-Wiener theorem for central functions on compactLie groups, Radon transforms and tomography (South Hadley, MA, 2000), Con-temp. Math., vol. 278, Amer. Math. Soc., Providence, RI, 2001, pp. 131–136, DOI10.1090/conm/278/04601. MR1851484 (2002f:43005)

[H-1] Sigurdur Helgason, Differential geometry, Lie groups, and symmetric spaces, Pure andApplied Mathematics, vol. 80, Academic Press Inc. [Harcourt Brace Jovanovich Pub-lishers], New York, 1978. MR514561 (80k:53081)

[H-2] Sigurdur Helgason, Groups and geometric analysis, Pure and Applied Mathematics,vol. 113, Academic Press Inc., Orlando, FL, 1984. Integral geometry, invariant differ-ential operators, and spherical functions. MR754767 (86c:22017)

[H-3] Sigurdur Helgason, Huygens’ principle for wave equations on symmetric spaces,J. Funct. Anal. 107 (1992), no. 2, 279–288, DOI 10.1016/0022-1236(92)90108-U.MR1172025 (93i:58151)

138 TOMOYUKI KAKEHI

[H-4] Sigurdur Helgason, Integral geometry and multitemporal wave equations, Comm.Pure Appl. Math. 51 (1998), no. 9-10, 1035–1071, DOI 10.1002/(SICI)1097-0312(199809/10)51:9/10¡1035::AID-CPA5¿3.3.CO;2-H. Dedicated to the memory ofFritz John. MR1632583 (99j:58207)

[H-5] Sigurdur Helgason : Integral Geometry and Radon Transforms. Springer, New York2010.

[H-Sch] S. Helgason and H. Schlichtkrull, The Paley-Wiener space for the multitemporal wave

equation, Comm. Pure Appl. Math. 52 (1999), no. 1, 49–52, DOI 10.1002/(SICI)1097-0312(199901)52:1¡49::AID-CPA2¿3.0.CO;2-S. MR1648417 (99j:58208)

[Hor-1] Lars Hormander, The analysis of linear partial differential operators. III, Grundlehrender Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sci-ences], vol. 274, Springer-Verlag, Berlin, 1994. Pseudo-differential operators; Correctedreprint of the 1985 original. MR1313500 (95h:35255)

[Hor-2] Lars Hormander, The analysis of linear partial differential operators. IV, Grundlehrender Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sci-ences], vol. 275, Springer-Verlag, Berlin, 1994. Fourier integral operators; Correctedreprint of the 1985 original. MR1481433 (98f:35002)

[K] Tomoyuki Kakehi, Support theorem for the fundamental solution to the Schrodingerequation on certain compact symmetric spaces, Adv. Math. 226 (2011), no. 3, 2739–2763, DOI 10.1016/j.aim.2010.10.003. MR2739792 (2011j:33022)

[Olaf-Sch] G. Olafsson and H. Schlichtkrull, Wave propagation on Riemannian symmetric spaces,J. Funct. Anal. 107 (1992), no. 2, 270–278, DOI 10.1016/0022-1236(92)90107-T.MR1172024 (93i:58150)

[Sol] L. E. Solomatina, Translation representation and Huygens’ principle for the invariantwave equation on a Riemannian symmetric space, Izv. Vyssh. Uchebn. Zaved. Mat. 6(1986), 72–74, 84 (Russian). MR865698 (87m:58168)

[W] Alan Weinstein, Asymptotics of eigenvalue clusters for the Laplacian plus a potential,Duke Math. J. 44 (1977), no. 4, 883–892. MR0482878 (58 #2919)

Department of Mathematics, Faculty of Science, Okayama University, Okayama,

700-8530, Japan



F -method for constructing equivariant differential operators

Toshiyuki Kobayashi

Dedicated to Professor Sigurdur Helgason for his 85th birthday.

Abstract. Using an algebraic Fourier transform of operators, we develop amethod (F -method) to obtain explicit highest weight vectors in the branch-ing laws by differential equations. This article gives a brief explanation ofthe F -method and its applications to a concrete construction of some natu-ral equivariant operators that arise in parabolic geometry and in automorphicforms.

Contents

1. Introduction2. Preliminaries3. A recipe of the F -methodReferences

1. Introduction

The aim of this article is to give a brief account of a method that helps usto find a closed formula of highest weight vectors in the branching laws of cer-tain generalized Verma modules, or equivalently, to construct explicitly equivariantdifferential operators from generalized flag varieties to subvarieties.

This method, which we call the F -method, transfers an algebraic problem offinding explicit highest weight vectors to an analytic problem of solving differentialequations (of second order) via the algebraic Fourier transform of operators (Def-inition 3.1). A part of the ideas of the F -method has grown in a detailed analysisof the Schrodinger model of the minimal representation of indefinite orthogonalgroups [8].

The F -method provides a conceptual understanding of some natural differen-tial operators which were previously found by a combinatorial approach based onrecurrence formulas. Typical examples that we have in mind are the Rankin–Cohen

2010 Mathematics Subject Classification. Primary 22E46; Secondary 53C35.Key words and phrases. Branching law, reductive Lie group, symmetric pair, parabolic ge-

ometry, Rankin–Cohen operator, Verma module, F -method, BGG category.Partially supported by IHES and Grant-in-Aid for Scientific Research (B) (22340026) JSPS.

c©2013 by the author

139

140 TOSHIYUKI KOBAYASHI

bidifferential operators

Rk1,k2n (f1, f2) =

n∑j=0

(−1)j(nj

)(k1 + n− 1)!(k2 + n− 1)!

(k1 + n− j − 1)!(k2 + j − 1)!

∂n−jf1∂xn−j

∂jf2∂yj

∣∣∣∣x=y

in automorphic form theory [2,3,11], and Juhl’s conformally equivariant operators[4] from C∞(Rn) to C∞(Rn−1):

Tλ,ν =∑

2j+k=ν−λ

1

2jj!(ν − λ− 2j)!

( ν−λ2 −j∏i=1

(λ+ ν − n− 1 + 2i)

)Δj

Rn−1

( ∂

∂xn

)k.

These examples can be reconstructed by the F -method by using a special caseof the fundamental differential operators , which are commuting family of secondorder differential operators on the isotropic cone, see [8, (1.1.3)].

In recent joint works with B. Ørsted, M. Pevzner, P. Somberg and V. Soucek [9,10], we have developed the F -method to more general settings, and have found newexplicit formulas of equivariant differential operators in parabolic geometry, and alsohave obtained a generalization of the Rankin–Cohen operators. To find those nicesettings where the F -method works well, we can apply the general theory [6,7] thatassures discretely decomposable and multiplicity-free restrictions of representationsto reductive subalgebras.

The author expresses his sincere gratitude to the organizers, J. Christensen, F.Gonzalez and T. Quinto, for their warm hospitality during the conference in honorof Professor Helgason for his 85th birthday in Boston 2012. The final manuscriptwas prepared when the author was visiting IHES.

2. Preliminaries

2.1. Induced modules. Let g be a Lie algebra over C, and U(g) its universalenveloping algebra. Suppose that h is a subalgebra of g and V is an h-module. Wedefine the induced U(g)-module by

indgh(V ) := U(g)⊗U(h) V.

If h is a Borel subalgebra and if dimC V = 1, then indgh(V ) is the standard Verma

module.

2.2. Extended notion of differential operators. We understand clearlythe notion of differential operators between two vector bundles over the same basemanifold. We extend this notion in a more general setting where there is a morphismbetween two base manifolds (see [10] for details).

Definition 2.1. Let VX → X and WY → Y be two vector bundles with asmooth map p : Y → X between the base manifolds. Denote by C∞(X,VX) andC∞(Y,WY ) the spaces of smooth sections to the vector bundles. We say that alinear map T : C∞(X,VX) → C∞(Y,WY ) is a differential operator if T is a localoperator in the sense that

(2.1) Supp(Tf) ⊂ p−1(Supp f), for any f ∈ C∞(X,V).

We write Diff(VX ,WY ) for the vector space of such differential operators.

F -METHOD FOR CONSTRUCTING EQUIVARIANT DIFFERENTIAL OPERATORS 141

Since any smooth map p : Y → X is given as the composition of a submersionand an embedding

Y ↪→ X × Y � X, y �→ (p(y), y) �→ p(y),

the following example describes the general situation.

Example 2.2. Let n be the dimension of X.

(1) Suppose p : Y � X is a submersion. Choose local coordinates {(xi, zj)}on Y such thatX is given locally by zj = 0. Then every T ∈ Diff(VX ,WY )is locally of the form ∑

α∈Nn

hα(x, z)∂|α|

∂xα,

where hα(x, z) are Hom(V,W )-valued smooth functions on Y .(2) Suppose i : Y ↪→ X is an embedding. Choose local coordinates {(yi, zj)}

onX such that Y is given locally by zj = 0. Then every T ∈ Diff(VX ,WY )is locally of the form ∑

(α,β)∈Nn

gαβ(y)∂|α|+|β|

∂yα∂zβ,

where gα,β(y) are Hom(V,W )-valued smooth functions on Y .

2.3. Equivariant differential operators. Let G be a real Lie group, g(R) =Lie(G) and g = g(R)⊗C. Analogous notations will be applied to other Lie groupsdenoted by uppercase Roman letters.

Let dR be the representation of U(g) on the space C∞(G) of smooth complex-valued functions on G generated by the Lie algebra action:

(2.2) (dR(A)f)(x) :=d

dt

∣∣∣t=0

f(xetA) for A ∈ g(R).

Let H be a closed subgroup of G. Given a finite dimensional representation Vof H we form a homogeneous vector bundle VX := G×H V over the homogeneousspace X := G/H. The space of smooth sections C∞(X,VX) can be seen as asubspace of C∞(G)⊗ V .

Let V ∨ be the (complex linear) dual space of V . Then the (G × g)-invariantbilinear map C∞(G) × U(g) → C∞(G), (f, u) �→ dR(u)f induces a commutativediagram of (G× g)-bilinear maps:

C∞(G)⊗ V × U(g)⊗C V ∨ −→ C∞(G)

↪→ � ‖C∞(X,VX)× indg

h(V ∨) −→ C∞(G).

In turn, we get the following natural g-homomorphism:

(2.3) indgh(V ∨) −→ HomG(C

∞(X,VX), C∞(G)).

Next, we take a connected closed subgroup H ′ of H. For a finite dimensionalrepresentation W of H ′ we form the homogeneous vector bundle WZ := G×H′ Wover Z := G/H ′. Taking the tensor product of (2.3) with W , and collecting allh′-invariant elements, we get an injective homomorphism:

(2.4) Homh′(W∨, indgh(V ∨)) −→ HomG(C

∞(X,VX), C∞(Z,WZ)), ϕ �→ Dϕ.


Finally, we take any closed subgroup G′ containing H ′ and form a homogeneousvector bundle WY := G′ ×H′ W over Y := G′/H ′. We note that WY is obtainedfrom WZ by restricting the base manifold Z to Y .

Let RZ→Y : C∞(Z,WZ)→ C∞(Y,WY ) be the restriction map. We set

(2.5) DX→Y (ϕ) := RX→Y ◦Dϕ.

Since there is a natural (G′-equivariant but not necessarily injective) morphismY → X, the extended notion of differential operators between VX and WY makessense (see Definition 2.1). We then have:

Theorem 2.3. The operator DX→Y (see (2.5)) induces a bijection:

DX→Y : Homh′(W∨, indgh(V ∨))

∼−→ DiffG′(VX ,WY ).(2.6)

Remark 2.4. We may consider a holomorphic version of Theorem 2.3 as fol-lows. Suppose GC, HC, G

′Cand H ′

Care connected complex Lie groups with Lie

algebras g, h, g′ and h′, and VXCand WYC

are homogeneous holomorphic vectorbundles over XC := GC/HC and YC := G′

C/H ′

C, respectively. Then Theorem 2.3

implies that we have a bijection:

(2.7) DXC→YC: Homh′(W∨, indg

h(V ∨))

∼−→ DiffholG′

C

(VXC,WYC

).

Here DiffholG′

C

denotes the space of G′C-equivariant holomorphic differential operators

with respect to the holomorphic map YC → XC. By the universality of the inducedmodule, (2.7) may be written as

(2.8) DXC→YC: Homg′(indg

′

h′(W∨), indg

h(V ∨))

∼−→ DiffholG′

C

(VXC,WYC

).

The isomorphism (2.8) is well-known when XC = YC is a complex flag variety.The proof of Theorem 2.3 is given in [10] in the generality that X �= Y .

2.4. Multiplicity-free branching laws. Theorem 2.3 says that ifHomh′(W∨, indg

h(V ∨)) is one-dimensional then G′-equivariant differential opera-

tors from VX toWY are unique up to scalar. Thus we may expect that such uniqueoperators should have a natural meaning and would be given by a reasonably simpleformula. Then we may be interested in finding systematically the examples whereHomh′(W∨, indg

h(V ∨)) is one-dimensional. This is a special case of the branching

problems that asks how representations decompose when restricted to subalgebras.In the setting where h is a parabolic subalgebra (to be denoted by p) of a reductiveLie algebra g, we have the following theorem:

Theorem 2.5. Assume the nilradical n+ of p is abelian and τ is an involutiveautomorphism of g such that τp = p. Then for any one-dimensional representationCλ of p and for any finite dimensional representation W of pτ := {X ∈ p : τX =X}, we have

dimHompτ (W∨, indgp(C∨λ)) ≤ 1.

There are two known approaches for the proof of Theorem 2.5. One is geometric— to use the general theory of the visible action on complex manifolds [5,6], andthe other is algebraic — to work inside the universal enveloping algebra [7].

Remark 2.6. Branching laws in the setting of Theorem 2.5 are explicitly ob-tained in terms of ‘relative strongly orthogonal roots’ on the level of the Grothen-dieck group, which becomes a direct sum decomposition when the parameter λ of


V is ‘generic’ or sufficiently positive, [6, Theorems 8.3 and 8.4] or [7]. The F -method will give a finer structure of branching laws by finding explicitly highestweight vectors with respective reductive subalgebras. The two prominent examplesin Introduction, i.e. the Rankin–Cohen bidifferential operators and the Juhl’s con-formally equivariant differential operators, can be interpreted in the framework ofthe F -method as a special case of Theorem 2.5.

3. A recipe of the F -method

The idea of the F -method is to work on the branching problem of represen-tations by taking the Fourier transform of the nilpotent radical. We shall explainthis method in the complex setting where HC is a parabolic subgroup PC withabelian unipotent radical (see Theorem 2.3 and Remark 2.4) for simplicity. A de-taild proof will be given in [10] (see also [9] for a somewhat different formulationand normalization).

3.1. Weyl algebra and algebraic Fourier transform. Let E be an n-dimensional vector space over C. The Weyl algebra D(E) is the ring of holomorphicdifferential operators on E with polynomial coefficients.

Definition 3.1 (algebraic Fourier transform). We define an isomorphism oftwo Weyl algebras on E and its dual space E∨:

(3.1) D(E)→ D(E∨), T �→ T ,

which is induced by

(3.2)∂

∂zj:= −ζj , zj :=

∂

∂ζj(1 ≤ j ≤ n),

where (z1, . . . , zn) are coordinates on E and (ζ1, . . . , ζn) are the dual coordinateson E∨.

Remark 3.2. (1) The isomorphism (3.1) is independent of the choice of coor-dinates.

(2) An alternative way to get the isomorphism (3.1) or its variant is to use theEuclidean Fourier transform F by choosing a real form E(R) of E. We then have

∂

∂z=√−1F ◦ ∂

∂x◦ F−1, z = −

√−1F ◦ z ◦ F−1

as operators acting on the space S ′(E∨) of Schwartz distributions. This was the

approach taken in [9]. In particular, T �= F◦T ◦F−1 in our normalization here. Theadvantage of our normalization (3.2) is that the commutative diagram in Theorem3.5 does not involve any power of

√−1 that would otherwise depend on the degrees

of differential operators. As a consequence, the final step of the F -method (see Step5 below) as well as actual computations becomes simpler.

3.2. Infinitesimal action on principal series. Let p = l + n+ be a Levidecomposition of a parabolic subalgebra of g, and g = n− + l + n+ the Gelfand–Naimark decomposition. Since the following map

n− × l× n+ → GC, (X,Z, Y ) �→ (expX)(expZ)(expY )


is a local diffeomorphism near the origin, we can define locally the projections p−and po from a neighbourhood of the identity to the first and second factors n− andl, respectively. Consider the following two maps:

α : g× n− → l, (Y,X) �→ d

dt

∣∣t=0

po(etY eX

),

β : g× n− → n−, (Y,X) �→ d

dt

∣∣t=0

p−(etY eX

).

We may regard β(Y, ·) as a vector field on n− by the identification β(Y,X) ∈ n− TXn−.

For l-module λ on V , we set μ := λ∨ ⊗ Λdimn+. Since Λdim n+n+ is one-dimensional, we can and do identify the representation space with V ∨. We inflateλ and μ to p-modules by letting n+ act trivially. Consider a Lie algebra homomor-phism

(3.3) dπμ : g→ D(n−)⊗ End(V ∨),

defined for F ∈ C∞(n−, V∨) as

(3.4) (dπμ(Y )F ) (X) := μ(α(Y,X))F (X)− (β(Y, · )F )(X).

If (μ, V ∨) lifts to the parabolic subgroup PC of a reductive group GC with Liealgebras p and g respectively, then dπμ is the differential representation of the

induced representation IndGC

PC(V ) (without ρ-shift). We note that the Lie algebra

homomorphism (3.4) is well-defined without integrality condition of μ. The F -method suggests to take the algebraic Fourier transform (3.1) on the Weyl algebraD(n−). We then get another Lie algebra homomorphism

(3.5) dπμ : g→ D(n+)⊗ End(V ∨).

Then we have (see [10])

Proposition 3.3. There is a natural isomorphism

Fc : indgp(λ

∨)∼−→ Pol(n+)⊗ V ∨

which intertwines the left g-action on U(g)⊗U(p) V∨ with dπμ.

3.3. Recipe of the F -method. Our goal is to find an explicit form of aG′-intertwining differential operator from VX to WY in the upper right corner ofDiagram 3.1. Equivalently, what we call the F -method yields an explicit homomor-

phism belonging to Homg′(indg′

p′(W∨), indgp(V∨)) Homp′(W∨, indgp(V

∨)) in thelower left corner of Diagram 3.1 in the setting that n+ is abelian.

The recipe of the F -method in this setting is stated as follows:

Step 0. Fix a finite dimensional representation (λ, V ) of p = l+ n+.Step 1. Consider a representation μ := λ∨ ⊗ Λdim n+n+ of the Lie algebra p.

Consider the restriction of the homomorphisms (3.3) and (3.5) to thesubalgebra n+:

dπμ : n+ → D(n−)⊗ End(V ∨),

dπμ : n+ → D(n+)⊗ End(V ∨).


Step 2. Take a finite dimensional representation W of the Lie algebra p′. Forthe existence of nontrivial solutions in Step 3 below, it is necessary andsufficient for W to satisfy

(3.6) Homg′(indg′

p′(W∨), indgp(V

∨)) �= {0}.

Choose W satisfying (3.6) if we know a priori an abstract branching lawof the restriction of indgp(V

∨) to g′. See [6, Theorems 8.3 and 8.4] or [7]for some general formulae. Otherwise, we take W to be any l′-irreduciblecomponent of S(n+)⊗ V ∨ and go to Step 3.

Step 3. Consider the system of partial differential equations for ψ ∈ Pol(n+) ⊗V ∨ ⊗W which is l′-invariant under the diagonal action:

dπμ(C)ψ = 0 forC ∈ n′+.(3.7)

Notice that the equations (3.7) are of second order. The solution spacewill be one-dimensional if we have chosen W in Step 2 such that

(3.8) dimHomg′(indg′

p′(W∨), indgp(V

∨)) = 1.

Step 4. Use invariant theory and reduce (3.7) to another system of differentialequations on a lower dimensional space S. Solve it.

Step 5. Let ψ be a polynomial solution to (3.7) obtained in Step 4. Compute(Symb⊗ Id)−1(ψ). Here the symbol map

Symb : Diffconst(n−)∼→ Pol(n+)

is a ring isomorphism given by the coordinates

C[∂

∂z1, · · · , ∂

∂zn]→ C[ξ1, · · · , ξn],

∂

∂zj�→ ξj .

In case the Lie algebra representation (λ, V ) lifts to a group PC, we form aGC-equivariant holomorphic vector bundle VXC

over XC = GC/PC. Likewise, incase W lifts to a group P ′

C, we form a G′

C-equivariant holomorphic vector bundle

WYCover YC = G′

C/P ′

C. Then (Symb⊗ Id)−1(ψ) in Step 5 gives an explicit formula

of a G′C-equivariant differential operator from VXC

to WYCin the coordinates of n−

owing to Theorem 3.5 below. This is what we wanted.

Remark 3.4. In Step 2 we can find all such W if we know a priori (abstract)explicit branching laws. This is the case, e.g., in the setting of Theorem 2.5. SeeRemark 2.6.

Conversely, the differential equations in Step 3 sometimes give a useful infor-mation on branching laws even when the restrictions are not completely reducible,see [9].

For concrete constructions of equivariant differential operators by using theF -method in various geometric settings, we refer to [9,10]. A further applicationof the F -method to the construction of non-local operators will be discussed inanother paper.

The key tool for the F-method is summarized as:

Theorem 3.5 ([10]). Let P ′C

be a parabolic subgroup of G′C

compatible witha parabolic subgroup PC of GC. Assume further the nilradical n+ of p is abelian.


Then the following diagram commutes:

HomC(W∨, indg

p(V∨)) � Pol(n+)⊗HomC(V,W )

Symb⊗Id∼←− Diffconst(n−)⊗ HomC(V,W )

∪ � ∪Homp′(W

∨, indgp(V

∨))DXC→YC−→ DiffG′

C(VXC

,WYC).

Diagram 3. 1

References

[1] Henri Cohen, Sums involving the values at negative integers of L-functions of quadraticcharacters, Math. Ann. 217 (1975), no. 3, 271–285. MR0382192 (52 #3080)

[2] Paula Beazley Cohen, Yuri Manin, and Don Zagier, Automorphic pseudodifferential operators,Algebraic aspects of integrable systems, Progr. Nonlinear Differential Equations Appl., vol. 26,Birkhauser Boston, Boston, MA, 1997, pp. 17–47. MR1418868 (98e:11054)

[3] Gerrit van Dijk and Michael Pevzner, Ring structures for holomorphic discrete series andRankin-Cohen brackets, J. Lie Theory 17 (2007), no. 2, 283–305. MR2325700 (2008e:11057)

[4] Andreas Juhl, Families of conformally covariant differential operators, Q-curvature andholography, Progress in Mathematics, vol. 275, Birkhauser Verlag, Basel, 2009. MR2521913(2010m:58048)

[5] Toshiyuki Kobayashi, Visible actions on symmetric spaces, Transform. Groups 12 (2007),no. 4, 671–694, DOI 10.1007/s00031-007-0057-4. MR2365440 (2008h:32026)

[6] Toshiyuki Kobayashi, Multiplicity-free theorems of the restrictions of unitary highest weightmodules with respect to reductive symmetric pairs, Representation theory and automor-phic forms, Progr. Math., vol. 255, Birkhauser Boston, Boston, MA, 2008, pp. 45–109, DOI10.1007/978-0-8176-4646-2 3. MR2369496 (2008m:22024)

[7] Toshiyuki Kobayashi, Restrictions of generalized Verma modules to symmetric pairs, Trans-form. Groups 17 (2012), no. 2, 523–546, DOI 10.1007/s00031-012-9180-y. MR2921076

[8] Toshiyuki Kobayashi and Gen Mano, The Schrodinger model for the minimal representationof the indefinite orthogonal group O(p, q), Mem. Amer. Math. Soc. 213 (2011), no. 1000,vi+132, DOI 10.1090/S0065-9266-2011-00592-7. MR2858535 (2012m:22016)

[9] T. Kobayashi, B. Ørsted, P. Somberg, V. Soucek, Branching laws for Verma modules andapplications in parabolic geometry, in preparation.

[10] T. Kobayashi, M. Pevzner, Rankin–Cohen operators for symmetric pairs, preprint,arXiv:1301.2111.

[11] R. A. Rankin, The construction of automorphic forms from the derivatives of a given form,J. Indian Math. Soc. (N.S.) 20 (1956), 103–116. MR0082563 (18,571c)

Kavli IPMU, and Graduate School of Mathematical Sciences, The University of

Tokyo



Schiffer’s conjecture, interior transmission eigenvalues andinvisibility cloaking: Singular problem vs. nonsingular

problem

Hongyu Liu

Abstract. In this note, we present some connections between Schiffer’s con-jecture, interior transmission eigenvalue problem and singular and non-singularinvisibility cloaking problems of acoustic waves.

1. Schiffer’s conjecture

Schiffer’s conjecture is a long standing problem in spectral theory, which isstated as follows:

Let Ω ⊂ R2 be a bounded domain. Does the existence of anontrivial solution u to the over-determined Neumann eigenvalueproblem

(1.1)

⎧⎪⎪⎨⎪⎪⎩−Δu = λu in Ω, λ ∈ R+,∂u

∂n= 0 on ∂Ω,

u = const on ∂Ω,

imply that Ω must be a ball?

The problem is equivalent to the Pompeiu’s problem in integral geometry. Adomain Ω ⊂ R2 is said to have the Pompeiu property iff the only continuous

function ϕ on R2 for which

∫σ(Ω)

ϕ(x, y) dxdy = 0 for every rigid motion σ of R2

is ϕ(x, y) = 0. It is shown in [B] that the failure of the Pompeiu property of adomain Ω is equivalent to the existence of a nontrivial solution to (1.1). We referto [Z] for an extensive survey on the current state of the problem.

For our subsequent discussion, we introduce the following theorem on the trans-formation invariance of the Laplace’s equation, whose proof could be found, e.g.,in [GKL,KOV,L].

1991 Mathematics Subject Classification. Primary 58J50, 35J05; Secondary 58C40, 35Q60 .Key words and phrases. Spectral geometry, Schiffer’s conjecture, Pompeiu property, interior

transmission eigenvalues, transformation optics, invisibility cloaking.The work is supported by NSF grant, DMS 1207784.


147

148 HONGYU LIU

Theorem 1.1. Let Ω and Ω be two bounded Lipschitz domains in RN and

suppose that there exists a diffeomorphism F : Ω→ Ω. Let u ∈ H1(Ω) satisfy

∇ · (g(x)∇u(x)) + λq(x)u(x) = f(x) x ∈ Ω,

where g(x) = (gij(x))2i,j=1, q(x), x ∈ Ω are uniformly elliptic and g is symmetric,

and f ∈ L2(Ω). Then one has that u = (F−1)∗u := u ◦ F−1 ∈ H1(Ω) satisfies

∇ · (g(y)∇u(y)) + λq(y)u(y) = f(y), y ∈ Ω,

where

F∗g(y) =DF (x) · g(x) · (DF (x))T

|det(DF (x))|

∣∣∣∣x=F−1(y)

,

F∗q(y) =q(x)

|det(DF (x))|

∣∣∣∣x=F−1(y)

, x ∈ Ω, y ∈ Ω,

(1.2)

and

f =

(f

| det(DF )|

)◦ F−1,

and DF denotes the Jacobian matrix of the transformation F .

Next, let u and Ω be the ones in (1.1) and suppose there exists a diffeomorphism

F : Ω→ Ω.

Further, we let

(1.3) g = F∗I and q = F∗1.

Then, according to Theorem 1.1, if one lets v = (F−1)∗u, we have

(1.4)

⎧⎪⎪⎪⎪⎨⎪⎪⎪⎪⎩∇ · (g∇v) + λqv = 0 in Ω,

2∑i,j=1

nigij∂jv = 0 on ∂Ω,

v = const on ∂Ω.

Hence, it is natural to propose the following generalized Schiffer’s conjecture:

Let (Ω; g, q) be uniformly elliptic and symmetric. Does the exis-tence of a nontrivial solution to

(1.5)

⎧⎪⎪⎪⎪⎨⎪⎪⎪⎪⎩∇ · (g∇u) + λqu = 0 in Ω, λ ∈ R+,

2∑i,j=1

nigij∂ju = 0 on ∂Ω,

u = const on ∂Ω.

imply that there must exist a diffeomorphism F such that

F : B → Ω, B is a ball,

and

g = F∗I and q = F∗1?

Definition 1.2. Let (Ω; g, q) be uniformly elliptic. Then, it is said to possessthe Pompeiu property if the corresponding over-determined system (1.5) has onlythe trivial solution.

SCHIFFER’S CONJECTURE 149

Later in Section 3, we shall see that the Pompeiu property of the parameters(Ω; g, p) would have important implications for invisibility cloaking in acoustics.

2. Interior transmission eigenvalue problem

Let (Ω; g1, q1) and (Ω; g2, q2) be uniformly elliptic and symmetric such that

(g1, q1) �= (g2, q2).

Consider the following interior transmission problem for a pair (u, v)

(2.1)

⎧⎪⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎪⎩

∇ · (g1∇u) + λq1u = 0 in Ω,

∇ · (g2∇v) + λq2v = 0 in Ω,

a11u+ a22v = 0 on ∂Ω,

a21

2∑i,j=1

nigij1 ∂ju+ a22

2∑i,j=1

nigij2 ∂jv = 0 on ∂Ω,

where A = (aij)2i,j=1 ∈ C(∂Ω). If there exists a nontrivial pair of solutions (u, v)

to (2.1), then λ is called a generalized interior transmission eigenvalue, and (u, v)are called generalized interior transmission eigenfunctions.

The interior transmission eigenvalue problem arises in inverse scattering theoryand has a long history in literature (cf. [CO,CP]). It was first introduced in [COL]in connection with an inverse scattering problem for the reduced wave equation.Later, it found important applications in inverse scattering theory, especially, forthe study of qualitative reconstruction schemes including linear sampling methodand factorization method (see, e.g., [KI]).

We would like to emphasize that the one (2.1) presented here is a generalizedformulation of the interior transmission eigenvalue problem that has been consid-ered in the literature.

3. Transformation optics and invisibility cloaking

In recent years, transformation optics and invisibility cloaking have receivedsignificant attentions; see, e.g., [C,GK,GKL,NO] and references therein. Thecrucial ingredient is the transformation invariance of the acoustic wave equations,which is actually Theorem 1.1. Let’s consider the well-known two-dimensionalcloaking problem for the time-harmonic acoustic wave governed by the Helmholtzequation (cf. [GLU,LE,PE])

(3.1) ∇ · (g∇u) + ω2qu = fχB1in B2,

where ω ∈ R+ is the wave frequency, g is symmetric matrix-valued denoting theacoustical density and q is the bulk modulus. In (3.1), the parameters are given as

(3.2) (g, q) =

{ga, qa in B1,

gc, qc in B2\B1,

and f ∈ L2(B1). In the physical situation, (gc, qc) is the cloaking medium which weshall specify later, and (ga, qa, fa) is the target object including a passive medium(ga, qa) and an active source or sink fa. The invisibility is understood in terms ofthe exterior measurement encoded either into the boundary Dirichlet-to-Neumann

150 HONGYU LIU

(DtN) map or the scattering amplitude far-away from the object. To ease ourexposition, we only consider the boundary DtN map. Let

(3.3) u = ϕ ∈ H1/2(∂B2)

and define the DtN operator by

Λc(ϕ) =

2∑i,j=1

nigijc ∂ju ∈ H−1/2(∂B2).

We also let Λ0 denote the “free” DtN operator on ∂B2, namely the DtN mapassociated with the Helmholtz equation (3.1) with g = I and q = 1 in B2. Next,we shall construct (gc, qc) to make

(3.4) Λc = Λ0.

To that end, we let

F (x) =

(1 +

|x|2

)x

|x| , x ∈ B2\{0}

It is verified that F maps B2\{0} to B2\B1. Let

gc(x) = F∗I =|x| − 1

|x| Π(x) +|x|

|x| − 1(I −Π(x)), x ∈ B2\B1,

and

qc(x) = F∗1 =4(|x| − 1)

|x| , x ∈ B2\B1,

where Π(x) : R2 → R2 is the projection to the radial direction, defined by

Π(x)ξ =

(ξ · x

|x|

)x

|x| ,

i.e., Π(x) is represented by the symmetric matrix |x|−2xxT . It can be seen that gcpossesses both degenerate and blow-up singularities at the cloaking interface ∂B1.Hence, one need to deal with the singular Helmholtz equation (3.1)–(3.3). It isnatural to consider physically meaningful solutions with finite energy. To that end,one could introduce the weighted Sobolev norm (cf. [LZH])

(3.5) ‖ψ‖g,q =

∣∣∣∣∫B2

(gij∂jψ∂iψ + ω2qψ2

)dx

∣∣∣∣1/2 .It is directly verified that for ψ ∈ E(B2)

‖ψ‖g,q <∞ iff∂ψ

∂θ

∣∣∣∣∂B1

= 0,

where E(B2) denotes the linear space of smooth functions in B2, and the standardpolar coordinate (x1, x2)→ (r cos θ, r sin θ) in R2 has been utilized. Hence, we set

T ∞(B2) :=

{ψ ∈ E(B2);

∂ψ

∂θ

∣∣∣∣∂B1

= 0

},

and

T ∞0 (B2) := T ∞(B2) ∩ D(B2),


which are closed subspaces of E(B2). Then, we introduce the finite energy solutionspace

(3.6) H1g,q(B2) := cl{T ∞(B2); ‖ · ‖g,q},

that is, the closure of the linear function space T ∞(B2) with respect to the singu-larly weighted Sobolev norm ‖ · ‖g,q.

(3.4) is justified in [LZH] by solving (3.1)–(3.3) in H1g,q(B2). Particularly, it is

shown that w = u|B1∈ H1(B1) is a solution to

(3.7)

⎧⎪⎪⎪⎨⎪⎪⎪⎩∇ · (ga∇w) + ω2qaw = fa in B1,

w|∂B1= const,∫

∂B1

∑i,j≤1

nigija ∂jw ds = 0.

In order to show the solvability of (3.7) by using Fredholm theory, it is necessaryto consider the following eigenvalue problem

(3.8)

⎧⎪⎪⎪⎨⎪⎪⎪⎩∇ · (ga∇w) + ω2qaw = 0 in B1,

w|∂B1= const,∫

∂B1

∑i,j≤1

nigija ∂jw ds = 0.

We let R(ga, qa) denote the set of solutions to (3.8). Clearly, (3.7) is uniquelysolvable iff fa ∈ R(ga, qa)

⊥. If (3.8) possess nontrivial solutions, then one cannotcloak a source f ∈ R(ga, qa) since in such a case, (3.7) has no finitely energysolution, and such a radiating source would break the cloaking. Next, we notethe close connection between the (3.8) and the generalized Schiffer’s conjecture(1.5). According to Definition 1.2, it is readily seen that if the target mediumparameters (B1, ga, qa) fails to have the Pompeiu property, then (3.8) has nontrivialsolutions. That is, one would encounter the interior resonance problem for theinvisibility cloaking discussed above. But we would also like to note that the system(3.8) arising from invisibility cloaking is still formally determined, whereas (1.5) ofthe generalized Schiffer’s conjecture is over-determined. The connection discussedabove between the singular cloaking problem and Schiffer’s conjecture is naturalwhen one is curious in whether and how such spherical cloaking design is generalizedto other shaped cloaked domain.

In order to avoid the singular structure of the perfect cloaking, it is natural toincorporate regularization into the context and consider the corresponding approx-imate cloaking. Let

Fε(x) =

(2(1− ε)

2− ε+|x|

2− ε

)x

|x| , ε ∈ R+,

which maps B2\Bε → B2\B1 and let

(3.9) gc = (Fε)∗I and qc = (Fε)∗1.

Now, we consider the nonsingular cloaking problem (3.1)–(3.3) with (gc, qc) given in(3.9). Let uε denote the corresponding solution in this case. The limiting behaviorof uε as ε → 0+ was considered in [LZ,NG]. In order to justify the near-cloak

152 HONGYU LIU

of the regularized construction, the following eigenvalue problem arose from thecorresponding study,

(3.10)

⎧⎪⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎪⎩

Δw = 0 in R2\B1,

∇ · (ga∇w) + ω2qaw = 0 in B1,

w+ = w− on ∂B1,∂w+

∂n=∑i,j≤2

nigija ∂jw

− on ∂B1.

It is shown in [NG] that if (3.10) has only trivial solution, then uε|R2\B1converges

to the “free-space” solution, whereas uε|B1converges to u implied in

(3.11)

⎧⎪⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎪⎩

Δw = 0 in R2\B1,

∇ · (ga∇w) + ω2qaw = fa in B1,

w+ = w− on ∂B1,∂w+

∂n=∑i,j≤2

nigija ∂jw

− on ∂B1.

Otherwise, if (3.10) has nontrivial solutions forming the set W , then it is shown thatif fa ∈ W , then the cloaking fails as ε → 0+. Now, let’s take a more careful lookat the eigenvalue problem (3.10). By letting z = x + iy and using the conformalmapping F : z → 1/z, we set v = w+ ◦ F with w+ = w|

R2\B1. One can verify

directly that w|B1∈ H1(B1) and v ∈ H1(B1) satisfy the following generalized

interior transmission eigenvalue problem

(3.12)

⎧⎪⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎪⎩

Δv = 0 in B1,

∇ · (ga∇w) + ω2qaw = 0 in B1,

v − w = 0 on ∂B1,∂v

∂n+∑i,j≤2

nigija ∂jw = 0 on ∂B1.

From our earlier discussions in this section, one can see that the invisibilitycloaking construction is very unstable, especially when there is interior resonanceproblem. Hence, in order to overcome this instability, cloaking schemes by incor-porating some damping mechanism through the adding of a lossy layer betweenthe cloaked and cloaking regions have been introduced and studied recently, see[KOV,LLS,LS]. But it is interesting to further note that for cloaking design witha lossy layer, one may encounter eigenvalues in the complex plan, i.e., poles. Infact, it is numerically observed in [LZU] for the cloaking of electromagnetic waves,the cloak effect will be deteriorated when the frequency is close to the poles.

Acknowledgement

The author would like to thank the anonymous referee for many insightful andconstructive comments.

References

[B] Leon Brown, Bertram M. Schreiber, and B. Alan Taylor, Spectral synthesis and the Pom-peiu problem, Ann. Inst. Fourier (Grenoble) 23 (1973), no. 3, 125–154 (English, with Frenchsummary). MR0352492 (50 #4979)


[C] Chen, H. and Chan, C. T., Acoustic cloaking and transformation acoustics, J. Phys. D:Appl. Phys., 43 (2010), 113001.

[CO] David Colton and Rainer Kress, Inverse acoustic and electromagnetic scattering the-ory, 2nd ed., Applied Mathematical Sciences, vol. 93, Springer-Verlag, Berlin, 1998.MR1635980 (99c:35181)

[COL] David Colton and Peter Monk, The inverse scattering problem for time-harmonic acousticwaves in an inhomogeneous medium, Quart. J. Mech. Appl. Math. 41 (1988), no. 1, 97–

125, DOI 10.1093/qjmam/41.1.97. MR934695 (89i:76080)[CP] David Colton, Lassi Paivarinta, and John Sylvester, The interior transmission problem,

Inverse Probl. Imaging 1 (2007), no. 1, 13–28, DOI 10.3934/ipi.2007.1.13. MR2262743(2008j:35027)

[GK] Allan Greenleaf, Yaroslav Kurylev, Matti Lassas, and Gunther Uhlmann, Invisibilityand inverse problems, Bull. Amer. Math. Soc. (N.S.) 46 (2009), no. 1, 55–97, DOI10.1090/S0273-0979-08-01232-9. MR2457072 (2010d:35399)

[GKL] Allan Greenleaf, Yaroslav Kurylev, Matti Lassas, and Gunther Uhlmann, Cloaking devices,electromagnetic wormholes, and transformation optics, SIAM Rev. 51 (2009), no. 1, 3–33,DOI 10.1137/080716827. MR2481110 (2010b:35484)

[GLU] Allan Greenleaf, Matti Lassas, and Gunther Uhlmann, On nonuniqueness for Calderon’sinverse problem, Math. Res. Lett. 10 (2003), no. 5-6, 685–693. MR2024725 (2005f:35316)

[KOV] Robert V. Kohn, Daniel Onofrei, Michael S. Vogelius, and Michael I. Weinstein, Cloakingvia change of variables for the Helmholtz equation, Comm. Pure Appl. Math. 63 (2010),no. 8, 973–1016, DOI 10.1002/cpa.20326. MR2642383 (2011j:78004)

[KI] Andreas Kirsch and Natalia Grinberg, The factorization method for inverse problems,Oxford Lecture Series in Mathematics and its Applications, vol. 36, Oxford UniversityPress, Oxford, 2008. MR2378253 (2009k:35322)

[LZ] Matti Lassas and Ting Zhou, Two dimensional invisibility cloaking for Helmholtz equa-tion and non-local boundary conditions, Math. Res. Lett. 18 (2011), no. 3, 473–488.MR2802581 (2012d:35062)

[LE] Ulf Leonhardt, Optical conformal mapping, Science 312 (2006), no. 5781, 1777–1780, DOI10.1126/science.1126493. MR2237569

[LLS] Jingzhi Li, Hongyu Liu, and Hongpeng Sun, Enhanced approximate cloaking by SHand FSH lining, Inverse Problems 28 (2012), no. 7, 075011, 21, DOI 10.1088/0266-5611/28/7/075011. MR2946799

[L] Hongyu Liu, Virtual reshaping and invisibility in obstacle scattering, Inverse Prob-lems 25 (2009), no. 4, 045006, 16, DOI 10.1088/0266-5611/25/4/045006. MR2482157(2010d:35044)

[LS] Hongyu Liu and Hongpeng Sun, Enhanced near-cloak by FSH lining, J. Math. Pures Appl.(9) 99 (2013), no. 1, 17–42, DOI 10.1016/j.matpur.2012.06.001. MR3003281

[LZH] Hongyu Liu and Ting Zhou, Two dimensional invisibility cloaking via transformation op-tics, Discrete Contin. Dyn. Syst. 31 (2011), no. 2, 525–543, DOI 10.3934/dcds.2011.31.525.MR2805818

[LZU] Hongyu Liu and Ting Zhou, On approximate electromagnetic cloaking by transforma-tion media, SIAM J. Appl. Math. 71 (2011), no. 1, 218–241, DOI 10.1137/10081112X.MR2776835 (2012f:78005)

[NG] Hoai-Minh Nguyen, Approximate cloaking for the Helmholtz equation via transformationoptics and consequences for perfect cloaking, Comm. Pure Appl. Math. 65 (2012), no. 2,155–186, DOI 10.1002/cpa.20392. MR2855543

[NO] Andrew N. Norris, Acoustic cloaking theory, Proc. R. Soc. Lond. Ser. A Math. Phys.Eng. Sci. 464 (2008), no. 2097, 2411–2434, DOI 10.1098/rspa.2008.0076. MR2429553(2009k:74044)

[PE] J. B. Pendry, D. Schurig, and D. R. Smith, Controlling electromagnetic fields, Science 312(2006), no. 5781, 1780–1782, DOI 10.1126/science.1125907. MR2237570

[Z] L. Zalcman, A bibliographic survey of the Pompeiu problem, (Hanstholm, 1991), NATOAdv. Sci. Inst. Ser. C Math. Phys. Sci., vol. 365, Kluwer Acad. Publ., Dordrecht, 1992,pp. 185–194. MR1168719 (93e:26001)

154 HONGYU LIU

Department of Mathematics and Statistics, University of North Carolina, Char-

lotte, North Carolina 28223



Approximate Reconstruction from Circular and SphericalMean Radon Transform Data

W. R. Madych

Abstract. Circular and spherical mean Radon transforms are important anduseful in pure mathematics and practical applications. The objective of thisarticle is to bring attention to a natural method for obtaining approximateinversions of these transforms that do not rely on any knowledge of or formulafor an exact inverse. Limiting cases of these approximations give rise to exactinversion procedures.

1. Introduction

The spherical mean Radon transform of a sufficiently regular scalar valuedfunction f on Rn, n ≥ 2, evaluated at ξ and r where ξ ∈ Rn and r > 0, is theaverage of f over the sphere of radius r centered at ξ and is denoted by Mf(ξ, r).It can be expressed as the following integral over the unit sphere in Rn

(1) Mf(ξ, r) =

∫Sn−1

f(ξ + ru)dσ(u)

where dσ(u) denotes the usual rotation invariant surface measure on the unit sphereSn−1 = {u ∈ Rn : |u| = 1}. In the special case n = 2 it is sometimes referred to asthe circular mean Radon transform.

Mf(ξ, r) is used to model the data in various models of thermoacoustic andphotoacoustic tomography where f represents the phantom and ξ the position of adetector. For background material in thermoacoustic and photoacoustic tomogra-phy see the handbook article [10] which also provides a comprehensive survey andextensive list of references.

The transformation f(x)→Mf(ξ, r) plays a significant role in the study of thewave equation and gives rise to many interesting mathematical questions. Severalsuch questions were addressed by Peter Kuchment in his plenary talk at the Janu-ary 2012 Tufts Workshop on Geometric Analysis on Euclidean and HomogeneousSpaces, see also [10] and the relevant references cited there. The introduction to[6] includes a succinct summary of such work prior to 2004. The forthcoming book[1] should provide additional information.

2010 Mathematics Subject Classification. Primary 40C10, 41A35, 44A12, 45Q05, 65R32,92C55.


155

156 W. R. MADYCH

2. Main subject matter

The objective of this article is to highlight a natural method for obtaining anapproximation to the phantom f(x) in terms of the available spherical mean dataMf(ξ, r).

If the nature of the data permits, an approximation of f(x) can be obtainedby regularizing known exact inversion formulas, for example see [4–9]. Such anapproach is not the subject of this article. On the contrary, limiting cases of theapproximation method under consideration here can lead to various exact inversiontype formulas, see Section 9.

We make use of the fact that if a sufficiently regular function g(x) dependsonly on the distance of x to some fixed point ξ in Rn, say g(x) = h(|x− ξ|), thenusing polar coordinates centered at ξ, x − ξ = ru where |u| = 1 and r = |x − ξ|,∫g(x)f(x)dx can be evaluated in terms of Mf(ξ, r), namely∫

Rn

g(x)f(x)dx =

∫Rn

h(|x− ξ|)f(ξ + x− ξ)dx

=

∫ ∞

0

∫Sn−1

h(r)f(ξ + ru)dσ(u) rn−1dr

=

∫ ∞

0

h(r)Mf(ξ, r)rn−1dr .

(2)

3. Summability kernels

3.1. Definition. Suppose Ω is a region in Rn. We say that Kε(x, y), ε > 0,(x, y) ∈ Ω× Ω, is a summability kernel on Ω if for each x in Ω

(s1) as a function of y the kernel Kε(x, y) is an integrable function on Ω for allpositive ε, and

(s2) limε→0

∫Ω

Kε(x, y)f(y)dy = f(x) for any bounded measurable function f that

vanishes outside a compact subset of Ω and is continuous at x.

3.2. Examples. Sufficient conditions for Kε(x, y) to be a summability kernelon Ω are the following: For each x in Ω(i) limε→0

∫ΩKε(x, y)dy = 1,

(ii)∫Ω|Kε(x, y)|dy ≤ C <∞ where C is a constant independent of ε, and

(iii) limε→0

∫{|x−y|>δ}∩Ω

|Kε(x, y)|dy = 0 for each positive δ.

Summability kernels are widely used in analysis and its applications. For con-venience, we recall two particularly well-known general formulations:

3.2.1. Orthogonal expansions. Summability kernels on Ω can be constructedfrom certain complete orthonormal systems {φn(x)}∞n=1 on Ω via

Kε(x, y) =∞∑n=1

λn(ε)φn(x)φn(y)

where the λn(ε) are appropriate constants such that limε→0 λn(ε) → 1. In manycases 0 ≤ λn(ε) ≤ 1 and only a finite number are non-zero when ε > 0.

3.2.2. Convolution kernels. If g(x) is an integrable function on Rn such that∫Rn g(x)dx = 1 then the convolution type kernel

Kε(x, y) =1

εng(x− y

ε

)

APPROXIMATE RECONSTRUCTION 157

is a summability kernel for any region Ω in Rn. We will make use of certain variantsof such kernels in what follows.

4. Reconstruction via summability

It follows from (2) that if f is sufficiently regular and Kε(x, y) is a summabilitykernel that as a function of y is expressible as a sum of radial functions whosecenters are in the measurable subset Ξ of Rn then

∫ΩKε(x, y)f(y)dy is computable

in terms of the spherical mean data Mf(ξ, r), ξ in Ξ and 0 < r < ∞ and, forsufficiently small ε, should be a good approximation of f(x). Namely, if Kε(x, y) isa summability kernel and

(3) Kε(x, y) =

∫Ξ

kε(x, ξ, |y − ξ|)dμ(ξ)

where kε(x, ξ, r) is a sufficiently regular function of all its variables and dμ(ξ) is anappropriate measure on Ξ then

(4)

∫Ω

Kε(x, y)f(y)dy =

∫Ξ

∫ ∞

0

kε(x, ξ, r)Mf(ξ, r)rn−1dr dμ(ξ)

is a good approximation of f(x) for sufficiently small ε.Thus, to obtain approximations of f(x) in terms of the spherical mean data

Mf(ξ, r), ξ ∈ Ξ and 0 < r < ∞, it remains to find summability kernels Kε(x, y)that, as function of the variable y, are expressible as sums of radial functions whosecenters are in Ξ, as described by (3).

5. First setup

Determining whether a given function is expressible as a sum of radial functionswith prescribed centers is, in general, not easy. It is related to the question ofinvertibility of the corresponding spherical mean transform, a question that wasaddressed by Peter Kuchment in his plenary talk at the Tufts Workshop mentionedin the introduction. Finding appropriate radial functions to do the job seems to bean even more daunting task.

To overcome this hurdle we reverse the process. That is to say, we select radialfunctions kε(x, ξ, |y − ξ|) with the goal of producing good summability kernels via(3). Our selections are based on analogous constructions that are valid for classicalRadon transform data [11].

Although our considerations are valid in far more general situations, for thesake of clarity, here we restrict our attention to the case when Ω = {x : |x| < 1}is the disk or ball centered at the origin and the set of centers Ξ is it’s boundarySn−1 = {x : |x| = 1}. This is also the case that seems to have attracted the mostattention in practical applications.

In this case we consider kernels of the form (3) where dμ(ξ) is the usual rotationinvariant surface measure dσ(ξ) on Sn−1 and

(5) kε(x, ξ, r) =1

εnh( |x− ξ| − r

ε

)where, in view of the development in [11], h(t) satisfies the following properties:

(6) h(t) is a locally integrable univariate function that is even and such that

158 W. R. MADYCH

H(x) =

∫Sn−1

h(〈x, ξ〉)dσ(ξ) is an integrable function of x

with total integral one, i. e.(7)

(8) H(x) = H0(|x|) with

∫Rn

H(x)dx = cn

∫ ∞

0

H0(r)rn−1dr = 1.

Here the constant cn is the surface area of Sn−1 and 〈x, ξ〉 denotes the inner productof x and ξ in Rn.

In summary, we consider kernels of the form

(9) Kε(x, y) =

∫Sn−1

1

εnh

(∣∣∣x− ξ

ε

∣∣∣− ∣∣∣y − ξ

ε

∣∣∣)dσ(ξ)where h(t) satisfies (6) and (7).

5.1. Rationale. The motivation for this choice of k is based on the fact that

(10) limρ→∞

{|y − ρξ| − |x− ρξ|

}= 〈x− y, ξ/|ξ|〉.

Identity (10) suggests that for x and y in the interior of the unit ball and not tooclose to the boundary Kε(x, y) should behave like ε−nH

((x − y)/ε

), which is a

summability kernel of convolution type, when ε is sufficiently small.

5.2. Examples. If h(t) and H0(r) are related via (6), (7), and (8) then H0(r)is uniquely determined by h(t) and vice versa. Indeed, the Funk-Hecke formulaleads to

H0(r) =c1

rn−2

∫ r

0

h(t)(r2 − t2)(n−3)/2dt

which can be inverted by standard integral equation techniques to yield

h(t) = c2t

(1

2t

d

dt

)n−1 ∫ t

0

H0(r)(t2 − r2)(n−3)/2rn−1dr.

Here c1 and c2 are constants that depend only on n. For more details see [11].As specific examples in the case n = 2 we have

H(x) =

{1/π if |x| ≤ 1

0 otherwisewith h(t) =

{1/(2π2) if |t| ≤ 11

2π2

{1− |t|√

t2−1

}otherwise

and

(11) H(x) =1

2π

1

(1 + |x|2)3/2 with h(t) =1

4π2

1− t2

(1 + t2)2.

Specific examples in the case n = 3 include

(12) H(x) =4√πe−|x|2 with h(t) =

1

π3/2{1− 2t2}e−t2

and the analogue of (11)

H(x) =1

π2

1

(1 + |x|2)2 with h(t) =1

4π3

1− 3t2

(1 + t2)3.


6. Consequences

With the choice of kernel (3) and (5) we may write∫Ω

Kε(x, y)f(y)dy

=

∫Ω

{∫Sn−1

1

εnh

(∣∣∣x− ξ

ε

∣∣∣− ∣∣∣y − ξ

ε

∣∣∣)dσ(ξ)}f(y)dy=

∫Ωx,ε

{∫Sn−1

h

(∣∣∣x− ξ

ε

∣∣∣− ∣∣∣z + x− ξ

ε

∣∣∣)dσ(ξ)}f(x+ εz)dz

=

∫Rn

{χx,ε(z)

∫Sn−1

h

(∣∣∣x− ξ

ε

∣∣∣− ∣∣∣z + x− ξ

ε

∣∣∣)dσ(ξ)}f(x+ εz)dz

(13)

where we used the change of variables z = y−xε to pass from the first identity to the

second and where χx,ε(z) is the indicator function of the set Ωx,ε = {z : |z+x/ε| <1/ε}, that is

χx,ε(z) =

{1 if |(z + x/ε)| < 1/ε

0 otherwise.

Now, if the last expression in braces is bounded by an integrable function, namelyif ∣∣∣∣χx,ε(z)

∫Sn−1

h

(∣∣∣x− ξ

ε

∣∣∣− ∣∣∣z + x− ξ

ε

∣∣∣)dσ(ξ)∣∣∣∣ ≤ g(z)

where g(z) is independent of ε and is integrable over Rn,

(14)

and f is continuous at x then the dominated convergence theorem implies that(15)

limε→0

∫Rn

{χx,ε(z)

∫Sn−1

h

(∣∣∣x− ξ

ε

∣∣∣− ∣∣∣z + x− ξ

ε

∣∣∣)dσ(ξ)}f(x+ εz)dz = δ(x)f(x)

where

(16) δ(x) =

∫Rn

{∫Sn−1

h

(⟨z,

x− ξ

|x− ξ|⟩)

dσ(ξ)

}dz.

The expression for δ(x) follows from

∣∣∣z + x− ξ

ε

∣∣∣− ∣∣∣x− ξ

ε

∣∣∣ =∣∣∣z + x−ξ

ε

∣∣∣2 − ∣∣∣x−ξε

∣∣∣2∣∣∣z + x−ξε

∣∣∣+ ∣∣∣x−ξε

∣∣∣ =

⟨z, εz + 2(x− ξ)

⟩|εz + x− ξ|+ |x− ξ|

and evaluating the limit as ε→ 0.If x = 0 then in view of (7) and (8) we may conclude that δ(x) = 1.If |x| < 1 but x �= 0 then assuming (14) we may conclude that the expression

in braces in (16) is integrable over Rn and use the polar coordinates z = ru, set

v(ξ) = x−ξ|x−ξ| , interchange order of integration, and write

δ(x) =

∫Sn−1

∫ ∞

0

{∫Sn−1

h(〈ru, v(ξ)〉)dσ(u)}rn−1drdσ(ξ)

=

∫Sn−1

∫ ∞

0

{∫Sn−1

h(〈rv(ξ), u〉)dσ(u)}rn−1drdσ(ξ)

=

∫Sn−1

∫ ∞

0

H0(r)rn−1drdσ(ξ) = 1.

160 W. R. MADYCH

The function Kε(x, y) can be viewed as the point or impulse response functionof the approximate reconstruction formula (4). The term impulse response functionis often used in the engineering literature to describe the output of an algorithmwhen data corresponding to an ideal Dirac measure is used as input. When h(t) isas in (11) the plots below suggest that, for sufficiently small ε, the correspondingKε(x, y) is a reasonably good impulse response function.

Figure 1 contains plots of z = Kε(x, y) as a function of y for, clockwise from topleft, x = (0, 0), − 1

2√2(1, 1), − 3

4√2(1, 1), and − 7

8√2(1, 1). Kε(x, y) is defined by (9)

with ε = 0.0625 and h(t) as in (11). The integral in (9) was evaluated numericallyby the trapezoid rule. Note the increased distortions from ε−2H

((x − y)/ε

)as x

approaches the boundary. All numerical work and graphics was done with Matlab.

Figure 1.

Numerical experiments indicate that in the cases n = 2 and n = 3 the choices(11) and (12) give rise to very effective summability kernels on the unit ball Ω notonly when Ξ = Sn−2 but for much more general sets Ξ that surround Ω. Some ofthese numerical experiments are documented in [3] and involve the reconstructionof piecewise constant phantoms f(x). However, (14) has not yet been verified. Inthe mean time an algebraically more tractable variation of the form (5) has beenconsidered, see Section 7.

6.1. δ(x) when n = 2. The fact that δ(x) = 1 can be verified in the casen = 2 without the a priori assumption (14).


In this case let u(θ) = (cos θ, sin θ) so that∫S1

h

(⟨z,

x− ξ

|x− ξ|

⟩)dσ(ξ) =

∫ 2π

0

h

(⟨z,

x− u(θ)

|x− u(θ)|

⟩)dθ

where, by a shift in the variable θ if necessary, we may and do assume that〈x, u(0)〉 = |x|. If φ is such that

u(φ) =x− u(θ)

|x− u(θ)|

then

θ = φ+ arcsin(|x| sinφ).The last identity can be verified by sketching two circles, centered at the origin andat x respectively, and observing that

sinφ

cosφ= tanφ =

sin θ

|x|+ cos θ.

This leads to

sin θ cosφ− cos θ sinφ = |x| sinφ,

sin(θ − φ) = |x| sinφ,and thus the desired identity.

Hence∫ 2π

0

h

(⟨z,

x− u(θ)

|x− u(θ)|

⟩)dθ =

∫ 2π

0

h(〈z, u(φ)〉)dθ

=

∫ 2π

0

h(〈z, u(φ)〉)(1 +

|x| cosφ√1− (|x| sinφ)2

)dφ.

Since h(t) is an even function of t we may conclude that

h(〈z, u(φ)〉) = h(〈z, u(φ+ π)〉),∫ 2π

π

h(〈z, u(φ)〉) |x| cosφ√1− (|x| sinφ)2

dφ

=

∫ π

0

h(〈z, u(φ+ π)〉) |x| cos(φ+ π)√1− (|x| sin(φ+ π))2

dφ

= −∫ π

0

h(〈z, u(φ)〉) |x| cosφ√1− (|x| sinφ)2

dφ

and ∫ 2π

0

h(〈z, u(φ)〉) |x| cosφ√1− (|x| sinφ)2

dφ = 0.

It follows that

δ(x) =

∫R2

∫ 2π

0

h((z, u(φ))dφdx =

∫R2

H(z)dz = 1.

162 W. R. MADYCH

7. Second setup

Using the same basic scenario as in Sections 5 and 6 where Ω = {x : |x| < 1}is the disk or ball centered at the origin and the set of centers Ξ is its boundarySn−1 = {x : |x| = 1} consider the following modification of (5):

kε(x, ξ, r) =γ(x)

εnh( |x− ξ|2 − r2

2ε

)=

γ(x)

εnh

(⟨y − x

ε, ξ − x+ y

2

⟩)if r = |y − ξ|,

(17)

that leads to the kernel

Kε(x, y) =

∫Ξ

kε(x, ξ, |y − ξ|)dμ(ξ)

=

∫Sn−1

γ(x)

εnh

(|x− ξ|2 − |y − ξ|2

2ε

)dσ(ξ)

=

∫Sn−1

γ(x)

εnh

(⟨y − x

ε, ξ − x+ y

2

⟩)dσ(ξ).

(18)

Here, as earlier, h(t) is a function that enjoys properties (6) and (7). The variableγ(x) is chosen so that

limε→0

∫Ω

Kε(x, y)dy = 1 for all x in Ω.

The rationale for this particular form for kε(x, ξ, r) is that it is sufficiently closeto (5) so that the corresponding kernel Kε(x, y) should behave in a manner similarto that which resulted in the earlier choice but the analytic nature of the argumentof h gives rise to an expression that might be easier to work with algebraically.

8. Consequences

The same reasoning and computations, mutatis mutandis, as those in Section6 give rise to the analogous conclusion. Namely if∣∣∣∣χx,ε(z)

∫Sn−1

h(〈z, ξ − x− ε2z〉)dσ(ξ)

∣∣∣∣ ≤ g(z)

where g(z) is independent of ε and is integrable over Rn,

(19)

then the kernel defined by (18) is a summability kernel for Ω.Note that (19) is the analogue of (14) for the modification (18).In the case n = 3 this modification is sufficient to allow for the verification of

(19). Namely, the Funk-Hecke formula gives rise to an expression for the integralin (19) that can be estimated and leads to both the verification of (19) and theevaluation of γ(x). Thus the kernel Kε(x, y) defined by (18) is a summabilitykernel for Ω whenever h(t) is a function that enjoys properties (6), (7), (8) andγ(x) = c(1− |x|2), where c is a constant. Details can be found in [2].

When n �= 3 the Funk-Hecke formula is not so convenient. Nevertheless inthe case when n = 2 and h(t) is an analytic function the integral in (19) can beexplicitly evaluated using complex contour integration and the residue theorem.This has been carried out for the function of example (11), namely when h(t) is the


analytic function

h(t) =1

4π2

1− t2

(1 + t2)2.

The result leads to a verification of (19) and an evaluation of γ(x). Here againγ(x) = c(1− |x|2), where c is a constant. The details can be found in [2].

The left hand side of Figure 2 shows a plot of z = Kε(x, y) as a functionof y defined by (9), while the right hand side shows a plot of z = Kε(x, y)/γ(x)where Kε(x, y) is defined by (18). In both cases ε = 0.0625, h(t) is as in (11), andx = − 7

8√2(1, 1). Both integrals were evaluated numerically by the trapezoid rule.

The point x was chosen relatively close to the boundary to highlight the increaseddistortion from ε−2H

((x− y)/ε

)in the plot on the right, that is in part due to the

deletion of the factor γ(x).

Figure 2.

9. Inversion type formulas as corollaries

By passing to the limit as ε→ 0 the results mentioned in Section 8 can lead tovarious inversion type formulas.

For example, in the case n = 2 the following two corollaries were detailed in[2]:

Corollary 1. If x is in Ω, and f is in C2(Ω) then

(20)f(x)

(1− |x|2)

=−1π2

∫S1

{1

2

∫ √2a

0

{M(r) +M

(√2a2 − r2

)− 2M(a)

} r(r2 − a2

)2 dr+

∫ ∞

√2a

M(r)r(

r2 − a2)2 dr − M(a)

a2

}dσ(ξ)

where M(r) =Mf(ξ, r) and a = |x− ξ|.

164 W. R. MADYCH

Corollary 2. If x is in Ω, and f is in C2(Ω) then

(21)f(x)

1− |x|2 = c

∫S1

{∫ ∞

0

log(|r2 − a2|

) ddr

(1

2r

dMf(ξ, r)

dr

)dr

}dσ(ξ)

where a = |x− ξ| and c is a constant independent of f .

Identities (20) and (21) can be compared to the inversion formulas found else-where, for example [1,4,8–10]. Note that (21) is similar to but not identical with[4, formula (4)].

In the case n = 3 we can show the following:

Corollary 3. Suppose x is in Ω, f is in C2(Ω), and F (ξ, r)=d

dr

1

r

d

drrMf(ξ, r).

Then

(22) f(x) = c(1− |x|2)∫S2

F (ξ, |x− ξ|)|x− ξ| dσ(ξ)

where c is a constant independent of f .

Note that (22) is similar to but not same as the n = 3 cases of the inversionformulas [6, Theorem 3].

9.1. Proof of Corollary 3. Use the function h(t) of example (12) and thefact that

(23) f(x) = limε→0

∫S2

∫ ∞

0

γ(x)

ε3h( |x− ξ|2 − r2

2ε

)Mf(ξ, r)r2drdσ(ξ)

with

γ(x) = c1(1− |x|2) and h(t) = c2{1− 2t2}e−t2

where c1 and c2 are positive constants. Next note that if g(t) is the Gaussian,

g(t) = c2e−t2 ,

then

g′′(t) = −2h(t)and

1

ε3h( |x− ξ|2 − r2

2ε

)= − 1

2ε

(1r

d

dr

)2g( |x− ξ|2 − r2

2ε

).

To simplify the notation use the abbreviation a = |x− ξ| and write∫ ∞

0

1

ε3h(a2 − r2

2ε

)Mf(ξ, r)r2drdσ(ξ) = −

∫ ∞

0

1

2εg(a2 − r2

2ε

)F (ξ, r)dr

= −∫ ∞

−a2

2ε

g(s)F (ξ,√2εs+ a2)

ds

2√2εs+ a2

= −∫ ∞

−∞g(s)F (ξ, a)

ds

2awhen ε→ 0.

(24)

The first equality follows from integration by parts twice, the second follows from

the change of variables s = a2−r2

2ε , and the third from dominated convergence.The desired result (22) now follows by substituting the result of the calculation

(24) into (23).


References

[1] M. Agranovsky, P. Kuchment, and E. T. Quinto, Spherical Mean Operators and Their Ap-plications, in preparation.

[2] M. Ansorg, F. Filbir, W. R. Madych, and R. Seyfried, Summability kernels for circular andspherical mean data, Inverse Problems 29, no. 1, (2013), 015002.

[3] F. Filbir, R. Hielscher, and W. R. Madych, Reconstruction from circular and sphericalmean data, Applied and Computational Harmonic Analysis 29, (2010), 111-120. MR2647016(2011k:65193)

[4] D. Finch, M. Haltmeier, and Rakesh, Inversion of spherical means and the wave equation ineven dimensions, SIAM J. Appl. Math. 68, no. 2, (2007), 392-412. MR2366991 (2008k:35494)

[5] D. Finch and Rakesh, The spherical mean value operator with centers on a sphere, InverseProblems 23, no. 6, (2007), 37-49. MR2440997 (2009k:35167)

[6] D. Finch, S. K. Patch, and Rakesh, Determining a function from its mean values over a familyof spheres, SIAM J. Math. Anal. 35, no. 5, (2004), 1213-1240. MR2050199 (2005b:35290)

[7] M. Haltmeier, T. Schuster, O. Scherzer, Filtered backprojection for thermoacoustic com-

puted tomography in spherical geometry, Math. Methods Appl. Sci., 28, (2005), 1919-1937.MR2170772 (2006d:92023)

[8] M. Haltmeier, O. Scherzer, P. Burgholzer, R. Nuster, G. Paltauf, Thermoacoustic tomographyand the circular Radon transform: exact inversion formula, Math. Models Methods Appl. Sci.17, no. 4, (2007), 635-655. MR2316302 (2009h:35248)

[9] M. Haltmeier, A mollification approach for inverting the spherical mean Radon transform,SIAM J. Appl. Math., 71 (5), (2011), 1637-1652. MR2835366 (2012g:45022)

[10] P. Kuchment and L. Kunyansky, Mathematics of Photoacoustic and Thermoacoustic Tomog-raphy, in Handbook of Mathematical Methods in Imaging, O. Scherzer, ed., Springer (2011),817-865.

[11] W. R. Madych, Summability and approximate reconstruction from Radon Transform data,Contemporary Mathematics, Vol 113 (1990), 189-219. MR1108655 (92i:44001)

Department of Mathematics, University of Connecticut, Storrs, Connecticut 06269



Analytic and group-theoretic aspectsof the Cosine transform

G. Olafsson, A. Pasquale, and B. Rubin

Abstract. This is a brief survey of recent results by the authors devoted toone of the most important operators of integral geometry. Basic facts aboutthe analytic family of cosine transforms on the unit sphere Sn−1 in Rn and thecorresponding Funk transform are extended to the “higher-rank” case for func-tions on Stiefel and Grassmann manifolds. Among the topics we consider arethe analytic continuation and the structure of the polar sets, the connectionwith the Fourier transform on the space of rectangular matrices, inversion for-mulas, spectral analysis, and the group-theoretic realization as an intertwiningoperator between generalized principal series representations of SL(n,R).

1. Introduction

The cosine transform has a long and rich history, with connections to severalbranches of mathematics. The name cosine transform was adopted by Lutwak[50, p. 385] for the spherical convolution

(1.1) (Cf)(u) =∫Sn−1

f(v)|u · v| dv, u ∈ Sn−1.

The motivation for this name is that the inner product u · v is nothing but thecosine of the angle between the unit vectors u and v.

The following list of references shows some branches of mathematics, where theoperator (1.1) and its generalizations arise in a natural way (sometimes implicitly,without naming) and play an important role.

• Convex geometry: [1,6,23,24,32,46,50,69,71,75].

• Pseudo-differential operators: [15,61].• Group representations: [2,3,11,12,57,60].

• Harmonic analysis and singular integrals: [4,21,22,27,48,52,58,59,63,66,73,74,79].

• Integral geometry: [5,20,26,30,62,64,65,68,70,76,86].

2010 Mathematics Subject Classification. Primary 43A80; secondary 47G10, 22E46.The authors are thankful to Tufts University for the hospitality and support during the Joint

AMS meeting and the Workshop on Geometric Analysis on Euclidean and Homogeneous Spaces

in January 2012. The research of G. Olafsson was supported by DMS-0801010 and DMS-1101337.A. Pasquale gratefully acknowledges travel support from the Commission de Colloques et CongresInternationaux (CCCI).


167

168 G. OLAFSSON, A. PASQUALE, AND B. RUBIN

• Stochastic geometry and probability: [29,49,51,77,78].

• Banach space theory: [38,45,47,54,72].

This list is far from being complete. In most of the publications cosine-liketransforms serve as a tool for certain specific problems. At the same time, there aremany papers devoted to the cosine transforms themselves. The present article isjust of this kind. Our aim is to give a short overview of our recent work [57,70] onthe cosine transform and explain some of the ideas and tools behind those results.

For a complex number λ, the λ-analogue of (1.1) is the convolution operator

(1.2) (Cλf)(u) =∫Sn−1

f(v)|u · v|λ dv, u ∈ Sn−1,

where the integral is understood in the sense of analytic continuation, if necessary.We adopt the name “the cosine transform” for (1.2) too. The same name will beused for generalizations of these operators to be defined below.

In recent years more general, higher-rank cosine transforms attracted consid-erable attention. This class of operators was inspired by Matheron’s injectivityconjecture [51], its disproval by Goodey and Howard [29], applications in grouprepresentations [7,12,57,60,86] and in algebraic integral geometry [3,5,20]. Tothe best of our knowledge, the higher-rank cosine transform was explicitly presented(without naming) for the first time in [26, formula (3.5)]. Our interest in this topicgrew up from specific problems of harmonic analysis and group representations.However, in this article we do not focus on those problems, and mention them onlyfor better explanation of the corresponding properties of the cosine transforms andrelated operators of integral geometry. We also restrict ourselves to the case of realnumbers, referring to [57] for the case of complex and quaternionic fields.

The paper is organized as follows. Section 2 contains basic facts about thecosine transforms on the unit sphere. More general transforms on Stiefel or Grass-mann manifolds are considered in Section 3, where the main tool is the classicalFourier analysis. In Sections 4 and 5 we discuss the connections to representationtheory, and more precisely to the spherical representations and the intertwiningproperties. Section 6 is devoted to the explicit spectral formulas.

2. The cosine transform on the unit sphere

In this section we discuss briefly the cosine transform on the sphere Sn−1. For theconvenience of analytic continuation, we normalize (1.2) by setting

(Cλf)(u) = γn(λ)

∫Sn−1

f(v)|u · v|λ dv, u ∈ Sn−1.

Here dv stands for the SO(n)-invariant probability measure on Sn−1,

(2.1) γn(λ)=π1/2 Γ(−λ/2)

Γ(n/2) Γ((1 + λ)/2), Re λ>−1, λ �=0, 2, 4, . . . .

The normalization is chosen so that

Cλ(1) = Γ (−λ/2)Γ((n+ λ)/2)

.

This normalization simplifies the formula for the spectrum of the cosine trans-form and is convenient in many occurrences, when harmonic analysis on Sn−1 is

THE COSINE TRANSFORM 169

performed in the multiplier language (in the same manner as analysis of pseudo-differential operators is performed in the language of their symbols).

The limit case λ = −1 gives, up to a constant, the well-known Funk transform.Specifically, if f ∈ C(Sn−1), then for every u ∈ Sn−1,

(2.2) limλ→−1

(Cλf)(u) = π1/2

Γ((n− 1)/2)(Ff)(u),

where

(2.3) (Ff)(u) =

∫{v∈Sn−1|u·v=0}

f(v) duv,

duv being the relevant probability measure; see, e.g., [69, Lemma 3.1].Since |u · v|λ is an even function of u and v, then Cλf = 0 whenever f is

odd. Similarly, Ff = 0 for all odd functions. As the projective space P(Rn) isthe quotient of Sn−1 by identifying the antipodal points u and −u, it follows thatfunctions on P(Rn) correspond to even functions on Sn−1. Thus, both the cosinetransform and the Funk transform can be viewed as integral transforms on P(Rn).

The operators Cλ and Cλ were investigated by different approaches. A first oneemploys the Fourier transform technique [45,63,76] and relies on the equality inthe sense of distributions

(2.4)

(Eλ Cλf

Γ((1 + λ)/2),Fω

)=c1

(E−λ−nf

Γ(−λ/2) , ω), c1=2n+λ π(n−1)/2 Γ(n/2).

Here ω is a test function belonging to the Schwartz space S(Rn),

(Fω)(y) =∫Rn

ω(x)eix·ydx,

and (Eλf)(x) = |x|λf(x/|x|) denotes the extension by homogeneity.A second approach is based on the Funk-Hecke formula, so that for each spher-

ical harmonic Yj of degree j,

(2.5) CλYj = mj,λ Yj ,

where

(2.6) mj,λ=

⎧⎨⎩ (−1)j/2 Γ(j/2− λ/2)

Γ(j/2 + (n+ λ)/2)if j is even,

0 if j is odd;

see, e.g., [63]. The Fourier-Laplace multiplier {mj,λ} forms the spectrum of Cλ.Note that the normalizing coefficient in Cλ was chosen so that only factors depend-ing on j are involved in the spectral functions {mj,λ}. The spectrum of Cλ encodesimportant information about this operator. For instance, since mj,λmj,−λ−n = 1,then for any f ∈ C∞(Sn−1) the following inversion formula holds:

(2.7) C−λ−nCλf = f,

provided

λ ∈ C, λ /∈ {−n,−n− 2,−n− 4, . . .} ∪ {0, 2, 4, . . .}.For the non-normalized transforms, (2.7) yields

(2.8) C−λ−nCλf = ζ(λ) f, ζ(λ) =Γ2(n/2) Γ((1 + λ)/2) Γ((1− λ− n)/2)

π Γ(−λ/2) Γ((n+ λ)/2),


λ ∈ C, λ /∈ {−1,−3,−5, . . .} ∪ {1− n, 3− n, 5− n, . . .}.Formula (2.6) reveals singularities, provides information about the kernel and

the image. Moreover, it plays a crucial role in the study of the cosine transformsof Lp functions. For instance, the following statement was proved in [64, p. 11],using the relevant results of Gadzhiev [21,22] and Kryuchkov [48] for symbols ofthe Calderon-Zygmund singular integral operators.

Theorem 2.1. Let Lpe(S

n−1) and Lγp,e(S

n−1) be the spaces of even functions (or

distributions) belonging to Lp(Sn−1) and the Sobolev space Lγp(S

n−1), respectively.Then

(2.9) Lδp,e(S

n−1) ⊂ Cλ(Lpe(S

n−1)) ⊂ Lγp,e(S

n−1)

provided

γ = Reλ+n+ 1

2−∣∣∣1p− 1

2

∣∣∣(n− 1), δ = Reλ+n+ 1

2+∣∣∣1p− 1

2

∣∣∣(n− 1),

λ /∈ {0, 2, 4, . . . } ∪ {−n− 1,−n− 3,−n− 5, . . . }.The embeddings ( 2.9) are sharp.

Finally, to study Cλ and Cλ, one can use tools from representation theory, aswe will discuss in more details in the second half of this article.

One can easily explain (2.5) – but not (2.6) – by the fact that the space ofharmonic polynomials of degree j is the underlying space of an irreducible rep-resentation of K = SO(n). Then (2.5) follows from Schur’s lemma and the factthat Cλ commutes with rotations. Note that the group K acts by the left regularrepresentation on L2(Sn−1) and, as a representation of K, we have the orthogonaldecomposition

(2.10) L2(Sn−1) K

⊕j∈N0

Yj ,

where the set Yj of all spherical harmonics of degree j is an irreducible K-space.As we shall see in Section 6, the spectral multiplier (2.6) can also be computed byidentifying Cλ as a standard intertwining operator between certain principal seriesrepresentations of the larger group SL(n,R), see [57].

We have already noted that Cλ should be viewed as an operator on functionson P(Rn). This is related to the fact that the analogue of (2.10) for P(Rn) is

L2(P(Rn)) K

⊕j∈2N0

Yj .

3. Cosine transforms on Stiefel and Grassmann manifolds

In this section we introduce the higher-rank cosine transforms and collect somebasic facts about these transforms. The main results are presented in Theorems3.2, 3.3, 3.6, 3.7, and 3.8.

3.1. Notation. We denote by Vn,m ∼ O(n)/O(n − m) the Stiefel manifoldof n × m real matrices, the columns of which are mutually orthogonal unit n-vectors. For v ∈ Vn,m, dv stands for the invariant probability measure on Vn,m;ξ = {v} denotes the linear subspace of Rn spanned by v. These subspaces form theGrassmann manifold Gn,m ∼ O(n)/(O(n−m)×O(m)) endowed with the invariant


probability measure dξ. We write Mn,m ∼ Rnm for the space of real matricesx = (xi,j) having n rows and m columns and set

dx =

n∏i=1

m∏j=1

dxi,j , |x|m = det(xtx)1/2,

xt being the transpose of x. If n = m, then |x|m is just the absolute value of thedeterminant of x; if m = 1, then |x|1 is the usual Euclidean norm of x ∈ Rn.

3.2. The Cos-function. We give two equivalent “higher-rank” substitutes for|u·v| in (1.1). The first one is “more geometric”, while the second is “more analytic”.For 1 ≤ m ≤ k ≤ n − 1, let η ∈ Gn,m and ξ ∈ Gn,k be linear subspaces of Rn ofdimension m and k, respectively. Following [2,3,57], we set

(3.1) Cos(ξ, η) = volm(prξE),

where volm(·) denotes the m-dimensional volume function, E is a convex subset ofη of volume one, prξ denotes the orthogonal projection onto ξ. By affine invariance,this definition is independent of the choice of E.

The second definition [31] gives precise meaning to the projection operatorprξ. Let u and v be arbitrary orthonormal bases of ξ and η, respectively. Weregard u and v as elements of the corresponding Stiefel manifolds Vn,k and Vn,m.If k = m = 1, then u and v are unit vectors, as in (1.1). The orthogonal projectionprξ is given by the k × k matrix uut, and we can define

(3.2) Cos(ξ, η) ≡ Cos({u}, {v}) = (det(vtuutv))1/2 ≡ |utv|m.

This definition is independent of the choice of bases in ξ and η and yields |u · v| ifk = m = 1.

Remark 3.1. Note that vtuutv is a positive semi-definite matrix, and therefore,det(vtuutv) ≡ det(utvvtu) ≥ 0. It means that Cos(ξ, η) = Cos(η, ξ) ≥ 0.

3.3. Non-normalized cosine transforms. According to (3.1) and (3.2), onecan use both Stiefel and Grassmannian language in the definition of the higher-rankcosine transform, namely,

(3.3) (Cλm,kf)(u) =

∫Vn,m

f(v) |utv|λm dv, u∈Vn,k,

(3.4) (Cλm,kf)(ξ) =

∫Gn,m

f(η) Cosλ(ξ, η) dη, ξ∈Gn,k,

where dv and dη stand for the relevant invariant probability measures. The impor-tant point here is that functions on the Grassmannian Gn,m correspond to O(m)-invariant functions on the Stiefel manifold Vn,m. For those functions the transformsin (3.3) and (3.4) agree. The fact that we have two ways of writing the same opera-tor, extends the arsenal of techniques for its study (some of them will be exhibitedbelow). Both operators agree with Cλ in (1.2), when k = m = 1. For brevity, weshall write Cλm = Cλm,m.

We remark that there are different shifts in the power λ in the literature, all fordifferent reasons. In particular, to make our statements in Sections 2-4 consistentwith those in [70], one should set λ = α− k. To adapt to the notation in [57] onehas to change λ to λ−n/2. For unifying the presentation of the results in [70] and[57] we have preferred to adopt the unshifted notation as in (3.3) and (3.4).


Following [16,28], the Siegel gamma function of the cone Ω of positive definitem×m real symmetric matrices is defined by

(3.5) Γm(α)=

∫Ω

exp(−tr(r))|r|α−(m+1)/2m dr = πm(m−1)/4

m−1∏j=0

Γ(α−j/2)

and represents a meromorphic function with the polar set

(3.6) {(m− 1− j)/2 | j = 0, 1, 2, . . .}.

Theorem 3.2. Let 1 ≤ m ≤ k ≤ n− 1.

(i) If f ∈L1(Vn,m) and Reλ > m − k − 1, then the integral ( 3.3) convergesfor almost all u∈Vn,k.

(ii) If f ∈ C∞(Vn,m), then for every u∈ Vn,k, the function λ �→ (Cλm,kf)(u)extends to the domain Reλ ≤ m− k − 1 as a meromorphic function withthe only poles m− k− 1,m− k− 2, . . . . These poles and their orders arethe same as those of the gamma function Γm((λ+ k)/2).

(iii) The normalized integral (Cλm,kf)(u)/Γm((λ + k)/2) is an entire function

of λ and belongs to C∞(Vn,k) in the u-variable.

A similar statement holds for (3.4). The proof of Theorem 3.2 can be foundin [70, Theorems 4.3, 7.1]. It relies on the fact that |utv|λm is a special case of thecomposite power function (utv)λ with the vector-valued exponent λ ∈ Cm [16,28].The corresponding composite cosine transforms were studied in [58,59,70].

An important ingredient of the proof of Theorem 3.2 is the connection betweenthe cosine transform Cλm,kf on Vn,m and the Fourier transform

(3.7) ϕ(y) = (Fϕ)(y) =∫Mn,m

etr(iytx)ϕ(x) dx, y ∈ Mn,m .

The corresponding Parseval equality has the form

(3.8) (ϕ, ω) = (2π)nm (ϕ, ω), (ϕ, ω) =

∫Mn,m

ϕ(x)ω(x) dx.

This equality, with ω in the Schwartz class S(Mn,m) of smooth rapidly decreasingfunctions, is used to define the Fourier transform of the corresponding distributions.

We will need polar coordinates on Mn,m: for n ≥ m, every matrix x ∈Mn,m of

rank m can be uniquely represented as x = vr1/2 with v ∈ Vn,m and r = xtx ∈ Ω.

Given a function f on Vn,m, we denote (Eλf)(x) = |r|λ/2m f(v). The followingstatement holds in the case k = m.

Theorem 3.3. Let f be an integrable right O(m)-invariant function on Vn,m,ω ∈ §(Mn,m), 1≤m≤n−1, Cλmf = Cλm,mf . Then for every λ ∈ C,

(3.9)

(EλCλmf

Γm((λ+m)/2),Fω

)= c

(E−λ−nf

Γm(−λ/2) , ω),

c =2m(n+λ) πnm/2 Γm(n/2)

Γm(m/2),

where both sides are understood in the sense of analytic continuation.

Formula (3.9) agrees with (2.4). The more general statement for arbitraryk ≥ m can be found in [70].


Remark 3.4. It is important to note that the domains, where the left-handside and the right-hand side of of (3.9) exist as absolutely convergent integrals, haveno points in common, when m > 1. This is the principal distinction from the casem = 1, when there is a common strip of convergence −1 < Reλ < 0. To performanalytic continuation, we have to switch from Cλm to the more general compositecosine transform Cλm with λ ∈ Cm and then take the restriction to the diagonalλ1 = · · · = λm = λ + m. This method of analytic continuation was first used byKhekalo (for another class of operators) in his papers [39–41] on Riesz potentialson the space of rectangular matrices.

3.4. The Funk transform. The higher-rank version of the classical Funktransform (2.3) sends a function f on Vn,m to a function Fm,kf on Vn,k by theformula

(3.10) (Fm,kf)(u) =

∫{v∈Vn,m| utv=0}

f(v) duv, u∈Vn,k.

The condition utv = 0 means that subspaces {u} ∈ Gn,k and {v} ∈ Gn,m aremutually orthogonal. Hence, necessarily, k+m ≤ n. The case k = m, when both fand its Funk transform live on the same manifold, is of particular importance andcoincides with (2.3) when k = m = 1. We denote Fm = Fm,m.

If f is right O(m)-invariant, (Fm,kf)(u) can be identified with a function onthe Grassmannians Gn,m or Gn,n−m, and can be written “in the Grassmannian

language”. For instance, setting ξ = {v} ∈Gn,m, η = {u}⊥ ∈Gn,n−k, and f(ξ) =f(v), we obtain

(3.11) (Fm,kf)(u) =

∫ξ⊂η

f(ξ) dηξ.

3.5. Normalized cosine transforms. Our next aim is to introduce a naturalgeneralization Cλm,kf of the normalized transform (2.1). “Natural” means that we

expect Cλm,kf to obey the relevant modifications of the properties (2.2)-(2.5).

Definition 3.5. Let 1 ≤ m ≤ k ≤ n− 1. For u∈Vn,k and v∈Vn,m, we define

(3.12) (Cλm,kf)(u) = γn,m,k(λ)

∫Vn,m

f(v) |utv|λm dv,

where

γn,m,k(λ) =Γm(m/2)

Γm(n/2)

Γm(−λ/2)Γm((λ+ k)/2)

, λ+m �= 1, 2, . . . .

We denote Cλm = Cλm,m. The integral (3.12) is absolutely convergent if Re λ >m − k − 1. The excluded values of λ belong to the polar set of Γm(−λ/2). Ifk = m = 1 this definition coincides with (2.1). Operators of this kind implicitlyarose in [26, pp. 367, 368].

Theorem 3.6. Let 1 ≤ m ≤ k ≤ n − 1, k + m ≤ n. If f is a C∞ rightO(m)-invariant function on Vn,m, then for every u∈Vn,k,

(3.13) a.c.λ=−k

(Cλm,kf)(u) =Γm(m/2)

Γm((n− k)/2)(Fm,kf)(u),

where “a.c.” denotes analytic continuation and Fm,kf is the Funk transform ( 3.10).


This statement follows from [70, Theorems 7.1 (iv) and 6.1]. Note that ifm = k = 1, then (3.13) yields (2.2). However, unlike (2.2), the proof of whichis straightforward, (3.13) requires a certain indirect procedure, which invokes theFourier transform on the space of matrices and the relevant analogue of (3.9).

We point out that a pointwise inversion of the Funk transform can be obtainedby means of the dual cosine transform, which is defined by

(3.14) (∗C λm,kϕ)(v) =

∫Vn,k

ϕ(u) |utv|λm du, v ∈ Vn,m.

Indeed, the following result holds.

Theorem 3.7. (cf. [70, Theorems 7.4]) Let ϕ = Fm,kf , where f is a C∞ rightO(m)-invariant function on Vn,m, 1 ≤ m ≤ k ≤ n−m. Then, for every v ∈ Vn,m,

(3.15) a.c.λ=m−n

(∗C λm,kϕ)(v)

Γm((λ+ k)/2)= c f(v), c=

Γm(n/2)

Γm(k/2) Γm(m/2).

Regarding other inversion methods of the higher-rank Funk transform (whichis also known as the Radon transform for a pair of Grassmannnians), see [31,85]and references therein.

In the case k = m the normalized cosine transform Cλm = Cλm,m has a number of

important features. If f ∈ C∞(Vn,m), then the analytic continuation of (Cλmf)(u)is well-defined for all complex λ /∈ {1 −m, 2 −m, . . .} and belongs to C∞(Vn,m).The following inversion formulas hold.

Theorem 3.8. (cf. [70, Theorems 7.7]) Let f ∈ C∞(Vn,m) be a right O(m)-invariant function on Vn,m, 2m ≤ n. Then, for every u ∈ Vn,m,

(3.16) (C−λ−nm Cλmf)(u) = f(u), λ,−λ− n /∈ {1−m, 2−m, . . .}.

In particular, for the non-normalized transforms,

(3.17) (C−λ−nm Cλmf)(u) = ζ(λ) f(u), λ+ n,−λ /∈ {1, 2, 3, . . .},

where

(3.18) ζ(λ) =Γ2m(n/2) Γm((m+ λ)/2) Γm((m− λ− n)/2)

Γ2m(m/2) Γm(−λ/2) Γm((n+ λ)/2)

.

Both equalities ( 3.16) and ( 3.18) are understood in the sense of analytic continu-ation.

In the case m = 1, the formulas (3.16) and (3.17) coincide with (2.7) and (2.8),respectively, but the method for proving them is different.

4. Connection to Representation Theory

The cosine transform is closely related to the representation theory of semisimpleLie groups. In particular, as we shall now discuss, it has an important group-theoretic interpretation as a standard intertwining operator between generalizedprincipal series representations of SL(n,R).

In the following we shall use the notation G = SL(n,R), K = SO(n), and

L = S(O(m)×O(n−m))=

{(A 00 B

) ∣∣∣∣ A ∈ O(m)B ∈ O(n−m)

, det(A) det(B) = 1

}


with m ≤ n −m. Then B ≡ K/L = Gn,m is the Grassmanian of m-dimensionallinear subspaces of Rn. We fix the base point

bo = {(x1, . . . , xm, 0, . . . , 0) | x1, . . . , xm ∈ R} ∈ B,

so that B = K · b0 and every function on B can be regarded as a right L-invariantfunction on K.

From now on, our main concern is the cosine transform (3.4) with equal lowerindices, that is, Cλm ≡ Cλm,m. We refer to [35, Chapter V] for the harmonic analysison compact symmetric spaces and [42] for the representation theory of semisimpleLie groups.

4.1. Analysis on B with respect to K. The first connection to representa-tion theory is related to the left regular action of the group K on L2(B) by(

�(k)f)(b) = f(k−1b), k ∈ K , b ∈ B.

For an irreducible unitary representation (π, Vπ) of K, we consider the subspace

V Lπ = {v ∈ Vπ | π(k)v = v ∀k ∈ L}, L = S(O(m)×O(n−m)).

The representation (π, Vπ) is said to be L-spherical if V Lπ �= {0}. As B = K/L is a

symmetric space, the following result is a consequence of [35, Ch. IV, Lemma 3.6].

Proposition 4.1. If (π, Vπ) is L-spherical, then dimV Lπ = 1.

Since V Lπ �= {0}, we can choose a unit vector eπ ∈ V L

π . Then we define a mapΦπ : Vπ → C∞(B) ⊂ L2(B) by the formula

(4.1) (Φπv)(b) = d(π)−1/2〈v, π(k)eπ〉 , v ∈ Vπ, b = k · bo ∈ B = K · bo,

where d(π) = dimVπ. This definition is meaningful because k · bo = kk′ · bo forevery k′ ∈ L and eπ remains fixed under the action of π(k′). We also set

Φπ(v; b) = (Φπv)(b).

Recall, if (π, Vπ) and (σ, Vσ) are two representations of a Hausdorff topological groupH, then an intertwining operator between π and σ is a bounded linear operatorT : Vπ → Vσ such that Tπ(h) = σ(h)T for all h ∈ H. If π is irreducible and Tintertwines π with itself, then Schur’s Lemma states that T = c id for some complexnumber c, [17], p. 71. The map Φπ is a K-intertwining operator in the sense thatit intertwines the representation π on Vπ and the left regular representation � onL2(B), so that for b = h · bo and k ∈ K we have

Φπ(π(k)v; b) = 〈π(k)v, π(h)eπ〉 = 〈v, π(k−1h)eπ〉 = �(k)Φπ(v; b) .

Furthermore, the left regular representation � on L2(B) is multiplicity free, seee.g. [84, Corollary 9.8.2]. Therefore, since (π, Vπ) is irreducible, any intertwiningoperator Vπ → L2(B) is by Schur’s Lemma of the form cΦπ for some c ∈ C.

We let L2π(B) = ImΦπ. Denote by KL the set of all equivalence classes of

irreducible L-spherical representations (π, Vπ) of K. Then, see [35, Chapter V,Thm. 4.3], the decomposition of L2(B) as a K-representation is as follows.

Theorem 4.2. L2(B) K

⊕π∈ KL

L2π(B) .


The cosine transform is, as mentioned before, a K-intertwining operator, i.e.,Cλm(�(k)f) = �(k)Cλm(f) for all k ∈ K and f ∈ L2(B). It follows by Schur’s Lemma

that for each π ∈ KL there exists a function ηπ on C such that

(4.2) Cλm|L2π= ηπ(λ) id .

Let f ∈ L2π(B) of norm one. Then ηπ(λ) = 〈Cλm(f), f〉 and it follows that ηπ(λ) is

meromorphic; cf. Theorem 3.2.

4.2. Generalized spherical principal series representations of G. Thefact that Cλm is a K-intertwining operator does not indicate how to determine thefunctions ηπ. In the case m = 1 and in some particular cases for the higher-rankcosine transforms [58,59] explicit expression for ηπ can be obtained using the Funk-Hecke Theorem or the Fourier transform technique. It is a challenging open problemto proceed the same way in the most general case, using, e.g., the relevant resultsof Gelbart, Strichartz, and Ton-That, see, e.g., [25,80,82]. Below we suggest analternative way and proceed as follows.

To find ηπ explicitly, we observe that the cosine transform is an intertwiningoperator between certain generalized principal series representations (πλ, L

2(B)) ofG = SL(n,R) induced from a maximal parabolic subgroup of G. We can then usethe bigger group G, or better its Lie algebra, to move between K-types. We invokethe spectrum generating technique introduced in [7] to build up a recursion relationbetween the spectral functions ηπ. This finally allows us to determine all of themby knowing ηtrivial, where trivial denotes the trivial representation of K.

The group G = SL(n,R) acts on B by

g · η = {gv | v ∈ η} ,where gv denotes the usual matrix multiplication. This action is transitive, as theK-action is already transitive. The stabilizer of bo is the group

P =

{(A X0 B

) ∣∣∣∣ X ∈ Mm,n−m ,A ∈ GL(m,R)B ∈ GL(n−m,R) and det(A) det(B) = 1

} S(GL(m)×GL(n−m))�Mm,n−m ,

where Mn,m is the space of n × m real matrices; see Section 3.1. We then haveB = G/P .

The K-invariant probability measure on B is not G-invariant. But there existsa function j : G× B → R+ such that for all f ∈ L1(B) we have

(4.3)

∫Bf(b) db =

∫Bf(g · b)j(g, b)n db , g ∈ G, b ∈ B .

We include the power n to adapt our notation to [57]. By the associativity of theaction we have j(gg′, b) = j(g, g′ · b)j(g′, b) for all g ∈ G and b ∈ B. Hence, for eachλ ∈ C we can define a continuous representation πλ of G on L2(B) by

(4.4) [πλ(g)f ](b) = j(g−1, b)λ+n/2f(g−1 · b) , g ∈ G, f ∈ L2(B), β ∈ B.A simple change of variables shows that

〈πλ(g)f, h〉 = 〈f, π−λ(g−1)h〉 , g ∈ G , f, h ∈ L2(B) .

In particular, πλ is unitary if and only if λ is purely imaginary. The representationsπλ are the so-called generalized (spherical) principal series representations (induced


from the maximal parabolic subgroup P ), in the compact picture. See e.g. [42], p.169.

To connect our exposition here to [70], we note that the representation πλ canalso be realized on the space of O(m)-invariant functions on the Stiefel manifold.The explicit construction goes as follows. According to [70, Section 7.4.3], weintroduce the radial and angular components of a matrix x ∈Mn,m of rank m by

rad(x) = (xtx)1/2 ∈ Ω, ang(x) = x(xtx)−1/2 ∈ Vn,m,

so that x = ang(x) rad(x). Given λ ∈ C, define

(4.5) πλ(g)f(v) = |rad(g−1v)|−(λ+n/2) f(ang(g−1v)).

This defines a representation πλ of GL(n,R) on L2(Vn,m)O(m) L2(B). Therestriction of πλ to SL(n,R) is equivalent to the representation πλ defined in (4.4).

4.3. The cosine transform as an intertwining operator. In this sectionwe follow the ideas in [57]. An alternative self-contained exposition (without usingthe representation theory of semisimple Lie groups), can be found in [70].

The gain of using the representations πλ is that we now have a meromorphicfamily of representations on L2(B). Moreover, these representations are irreduciblefor almost all λ and closely related to the cosine transform. For all this, we needto recall some results from [83].

Theorem 4.3 (Vogan-Wallach). There exists a countable collection {pn} ofnon-zero holomorphic polynomials on C such that if pn(λ) �= 0 for all n then πλ isirreducible. In particular, πλ is irreducible for almost all λ ∈ C.

Proof. This is Lemma 5.3 in [83]. �

Let θ : G→ G be the involutive automorphism θ(g) = (g−1)t. We remark that

in [57] the notation Cosλ = Cλ−n/2m was used.

Theorem 4.4. The cosine transform intertwines πλ and π−λ ◦ θ, namely,

(4.6) Cλm ◦ πλ+n/2 = (π−λ−n/2 ◦ θ) ◦ Cλm,

whenever both sides of this equality are analytic functions of λ.

Proof. We refer to Theorem 2.3 and (4.10) in [57]. �

In fact, it is shown in [57], Lemma 2.5 and Theorem 4.2, that Cλ−n/2m = J(λ),

where J(λ) is a standard intertwining operator, studied in detail among others byKnapp and Stein in [43,44] and Vogan and Wallach in [83]. These authors show,in particular, that λ �→ J(λ) has a meromorphic extension to all of C. Furthermore,Vogan and Wallach show that if f ∈ C∞(B), then the map

{ν ∈ C | Re (ν) > −1 + n/2} � λ �−→ J(λ)f ∈ C∞(B)

is holomorphic. As a consequence of Cλ−n/2m = J(λ) and [83, Theorem 1.6], we get

the following theorem.

Theorem 4.5. The map λ �→ Cλm extends meromorphically to C. In particular,for f ∈ C∞(B) and b ∈ B the function λ �→ (Cλmf)(b) extends to a meromorphicfunction on C and the set of possibles poles is independent of f . In the complementof the singular set we have Cλmf ∈ C∞(B).


Notice that precise information about the analiticity of more general cosinetransforms, including the structure of polar sets, is presented in Theorem 3.2 above.

The implication of (4.6) is that C−λ−n/2m ◦ Cλ−n/2

m intertwines πλ with itself (inthe sense of a meromorphic family of operators). By Theorem 4.3 there exists ameromorphic function η on C such that

(4.7) C−λ−n/2m ◦ Cλ−n/2

m = η(λ) idC∞(B)

for all λ ∈ C for which the left-hand side is well defined. The shift by n/2 in thedefinition is chosen so that the final formulas agree with those in [57] and makesome formulas more symmetric. The fact that η is meromorphic follows by noting

that η(λ) = 〈C−λ−n/2m ◦ Cλ−n/2

m (1), 1〉.Formula (4.7) is a symmetric version of (3.18) with λ replaced by λ − n/2.

The explicit value of η(λ) can be easily obtained from (3.17). An alternative,representation-theoretic method to compute the function η(λ), is presented in Sec-tion 6. The first step is the following lemma.

Lemma 4.6. Let c(λ) = Cλ−n/2m (1). Then η(λ) = c(λ)c(−λ).

Note that c(λ) is the function ηπ(λ) in (4.2) with π equal to the trivial repre-sentation of K.

Remark 4.7. There are several ways to prove the meromorphic extension ofthe standard intertwining operators. The proof in [83] uses tensoring with finitedimensional representations of G to deduce a relationship between Cλm and Cλ+2n

m .In fact, there exists a family of (non-invariant) differential operators Dλ on B anda polynomial b(λ), the Bernstein polynomial, such that

(4.8) b(λ)Cλm(f) = Cλ+2nm (Dλ(f))

[83, Theorem 1.4]. Another way to derive an equation of the form (4.8) is to convertthe integral defining Cλm into an integral over the orbit of certain nilpotent groupN , as usually done in the study of standard intertwining operators, and then usethe ideas from [8,55,56]. In the case where G/P is a symmetric R-space (whichcontains the case of Grassmann manifolds), the standard intertwining operatorsJ(λ) have been recently studied by Clerc in [9], using Loos’ theory of positive Jordantriple systems. In particular, Clerc explicitly computes the Bernstein polynomialsb(λ) in (4.8), and, hence, proves the meromorphic extension of J(λ) for this classof symmetric spaces.

Finally, one can stick with the domain where λ �→ Cλm is holomorphic anddetermine the K-spectrum functions ηπ(λ) in (4.2). As rational functions of Γ-factors, these functions have meromorphic extension to C. Hence, λ �→ Cλm itselfhas meromorphic extension by (4.2); see also Remark 6.8.

4.4. Historical remarks. We conclude this section with a few historical re-marks. The standard intertwining operators J(λ), as a meromorphic family ofsingular integral operators on K or N , have been central objects in the study ofrepresentation theory of semimisimple Lie groups since the fundamental works ofKnapp and Stein [43,44], Harish-Chandra [33], and several others. In our case

N =

{(Im 0X In−m

) ∣∣∣∣ X ∈ Mm,n−m

}.


Then, in the realization of the generalized principal series representations on L2(B),the kernel of J(λ) is Cosλ−n/2(b, c). But in most cases there is neither an explicitformula nor geometric interpretation of the kernel defining J(λ).

Apart of customary applications of the cosine transform in convex geometry,probability, and the Banach space theory, similar integrals turned up independentlyas standard intertwining operators between generalized principal series representa-tions of SL(n,K), where K = R,C or H.

The real case was studied in [12], the complex case in [14], and the quaternioniccase in [60]. In these articles it was shown that integrals of the form∫

B|x · y|λ−n/2f(x) dx,

with some modification for K = C or H, define intertwining operators betweengeneralized principal series representations induced from a maximal parabolic sub-group in SL(n + 1,K). The K-spectrum was determined, yielding the cases ofirreducibility and, more generally, the composition series of those representations.Among the applications, there were some embeddings of the complementary seriesand the study of the so-called canonical representations on some Riemannian sym-metric spaces of the noncompact type, [10, 11, 13]. However the connections ofthese considerations to convex geometry, to the cosine transform and to the Funkand Radon transforms was neither discussed nor mentioned. These connectionswere first published in [57] in the context of the Grassmannians over R, C and H.However, it was probably S. Alesker who first remarked in his unpublished manu-script [2] that over R the cosine transform is a SL(n,R)-intertwining operator; seealso [3] for the case λ = 1. 1

It was also shown in [86] that the Sinλ-transform (a transform related to thesine transform) can be viewed as a Knapp-Stein intertwining operator. This was

used to construct complementary series representations for GL(2n,R). The Sinλ-transform is then also naturally linked to reflection positivity, which relates com-plementary series representations of GL(2n,R) to the highest weight representa-tions of SU(n, n), [18, 19,36, 37,53]. Notice, however, that the definition of the

Sinλ-transform in [86] differs from the one in [66], [70]; see also [67] for the sinetransform on the hyperbolic space.

5. The spherical representations

The functions ηπ(λ) in (4.2) are parametrized by the L-spherical representationsof K. The main purpose of this section is to present this parametrization, whichis given by a semilattice in a finite dimensional Euclidean space associated with amaximal flat submanifold of B. We will, therefore, have to study the structure ofthe symmetric space B. We refer to [81] and the books by Helgason [34,35] formore detailed discussions and proofs. To bring the discussion closer to standardreferences in Lie theory we also introduce some Lie theoretical notation which wehave avoided so far.

Let

g = {X ∈ Mn,n | tr(X) = 0} ,k = {X ∈ Mn,n | Xt = −X} ,

1The authors are grateful to Professor S. Alesker for pointing out these references.


be the Lie algebras of G = SL(n,R) and K = SO(n), respectively. The derivedinvolution of θ on g, still denoted θ, is given by θ(X) = −Xt. Hence k = g(1, θ),the eigenspace of θ on g with eigenvalue 1. We fix once and for all the G-invariantbilinear form β(X,Y ) = n

m(n−m)tr(XY ) on g. Note that β is negative definite on k

and 〈X,Y 〉 = −β(X, θ(Y )) is an inner product on g such that ad(X)t = −ad(θ(X)),where, as usual, ad(X)Y = [X,Y ] = XY − Y X. The normalization of β is chosenso that it agrees with [57].

We recall that B is a symmetric space corresponding to the involution

τ (x) =

(Im 00 −In−m

)x

(Im 00 −In−m

)=

(A −B−C D

)for x =

(A BC D

),

where for r ∈ N we denote by Ir the r × r identity matrix. Note that τ in factdefines an involution of G and that the derived involution on the Lie algebra g isgiven by the same form.

We have k = l⊕ q where l so(m)× so(n−m) is the Lie algebra of L and

q = k(−1, τ ) ={Q(X) =

(0mm X−Xt 0n−m,n−m

) ∣∣∣∣ X ∈ Mm,n−m

}.

Let Eν,μ = (δiνδjμ)i,j denote the matrix in Mm,n−m with all entries equal to 0 butthe (ν, μ)-th which is equal to 1. For t = (t1, . . . , tm)t ∈ Rm we set

X(t) = −m∑j=1

tjEj,n−2m+j ∈ Mm,n−m ,

Y (t) = Q(X(t)) ∈ q .

Then b = {Y (t) | t ∈ Rm} Rm is a maximal abelian subspace of q.

To describe the set KL we note first that B is not simply connected. So wecannot use the Cartan-Helgason theorem [35, p. 535] directly, but only a slightmodification is needed. Define εj(Y (t)) = itj . We will identify the element λ =∑m

j=1 λjεj ∈ b∗Cwith the corresponding vector λ = (λ1, . . . , λm).

If H ∈ b, then ad(H) is skew-symmetric on k with respect to the inner product〈 · , · 〉. Hence ad(H) is diagonalizable over C with purely imaginary eigenvalues.For α ∈ ib∗ let

kαC = {X ∈ kC | (∀H ∈ b) ad(H)X = α(H)X}be the joint α-eigenspace. Let

Δk = {α ∈ ib∗ | α �= 0 and kαC �= {0}} .The dimension of kα

Cis called the multiplicity of α (in kC).

Lemma 5.1. We have

Δk = {±εi ± εj (1 ≤ i �= j ≤ m,± independently), ±εi (1 ≤ i ≤ m) }with multiplicities respectively 1 (and not there if m = 1), 2n−m (and not there ifm = n−m).

Proof. The statement follows from [34]: the table on page 518, the descriptionof the simple root systems on page 462 ff. and the Satake diagrams on pages 532–533. �

We letΔ+

k= {εi ± εj (1 ≤ i < j ≤ m ), εi (1 ≤ i ≤ m) } .


Lemma 5.2. Let ρk =1

2

∑α∈Δ+

k

dim(kαC)α ∈ ib∗. Then ρk =m∑j=1

(n2− j)εj .

Let now (π, Vπ) be a unitary irreducible representation of K. Then Vπ is

finite dimensional. Moreover, π(H) =d

dt

∣∣∣t=0

π(exp(tH)) is skew-symmetric, hence

diagonalizable, for all H ∈ b (in fact, π(H) is diagonalizable for all H ∈ k). LetΓ(π) ⊂ ib∗ be the finite set of joint eigenvalues of π(H) with H ∈ b. For μ ∈ Γ(π),let V μ

π ⊂ Vπ denote the joint eigenspace of eigenvalue μ. If X ∈ kαCand v ∈ V μ

π ,then π(X)v ∈ V μ+α

π . Thus, there exists a μ = μπ ∈ Γ(π) such that π(kαC)V μ

π = {0}for all α ∈ Δ+

k. This only uses that π is finite dimensional, but the irreducibility

implies that this μ is unique. It is called the highest weight of π. Finally we haveπ σ if and only if μπ = μσ.

Let K be the universal covering group of K. Then τ lifts to an involution τ on

K, L = K τ is connected, and B = K/L is the universal covering of B. Replacing

K by K etc., we can talk about L-spherical representations of K and their highestweights. The following theorem is a consequence of the Cartan-Helgason theorem[35, p. 535].

Theorem 5.3. The map π �→ μπ sets up a bijection between the set of L-

spherical representations of K and the semi-lattice

(5.1) Λ+(B) ={μ ∈ ib∗

∣∣∣∣ (∀α ∈ Δ+k)〈μ, α〉〈α, α〉 ∈ Z+

}.

Furthermore, if m = n/2, then

Λ+(B) = {(μ1, . . . , μm) ∈ Zm | μ1 ≥ μ2 ≥ · · · ≥ μm−1 ≥ |μm|} .

Otherwise,

Λ+(B) = {(μ1, . . . , μm) ∈ Zm | μ1 ≥ μ2 ≥ · · · ≥ μm−1 ≥ μm ≥ 0} .

If μ ∈ Λ+(B), then we write (πμ, Vμ) for the corresponding L-spherical repre-sentation. Recall the notation Φπμ

from (4.1). Let Λ+(B) denote the sublattice in

Λ+(B) which corresponds to L-spherical representations of K. Then μ ∈ Λ+(B)if and only if the functions Φπμ

(v), which are originally defined on B, factor tofunctions on B. For that, let v ∈ V μ

μ and H ∈ b. We can normalize v and eπμso

that

Φπμ(v; expH) = eμ(H) .

The same argument as for the sphere [81, Ch. III.12] proves the following theorem.

Theorem 5.4. If m = n−m, then

Λ+(B) = {μ =m∑j=1

μjεj | μj ∈ 2N0andμ1 ≥ . . . ≥ μm−1 ≥ |μm| } .

In all other cases,

Λ+(B) = {μ =

m∑j=1

μjεj | μj ∈ 2N0and μ1 ≥ . . . ≥ μm ≥ 0 } .


6. The generation of the K-spectrum

Recall from Section 5 the involution θ(X) = −Xt on g. The Lie algebra g decom-poses into eigenspaces of θ as g = k⊕ s, where

s = g(−1, θ) = {X ∈Mn,n | θ(X) = −X and Tr(X) = 0} .Then, except in the case n = 2, the complexification sC of s is an irreducible L-spherical representation of K. For n = 2 this representation decomposes into twoone-dimensional representations.

Let

Ho =

(n−mn Im 00 −m

n In−m

)∈ s .

Then Ho is L-fixed and 〈H0, H0〉 = 1. Define a = RHo. The operator ad(H0) hasspectrum {0, 1,−1} and n = g(1, ad(H0)).

Let Ad(k) denote the conjugation by k. Define a map ω : sC → C∞(B) byω(Y )(k) = 〈Y,Ad(k)Ho〉 = β(Y,Ad(k)Ho) =

nm(n−m)Tr(Y kHok

−1)

and note that

ω(Ad(h)Y )(k) = 〈Ad(h)Y,Ad(k)Ho〉 = 〈Y,Ad(h−1k)Ho〉 = ω(Y )(h−1k) .

Thus ω is a K-intertwining operator.Fix an orthonormal basis X1, . . . , Xdim q of q such that X1, . . . , Xm, is an or-

thonormal basis of b. Denote by Ω = −∑

j X2j the corresponding positive definite

Laplace operator on B. ThenΩ|L2

μ(B) = ω(μ) id ,

whereω(μ) = 〈μ+ 2ρk, μ〉 .

A simple calculation then gives:

Lemma 6.1. Let μ = (μ1, . . . , μm) ∈ Λ+(B). Then

ω(μ) =m(n−m)

2n

m∑j=1

(μ2j + μj(n− 2j)

).

For f ∈ C∞(B) denote by M(f) : L2(B) → L2(B) the multiplication operatorg �→ fg. Recall the notation π0 for the finite dimensional spherical representationof highest weight 0 ∈ Λ+(B).

Theorem 6.2. Let Y ∈ s. Then [Ω,M(ω(Y ))] = 2π0(Y ).

Proof. This is Theorem 2.3 in [7]. �For μ ∈ Λ+(B) define Ψμ : L2

μ(B)⊗ sC → L2(B) byΨμ(ϕ⊗ Y ) = M(ω(Y ))ϕ .

Observe that for k ∈ K, Y ∈ sC, and ϕ ∈ L2μ(B) we have

�(k)(M(ω(Y )ϕ)

)=(�(k)ω(Y )

)(�(k)ϕ) = M

(ω(Ad(k)Y )

)(�(k)ϕ)

with Ad(k)Y ∈ sC and �(k)ϕ ∈ L2μ(B). Hence Ψμ is K-equivariant and ImΨμ is

K-invariant. Define a finite subset S(μ) ⊂ Λ+(B) by

ImΨμ K

⊕σ∈S(μ)

L2σ(B) .


Lemma 6.3. Let μ ∈ Λ+(B). Then

S(μ) = {μ± 2εj | j = 1, . . . ,m} ∩ Λ+(B) .

These representations occur with multiplicity one.

Denote by prσ the orthogonal projection L2(B) → L2σ(B). The first spectrum

generating relation which follows from Theorem 6.2, see also [7, Cor. 2.6], states:

Lemma 6.4. Assume that μ ∈ Λ+(B). Let σ ∈ S(μ), Y ∈ sC, and λ ∈ C. Let

(6.1) ωσμ(Y ) := prσ ◦M(ω(Y ))|L2μ(B) : L

2μ(B)→ L2

σ(B) .

Then

(6.2) prσ ◦ πλ(Y )|L2μ(B) =

1

2(ω(σ)− ω(μ) + 2m(n−m)

n λ)ωσμ(Y ) .

The spectrum generating relation that we are looking for can now easily bededucted and we get:

Lemma 6.5. Let μ = (μ1, . . . , μm) ∈ Λ+(B) and λ ∈ C. Then

(6.3)ημ+2εj (λ)

ημ(λ)=

λ− μj + j − 1

λ+ μj + n− j + 1= − −λ+ μj − j + 1

λ+ μj + n− j + 1

and η0(λ) = c(λ).

Proof. First we apply Cλ−n/2m to (6.2) from the left, using that Cλ−n/2

m com-

mutes with prσ and that Cλ−n/2m ◦πλ(Y ) = π−λ◦θ(Y )◦Cλ−n/2

m = −π−λ(Y )◦Cλ−n/2m .

We then get:(ω(σ)− ω(μ) + 2m(n−m)

n λ)ησ(λ− n/2)ωσμ(Y ) =

−(ω(σ)− ω(μ)− 2m(n−m)

n λ)ημ(λ− n/2)ωσμ(Y ) .

As ωσδ(Y ) is non-zero, for generic λ it can be canceled out. Now insert the expres-sion from Lemma 6.1 to get

ω(μ+ 2εj)− ω(μ) = 2m(n−m)n (μj + n/2− (j − 1))

and the claim follows. The statement follows from the fact that πλ is irreduciblefor generic λ, hence, iterated application of (6.1) will in the end reach all K-typesstarting from the trivial K-type. �

Lemma 6.5 tells us that the evaluation of ημ(λ) can be done in two steps. Firstwe determine the function η0(λ) and then use (6.3) as an inductive procedure todetermine the rest. The final result is given in the following theorem. It is presentedin terms of Γ-functions associated to the cone Ω of m×m positive definite matrices,namely,

(6.4) ΓΩ(λ) = πm(m−1)/4m∏j=1

Γ(λj − (j − 1)/2), λ = (λ1, . . . , λm) ∈ Cm.

This integral is a generalization of Γm(λ) in (3.5); cf. [16, p. 123], [70, Sec. 2.2].In the following the scalar parameters, which occur in the argument of ΓΩ, areinterpreted as vector valued, for instance, n ∼ (n, . . . , n), λ ∼ (λ, . . . , λ).


Theorem 6.6 ([57]). Let Λ+(B) be the sublattice in Theorem 5.4 parametrizingthe L-spherical representations of K, let μ = (μ1, . . . , μm) ∈ Λ+(B), and λ ∈ C.Then the K-spectrum of the cosine transform Cλm is given by:

(6.5) ημ(λ) = (−1)|μ|/2 Γm (n/2)

Γm (m/2)

Γm ((λ+m)/2))

Γm (−λ/2)ΓΩ ((μ− λ)/2)

ΓΩ ((λ+ n+ μ)/2).

Remark 6.7. Owing to (3.12), the spectrum of the normalized cosine transformCλm has the simpler form

(6.6) ημ(λ) = (−1)|μ|/2 ΓΩ ((μ− λ)/2)

ΓΩ ((λ+ n+ μ)/2).

In the case m = 1 this formula coincides with (2.6).

Remark 6.8. In Section 4 we referred to the result of Vogan and Wallach onthe meromorphic continuation of the intertwining operator J(λ). This result is notneeded for the computation of ημ(λ). Indeed, it is enough to know that J(λ) isholomorphic on some open subset of C as that is all what is needed to determineημ(λ) in Theorem 6.6. We can then extend Cλm meromorphically on each K-type.Note, however, that this is weaker than the statement in [83] which extends Cλmffor all smooth functions.

References

[1] A. D. Aleksandrov, On the theory of mixed volumes of convex bodies. II. New inequalitiesbetween mixed volumes and their applications, Mat. Sbornik N.S. 2 (1937), 1205–1238, inRussian.

[2] S. Alesker, The α-cosine transform and intertwining integrals on real Grassmannians, Un-published manuscript, 2003.

[3] S. Alesker and J. Bernstein, Range characterization of the cosine transform on higher Grass-mannians, Adv. Math. 184 (2004), no. 2, 367–379, DOI 10.1016/S0001-8708(03)00149-X.

MR2054020 (2005b:22024)[4] R. Askey and S. Wainger, On the behavior of special classes of ultraspherical expansions. I,

II, J. Analyse Math. 15 (1965), 193–220. MR0193290 (33 #1510)[5] A. Bernig, Integral geometry under G2 and Spin(7), Israel J. Math. 184 (2011), 301–316,

DOI 10.1007/s11856-011-0069-6. MR2823979 (2012h:53169)[6] W. Blaschke, Kreis und Kugel, Chelsea Publishing Co., New York, 1949 (German).

MR0076364 (17,887b)

[7] T. Branson, G. Olafsson, and B. Ørsted, Spectrum generating operators and intertwiningoperators for representations induced from a maximal parabolic subgroup, J. Funct. Anal.

135 (1996), no. 1, 163–205, DOI 10.1006/jfan.1996.0008. MR1367629 (97g:22009)[8] J.-L. Brylinski and P. Delorme, Vecteurs distributions H-invariants pour les series princi-

pales generalisees d’espaces symetriques reductifs et prolongement meromorphe d’integralesd’Eisenstein, Invent. Math. 109 (1992), no. 3, 619–664, DOI 10.1007/BF01232043 (French).MR1176208 (93m:22016)

[9] J.-L. Clerc, Intertwining operators for the generalized principal series on symmetric R-spaces,arxiv:1209.0691v1, 2012.

[10] G. van Dijk and S. C. Hille, Canonical representations related to hyperbolic spaces, J. Funct.Anal. 147 (1997), no. 1, 109–139, DOI 10.1006/jfan.1996.3057. MR1453178 (98k:22053)

[11] G. van Dijk and S. C. Hille, Maximal degenerate representations, Berezin kernels and canon-ical representations, Lie groups and Lie algebras, Math. Appl., vol. 433, Kluwer Acad. Publ.,Dordrecht, 1998, pp. 285–298.

[12] G. Van Dijk and V. F. Molchanov, Tensor products of maximal degenerate series repre-sentations of the group SL(n,R), J. Math. Pures Appl. (9) 78 (1999), no. 1, 99–119, DOI10.1016/S0021-7824(99)80011-7. MR1671222 (2000a:22020)


[13] G. van Dijk and A. Pasquale, Canonical representations of Sp(1, n) associated withrepresentations of Sp(1), Comm. Math. Phys. 202 (1999), no. 3, 651–667, DOI10.1007/s002200050600. MR1690958 (2000g:22018)

[14] A. H. Dooley and G. Zhang, Generalized principal series representations of SL(1 + n,C),Proc. Amer. Math. Soc. 125 (1997), no. 9, 2779–2787, DOI 10.1090/S0002-9939-97-03877-X.MR1396975 (97j:22029)

[15] G. I. Eskin, Boundary value problems for elliptic pseudodifferential equations, Translations of

Mathematical Monographs, vol. 52, American Mathematical Society, Providence, R.I., 1981.Translated from the Russian by S. Smith. MR623608 (82k:35105)

[16] J. Faraut and A. Koranyi, Analysis on symmetric cones, Oxford Mathematical Monographs,The Clarendon Press Oxford University Press, New York, 1994. Oxford Science Publications.MR1446489 (98g:17031)

[17] G. B. Folland, A course in abstract harmonic analysis, Studies in Advanced Mathematics,CRC Press, Boca Raton, FL, 1995. MR1397028 (98c:43001)

[18] R. L. Frank and E. H. Lieb, Inversion positivity and the sharp Hardy-Littlewood-Sobolevinequality, Calc. Var. Partial Differential Equations 39 (2010), no. 1-2, 85–99, DOI10.1007/s00526-009-0302-x. MR2659680 (2012a:26027)

[19] R. L. Frank and E. H. Lieb, Spherical reflection positivity and the Hardy-Littlewood-Sobolevinequality. concentration, functional inequalities and isoperimetry, Contemp. Math. 545(2011), 89–102.

[20] J. H. G. Fu, Algebraic integral geometry, arXiv: 1103.6256v2, 2012.[21] A. D. Gadzhiev, Differential properties of the symbol of a singular operator in spaces of Bessel

potentials on a sphere, Izv. Akad. Nauk Azerbaıdzhan. SSR Ser. Fiz.-Tekhn. Mat. Nauk3 (1982), no. 1, 134–140 (Russian, with English and Azerbaijani summaries). MR675522(84a:46073)

[22] A. D. Gadzhiev, Exact theorems on multipliers of spherical expansions and some of their

applications, Special problems in function theory, No. IV (Russian), “Elm”, Baku, 1989,pp. 73–100 (Russian). MR1224483

[23] R. J. Gardner, Geometric tomography, Encyclopedia of Mathematics and its Applications,vol. 58, Cambridge University Press, Cambridge, 1995. MR1356221 (96j:52006)

[24] R. J. Gardner and A. A. Giannopoulos, p-cross-section bodies, Indiana Univ. Math. J. 48(1999), no. 2, 593–613, DOI 10.1512/iumj.1999.48.1689. MR1722809 (2000i:52002)

[25] S. S. Gelbart, A theory of Stiefel harmonics, Trans. Amer. Math. Soc. 192 (1974), 29–50.MR0425519 (54 #13474)

[26] I. M. Gel′fand, M. I. Graev, and R. Rosu, The problem of integral geometry and intertwiningoperators for a pair of real Grassmannian manifolds, J. Operator Theory 12 (1984), no. 2,359–383. MR757440 (86c:22016)

[27] I. M. Gel′fand and Z. Ya. Sapiro, Homogeneous functions and their extensions, Uspehi Mat.Nauk (N.S.) 10 (1955), no. 3(65), 3–70, in Russian.

[28] S. G. Gindikin, Analysis in homogeneous domains, Uspehi Mat. Nauk 19 (1964), no. 4 (118),3–92 (Russian). MR0171941 (30 #2167)

[29] P. Goodey and R. Howard, Processes of flats induced by higher-dimensional processes,Adv. Math. 80 (1990), no. 1, 92–109, DOI 10.1016/0001-8708(90)90016-G. MR1041885

(91d:60025)[30] P. Goodey, V. Yaskin, and M. Yaskina, Fourier transforms and the Funk-Hecke the-

orem in convex geometry, J. Lond. Math. Soc. (2) 80 (2009), no. 2, 388–404, DOI10.1112/jlms/jdp035. MR2545259 (2010j:52006)

[31] E. L. Grinberg and B. Rubin, Radon inversion on Grassmannians via Garding-Gindikinfractional integrals, Ann. of Math. (2) 159 (2004), no. 2, 783–817, DOI 10.4007/an-nals.2004.159.783. MR2081440 (2005f:58042)

[32] H. Groemer, Geometric applications of Fourier series and spherical harmonics, Encyclopediaof Mathematics and its Applications, vol. 61, Cambridge University Press, Cambridge, 1996.MR1412143 (97j:52001)

[33] Harish-Chandra, Harmonic analysis on real reductive groups. III. The Maass-Selberg rela-tions and the Plancherel formula, Ann. of Math. (2) 104 (1976), no. 1, 117–201. MR0439994(55 #12875)


[34] S. Helgason, Differential geometry, Lie groups, and symmetric spaces, Pure and AppliedMathematics, vol. 80, Academic Press Inc. [Harcourt Brace Jovanovich Publishers], NewYork, 1978. MR514561 (80k:53081)

[35] S. Helgason, Groups and geometric analysis, Mathematical Surveys and Monographs, vol. 83,American Mathematical Society, Providence, RI, 2000. Integral geometry, invariant differen-tial operators, and spherical functions; Corrected reprint of the 1984 original. MR1790156(2001h:22001)

[36] P. E. T. Jorgensen and G. Olafsson, Unitary representations of Lie groups with reflectionsymmetry, J. Funct. Anal. 158 (1998), no. 1, 26–88, DOI 10.1006/jfan.1998.3285. MR1641554(99m:22013)

[37] P. E. T. Jorgensen and G. Olafsson, Unitary representations and Osterwalder-Schrader dual-ity, The mathematical legacy of Harish-Chandra (Baltimore, MD, 1998), Proc. Sympos. PureMath., vol. 68, Amer. Math. Soc., Providence, RI, 2000, pp. 333–401.

[38] M. Kanter, The Lp norm of sums of translates of a function, Trans. Amer. Math. Soc. 79(1973), 35–47. MR0361617 (50 #14062)

[39] S. P. Khekalo, Riesz potentials in the space of rectangular matrices, and the iso-Hyugensdeformation of the Cayley-Laplace operator, Dokl. Akad. Nauk 376 (2001), no. 2, 168–170(Russian). MR1840936

[40] S. P. Khekalo, The Cayley-Laplace differential operator on the space of rectangu-lar matrices, Izv. Ross. Akad. Nauk Ser. Mat. 69 (2005), no. 1, 195–224, DOI10.1070/IM2005v069n01ABEH000528 (Russian, with Russian summary); English transl., Izv.Math. 69 (2005), no. 1, 191–219. MR2128187 (2005m:35009)

[41] S. P. Khekalo, The Igusa zeta function associated with a complex power function on the spaceof rectangular matrices, Mat. Zametki 78 (2005), no. 5, 773–791, DOI 10.1007/s11006-005-0175-z (Russian, with Russian summary); English transl., Math. Notes 78 (2005), no. 5-6,

719–734. MR2252957 (2007j:15006)[42] A. W. Knapp, Representation theory of semisimple groups, Princeton Mathematical Series,

vol. 36, Princeton University Press, Princeton, NJ, 1986. An overview based on examples.MR855239 (87j:22022)

[43] A. W. Knapp and E. M. Stein, Intertwining operators for semisimple groups, Ann. of Math.(2) 93 (1971), 489–578. MR0460543 (57 #536)

[44] A. W. Knapp and E. M. Stein, Intertwining operators for semisimple groups. II, Invent.Math. 60 (1980), no. 1, 9–84, DOI 10.1007/BF01389898. MR582703 (82a:22018)

[45] A. Koldobsky, Inverse formula for the Blaschke-Levy representation, Houston J. Math. 23(1997), no. 1, 95–108. MR1688843 (2000b:42005)

[46] A. Koldobsky, Fourier analysis in convex geometry, Mathematical Surveys and Monographs,vol. 116, American Mathematical Society, Providence, RI, 2005. MR2132704 (2006a:42007)

[47] A. Koldobsky and H. Konig, Aspects of the isometric theory of Banach spaces, Handbook ofthe geometry of Banach spaces, Vol. I, North-Holland, Amsterdam, 2001, pp. 899–939.

[48] V. S. Kryuchkov, Differential properties of the symbol of the singular integral Calderon-Zygmund operator, Trudy Mat. Inst. Steklov. 170 (1984), 148–160, 276 (Russian). Studies inthe theory of differentiable functions of several variables and its applications, X. MR790334(86m:42024)

[49] P. Levy, Theorie de l’addition des variables aleatoires, Gauthier-Villars, 1937.[50] E. Lutwak, Centroid bodies and dual mixed volumes, Proc. London Math. Soc. (3) 60 (1990),

no. 2, 365–391, DOI 10.1112/plms/s3-60.2.365. MR1031458 (90k:52024)[51] G. Matheron, Un theoreme d’unicite pour les hyperplans poissoniens, J. Appl. Probability

11 (1974), 184–189 (French, with English summary). MR0372940 (51 #9144)

[52] S. Meda and R. Pini, Spherical convolution with kernels having singularities on an equa-tor, Boll. Un. Mat. Ital. B (7) 5 (1991), no. 2, 275–290 (English, with Italian summary).MR1111123 (92j:42017)

[53] K.-H. Neeb and G. Olafsson, Reflection positivity and conformal symmetry, arXiv:1206.2039,2012.

[54] A. Neyman, Representation of Lp-norms and isometric embedding in Lp-spaces, Israel J.Math. 48 (1984), no. 2-3, 129–138, DOI 10.1007/BF02761158. MR770695 (86g:46033)

[55] G. Olafsson, Fourier and Poisson transformation associated to a semisimple symmetric space,Invent. Math. 90 (1987), no. 3, 605–629, DOI 10.1007/BF01389180. MR914851 (89d:43011)


[56] G. Olafsson and A. Pasquale, On the meromorphic extension of the spherical functions onnoncompactly causal symmetric spaces, J. Funct. Anal. 181 (2001), no. 2, 346–401, DOI10.1006/jfan.2000.3721. MR1821701 (2002k:43006)

[57] G. Olafsson and A. Pasquale, The Cosλ and Sinλ transforms as intertwining operators be-tween generalized principal series representations of SL(n + 1,K), Adv. Math. 229 (2012),no. 1, 267–293, DOI 10.1016/j.aim.2011.08.015. MR2854176

[58] E. Ournycheva and B. Rubin, Composite cosine transforms, Mathematika 52 (2005), no. 1-2,53–68 (2006), DOI 10.1112/S0025579300000334. MR2261842 (2008e:42013)

[59] E. Ournycheva and B. Rubin, The composite cosine transform on the Stiefel manifold andgeneralized zeta integrals, Integral geometry and tomography, Contemp. Math., vol. 405,Amer. Math. Soc., Providence, RI, 2006, pp. 111–133.

[60] A. Pasquale, Maximal degenerate representations of SL(n + 1,H), J. Lie Theory 9 (1999),no. 2, 369–382. MR1718229 (2002e:22018)

[61] B. A. Plamenevskiı, Algebras of pseudodifferential operators, Mathematics and its Applica-tions (Soviet Series), vol. 43, Kluwer Academic Publishers Group, Dordrecht, 1989. Translatedfrom the Russian by R. A. M. Hoksbergen. MR1026642 (90h:47098)

[62] B. Rubin, Fractional calculus and wavelet transforms in integral geometry, Fract. Calc. Appl.Anal. 1 (1998), no. 2, 193–219. MR1656315 (99i:42054)

[63] B. Rubin, Inversion of fractional integrals related to the spherical Radon transform, J. Funct.Anal. 157 (1998), no. 2, 470–487, DOI 10.1006/jfan.1998.3268. MR1638340 (2000a:42019)

[64] B. Rubin, Fractional integrals and wavelet transforms associated with Blaschke-Levy rep-resentations on the sphere, Israel J. Math. 114 (1999), 1–27, DOI 10.1007/BF02785570.MR1738672 (2001b:42054)

[65] B. Rubin, Inversion and characterization of the hemispherical transform, J. Anal. Math. 77(1999), 105–128, DOI 10.1007/BF02791259. MR1753484 (2001m:44004)

[66] B. Rubin, Inversion formulas for the spherical Radon transform and the generalized cosinetransform, Adv. in Appl. Math. 29 (2002), no. 3, 471–497, DOI 10.1016/S0196-8858(02)00028-3. MR1942635 (2004c:44006)

[67] B. Rubin, Radon, cosine and sine transforms on real hyperbolic space, Adv. Math. 170(2002), no. 2, 206–223, DOI 10.1006/aima.2002.2074. MR1932329 (2004b:43007)

[68] B. Rubin, Notes on Radon transforms in integral geometry, Fract. Calc. Appl. Anal. 6 (2003),no. 1, 25–72. MR1992465 (2004e:44003)

[69] B. Rubin, Intersection bodies and generalized cosine transforms, Adv. Math. 218 (2008),no. 3, 696–727, DOI 10.1016/j.aim.2008.01.011. MR2414319 (2009m:44010)

[70] B. Rubin, Funk, cosine, and sine transforms on Stiefel and Grassmann manifolds, To appearin J. of Geom. Anal., 2012.

[71] B. Rubin and G. Zhang, Generalizations of the Busemann-Petty problem for sections ofconvex bodies, J. Funct. Anal. 213 (2004), no. 2, 473–501, DOI 10.1016/j.jfa.2003.10.008.MR2078635 (2005c:52005)

[72] W. Rudin, Lp-isometries and equimeasurability, Indiana Univ. Math. J. 25 (1976), no. 3,215–228. MR0410355 (53 #14105)

[73] S. G. Samko, Generalized Riesz potentials and hypersingular integrals with homogeneouscharacteristics; their symbols and inversion, Trudy Mat. Inst. Steklov. 156 (1980), 157–222,263 (Russian). Studies in the theory of differentiable functions of several variables and itsapplications, VIII. MR622233 (83a:45004)

[74] S. G. Samko, Singular integrals over a sphere and the construction of the characteristicfrom the symbol, Izv. Vyssh. Uchebn. Zaved. Mat. 4 (1983), 28–42 (Russian). MR706786(85f:42027)

[75] R. Schneider, Convex bodies: the Brunn-Minkowski theory, Encyclopedia of Mathematicsand its Applications, vol. 44, Cambridge University Press, Cambridge, 1993. MR1216521(94d:52007)

[76] V. I. Semjanistyı, Some integral transformations and integral geometry in an elliptic space,Trudy Sem. Vektor. Tenzor. Anal. 12 (1963), 397–441 (Russian). MR0166606 (29 #3879)

[77] E. Spodarev, On the rose of intersections of stationary flat processes, Adv. in Appl. Probab.33 (2001), no. 3, 584–599, DOI 10.1239/aap/1005091354. MR1860090 (2002i:60022)

[78] E. Spodarev, Cauchy-Kubota-type integral formulae for the generalized cosine transforms,Izv. Nats. Akad. Nauk Armenii Mat. 37 (2002), no. 1, 52–69 (2003); English transl., J.Contemp. Math. Anal. 37 (2002), no. 1, 47–63. MR1964588 (2004c:44007)


[79] R. S. Strichartz, Convolutions with kernels having singularities on a sphere, Trans. Amer.Math. Soc. 148 (1970), 461–471. MR0256219 (41 #876)

[80] R. S. Strichartz, The explicit Fourier decomposition of L2(SO(n)/SO(n − m)), Canad. J.Math. 27 (1975), 294–310. MR0380277 (52 #1177)

[81] M. Takeuchi, Modern spherical functions, Translations of Mathematical Monographs,vol. 135, American Mathematical Society, Providence, RI, 1994. Translated from the 1975Japanese original by Toshinobu Nagura. MR1280269 (96d:22009)

[82] T. T. That, Lie group representations and harmonic polynomials of a matrix variable, Trans.Amer. Math. Soc. 216 (1976), 1–46. MR0399366 (53 #3210)

[83] D. A. Vogan Jr. and N. R. Wallach, Intertwining operators for real reductive groups,Adv. Math. 82 (1990), no. 2, 203–243, DOI 10.1016/0001-8708(90)90089-6. MR1063958(91h:22022)

[84] J. A. Wolf, Harmonic analysis on commutative spaces, Mathematical Surveys and Mono-graphs, vol. 142, American Mathematical Society, Providence, RI, 2007. MR2328043(2008f:22008)

[85] G. Zhang, Radon transform on real, complex, and quaternionic Grassmannians, DukeMath. J. 138 (2007), no. 1, 137–160, DOI 10.1215/S0012-7094-07-13814-6. MR2309157(2008c:44002)

[86] G. Zhang, Radon, cosine and sine transforms on Grassmannian manifolds, Int. Math. Res.Not. IMRN (2009), no. 10, 1743–1772.


70803


Universite de Lorraine, Institut Elie Cartan de Lorraine, UMR CNRS 7502, Metz,

F-57045, France.



70803



Quantization of linear algebra and its application to integralgeometry

Hiroshi Oda and Toshio Oshima

Abstract. In order to construct good generating systems of two-sided idealsin the universal enveloping algebra of a complex reductive Lie algebra, wequantize some notions of linear algebra, such as minors, elementary divisors,and minimal polynomials. The resulting systems are applied to the integralgeometry on various homogeneous spaces of related real Lie groups.

1. Introduction

When a real Lie group GR acts on a homogeneous space, the space of func-tions or sections of line bundle on the homogeneous space is naturally an infinitedimensional representation of GR. One knows many important representations arerealized as subrepresentations of such spaces. Here it is quite usual that those sub-representations are characterized as the solutions of certain systems of differentialequations. In the first half of this article, we explain many such systems of equationscan be obtained through a quantization of elementary geometrical objects. For themost part our discussion is based on examples for GL(n,C), where our differentialequations are quantizations of some notions in linear algebra because the geometryof GL(n,C) is directly linked to linear algebra. In the second half, we show thesedifferential equations for GL(n,C) are equally applicable to the integral geometryof each real form of GL(n,C).

Let gR be the Lie algebra of GR, g its complexification, and U(g) the universalenveloping algebra of g. In general, the annihilator of a representation is a two-sidedideal in U(g). If G is the adjoint group of g (or a connected complex Lie groupwith Lie algebra g), then a two-sided ideal in U(g) is a left ideal which is stableunder the adjoint action of G. Hence, in the symmetric algebra S(g) of g, which isconsidered as the classical limit of U(g), a G-stable ideal is the classical counterpartof a two-sided ideal in U(g). Now suppose S(g) can be identified in a natural waywith the algebra P (g) of polynomial functions on g. Thus to a conjugacy class ofany A ∈ g there corresponds a big G-stable ideal of S(g). We regard a certainprimitive ideal in U(g) as a quantization of this ideal. Our systems of differentialequations are some good generating systems of these primitive ideals.

2010 Mathematics Subject Classification. Primary .Key words and phrases. Integral geometry, representation theory.The second author was supported in part by Grant-in-Aid for Scientific Researches (A), No.

20244008, Japan Society of Promotion of Science.


189

190 HIROSHI ODA AND TOSHIO OSHIMA

2. Conjugacy classes and scalar generalized Verma modules

For a while assume G = GL(n,C). As usual, we relate an n × n matrixA ∈M(n,C) to the left invariant holomorphic vector field on G defined by ϕ(x) �→ddtϕ(xe

tA)∣∣t=0

= ddtϕ(x+ txA)

∣∣t=0

. The Lie algebra g = gln of G is thus identifiedwith M(n,C). More explicitly, if Eij ∈ M(n,C) is the matrix with 1 in the (i, j)position and 0 elsewhere, the identification is written as

Eij =n∑

ν=1

xνi∂

∂xνj.

Then the adjoint action of g ∈ G on g reduces to Ad(g) : A �→ gAg−1. We denotethe algebra automorphisms of U(g), S(g) and P (g) induced from Ad(g) by the samesymbol. In this section we study an Ad(G)-stable ideal in U(g) which is consideredas a quantization of the defining ideal for the conjugacy class VA =

⋃g∈G Ad(g)A

(or its closure V A).Using the nondegenerate symmetric bilinear form

(2.1) 〈X,Y 〉 = TraceXY ,

we identify g with its dual space g∗, and S(g) with P (g) = S(g∗). The followingscheme shows our standpoint:

VA =⋃

g∈G Ad(g)A −−−−→ (G-stable) defining ideal of V A

...⏐⏐Cquantization

rep’s of U(g) or a real form GR of G ←−−−− G-stable ideal of U(g)

In order to study the classical object S(g) and its quantization U(g) at one time,the notion of homogenized enveloping algebra was introduced by [Os4]. It is analgebra defined by

(2.2) U ε(g) :=

(C[ε]⊗

∞∑m=0

m⊗g

) / ⟨X ⊗ Y − Y ⊗X − ε[X,Y ]; X,Y ∈ g

⟩.

Here ε is a complex number or an indeterminant which commutes with all elements.Clearly U(g) = U1(g), S(g) = U0(g). If ε ∈ C× then the map g � X �→ ε−1X ∈U ε(g) extends to an algebra isomorphism of U1(g) onto U ε(g). On the other hand,when ε is an indeterminant, a choice of Poincare-Birkhoff-Witt basis naturally in-duces an isomorphism U(g)⊗C[ε] ∼−→ U ε(g) of linear spaces. Furthermore, since thegenerators of the denominator of (2.2) are homogeneous of degree 2 with respect toε and X ∈ g \ {0}, we can endow U ε(g) with a graded algebra structure such thatε as well as any X ∈ g \ {0} has degree 1.

For a sequence {n′1, . . . , n

′L} of positive integers whose sum is n, put⎧⎪⎨⎪⎩

nk = n′1 + · · ·+ n′

k (1 ≤ k ≤ L), n0 = 0,

Θ = {n1, . . . , nL},ιΘ(ν) = k if nk−1 < ν ≤ nk (1 ≤ k ≤ L).

Clearly Θ is a strictly increasing sequence of positive integers terminating at n andto such a sequence Θ there corresponds a unique {n′

1, . . . , n′L}. Let us define some

QUANTIZATION OF LINEAR ALGEBRA 191

Lie subalgebras of g = gln as follows:

n =∑i>j

CEij , n =∑i<j

CEij , a =∑i

CEii,

b = a+ n, nΘ =∑

ιΘ(i)>ιΘ(j)

CEij , nΘ =∑

ιΘ(i)<ιΘ(j)

CEij ,

mΘ =∑

ιΘ(i)=ιΘ(j)

CEij , mkΘ =

∑ιΘ(i)=ιΘ(j)=k

CEij , bΘ = mΘ + nΘ.

One knows that bΘ is a standard parabolic subalgebra containing the Borel sub-algebra b and any standard parabolic subalgebra equals bΘ for some unique Θ.

Notice that mΘ =⊕L

k=1mkΘ and bΘ = {X ∈ g; 〈X,Y 〉 = 0 (∀Y ∈ nΘ)}.

For a fixed λ = (λ1, . . . , λL) ∈ CL, let us consider the affine subspace

AΘ,λ :=

n∑i=1

λιΘ(i)Eii + nΘ

=

⎧⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎩

⎛⎜⎜⎜⎜⎜⎝λ1In′

1 0A21 λ2In′2

A31 A32 λ3In′3

......

.... . .

AL1 AL2 AL3 · · · λLIn′L

⎞⎟⎟⎟⎟⎟⎠ ; Aij ∈M(n′i, n

′j ;C)

⎫⎪⎪⎪⎪⎪⎬⎪⎪⎪⎪⎪⎭of g. Here Im is the identity matrix of size m and M(k, �;C) is the set of k × �matrices.

Remark 2.1. A generic element of AΘ,λ belongs to a common conjugacy class,whose Jordan normal form is given by⊕

μ∈C, 1≤k≤n

J(#{i; λi = μ and n′

i ≥ k}, μ)

where J(m,μ) =

⎛⎜⎜⎜⎝μ 01 μ

. . .. . .

0 1 μ

⎞⎟⎟⎟⎠ ∈M(m,C).

Hereafter this conjugacy class is referred to as the conjugacy class of AΘ,λ. AnyJordan normal form is an element of such a conjugacy class for some choice of Θand λ. The closure of the conjugacy class of AΘ,λ is

VAΘ,λ:=⋃g∈G

Ad(g)AΘ,λ.

In the classical case, the condition that a function f ∈ P (g) = S(g) = U0(g)vanishes on the conjugacy class of AΘ,λ is equivalent to any of the following withε = 0:

(2.3)

f(VAΘ,λ) = {0} ⇐⇒

(Ad(g)f

)(AΘ,λ) = {0} (∀g ∈ G)

⇐⇒ Ad(g)f ∈ JεΘ(λ) (∀g ∈ G)

⇐⇒ f ∈⋂g∈G

Ad(g)JεΘ(λ)

⇐⇒ f ∈ AnnG(M ε

Θ(λ)).


Here for ∀ε ∈ C we set

JεΘ(λ) :=

L∑k=1

∑X∈mk

Θ

U ε(g)(X − λk Trace(X)) + U ε(g)nΘ,

M εΘ(λ) := U ε(g)/Jε

Θ(λ),

Ann(M ε

Θ(λ)):={D ∈ U ε(g); DM ε

Θ(λ) = 0}

IεΘ(λ) := AnnG(M ε

Θ(λ)):={D ∈ U ε(g); Ad(g)D ∈ Ann

(M ε

Θ(λ))(∀g ∈ G)

}.

When ε = 1 we omit the superscript 1 and use such notation as MΘ(λ) = M1Θ(λ).

Similarly, when Θ = {1, . . . , n} we omit the subscript Θ and use such notation asM ε(λ) = M ε

Θ(λ). MΘ(λ) is called a generalized Verma module of the scalar typeand is a quotient g-module of the Verma module M(λΘ) for the parameter

(2.4) λΘ := (λ1, . . . , λ1︸︷︷︸n′1

, λ2, . . . , λ2︸︷︷︸n′2

, . . . , λL, . . . , λL︸︷︷︸n′L

) ∈ Cn.

Since we realized that the defining ideal of VAΘ,λis I0Θ(λ) = AnnG

(M ε

Θ(λ)), it

is natural to think its quantization is IΘ(λ) = AnnG(MΘ(λ)

)= Ann

(MΘ(λ)

). In

fact the last two equivalences in (2.3) are valid for any ε ∈ C and any f ∈ U ε(g).Now let us formulate the main problem in the first half of this article.

Problem 2.2. For ε = 0, 1 construct good generating systems of IεΘ(λ).

In the following sections we shall give some concrete answers. Our generatingsystems will always be in U ε(g) and they are valid for any ε.

3. Eigenvalues and determinants

The space a =∑n

i=1 CEii of diagonal matrices is isomorphic to the linear spaceCn = {(x1, . . . , xn)} on which the n-th symmetric group Sn acts by permutationof coordinates. If we identify S(a) with P (a) by (2.1), then the restriction mapS(g) → S(a) is naturally defined and the Chevalley restriction theorem asserts itinduces the algebra isomorphism

Γ0 : S(g)G S(a)Sn .

One knows the elementary symmetric polynomials

sm(x) =∑

1≤i1<···<im≤n

xi1 · · ·xim (m = 1, . . . , n)

generate S(a)Sn and so do the power sum polynomials

Sm(x) =n∑

i=1

xmi (m = 1, 2, . . .).

The eigenvalues of any matrix in VAΘ,λ, counted with multiplicities, coincide

with the entries of λΘ given by (2.4). Thus the collection of them is an invariantof VAΘ,λ

. We note it is completely determined by the values at λΘ of the elements

in a generating system of S(a)Sn , e.g. the sequence {s1(λΘ), . . . , sn(λΘ)}, or thesequence {S1(λΘ), S2(λΘ), . . .}. Now any f ∈ S(g)G takes the value Γ0(f)(λΘ)constantly on VAΘ,λ

. Analogously, any D ∈ U(g)G acts on MΘ(λ) by a scalar.(Namely, MΘ(λ) has an infinitesimal character.) These are special cases of the


general fact that D ∈ U ε(g)G acts on M εΘ(λ) by the scalar γε(D)(λΘ). Here γε

denotes the quantization of the restriction map S(g)→ S(a) defined by

γε : U ε(g) � D �→ γε(D) ∈ U ε(a) = S(a)(D − γε(D) ∈ nU ε(g) + U ε(g)n

).

Note that U ε(g) =(nU ε(g) + U ε(g)n

)⊕ U ε(a) is a direct sum decomposition and

U ε(a) = S(a) by the commutativity of a. If we put ρ =∑n

i=1(i − n+12 )Eii and

define the translation Tερ : S(a) � D �→ D( · − ερ) ∈ S(a) under the identificationS(a) = P (a), then Γε := Tερ ◦ γε induces the algebra isomorphism

(3.1) Γε : U ε(g)G S(a)Sn .

If ε �= 0, then (3.1) is the celebrated Harish-Chandra isomorphism. So we refer toγε or Γε as the Harish-Chandra map.

Put E =(Eij

)∈M(n, U ε(g)). Since

(3.2) Ad(g)E = tgE tg−1 (∀g ∈ G),

we have

(3.3) Zm := TraceEm (m = 1, 2, . . .)

in U ε(g)G (cf. Gelfand’s construction in [Ge]). It is easy to see that the highesthomogeneous part of Γε(Zm) equals Sm(x). Hence U ε(g)G = C[Z1, . . . , Zn] (∀ε ∈C). Although the equality Γ0(Zm) = Sm(x) is immediate, a nontrivial calculationis necessary to write Γ1(Zm) down explicitly (cf. §7 Remark 7.2 ii)).

Now, for t ∈ C we define a quantized determinant by

(3.4) D(t) := det(Eij +

(ε(n− j)− t

)δij

)∈ U ε(g),

where the determinant in the right-hand side is a so-called column determinant. Inthis article, the determinant of a square matrix

(Aij

)with non-commutative entries

means the column determinant given by

det(Aij

)=∑

σ∈Sn

sgn(σ)Aσ(1)1 · · ·Aσ(n)n.

The G-invariance of D(t) with ε = 1 is a well-known classical result (cf. [Ca1]),which in fact follows from the Capelli identity

det(Eij + (n− j)δij

)= det

(xij

)det( ∂

∂xij

)and the algebra automorphism of U(g) defined by Eij �→ Eij− tδij . More generallywe have D(t) ∈ U ε(g)G. The image of D(t) under the Harish-Chandra map is easilycalculated:

γε(D(t)) =n∏

i=1

(Eii − t+ ε(i− 1)

), Γε(D(t)) =

n∑m=0

sm(x)(n− 1

2ε− t

)n−m

.

(Here we let s0(x) = 1.) Hence if we consider D(t) ∈ U ε(g)G[t] and denote itscoefficient of tm by Δm, then U ε(g)G = C[Δ0, . . . ,Δn−1] (∀ε ∈ C).

Remark 3.1. i) When ε is an indeterminant, then U ε(g)G = C[ε, Z1, . . . , Zn] =C[ε,Δ0, . . . ,Δn−1].ii) Various relations between {Zm}, D(t) and other central elements in U(g) arestudied by T. Umeda [U] and M. Ito [I].


iii) The construction method (3.3) of central elements applies to general complexreductive Lie algebras (cf. §7). On the other hand, there is a version of (3.4) forg = on, the Lie algebra of O(n,C) (cf. [HU], [Wa]).

Suppose λ ∈ Cn. Since M ε(λ) has infinitesimal character U ε(g)G � D �→γε(D)(λ) ∈ C, we have D − γε(D)(λ) ∈ Iε(λ) (∀D ∈ U ε(g)G). More strongly,

(3.5) Iε(λ) =∑

D∈Uε(g)G

U ε(g)(D − γε(D)(λ)

).

When λ = 0 and ε = 0, the above equality reduces to the assertion that the definingideal of VA{1,...,n},0 is generated by the G-invariant polynomials without constantterm. Here we note VA{1,...,n},0 is the set of all nilpotent matrices. This assertion is

proved by B. Kostant [Ko] for all complex reductive Lie algebras, from which (3.5)in the general case is readily deduced.

For λ ∈ Cn put

Iελ :={D ∈ U ε(g); γε(Ad(g)D)(λ) = 0 (∀g ∈ G)

}.

This is a two-sided ideal of U ε(g). If ε = 0, I0λ is the defining ideal of the conjugacyclass Vλ of a diagonalizable matrix whose eigenvalues are the entries of λ. If ε = 1,Iλ = Ann

(L(λ)

):={D ∈ U(g); DL(λ) = 0

}where L(λ) is the unique irreducible

quotient of the Verma module M(λ). Thus Iλ is a primitive ideal of U(g). Con-versely, {Iλ; λ ∈ Cn} equals the set of all primitive ideals (cf. [Du]). For w ∈ Sn

define its shifted action by w.λ = w(λ+ ερ)− ερ. Then it holds that Iεw.λ = Iελ fora generic λ. This is not true for some λ (for example, when λ = 0).

Now suppose Θ and λ ∈ C#Θ are arbitrary. Since M εΘ(λ) is a quotient of

M ε(λΘ), IεΘ(λ) ⊃ Iε(λΘ). On the other hand, we assert IεΘ(λ) ⊂ IελΘand the

equality holds for a generic λ ∈ C#Θ. In fact, when ε = 0, we have VAΘ,λ⊃ VλΘ

and VAΘ,λ= VλΘ

if each entry of λ ∈ C#Θ is distinct. So I0Θ(λ) ⊂ I0λΘand both are

equal for a generic λ. When ε = 1, since L(λΘ) is the unique irreducible quotient ofMΘ(λ) and since MΘ(λ) is irreducible for a generic λ, the assertion holds. Finallywe remark IΘ(λ) is always a primitive ideal because even if MΘ(λ) is reducible, wecan choose a w ∈ Sn so that IΘ(λ) = Iw.λΘ

.

4. Restriction to the diagonal part and completely integrable quantumsystems

When one wants to construct a generating system of the defining ideal I0λ of asemisimple conjugacy class Vλ (λ ∈ Cn), it is very useful to consider the restrictionof I0λ to the diagonal part a, that is, the ideal of S(a) = C[x1, . . . , xn] defined by

γ0(I0λ):={γ0(f) = f |a; f ∈ I0λ

}.

Let I(Snλ) denote the defining ideal of the finite subset Snλ in Cn.

Lemma 4.1. Suppose E ⊂ S(g) is a G-stable linear subspace. Then E := γ0(E)

is an Sn-stable linear subspace of S(a) and

E generates I0λ ⇐⇒ E generates I(Snλ).

In particular γ0(I0λ)= I(Snλ).


For example, I(Snλ) contains

(4.1) sm(x)− sm(λ) (m = 1, . . . , n).

If each entry of λ = (λ1, . . . , λn) is distinct, then Snλ consists of n! points and(4.1) generates I(Snλ) (so does {Sm(x) − Sm(λ); m = 1, . . . , n}). In this case,I0λ = I0(λ) and by Lemma 4.1 the assertion above is equivalent to (3.5) with ε = 0.

In addition to (4.1), I(Snλ) contains

n∏i=1

(xi − λj) (j = 1, . . . , n),(4.2)

n∏j=1

(xi − λj) (i = 1, . . . , n).(4.3)

If each entry of λ is distinct, (4.2) is also a generating system. But in other cases,even the combination of (4.2) with (4.3) does not generate I(Snλ). Suppose λ =(μ, . . . , μ︸︷︷︸

k

, ν, . . . , ν︸︷︷︸n−k

), μ �= ν, for example. In this case Snλ consists of n!k!(n−k)! points

and instead of (4.2) or (4.3), we should consider the following elements in I(Snλ):{(xi1 − μ) · · · (xin−k+1

− μ) (1 ≤ i1 < · · · < in−k+1 ≤ n),

(xj1 − ν) · · · (xjk+1− ν) (1 ≤ j1 < · · · < jk+1 ≤ n),

(4.4)

(xi − μ)(xi − ν) (i = 1, . . . , n).(4.5)

Then (4.4) generates I(Snλ). Also, the system (4.5), together with (4.1) for m = 1,generates I(Snλ). Note that both (4.4) and (4.5) span Sn-stable linear subspaces.

Remark 4.2. The Sn-invariant completely integrable quantum systems arethose systems of differential equations on a which are classified by [OSe]. Theyare considered as quantizations of (4.1) (cf. [OP], [Os5]). In general, solutions(wave functions) of these systems are not so well understood. But on the Heckman-Opdam hypergeometric functions (cf. [HO]) and on the generalized Bessel functions(cf. [Op]), there are many results. The most trivial way of quantization is tosimply replace xi with

∂∂xi

in (4.1). The solution space of this system is spannedby exponential polynomials and as an Sn-module it is isomorphic to the regularrepresentation of Sn. When λ = 0 a solution of the system is a so-called Sn-harmonic polynomial. A basis of the solution space which is entirely holomorphicin (x, λ) is given by [Os2].

Remark 4.3. For any Sn-stable linear subspace E ⊂ S(a) there exists a

GL(n,C)-stable linear subspace E ⊂ S(g) such that E = γ0(E). But the corre-

sponding assertion is not always true for a general complex reductive Lie group orin the similar setting for a Riemannian symmetric space (cf. [Br], [Od4]).

5. Minors

The rank of a matrix is also a basic invariant of a conjugacy class. Recall it isdescribed in terms of the minors. For example, suppose Θ = {k, n} and λ = (μ, ν).Then

(5.1) AΘ,λ =

{(μIk 0∗ νIn−k

)},


and for any A ∈ VAΘ,λwe have rank(A − μ) ≤ n − k and rank(A − ν) ≤ k.

Hence the minors of(Eij − μ

)∈M(n, S(g)) with size n− k + 1 and the minors of(

Eij − ν)∈M(n, S(g)) with size k + 1 vanish on VAΘ,λ

.

For t ∈ C let us define a quantization of the minors of(Eij − t

)to be

D{i1,...,im}{j1,...,jm}(t) := det(Eipjq +

(ε(m− q)− t

)δipjq

)1≤p≤m1≤q≤m

∈ U ε(g)({i1, . . . , im}, {j1, . . . , jm} ⊂ {1, . . . , n}

)and call them the generalized Capelli elements. As in the classical case, they changetheir sign by a transposition of row or column indices, and for any fixed t and ε,the elements of

{DIJ (t); #I = #J = m, I, J ⊂ {1, . . . , n}

}span a G-stable linear

space. Moreover, if ε = 1 we have the following generalized Capelli identity :

(5.2) D{i1,...,im}{j1,...,jm}(0) =∑

1≤ν1<···<νm≤n

det(xνpiq

)1≤p≤m1≤q≤m

· det( ∂

∂xνpjq

)1≤p≤m1≤q≤m

.

Now suppose Θ = {k, n} and λ = (μ, ν) again. Then for any ε

(5.3) DIJ (μ), DI′J′(ν + kε) (#I = #J = n− k + 1, #I ′ = #J ′ = k + 1)

belong to IεΘ(λ). This can be shown by calculating their images under the Harish-Chandra map γε. In the classical case where ε = 0, since γ0 maps (5.3) to (4.4),Lemma 4.1 implies that if μ �= ν, then (5.3) generate I0Θ(λ) = I0λΘ

. For a generalε, we can show (5.3) generate IεΘ(λ) if μ − ν /∈ {ε, 2ε, . . . , (n − 1)ε}. But in orderto obtain generating systems of IεΘ(λ) for all cases without exception, it is notsufficient to consider only the generalized Capelli elements. Besides them, we needthe notion of elementary divisors and their quantization, which are discussed in thenext section.

Remark 5.1. i) For any Θ and a generic λ ∈ C#Θ we can construct a generatingsystem of IεΘ(λ) which consists only of generalized Capelli elements (see (6.1)).ii) When g = on, we can use a suitable quantization of the minor Pfaffians andthe minor versions of the quantized determinant given by Howe–Umeda [HU] toconstruct a generating system of (the corresponding object to) IεΘ(λ) (cf. [Od1],[Od2]).

6. Elementary divisors

Let g = gln and suppose Θ, λ ∈ CL (L = #Θ) and ε ∈ C are arbitrary.

Definition 6.1 ([Os4]). For m = 1, . . . , n define⎧⎪⎪⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎪⎪⎩

dεm(t; Θ, λ) :=

L∏k=1

(t− λk − εnk−1)(n′

k+m−n),

dm(Θ) := degt dεm(t; Θ, λ) =

L∑k=1

max{n′k +m− n, 0},

eεm(t; Θ, λ) := dεm(t; Θ, λ)/dεm−1(t; Θ, λ).

Here dε0(t; Θ, λ) = 1 and

z(i) :=

{z(z − ε

)· · ·(z − ε(i− 1)

)if i > 0,

1 if i ≤ 0.


We call {eεm(t; Θ, λ); 1 ≤ m ≤ n} the elementary divisors of M εΘ(λ).

If ε = 0 and A is a generic in AΘ,λ, then d0m(t; Θ, λ) is the greatest commondivisor of the minors of tIn − A with size m. Hence e0m(t; Θ, λ) is nothing but them-th elementary divisor of a generic element of AΘ,λ in the sense of linear algebra.

Theorem 6.2 ([Os4]). Write dεm(t; Θ, λ) =∏�m

i=1(t − λm,i)Nm,i (i �= i′ ⇒

λm,i �= λm,i′) and put

EεΘ(λ) :=n∑

m=1

�m∑i=1

Nm,i−1∑j=0

∑#I=#J=m

C

(dj

dtjDIJ (t)

∣∣∣t=λm,i

).

Then IεΘ(λ) = U ε(g)EεΘ(λ). Moreover if all the roots of dεn(t; Θ, λ) are simple (inother words if w.λΘ (w ∈ Sn) are all distinct), it holds that

(6.1) IεΘ(λ) =

L∑k=1

∑#I=#J=n−n′

k+1

U ε(g)DIJ(λk + εnk−1).

Remark 6.3. i) An inclusion relation between annihilator ideals reduces to adivisibility relation between the elementary divisors as follows:

IεΘ(λ) ⊂ IεΘ′(λ′)⇐⇒ dεm(t; Θ, λ) | dεm(t; Θ′, λ′) (m = 1, . . . , n).

If ε = 0, the left-hand side is equivalent to the closure relation VAΘ,λ⊃ VAΘ′,λ′ .

In particular if ε = 0 and λ = 0, this is a closure relation between conjugacyclasses of nilpotent matrices (nilpotent orbits), which is equivalent to the well-known condition dm(Θ) ≤ dm(Θ′) (m = 1, . . . , n).ii) The special case of Theorem 6.2 where ε = 0 and λ = 0 is conjectured byT. Tanisaki [Ta1] and is proved by J. Weymann [We]. Theorem 6.2 in the generalcase can be considered as its quantization.iii) The special case of Theorem 6.2 where Θ = {1, . . . , n} is equivalent to (3.5).

7. Characteristic polynomials and minimal polynomials

Suppose AΘ,λ is as in (5.1). Because the minimal polynomial for a genericelement of AΘ,λ is (t− μ)(t− ν), all entries of (E− μ)(E− ν) ∈M(n, S(g)) vanish

on VAΘ,λ. Let E be the linear subspace in S(g) spanned by these entries. It is

G-stable by (3.2) and γ0(E) is spanned by (4.5). Hence it follows from Lemma 4.1

that if μ �= ν, the system E together with TraceE − kμ − (n − k)ν generatesI0Θ(λ) = I0λΘ

. A quantization of minimal polynomials can be formulated for generalcomplex reductive Lie algebras:

Definition 7.1 ([Os7]). Suppose g is a complex reductive Lie algebra andπ : g→M(N,C) is its faithful representation. By the faithfulness we identify g withπ(g) ⊂ M(N,C). Suppose moreover that the symmetric bilinear form 〈X,Y 〉 =TraceXY (X,Y ∈ π(g)) is nondegenerate on g × g (the assumption is automaticif g is semisimple). Let π∨ denote the orthogonal projection of M(N,C) onto g

with respect to 〈 · , · 〉. If we put Fπ =(π∨(Eij)

), then Fπ can be regarded as an

element of M(N,U ε(g)) (∀ε ∈ C). We call a monic polynomial q(t) ∈ U ε(g)G[t] thecharacteristic polynomial of Fπ (or π) if it satisfies q(Fπ) = 0 with the lowest degreeand denote it by qεπ(t). Also, letM be a U ε(g)-module. We call a monic polynomialq(t) ∈ C[t] the minimal polynomial of the pair (π,M) if it satisfies q(Fπ)M = 0


(namely each entry of q(Fπ) annihilates M) with the lowest degree and denote itby qπ,M(t).

Remark 7.2. i) There always exists the characteristic polynomial qεπ(t). Forany ε ∈ C, U ε(g)G S(a)W by the Harish-Chandra isomorphism Γε where S(a)W

is the algebra of Weyl group invariants in the symmetric algebra of a Cartan sub-algebra a (cf. (3.1)). The explicit formula of qεπ(t), regarded as an element ofS(a)W [t], is calculated by M. D. Gould [Go2]. In this formula we can interpretε as an indeterminant because qεπ(t) ∈ S(a)W [t] polynomially depends on ε. Letqπ(t) ∈ S(a)W [t, ε] be such an interpretation of qεπ(t).ii) TraceFm

π ∈ U ε(g)G (m = 0, 1, . . .) and Γε(TraceFm

π

)are explicitly calculated by

M. D. Gould [Go1].iii) If a U ε(g)-module M has a finite length or if it has an infinitesimal character,then there exists the minimal polynomial qπ,M(t).

For example, if g = on and π is its natural representation, then Fπ =(

Eij+Eji

2

).

If g = gln and π is its natural representation, then Fπ = E and for any increasingsequence Θ and λ ∈ C#Θ the polynomial qπ,M0

Θ(λ)(t) coincides with the minimalpolynomial of a generic element of AΘ,λ in the sense of linear algebra.

Theorem 7.3 ([Os7]). Suppose g = gln and π is its natural representation.i) The characteristic polynomial of E is given by

(7.1) qπ(t) = det(−Eij +

(t− ε(n− j)

)δij

)= (−1)n ×

(D(t) defined by (3.4)

)and it holds that qπ(E) = 0 (a quantized Cayley-Hamilton theorem).ii) For any Θ and λ ∈ CL (L = #Θ), the minimal polynomial of (π,M ε

Θ(λ)) is

(7.2) qπ,MεΘ(λ) =

L∏k=1

(t− λk − εnk−1).

Let Eε be the linear subspace of U ε(g) spanned by the n2 entries of qπ,MεΘ(λ)(E).

Then Eε is G-stable and Eε together with{Zm − γε(Zm)(λΘ); m = 1, . . . , L − 1

}generates IεΘ(λ) for a generic λ (it is sufficient if all w.λΘ (w ∈ Sn) are distinctor if ε = 0 and all entries of λ ∈ CL are distinct). Here Zm(m = 1, . . . , L− 1) arethe elements of U ε(g)G defined by (3.3).

Remark 7.4. Under the assumption in the above theorem the ideal of S(a)generated by the entries of Γε

(f(E)

)is calculated in [Os7, Theorem 4.19] for any

polynomial f(t) ∈ C[t].

For a general g we have the following:

Theorem 7.5. Suppose g and π are as in Definition 7.1. Let M εΘ(λ) be the

scalar generalized Verma module for a standard parabolic subalgebra bΘ and itscharacter λ ∈ (bΘ/[bΘ, bΘ])

∗ (the subscript Θ is a suitable parameter specifying thestandard parabolic subalgebra). Then there exists a polynomial qεπ,Θ(t;λ) in t, λ and

ε such that qπ,MεΘ(λ)(t) = qεπ,Θ(t;λ) for a generic λ (the equality holds if all the roots

of qεπ,Θ(t;λ) as a polynomial in the single variable t are simple). Moreover, for any

fixed ε ∈ C and λ the divisibility relation qπ,MεΘ(λ)(t) | qεπ,Θ(t;λ) holds in C[t]. We

call qεπ,Θ(t;λ) the global minimal polynomial of (π,Θ).


Remark 7.6. i) The explicit form of qεπ,Θ(t;λ) is determined by [Os7] in thecase where g is any classical Lie algebra and π is its natural representation. Thatin the fully general case is determined by [OO].ii) If g = on or spn and π is its natural representation, then the explicit form ofqπ,Mε

Θ(λ)(t) for any λ is determined by [Od3]. (That for g = gln is given by (7.2).)

iii) Let Θ0 denote the Θ specifying the Borel subalgebra b. Then qεπ,Θ0(t;λ) (λ ∈

a∗ (b/[b, b])∗) equals the polynomial obtained by evaluating each coefficient of thecharacteristic polynomial qπ(t) ∈ S(a)W [t, ε] at λ+ ερ. Here ερ = 1

2 Trace(ad[b,b]

).

iv) If π satisfies a certain additional condition, then for any Θ and a generic λ wecan construct a generating system of IΘ(λ) = Ann

(MΘ(λ)

)in the same way as

Theorem 7.3 ii). (For instance, this is possible if g is simple and π is the adjointrepresentation or a faithful representation with the lowest dimension.)

8. Integral geometry — Poisson transform and Penrose transform

In the case of g = gln, we have two different generating systems for the two-sided ideal IεΘ(λ) in U ε(g), which are respectively given by Theorem 6.2 and byTheorem 7.3 ii). (As for their relation, see [Sak].) The next theorem says the idealIεΘ(λ) has the role of filling the gap between the two left ideals of U ε(g), Jε

Θ(λ) (thedenominator of M ε

Θ(λ)) and Jε(λΘ) (the denominator of M ε(λΘ)). This propertyis important in applications to integral geometry.

Theorem 8.1. It holds for a generic λ that

(8.1) JεΘ(λ) = IεΘ(λ) + Jε(λΘ) (GAP).

Remark 8.2. i) The theorem is valid for all complex reductive Lie algebras.In the case where ε = 1, there is a sufficient condition for (8.1) given by Bernstein–Gelfand [BG] and A. Joseph [Jos], while [OO] obtains some conditions finer thanit through the explicit calculations of qεπ,Θ(t;λ) for various π. When ε = 0, (8.1)

holds if I0Θ(λ) = I0λΘ.

ii) In the case when g = gln a necessary and sufficient condition for (8.1) is givenby [Os4]. For example, (8.1) is valid if w.λΘ (w ∈ Sn) are all distinct.

Hereafter, we assume that ε = 1 and that GR is a real form of G = GL(n,C)or G = SL(n,C) such as GL(n,R), U(p, q) and SU∗(2m), or GR = GL(n,C) asa real form of G = GL(n,C) × GL(n,C). (More generally we may assume GR isa real form of a connected complex reductive Lie group G.) Let P be a minimalparabolic subgroup of GR and PΞ a parabolic subgroup containing P . Thus GR/PΞ

is a generalized flag variety. Let K be a maximal compact subgroup of GR and λa one-dimensional representation of PΞ such that λ(PΞ ∩K) = {1}. Put

B(GR/PΞ;λ) :={f ∈ B(GR); f(xp) = λ(p)f(x) (∀p ∈ PΞ)

}.

This is the space of hyperfunction sections of the line bundle on G/PΞ associatedto λ−1. When the action of g ∈ GR on B(GR/PΞ;λ) is given by Lg : f(x) �→f(g−1x), it is called a degenerate principal series representation. Since the Liealgebra g of G is the complexification of the Lie algebra gR of GR, the differentialaction LD ∈ EndB(GR/PΞ;λ) is defined for any D ∈ U(g). Now we note thatthe complexification of the Lie algebra of PΞ equals bΘ for some Θ and that thedifferential representation of λ is a character of bΘ (also denoted by λ). It is easy


to show the following equality holds:(8.2)

Ann(B(GR/PΞ;λ)

):={D ∈ U(g); LDf = 0 (∀f ∈ B(GR/PΞ;λ))

}= tIΘ(λ).

Here the rightmost side is the image of IΘ(λ) under the antiautomorphism D �→ tDof U(g) induced by g � X �→ −X ∈ g. Therefore, for any given GR-map ofB(GR/PΞ;λ) into the space of functions or line bundle sections on some other GR-homogeneous space, the image always satisfies the system of differential equationscorresponding to tIΘ(λ). We remark such a GR-map is usually given by an integraloperator since GR/PΞ is compact.

Example 8.3 (Grassmannians). Let F = R,C, or H. The manifold Grk(Fn)which consists of all k-dimensional linear subspace in Fn is called the Grassmannmanifold and is an important example of generalized flag varieties. For example, ifF = R,

Grk(Rn) := {k-dimensional linear subspace ⊂ Rn} (real Grassmann manifold)

= M◦(n, k;R)/GL(k,R)

where M◦(n, k;R) :={X ∈ M(n, k;R); rankX = k

}. In addition, if we let GR =

GL(n,R) act on Grk(Rn) by g ◦X = tg−1X, then we have

Grk(Rn) = GL(n,R)/Pk,n O(n)/O(k)×O(n− k)

where

Pk,n :=

{p =

(g1 0y g2

); g1 ∈ GL(k,R), g2 ∈ GL(n− k,R), y ∈M(n− k, k;R)

}.

Now we identify λ = (μ, ν) ∈ C2 with the character p �→ | det g1|μ| det g2|ν of Pk,n

and consider(8.3)B(GR/Pk,n;λ) =

{f ∈ B(GR); f(xp) = f(x)| det g1|μ| det g2|ν (∀p ∈ Pk,n)

} B(O(n)/O(k)× O(n− k)).

In this case, Θ = {k, n} and the ideal IΘ(λ) = t Ann(B(GR/Pk,n;λ)

)contains

the determinant-type differential operators of order k + 1 and n − k + 1 given by(5.3), the second order differential operators of Theorem 7.3 ii), and the first orderdifferential operator Z1 − γ(Z1)(λΘ) coming from Trace. Notice that if ν = 0 then(8.3) is also isomorphic to{

f ∈ B(M◦(n, k;R)); f(Xg1) = f(X)| det g1|−μ (∀g1 ∈ GL(k,R))}

as a GR-module.

Poisson transform.We call the GR-map

PΞ,λ : B(GR/PΞ;λ) −→(⊂ B(GR/P ;λ)

Pλ−−→)A(GR/K;Mλ)

∈ ∈

f �−→ (Pλf)(x) =

∫K

f(xk)dk

a Poisson transform. Here, Mλ is a maximal ideal attached to λ in the algebraof invariant differential operators on the Riemannian symmetric space GR/K, and


A(GR/K;Mλ) is the solution space for it. We remark GR/PΞ is isomorphic to apart of the boundary of a certain realization of GR/K.

Suppose PΞ = P for a while. Then for a suitable λ the Poisson transform Pλ isa topological isomorphism of B(GR/P ;λ) ontoA(GR/K;Mλ). This fact is observedby Helgason in some special cases. (For instance, if G = SL(2,R) and λ = 0, thena harmonic functions on the unit disk is the Poisson integral of a hyperfunction onthe unit circle.) He gives in the general case a necessary and sufficient conditionon λ for the injectivity of Pλ (cf. [He1]) and conjectures that the surjectivity alsoholds under the same condition. Helgason’s conjecture is proved by [K-].

Now suppose PΞ is arbitrary and λ satisfies the condition for the bijectivity ofPλ (it is sufficient if λ is trivial). Let us concentrate on the problem of giving aconcrete system of differential equations on GR/K which characterizes the imageof PΞ,λ,

PΞ,λ

(B(GR/PΞ;λ)

)= Pλ

(B(GR/PΞ;λ)

)⊂ A(GR/K;Mλ).

When λ is trivial, the problem is known as Stein’s problem and there are many stud-ies on it. In particular when GR/K is a bounded symmetric domain and PΞ is itsShilov boundary, various systems of differential equations are constructed (cf. [BV],[La], [KM]). Also, K. D. Johnson [Joh] gives a unified method of constructing dif-ferential equations which applies to any GR/K and PΞ when λ is trivial. But thismethod is not explicit enough to give the concrete form of differential operators. Onthe other hand, N. Shimeno [Sh] studies a generalized version of this problem for abounded symmetric domain GR/K, certain types of PΞ, and any λ. The systems ofdifferential equations given in these works are called Hua systems after L. K. Hua[Hu], the mathematician who first studied this problem.

We now return to the general setting and assume (8.1) holds. Then we have

B(GR/PΞ;λ) ={f ∈ B(GR/P ;λ); LDf = 0 (∀D ∈ tIΘ(λ))

}.

It follows that the image of PΞ,λ is characterized by tIΘ(λ) (and Mλ). Hence byapplying t· to any of those generating systems of IΘ(λ) constructed in the previoussections, we obtain a concrete system of differential equations characterizing theimage of PΞ,λ.

Remark 8.4. i) Since most of the known Hua systems coincide with systemscoming from minimal polynomials in §7, we can treat them from such a unifiedpoint of view. For example, in the case of the Shilov boundary of a boundedsymmetric domain, the system of differential equations has order 2 or 3 accordingas the domain is of tube type or not. We can explain the reason by the degreeof minimal polynomials. In the case of the Shilov boundary of SU(p, q)/S(U(p)×U(q)), the degree of the minimal polynomial is 2 if p = q and 3 otherwise. Butthere always exists a second order system even if p �= q (cf. [BV]). We can clarifythis phenomenon by decomposing the GR-stable generating system coming fromthe minimal polynomial into the sum of K-submodules (cf. [OSh]). Moreover, ourapproach enables us to determine at least which elements fromMλ we should addto the system.ii) When GR is a classical Lie group and PΞ is a maximal parabolic subalgebra,the generating system of (8.2) coming from the minimal polynomial of the naturalrepresentation has order ≤ 3. But this is not the case when GR is of exceptionaltype, PΞ is maximal, and the minimal polynomial is that of a faithful representationwith the lowest dimension (cf. [OO]).


iii) The image of a function space such as D′, Lp, C∞, Cm under the transformationPλ is studied by [BOS].

Penrose transform.Let bΘ ⊂ g be a parabolic subgroup and BΘ the corresponding parabolic sub-

group of the complex Lie group G. Let Oλ denote the sheaf of holomorphic sectionsof the line bundle on G/BΘ associated to a one-dimensional holomorhic representa-tion λ of BΘ (or bΘ). With respect to the natural action of U(g), Oλ is annihilatedby tIΘ(−λ). Hence, for any GR-orbit V in G/BΘ the local cohomology Hm

V (Oλ)is a GR-module which is annihilated by tIΘ(−λ). Accordingly, if TPen is a map ofHm

V (Oλ) into the space S of line bundle sections on a GR-homogeneous space suchas the Riemannian symmetric space (a Penrose transform), then the image of TPensatisfies the system of differential equations corresponding to tIΘ(−λ).

For example, suppose G = GL(2n,C), GR = U(n, n) and V is the closedorbit of G/BΘ with Θ = {k, 2n} (thus G/BΘ is the complex Grassmann manifoldGrk(C2n)). In the additional setting such that S is the space of sections of aline bundle on the bounded symmetric domain GR/K = U(n, n)/U(n) × U(n),H. Sekiguchi [Se] examines a Penrose transform and in particular proves the imagecoincieds with the space of holomorphic solutions for the system of differentialequations based on (6.1). This system can be expressed by some determinant-type differential operators with constant coefficients because (5.2) holds for thegeneralized Capelli elements in the generating system.

9. Integral geometry — Radon transform, hypergeometric functions

Radon transform.In general, a GR-map between B(GR/PΞ; λ) and B(GR/PΞ′ ; λ′) is an integral

transform. When it is the integration over a family of submanifolds in GR/PΞ, wecall it a GR-map of Radon transform type. Suppose 0 < k < � < n. If we identifythe Radon transform between real Grassmann manifolds

Rk� : B(Grk(Rn)) → B(Gr�(Rn))

∈ ∈

φ �→ (Rk�φ)(x) =

∫O(�)/O(k)×O(�−k)

φ(xy)dy

with the linear map

Rk� : B(GR/Pk,n; (�, 0))→ B(GR/P�,n; (k, 0)),

then remarkably the latter is a GR-map. If k + � < n, then Rk� is injective and its

image is characterized by tI{k,n}((�, 0)) = Ann(B(GR/Pk,n; (�, 0))

). More precisely

Theorem 9.1 ([Os3]). Suppose 0 < k < � < n and k + � < n. Then Rk� is a

topological isomorphism of B(Grk(Rn)) onto⎧⎪⎪⎪⎨⎪⎪⎪⎩Φ(X) ∈ B(M◦(n, �;R));

i) Φ(Xg) = Φ(X)| det g|−k ( ∀g ∈ GL(�,R) ),

ii) det( ∂

∂xiμjν

)1≤μ≤k+11≤ν≤k+1

Φ(X) = 0

( 1 ≤ i1 < · · · < ik+1 ≤ n, 1 ≤ j1 < · · · < jk+1 ≤ � )

⎫⎪⎪⎪⎬⎪⎪⎪⎭where

(9.1) M◦(n, �;R) ={X = (xij)1≤i≤n

1≤j≤�∈M(n, �;R); rankX = �

}.


Remark 9.2. i) A similar result for the complex Grassmann manifolds is ob-tained by T. Higuchi [Hi].ii) Another characterization of the image is given by T. Kakehi [Ka]. The inversemap is studied in some works such as [Ka], [GR].

Hypergeometric functions.For general GR and PΞ, we assume that bΘ is the complexification of the Lie

algebra of PΞ as in §8. Thus the nilradical nΘ of bΘ is stable under the adjointaction of PΞ and the map PΞ � p �→ detAdnΘ

(p) ∈ C× defines a one-dimensionalrepresentation of PΞ. It is known that for any f ∈ B(GR/PΞ; detAd−1

nΘ) the integral∫

K

f(xk)dk (x ∈ GR)

does not depend on x.

Definition 9.3 ([Os3]). Retain the setting above. Let Qj (j = 1, 2) be closedsubgroups of GR such that each of them has an open orbit in GR/PΞ. Let λ andμj be one-dimensional representations of PΘ and Qj , respectively. Suppose that λis trivial on K ∩ PΞ and that φj (j = 1, 2) are functions on GR satisfying

φ1(q1xp) = μ1(q1)λ(p)φ1(x) (q1 ∈ Q1, p ∈ PΞ),(9.2)

φ2(q2xp) = μ2(q2)λ∗(p)φ2(x) (q2 ∈ Q2, p ∈ PΞ, λ

∗ = λ−1 detAd−1nΘ

).(9.3)

Then we call the function

(9.4) Φφ1,φ2(x) :=

∫K

φ1(xk)φ2(k)dk

(=

∫K

φ1(k)φ2(x−1k)dk

)a hypergeometric function.

Remark 9.4. i) Φφ1,φ2(x) satisfies many differential equations. That is, the

action of the Lie algebra of Q1 on the left, the action of the Lie algebra of Q2 onthe right, and the action of IΘ(λ) on the right (or equivalently, the action of tIΘ(λ)on the left). We call the system consisting of all these differential equations a zonalhypergeometric differential system. In many cases or instances we can expect itssolution space has finite dimension and is spanned by the hypergeometric functions(9.4).ii) Suppose Q1 = Q2 = K and μj (j=1, 2) are trivial. Then Φφ1,φ2

(x) is a sphericalfunction. It is characterized by the hypergeometric differential system and (9.4)gives its integral representation. When PΞ is a maximal parabolic subgroup so thatGR/PΞ is a projective space, Φφ1,φ2

(x) is written by using Lauricella’s hypergeo-metric function FD (cf. [Kr]).iii) When Q1 = K and Q2 = N , Φφ1,φ2

(x) is a Whittaker vector, which is discussedin §10.iv) If Qi (i = 1, 2) is disconnected, the relative invariance under the action of Qi

cannot be expressed in terms of the Lie algebra action. In order to fill this gap wesometimes append some additional conditions to the hypergeometric differentialsystem. For example suppose GR = GL(n,R), Q2 = P�,n (1 < � < n). Then Q2

consists of 4 connected components, each containing one of the following:

(9.5) m1 := In, m2 :=

(−1 00 In−1

), m3 :=

(In−1 00 −1

), m4 := m2m3.


If λ = (μ, 0) and a solution Φ of the hypergeometric differential system satisfies

(9.6) Φ(xm3) = Φ(x),

then we can regard Φ as a function on M◦(n, �;R) by letting Φ(X) = Φ(x) for

X = tx−1

(E�

0

). Hence in such a case (for example in the next theorem) we can

realize the hypergeometric differential system on M◦(n, �;R) by forcing (9.6) onthe solutions.

Now in the setting of Theorem 9.1 we suppose PΞ = Pk,n, Q2 = P�,n, λ =(�, 0) and μ2 = (−k, 0). Let φ2 be the kernel functions of Rk

� . Thus for anyφ ∈ B(GR/Pk,n; (�, 0))

Rk�φ(x) =

∫O(n)

φ(k)φ2(x−1k)dk.

Theorem 9.1 immediately implies the following theorem.

Theorem 9.5. Let PΞ, Q2, λ, μ2 and φ2 be as above. Let mi be as in (9.5)(i = 1, 2, 3, 4). Suppose HR is a connected closed subgroup of GR = GL(n,R) withcomplexification H such that (H ×GL(k,C),Cn �Ck) is a prehomogeneous vectorspace. (When k = 1 the assumption is satisfied by any prehomogeneous vector space(H,V ) defined over R as long as dimV = n.) Put Q1 = tHR and let μ1 be anycharacter of Q1. Then each solution Φ of the hypergeometric differential systemthat additionally satisfies

Φ(xmi) = Φ(x) (i = 1, 2, 3, 4)

has a unique integral representation in the form of (9.4), in which φ1 is a rela-tive invariant hyperfunction on M◦(n, k;R) corresponding to the character HR ×GL(k,R) � (h, g) �→ μ1(

th−1)| det g|�. Here we used the natural identificationM(n, k;C) Cn �Ck. Conversely, such a relative invariant φ1 on M(n, k;R)◦, oron M(n, k;R), gives a solution of the hypergeometric differential system.

In the special case of Theorem 9.5 where k = 1, HR = R>0 × · · · × R>0 andμ1(h

−11 , . . . , h−1

n ) =∏n

i=1 hαj

i (∑n

i=1 αi = −� ), the hypergeometric differentialsystem on (9.1) is explicitly written as(9.7)⎧⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎩

�∑j=1

xij∂Φ

∂xij= αiΦ ( 1 ≤ i ≤ n ) · · · the left action of HR,

n∑ν=1

xνi∂Φ

∂xνj= −δijΦ ( 1 ≤ i, j ≤ � ) · · · the right action of gl�,

∂2Φ

∂xi1j1∂xi2j2

=∂2Φ

∂xi2j1∂xi1j2

(1 ≤ i1 < i2 ≤ n

1 ≤ j1 < j2 ≤ �

)· · · Capelli type.

Moreover, if εi = ± for i = 1, . . . , n then the integral representation of the hyper-geometric function corresponding to the distribution φ1(x1, . . . , xn) =

∏ni=1 x

αiεi +∏n

i=1 xαi−εi is reduced to

(9.8) Φ(α, x) =

∫t21+···+t2�=1

n∏i=1

( �∑ν=1

tνxiν

)αi

εiω (ω is the surface element).


This function is known as Aomoto-Gelfand’s hypergeometric function (cf. [Ao],[GG]).

Remark 9.6. i) If n = 4 and � = 2 in the last example, then (9.8) is essentiallyreduced to Gauss’s hypergeometric function.ii) If the number of the H ×GL(k,C)-orbits in

M◦(n, k;C) :={X ∈M(n, k;C); rankX = k

}is finite in Theorem 9.5, it is proved by T. Tanisaki [Ta2, Proposition 4.5] that thehypergeometric differential system is holonomic and the dimension of the space ofits local solutions is finite. This condition is satisfied if the prehomogeneous vectorspace (H × GL(k,C),Cn � Ck) has finite orbits and such prehomogeneous vectorspaces are classified by T. Kimura and others [KK].iii) There are a version of hypergeometric functions attached to Penrose transforms.They are studied by H. Sekiguchi [Se].

10. Whittaker vectors

Suppose GR = KAN is an Iwasawa decomposition and χ is a one-dimensionalrepresentation of N . In this section we consider the realization of a GR-module Vas a submodule of

B(GR/N ;χ) :={f ∈ B(GR); f(xn) = χ−1(n)f(x) (∀n ∈ N)

}.

When GR = GL(n,R), we may assume

K = O(n), A ={exp( n∑i=1

xiEii

); xi ∈ R

}, N =

{exp(∑i>j

xijEij

); xij ∈ R

}and

χ(exp(∑i>j

xijEij

); xij ∈ R

)= ec1x21+c2x32+···+cn−1xnn−1

for some cj ∈ C (j = 1, . . . , n− 1). If V = B(GR/PΞ;λ) is realized in B(GR/N ;χ),then a K-fixed vector u in the realization satisfies

(10.1)

{u(kxn) = χ(n)−1u(x) (∀k ∈ O(n), ∀n ∈ N),

LDu = 0 (∀D ∈ tIΘ(λ)).

We generally call a solution of the above system of equations a Whittaker vector.Owing to the Iwasawa decomposition a Whittaker vector u is determined by itsrestriction v := u|A to A. Since we know the concrete form of a generating systemof tIΘ(λ), we can explicitly write down the system of equations which v shouldsatisfy.

Now suppose V = B(GR/Pk,n; (μ, ν)) (2 ≤ 2k ≤ n), a degenerate principalseries representation on the real Grassmann manifold GR/Pk,n. Then we can seefrom the explicit form of the system for v that the condition for the existence ofnontrivial v is

cici+1 = ci1ci2 · · · cik+1= 0 (1 ≤ i < n, 1 ≤ i1 < i2 < · · · < ik+1 < n).

For example, if {ci = 0 (i = 2, 4, . . . , 2k, 2k + 1, 2k + 2, . . .),

c2j−1 �= 0 (j = 1, . . . , k),


then the system of differential equations for v is written as⎧⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎩

Eiv = νv (i = 2k + 1, 2k + 2, . . . , n),

(E2j−1 + E2j)v = (μ+ ν − 2j + k + 1)v,((E2j−1−E2j

2

)2 − (E2j−1−E2j

2

)+ c22j−1e

2(x2j−1−x2j))v = μ−ν−k+1

2

(μ−ν−k+1

2 − 1)v,

where j = 1, . . . , k, Ep = ∂∂xp

(p = 1, . . . , n).

From this we can deduce the multiplicity of the realization is 2k, while the realiza-tion satisfying the moderate growth condition has multiplicity one. A Whittakervector with moderate growth is thus unique up to a scalar multiple and is expressedby a modified Bessel function of the second kind.

Remark 10.1. Further studies on Whittaker vectors are given in [Os6].

References

[Ao] Kazuhiko Aomoto, On the structure of integrals of power product of linear functions, Sci.Papers College Gen. Ed. Univ. Tokyo 27 (1977), no. 2, 49–61. MR0590052 (58 #28651)

[BG] J. N. Bernstein and S. I. Gel′fand, Tensor products of finite- and infinite-dimensionalrepresentations of semisimple Lie algebras, Compositio Math. 41 (1980), no. 2, 245–285.MR581584 (82c:17003)

[BOS] S. Ben Saıd, T. Oshima and N. Shimeno, Fatou’s theorems and Hardy-type spaces foreigenfunctions of the invariant differential operators on symmetric spaces, Internat. Math.Research Notice 16 (2003), 913–931.

[BV] Nicole Berline and Michele Vergne, Equations de Hua et noyau de Poisson, Noncommuta-tive harmonic analysis and Lie groups (Marseille, 1980), Lecture Notes in Math., vol. 880,Springer, Berlin, 1981, pp. 1–51 (French). MR644825 (83h:22019)

[Br] Abraham Broer, The sum of generalized exponents and Chevalley’s restriction theorem formodules of covariants, Indag. Math. (N.S.) 6 (1995), no. 4, 385–396, DOI 10.1016/0019-3577(96)81754-X. MR1365182 (96j:20058)

[Ca1] Alfredo Capelli, Ueber die Zuruckfuhrung der Cayley’schen Operation Ω auf gewohnlichePolar-Operationen, Math. Ann. 29 (1887), no. 3, 331–338, DOI 10.1007/BF01447728(German). MR1510419

[Ca2] Alfredo Capelli, Sur les operations dans la theorie des formes algebriques, Math. Ann. 37(1890), 1–37.

[Du] Michel Duflo, Sur la classification des ideaux primitifs dans l’algebre enveloppante d’unealgebre de Lie semi-simple, Ann. of Math. (2) 105 (1977), no. 1, 107–120. MR0430005(55 #3013)

[Ge] I. M. Gel′fand, The center of an infinitesimal group ring, Mat. Sbornik N.S. 26(68) (1950),103–112 (Russian). MR0033831 (11,498a)

[GG] I. M. Gelfand and S. I. Gelfand, Generalized hypergeometric equations, Soviet Math. Dokl

33 (1986), 643–646.[Go1] M. D. Gould, A trace formula for semisimple Lie algebras, Ann. Inst. H. Poincare Sect. A

(N.S.) 32 (1980), no. 3, 203–219. MR579960 (82b:17011)[Go2] M. D. Gould, Characteristic identities for semisimple Lie algebras, J. Austral. Math. Soc.

Ser. B 26 (1985), no. 3, 257–283, DOI 10.1017/S0334270000004501. MR776316 (86h:17005)[GR] Eric L. Grinberg and Boris Rubin, Radon inversion on Grassmannians via Garding-

Gindikin fractional integrals, Ann. of Math. (2) 159 (2004), no. 2, 783–817, DOI10.4007/annals.2004.159.783. MR2081440 (2005f:58042)

[Hi] T. Higuchi, An application of generalized Capelli operators to the image characterizationof Radon transforms for Grassmann manifolds, Master thesis presented to the Universityof Tokyo, 2004, in Japanese.

[HO] G. J. Heckman and E. M. Opdam, Root systems and hypergeometric functions. I, Compo-sitio Math. 64 (1987), no. 3, 329–352. MR918416 (89b:58192a)

[HC] Harish-Chandra, Harmonic analysis on real reductive groups. I. The theory of the constantterm, J. Functional Analysis 19 (1975), 104–204. MR0399356 (53 #3201)


[He1] Sigurdur Helgason, A duality for symmetric spaces with applications to group represen-tations. II. Differential equations and eigenspace representations, Advances in Math. 22(1976), no. 2, 187–219. MR0430162 (55 #3169)

[HU] Roger Howe and Toru Umeda, The Capelli identity, the double commutant theo-rem, and multiplicity-free actions, Math. Ann. 290 (1991), no. 3, 565–619, DOI10.1007/BF01459261. MR1116239 (92j:17004)

[Hu] L. K. Hua, Harmonic analysis of functions of several complex variables in the classical

domains, Translated from the Russian by Leo Ebner and Adam Koranyi, American Math-ematical Society, Providence, R.I., 1963. MR0171936 (30 #2162)

[I] Minoru Itoh, Explicit Newton’s formulas for gln, J. Algebra 208 (1998), no. 2, 687–697,DOI 10.1006/jabr.1998.7510. MR1655473 (99j:17016)

[Joh] Kenneth D. Johnson, Generalized Hua operators and parabolic subgroups, Ann. of Math.(2) 120 (1984), no. 3, 477–495, DOI 10.2307/1971083. MR769159 (86e:22016)

[Jos] A. Joseph, Dixmier’s problem for Verma and principal series submodules, J. London Math.Soc. (2) 20 (1979), no. 2, 193–204, DOI 10.1112/jlms/s2-20.2.193. MR551445 (81c:17016)

[K-] M. Kashiwara, A. Kowata, K. Minemura, K. Okamoto, T. Oshima, and M. Tanaka, Eigen-functions of invariant differential operators on a symmetric space, Ann. of Math. (2) 107(1978), no. 1, 1–39, DOI 10.2307/1971253. MR485861 (81f:43013)

[Ka] Tomoyuki Kakehi, Integral geometry on Grassmann manifolds and calculus of invariantdifferential operators, J. Funct. Anal. 168 (1999), no. 1, 1–45, DOI 10.1006/jfan.1999.3459.MR1717855 (2000k:53069)

[KK] Tatsuoo Kimura, Tomohiro Kamiyoshi, Norimichi Maki, Masaya Ouchi, and MishuoTakano, A classification of representations ρ⊗Λ1 of reductive algebraic groups G×SLn(n ≥2) with finitely many orbits, Algebras Groups Geom. 25 (2008), no. 2, 115–159. MR2444306(2009f:20066)

[KM] A. Koranyi and P. Malliavin, Poisson formula and compound diffusion associated to anoverdetermined elliptic system on the Siegel halfplane of rank two, Acta Math. 134 (1975),no. 3-4, 185–209. MR0410278 (53 #14028)

[Ko] Bertram Kostant, Lie group representations on polynomial rings, Amer. J. Math. 85(1963), 327–404. MR0158024 (28 #1252)

[Kr] Adam Koranyi, Hua-type integrals, hypergeometric functions and symmetric polynomials,International Symposium in Memory of Hua Loo Keng, Vol. II (Beijing, 1988), Springer,Berlin, 1991, pp. 169–180. MR1135834 (92h:33036)

[La] Michel Lassalle, Les equations de Hua d’un domaine borne symetrique du type tube, In-vent. Math. 77 (1984), no. 1, 129–161, DOI 10.1007/BF01389139 (French). MR751135(85m:32033)

[Od1] Hiroshi Oda, On annihilator operators of the degenerate principal series for orthogonal Liegroups, Surikaisekikenkyusho Kokyuroku 1183 (2001), 74–93 (Japanese). Representationtheory of groups and rings and non-commutative harmonic analysis (Japanese) (Kyoto,2000). MR1840404

[Od2] Hiroshi Oda, Determinant-type operators for orthogonal Lie algebras, SurikaisekikenkyushoKokyuroku 1245 (2002), 16–24 (Japanese). Representation theory and harmonic analysistoward the new century (Japanese) (Kyoto, 2001). MR1913488

[Od3] Hiroshi Oda, Calculation of Harish-Chandra homomorphism for certain non-central el-ements in the enveloping algebras of classical Lie algebras, RIMS Kokyuroku, KyotoUniv. 1508 (2006), 69–96.

[Od4] Hiroshi Oda,Generalization of Harish-Chandra’s basic theorem for Riemannian symmetricspaces of non-compact type, Adv. Math. 281 (2007), 549–596.

[OO] Hiroshi Oda and Toshio Oshima, Minimal polynomials and annihilators of generalizedVerma modules of the scalar type, J. Lie Theory 16 (2006), no. 1, 155–219. MR2196421(2006m:22022)

[Op] E. M. Opdam, Dunkl operators, Bessel functions and the discriminant of a finite Coxeter

group, Compositio Math. 85 (1993), no. 3, 333–373. MR1214452 (95j:33044)[OP] M. A. Olshanetsky and A. M. Perelomov, Quantum integrable systems related to Lie alge-

bras, Phys. Rep. 94 (1983), no. 6, 313–404, DOI 10.1016/0370-1573(83)90018-2. MR708017(84k:81007)

[Os1] T. Oshima, Boundary value problems for various boundaries of symmetric spaces, RIMSKokyuroku, Kyoto Univ. 281 (1976), 211–226, in Japanese.


[Os2] Toshio Oshima, Asymptotic behavior of spherical functions on semisimple symmetricspaces, Representations of Lie groups, Kyoto, Hiroshima, 1986, Adv. Stud. Pure Math.,vol. 14, Academic Press, Boston, MA, 1988, pp. 561–601. MR1039853 (91c:22018)

[Os3] Toshio Oshima, Generalized Capelli identities and boundary value problems for GL(n),Structure of solutions of differential equations (Katata/Kyoto, 1995), World Sci. Publ.,River Edge, NJ, 1996, pp. 307–335. MR1445347 (98f:22021)

[Os4] Toshio Oshima, A quantization of conjugacy classes of matrices, Adv. Math. 196 (2005),

no. 1, 124–146, DOI 10.1016/j.aim.2004.08.013. MR2159297 (2006f:17011)[Os5] Toshio Oshima, A class of completely integrable quantum systems associated with classi-

cal root systems, Indag. Math. (N.S.) 16 (2005), no. 3-4, 655–677, DOI 10.1016/S0019-3577(05)80045-X. MR2313643 (2008d:81100)

[Os6] Toshio Oshima, Whittaker models of degenerate principal series, RIMS Kokyuroku, KyotoUniv. 1467 (2006), 71–78, in Japanese.

[Os7] Toshio Oshima, Annihilators of generalized Verma modules of the scalar type for clas-sical Lie algebras, Harmonic analysis, group representations, automorphic forms and in-variant theory, Lect. Notes Ser. Inst. Math. Sci. Natl. Univ. Singap., vol. 12, World Sci.Publ., Hackensack, NJ, 2007, pp. 277–319, DOI 10.1142/9789812770790 0009. MR2401816(2009m:17005)

[OSe] Toshio Oshima and Hideko Sekiguchi, Commuting families of differential operators invari-ant under the action of a Weyl group, J. Math. Sci. Univ. Tokyo 2 (1995), no. 1, 1–75.MR1348022 (96k:35006)

[OSh] T. Oshima and N. Shimeno, Boundary value problems on Riemannian symmetric spacesof noncompact type, to appear in a book celebrating J. Wolf’s 75th birthday, Prog. Math.Birkhauser.

[Sak] H. Sakaguchi, U(g)-modules associated to Grassmannian manifolds, Master thesis pre-sented to the University of Tokyo, 1999, in Japanese.

[Sat] Ichiro Satake, On representations and compactifications of symmetric Riemannian spaces,Ann. of Math. (2) 71 (1960), 77–110. MR0118775 (22 #9546)

[Sc] Henrik Schlichtkrull, Hyperfunctions and harmonic analysis on symmetric spaces, Progressin Mathematics, vol. 49, Birkhauser Boston Inc., Boston, MA, 1984. MR757178 (86g:22021)

[Se] Hideko Sekiguchi, The Penrose transform for certain non-compact homogeneous manifoldsof U(n, n), J. Math. Sci. Univ. Tokyo 3 (1996), no. 3, 655–697. MR1432112 (98d:22011)

[Sh] Nobukazu Shimeno, Boundary value problems for various boundaries of Hermitian sym-metric spaces, J. Funct. Anal. 170 (2000), no. 2, 265–285, DOI 10.1006/jfan.1999.3529.MR1740653 (2001c:22018)

[Ta1] Toshiyuki Tanisaki, Defining ideals of the closures of the conjugacy classes and repre-sentations of the Weyl groups, Tohoku Math. J. (2) 34 (1982), no. 4, 575–585, DOI10.2748/tmj/1178229158. MR685425 (84g:14049)

[Ta2] Toshiyuki Tanisaki, Hypergeometric systems and Radon transforms for Hermitian sym-metric spaces, Analysis on homogeneous spaces and representation theory of Lie groups,Okayama–Kyoto (1997), Adv. Stud. Pure Math., vol. 26, Math. Soc. Japan, Tokyo, 2000,pp. 235–263. MR1770723 (2001m:22020)

[U] Toru Umeda, Newton’s formula for gln, Proc. Amer. Math. Soc. 126 (1998), no. 11, 3169–3175, DOI 10.1090/S0002-9939-98-04557-2. MR1468206 (99a:17018)

[Wa] Akihito Wachi, Central elements in the universal enveloping algebras for the split real-ization of the orthogonal Lie algebras, Lett. Math. Phys. 77 (2006), no. 2, 155–168, DOI10.1007/s11005-006-0082-6. MR2251303 (2007e:17009)

[We] J. Weyman, The equations of conjugacy classes of nilpotent matrices, Invent. Math. 98(1989), no. 2, 229–245, DOI 10.1007/BF01388851. MR1016262 (91g:20070)

Faculty of Engineering, Takushoku University, 815-1, Tatemachi, Hachioji-shi, Tokyo

193-0985, Japan


Graduate School of Mathematical Sciences, University of Tokyo, 7-3-1, Komaba,

Meguro-ku, Tokyo 153-8914, Japan



Mean value theorems on symmetric spaces

Francois Rouviere

kæri Sigurður Helgason, til hamingju með 85 ara afmælið

Abstract. Revisiting some mean value theorems by F. John, respectively S.Helgason, we study their extension to general Riemannian symmetric spaces,resp. their restatement in a more detailed form, with emphasis on their linkwith the infinitesimal structure of the symmetric space.

1. An old formula by Fritz John

In his inspiring 1955 book Plane waves and spherical means [7], Fritz Johnconsiders the mean value operator on spheres in the Euclidean space Rn:

(1.1) MXu(p) :=

∫K

u(p+ k ·X)dk = Mxu(p) with x = ‖X‖

where X, p ∈ Rn, u is a continuous function on Rn, dk is the normalized Haarmeasure on the orthogonal group K = SO(n) and dot denotes the natural actionof this group on Rn. This average of u over the sphere with center p and radiusx = ‖X‖ (the Euclidean norm of Rn) only depends on p and this radius; it may bewritten Mxu(p) as well.

For X,Y, p ∈ Rn the iterated spherical mean is

(1.2) MXMY u(p) =

∫K

MX+k·Y u(p)dk,

as easily checked. Taking z = ‖X + k · Y ‖ as the new variable this transforms into

MxMyu(p) =

∫ x+y

|x−y|Mzu(p)a(x, y, z)zn−1dz ,(1.3)

a(x, y, z)=Cn

(xyz)n−2

((x+ y + z

2

)(x+ y − z

2

)(x− y + z

2

)(−x+ y + z

2

))(n−3)/2

for x, y > 0, a formula first proved by John; here Cn = 2n−3Γ(n2

)/Γ(n−12

)Γ(12

).

A nice proof is given in Chapter VI of Helgason’s book [6].

2000 Mathematics Subject Classification. Primary 43A85, 53C35; Secondary 33C80, 43A90.Key words and phrases. Symmetric space, mean value.Many thanks to Jens Christensen, Fulton Gonzalez and Eric Todd Quinto, organizers of this

AMS special session in Boston and the satellite workshop in Tufts, for inviting me to participate- and to Tufts University for its support.


209

210 FRANCOIS ROUVIERE

Is there a similar result for symmetric spaces? The purpose of this note is toprove an analog of John’s formula for general Riemannian symmetric spaces and,focusing on the formula replacing (1.2), to show some interesting relations with themean value operator itself, leading to an explicit series expansion of this operator.

From now on we shall work on a Riemannian symmetric space G/K, whereG is a connected Lie group and K a compact subgroup. We use the customarynotation g = k ⊕ p for the decomposition of the Lie algebra of G given by thesymmetry, where k is the Lie algebra of K and p identifies with the tangent spaceat the origin o of G/K. Let ‖.‖ be the K-invariant norm on p corresponding tothe Riemannian structure of G/K. Let X �→ eX denote the exponential mappingof the Lie group G and Exp : p→ G/K the exponential mapping of the symmetricspace at o. For X ∈ p the natural generalization of MX above should be averagingover Riemannian spheres of radius ‖X‖ in G/K. However we shall rather useaverages over K-orbits (which are included in spheres), easier to handle in ourgroup-theoretic framework; both notions agree if G/K is isotropic, i.e. K actstransitively on the unit sphere of p. Thus let

(1.4) MXu(gK) :=

∫K

u(g · Exp (k ·X))dk =

∫K

u(gk · ExpX)dk

where u : G/K → C is continuous, X ∈ p, g ∈ G and dots denote the natural actionof G on G/K, respectively the adjoint action of K on p. Then (1.2) generalizes asfollows. Since Exp is a diffeomorphism between neighborhoods of the origins in p

and G/K we may define Z(X,Y ) ∈ p (for X,Y near the origin of p at least) by

(1.5) ExpZ(X,Y ) = eX · ExpY,that is eZ(X,Y )K = eXeY K, and we have

(1.6) MXMY u(gK) =

∫K

MZ(X,k·Y )u(gK)dk.

Remark. From (1.5) and (1.6) an elementary proof of the commutativity MXMY =MY MX for symmetric spaces is easily obtained1. Indeed, let k(X,Y ) ∈ K be de-fined by k(X,Y ) := e−Z(X,Y )eXeY hence, applying the involution of G, k(X,Y ) =eZ(X,Y )e−Xe−Y and, combining both expressions, e2Z(X,Y ) = eXe2Y eX . Therefore

eXeY K = e−Z(X,Y )e2Z(X,Y )K =(e−Z(X,Y )eXeY

)eY eXK = k(X,Y )eY eXK

and it follows thatZ(X,Y ) = k(X,Y ) · Z(Y,X).

Then, for any k′ ∈ K,

Z(X, k′ · Y ) = k(X, k′ · Y ) · Z(k′ · Y,X) = (k(X, k′ · Y )k′) · Z(Y, k′−1 ·X)

and the K-invariance of X �→ MXu(gK) gives MZ(X,k′·Y ) = MZ(Y,k′−1·X) andMXMY = MY MX by (1.6), as claimed.

This preliminary result (1.6) will now be improved in two directions:- an analog of John’s formula (1.3) for rank one spaces (Theorem 2.1)- a proof that MXMY u(gK) =

∫KMX+k·Y u(gK) f(X, k · Y ) dk for general Rie-

mannian symmetric spaces (Theorem 3.1), with a specific factor f which turns out

1More ”sophisticated” proofs are suggested by the last remarks of sections 3.1 and 3.2. Moregenerally, this commutativity property holds true for all Gelfand pairs (G,K): see [5, p. 80]

MEAN VALUES ON SYMMETRIC SPACES 211

to have independent interest and will play a part in an expansion of MX itself(Theorem 3.2).

2. Mean values on rank one spaces

In this section let G/K denote a Riemannian symmetric space of the noncom-pact type and of rank one, that is one of the hyperbolic spaces (real, complex,quaternionic, or exceptional). This space is isotropic and MXu(gK), which is aK-invariant function of X ∈ p, only depends on the (K-invariant) norm x = ‖X‖; as in the Euclidean case we may thus write MXu(gK) = Mxu(gK). Note thatExp is here a global diffeomorphism of p onto G/K, so that Z(., .) is globallydefined. Let p ≥ 1 and q ≥ 0 denote the mutiplicities of the positive roots,n = p+ q + 1 = dimG/K and let Cn be as in (1.3).

Theorem 2.1. For x, y > 0 and u continuous on G/K

MxMyu(gK) =

∫ x+y

|x−y|Mzu(gK)b(x, y, z)δ(z)dz ,

where δ(z) = (sh z)n−1 (ch z)q,

b(x, y, z) = Cn(chx ch y ch z)

(p/2)−1

(sh x sh y sh z)n−2 B(n−3)/22F1

(1− q

2,q

2;n− 1

2;B

)and

B=1

chx ch y ch zsh

(x+ y + z

2

)sh

(x+ y − z

2

)sh

(x− y + z

2

)sh

(−x+ y + z

2

).

One has b(x, y, z) > 0 for x, y > 0 and |x− y| < z < x+ y.

Nice function, isn’t it?Remark 1. The hypergeometric factor is identically 1 if q = 0 that is G/K =Hn(R).Remark 2. tb(tx, ty, tz)δ(tz) tends to the Euclidean factor a(x, y, z)zn−1 as t→ 0.John’s formula is thus a ”flat limit” of Theorem 2.1.Remark 3. In [7] John inverts (1.3), an Abel type integral equation, so as toexpress Mzu by means of MxMyu and finally, taking z = 0, u itself as a sum ofderivatives of its iterated mean values. This seems unworkable here however, withthe present kernel b.Proof. The main point is to prove

(2.1)

∫K

ϕ (‖Z(X, k · Y )‖) dk =

∫ x+y

|x−y|ϕ(z)b(x, y, z)δ(z)dz

for any continuous function ϕ : [0,∞[→ C. Then taking ϕ(t) = M tu(gK) (withgK fixed) our claim will follow in view of (1.6).To prove (2.1) one may use a Cartan type decomposition of K, which reducesthe problem to a 2- or 1- dimensional subgroup of K. The classical technique ofSU(2, 1)-reduction then allows an explicit computation of z = ‖Z(X, k · Y )‖ and(2.1) is finally obtained by taking z as the new variable in the integral. Euler’sintegral representation of the hypergeometric function shows that b(x, y, z) > 0 forx, y > 0 and |x − y| < z < x + y. Full details, lengthy but easy, will appear in[9, Chapter 3 and Appendix B].


Remark 4. Formula (2.1) for spherical functions can be derived from the work [1]of Flensted-Jensen and Koornwinder (or [8, §7.1]). Indeed the following productformula is proved in §4 of [1] (in the more general framework of Jacobi functions):

(2.2) ϕλ(x)ϕλ(y) =

∫ x+y

|x−y|ϕλ(z)b(x, y, z)δ(z)dz

for x, y > 0, λ∈ C; see also the last pages of Chapter III in [5]. Here b and δ are thefunctions defined above, ϕλ is one of Harish-Chandra’s spherical functions of G/Kand ϕλ(x) is, abusing notation, ϕλ(ExpX) for X ∈ p and ‖X‖ = x. Actually, todeduce (2.2) from (4.2) and (4.19) in [1] (or (7.11) in [8]) some minor changes needto be made: the parameters (α, β) of general Jacobi functions are here α = (n−2)/2,β = (q−1)/2 (corresponding to the group case), the function B of [1] is our 1−2Band the hypergeometric formula

2F1 (a, b; c; t) = (1− t)c−a−b

2F1 (c− a, c− b; c; t)

is applied. Finally the left-hand side of (2.2) is∫Kϕλ (‖Z(X, k · Y )‖) dk, by the

functional equation of spherical functions.

Similarly, in the Euclidean setting of p, John’s formula (1.3) follows from∫K

ϕ (‖X + k · Y ‖) dk =

∫ x+y

|x−y|ϕ(z)a(x, y, z)zn−1dz

for any continuous function ϕ : [0,∞[→ C. Combining this with (2.1) we have∫K

ϕ (‖Z(X, k · Y )‖) dk =∫K

ϕ (‖X + k · Y ‖) ba(‖X‖ , ‖Y ‖ , ‖X + k · Y ‖) δ (‖X + k · Y ‖)

‖X + k · Y ‖n−1 dk,

thus, for any continuous K-invariant function F on p,∫K

F (Z(X, k · Y )) dk =

∫K

F (X + k · Y ) f(X, k · Y )dk(2.3)

with f(X,Y ) :=b

a(x, y, z)

δ(z)

zn−1, x = ‖X‖ , y = ‖Y ‖ , z = ‖X + Y ‖ .

Applying this to the mean value F (X) = MXu(gK) we obtain from (1.6)

MXMY u(gK) =

∫K

MX+k·Y u(gK)f(X, k · Y )dk.

The latter formula extends to all Riemannian symmetric spaces, as will be shownnow.

3. Mean values on Riemannian symmetric spaces

Throughout this section G is a connected Lie group and K a compact subgroup,such that G/K is a Riemannian symmetric space.


3.1. Iterated mean values.

Theorem 3.1. (i) There exists a neighborhood U of the origin in p × p andan analytic map (X,Y ) �→ c(X,Y ) from U into K such that c(k · X, k · Y ) =kc(X,Y )k−1 for all k ∈ K and, letting γX(Y ) := c(X,Y ) · Y ,

F (Z(X, γX(Y ))) = F (X + Y )

for any K-invariant continuous function F on p.(ii) Let f(X,Y ) := detp DγX(Y ) be the Jacobian of γX with respect to Y . Then,for (X,Y ) ∈ U , one has f(k ·X, k · Y ) = f(X,Y ) > 0 and

(3.1)

∫K

F (Z(X, k · Y ))dk =

∫K

F (X + k · Y )f(X, k · Y )dk .

In particular, for u continuous on G/K, g ∈ G and (X,Y ) ∈ U ,

(3.2) MXMY u(gK) =

∫K

MX+k·Y u(gK)f(X, k · Y )dk .

Proof. We shall only sketch the main steps and refer to Chapter 4 of [9] for details.(i) Let Zt(X,Y ) := t−1Z(tX, tY ) for 0 < t ≤ 1 and Z0(X,Y ) := X + Y . Themap (t,X, Y ) �→ Zt(X,Y ) is analytic in an open subset of R × p × p containing[0, 1]×U , where U is a suitably chosen neighborhood of (0, 0) in p× p. In order torelate Z(X,Y ) = Z1(X,Y ) to its flat analog Z0(X,Y ) we shall solve a differentialequation with respect to t.One can construct two series of even Lie brackets

A(X, Y ) = −1

3[X,Y ] +

1

90(7[X, [X, [X,Y ]]] + 12[Y, [X, |X,Y ]]] + 4[Y, [Y, [X,Y ]]]) + · · ·

C(X,Y ) = −1

3[X,Y ] +

1

45(2[X, [X, [X,Y ]]] + 3[Y, [X, |X,Y ]]] + 2[Y, [Y, [X,Y ]]]) + · · ·

such that C(Y,X) = −C(X,Y ) and

(3.3) ∂tZt = [Zt, At] + (∂Y Zt) [Y,Ct]

withAt(X,Y ) = t−1A(tX, tY ), Ct(X,Y ) = t−1C(tX, tY ). All functions in (3.3) areevaluated at (X,Y ) ∈ U and, for V ∈ p, (∂Y Zt)V means ∂s (Zt(X,Y + sV ))|s=0.Since [p, p] ⊂ k and [k, p] ⊂ p, both At and Ct map U into k. Such series of Liebrackets are obtained by manipulating the Campbell-Hausdorff formula for the Liealgebra g, written in the specific form introduced by Kashiwara and Vergne.Now let ct = ct(X,Y ) ∈ K denote the solution of the differential equation

∂tct = DeRct (Ct(X, ct · Y )) , c0 = e,

where Rc denotes the right translation by c in K and DeRc its differential at theorigin e of K. The K-invariance ct(k · X, k · Y ) = kct(X,Y )k−1 follows fromCt(k ·X, k · Y ) = k · Ct(X,Y ) and the uniqueness of the solution.Setting Vt = ct · Y we have

(3.4) ∂tVt = [Ct(X,Vt), Vt] , V0 = Y

and, for any smooth function F on p,

∂t (F (Zt(X,Vt)) = DF (Zt(X,Vt)) {(∂tZt) (X,Vt) + (∂Y Zt) (X,Vt)∂tVt}= DF (Zt(X,Vt)) [Zt, At](X,Vt)

by (3.3). But if F is K-invariant we have F (esA · X) = F (X) for any s ∈ R,A ∈ k, X ∈ p, hence ∂s

(F (esA ·X)

)∣∣s=0

= DF (X) [A,X] = 0 and finally


∂t (F (Zt(X,Vt)) = 0 for all t ∈ [0, 1]. Thus F (Z0(X,V0)) = F (Z(X,V1)). SinceV0 = Y this implies our claim (with c(X,Y ) = c1, γX(Y ) = V1 = c(X,Y ) · Y ) forsmooth F , and the general case follows by approximation.(ii) The K-invariance of the Jacobian f is an easy consequence of the correspondingproperty of c. Let ft = ft(X,Y ) denote the Jacobian of the map Y �→ Vt =ct(X,Y ) · Y constructed above. From (3.4) we infer

∂tft = −ft trp (adVt ◦ (∂Y Ct) (X,Vt)) , f0 = 1,

where trp(..) means the trace of (..) restricted to p. It follows that ft > 0 for0 ≤ t ≤ 1; in particular f1 = f is positive on U .In order to prove (3.1) let ϕ be anyK-invariant continuous function on p, compactlysupported near the origin and let X be fixed in p (near 0). Then∫

p

ϕ(Y )dY

∫K

F (Z(X, k · Y ))dk =

∫p

ϕ(Y )F (Z(X,Y ))dY

=

∫p

ϕ(γX(Y ))F (X + Y ) detp

DγX(Y )dY

=

∫p

ϕ(Y )F (X + Y )f(X,Y )dY,

using the K-invariance of ϕ and the change Y �→ γX(Y ). Then (3.1) follows sinceϕ is an arbitrary K-invariant function, hence (3.2) taking F (X) = MXu(gK) andremembering (1.6). �Remark 1. For any X,Y close enough to the origin of p we have f(X, 0) =f(0, Y ) = 1. Indeed, by (3.1) with F = 1 and the K-invariance of f we obtain∫Kf(X, k · Y )dk =

∫Kf(k−1 ·X,Y )dk = 1.

Remark 2. The construction of γX(Y ) in (i) is universal and only depends onthe infinitesimal structure of the symmetric space (the structure of the correspond-ing Lie triple system). The factor f introduced in Theorem 3.1 turns out to havedeeper relations with analysis on G/K; some of them will appear during the proofof Theorem 3.2. Actually f is related to the ”e-function” of [9] by e(X,Y ) =

(J(X)J(Y )/J(X + Y ))1/2

f(X,Y ) where J(X) = detp (sh(adX)/ adX) is the Ja-cobian of Exp. In particular, if G is a complex Lie group and K a compact real form

ofG it can be shown that e(X,Y ) = 1 hence2 f(X,Y ) = (J(X + Y )/J(X)J(Y ))1/2.A direct proof of Theorem 3.1 could be given for spherical functions, under thefurther assumption that G is a complex semisimple Lie group with finite center andK a maximal compact subgroup. Indeed, by Helgason’s Theorem 4.7 in [4, ChapterIV], the spherical functions of G/K are then given by

J(X)1/2ϕλ(ExpX) =

∫K

ei〈λ,k·X〉dk,

2By [9] again the same holds for a compact Lie group G viewed as the symmetric space(G×G) /K where K is the diagonal subgroup.


where λ is an arbitrary linear form on p. Therefore∫K

ϕλ(ExpZ(X, k · Y ))dk =

∫K

ϕλ

(eXk · ExpY

)dk = ϕλ(ExpX)ϕλ(ExpY )

=

∫K×K

ei〈λ,k·X+k′·Y 〉 (J(X)J(Y ))−1/2

dkdk′

=

∫K×K

ei〈λ,k·(X+k′·Y )〉 (J(X)J(Y ))−1/2

dkdk′

=

∫K

ϕλ(Exp(X + k′ · Y ))

(J(X + k′ · Y )

J(X)J(k′ · Y )

)1/2

dk′,

hence (3.1) with F = ϕλ ◦ Exp and f(X,Y ) = (J(X + Y )/J(X)J(Y ))1/2.Remark 3. A close look at the construction of c shows that f(X,Y ) = f(Y,X)([9, Chapter 4]); in the rank one case this was clear from the explicit expression off . Thus Theorem 3.1 gives another proof of MXMY = MY MX .

3.2. Expansion of the mean value operator. In the isotropic case a K-

invariant analytic function F on p can be expanded as F (X) =∑∞

m=0 am ‖X‖2m

and the coefficients are given, for m ≥ 1, by

ΔmF (0) = amcm with cm = Δm ‖X‖2m = 2mm!n(n+ 2) · · · (n+ 2m− 2),

where Δ is the Euclidean Laplace operator on p and n = dim p.A similar expansion is valid in an arbitrary Riemannian symmetric space G/K,

as we shall now see. Applying it to F (X) = MXu(gK) we shall obtain an expansionof the mean value operator in terms of differential operators on p. It is moreinteresting however to replace them by elements of D(G/K), the algebra of G-invariant differential operators on G/K. This is the goal of the next theorem,where the function f of Theorem 3.1 will play again a significant role.

Let D(p)K = S(p)K denote the algebra of K-invariant differential operatorson p with constant (real) coefficients, canonically isomorphic to the algebra of K-invariant elements in the symmetric algebra of p, that is K-invariant polynomialson its dual p∗. This algebra is finitely generated ([4, Chapter III, Theorem 1.10]).Thus let P1, ..., Pl be a system of homogeneous generators of this algebra: from theset of all

Pα := Pα11 · · ·Pαl

l , α = (α1, ..., αl) ∈ Nl,

we can thus extract a basis of D(p)K , say (Pα)α∈B where B is some subset of Nl.

We shall denote by (P ∗α)α∈B the dual basis of S(p∗)K (the K-invariant polynomials

on p) with respect to the Fischer product :⟨Pα|P ∗

β

⟩= Pα(∂X)P ∗

β (0) = δαβ .

Each Pα is homogeneous of degree α · d = α1d1 + · · ·+αldl (with dj = degPj) andit follows easily that P ∗

α is homogeneous of the same degree.Example 1. Trivial case: K = {e}, B = Nn, Pα = ∂α

X , P ∗α = Xα/α! in multi-index

notation.Example 2. Isotropic case: α = m ∈ B = N, Pα = Δm, P ∗

α = ‖X‖2m /cm.

Any K-invariant analytic function on p can now be expanded by means of theP ∗α’s. In particular, we have the following K-invariant Taylor expansion of the mean


value of an analytic function u on G/K

MXu(gK) =∑α∈B

aα(gK)P ∗α(X)

(for X near the origin of p), with coefficients given by

aα(gK) = Pα(∂X)(MXu(gK)

)X=0

= Pα(∂X)

(∫K

u(g · Exp(k ·X))dk

)X=0

= Pαu(gK),

introducing P ∈ D(G/K), theG-invariant differential operator on G/K correspond-ing to P ∈ D(p)K under

(3.5) P u(gK) := P (∂X) (u(g · ExpX))X=0 .

The map P �→ P is a linear bijection of D(p)K onto D(G/K) (not in general anisomorphism of algebras). Thus

(3.6) MXu(gK) =∑α∈B

Pαu(gK)P ∗α(X).

Our final step is to express this by means of the generators P1, ..., Pl of D(G/K).Let

f(X,Y ) =∑

σ,τ∈Nn

fστXσY τ = 1 +

∑|σ|≥1,|τ |≥1

fστXσY τ

denote the Taylor series of f at the origin (with respect to some basis of p); thelatter expression of this series follows from Remark 1 in 3.1.

Theorem 3.2. Let u be an analytic function on a neighborhood of a point g0Kin G/K. Then there exist a neighborhood V of g0 in G, a radius R > 0 and asequence of polynomials (Aα)α∈B such that, for g ∈ V , X ∈ p and |X| < R,

MXu(gK) =∑α∈B

P ∗α(X)Aα

(f ; P1, ..., Pl

)u(gK) .

Each Aα (f ;λ1, ..., λl) is a polynomial in the λj (with 1 ≤ j ≤ l) and the fστ (with|σ| + |τ | ≤ α · d), homogeneous of degree α · d if one assigns deg λj = degPj = djand deg fστ = |σ|+ |τ |. Furthermore

Aα (f ;λ1, ..., λl) = λα11 · · ·λαl

l + lower degree in λ .

The coefficients of Aα only depend on the structure of the algebra D(p)K and thechoice of generators Pj.

This theorem provides a precise form of a result proved in 1959 by Helgason ([3,Chapter IV], or [5, Chapter II, Theorem 2.7]). Helgason’s theorem states that, forarbitrary Riemannian homogeneous spaces,

MXu(gK) =

∞∑n=0

pn (D1, ..., Dl)u(gK)

where the pn’s are polynomials (with p0 = 1) and the Dj ’s are generators of thealgebra D(G/K). Here, restricting to the case of a symmetric space, we obtaina more explicit expression of pn, showing its dependence on X, and an inductiveprocedure to compute the Aα’s, relating them to the infinitesimal structure of thespace (the coefficients fστ ).


For a spherical function ϕ we have MXϕ(gK) = ϕ(ExpX)ϕ(gK) by the func-tional equation and we immediately infer the following expansion. Again, a moregeneral but less precise form of this expansion is given in Helgason [5, Chapter II,Corollary 2.8].

Corollary 1. Let ϕ be a spherical function on G/K and let λj denote the

eigenvalues corresponding to our generators of D(G/K): Pjϕ = λjϕ for j = 1, ..., l.Then there exists R > 0 such that, for |X| < R,

ϕ(ExpX) =∑α∈B

Aα(f ;λ1, ..., λl)P∗α(X) .

Proof of Theorem 3.2. We shall prove the equality

(3.7) Pα = Aα

(f ; P1, ..., Pl

)for all α ∈ Nl by induction on the degree α · d of Pα. First P 0 = 1 and our claim

holds true with A0 = 1. The main step is to compare PαPj with Pα ◦ Pj and thisis where f enters the picture.Indeed let u be an arbitrary smooth function on G/K. By Theorem 3.1 applied tothe K-invariant function F (X) =

∫Ku(Exp (k ·X))dk we have∫

K×K

u(ek·X · Exp(k′ · Y )

)dkdk′ =

∫K×K

u(Exp(k ·X+k′ ·Y ))f(k ·X, k′ ·Y )dkdk′.

Now let P,Q ∈ D(p)K . Applying to both sides of this equality the differentialoperator P (∂X)Q(∂Y ) we obtain, at X = Y = 0,

P (∂X)Q(∂Y )(u(eX · ExpY

))X=Y =0

= P (∂X)Q(∂Y ) (u (Exp(X + Y )) f(X,Y ))X=Y =0

in view of theK-invariance of P and Q. In other words, remembering the definition(3.5) and the expansion of f ,

P ◦ Qu(o) = P (∂X)Q(∂Y ) (u (Exp(X + Y )) f(X,Y ))X=Y=0(3.8)

= P (∂X)Q(∂X) (uExp(X))X=0 +

+∑

|σ|≥1,|τ |≥1

fστ P (σ)(∂X)Q(τ)(∂X)(u(ExpX))X=0

by Leibniz’ formula, where P (σ)(ξ) = ∂σξ P (ξ) for ξ ∈ p∗. To put it briefly we

have PQu(o) = P ◦ Qu(o)− Ru(o), where R =∑

|σ|≥1,|τ |≥1 fστ P (σ)(∂X)Q(τ)(∂X)

(finite sum) belongs to D(p)K and has lower order than PQ. The G-invariance ofall operators allows replacing the origin o by any point in G/K, hence

(3.9) PQ = P ◦ Q− R,

an equality in D(G/K).Now let α ∈ Nl with αj ≥ 1 for some j. We may write Pα = P βPj (composition of

operators in D(p)K) with β ∈ Nn and (3.9) gives P βPj = P β ◦ Pj − R. In D(p)Kthe operator R decomposes as R =

∑γ rγP

γ , where the rγ ’s are scalars and∑

runs over γ ∈ B, γ · d ≤ β · d+ dj − 2 = α · d− 2. Therefore, assuming (3.7) for allβ with β · d < α · d we obtain

Pα = P βPj = Aβ

(f ; P1, ..., Pl

)◦ Pj −

∑γ

rγAγ

(f ; P1, ..., Pl

),


and the result follows by induction on α · d. �Remark 1. In [2] Gray and Willmore proved the following mean value expansionfor an arbitrary Riemannian manifold M . Let u be an analytic function on aneighborhood of a point a ∈ M and let Mru(a) denote the mean value of u overthe (Riemannian) sphere S(a, r) with center a and radius r. Besides let J be theJacobian of the exponential map Expa, let Δ be (as above) the Euclidean Laplaceoperator on the tangent space to M at a, and Δa the differential operator on aneighborhood of a in M defined by (Δau) ◦ Expa = Δ(u ◦ Expa). Then

(3.10) Mru(a) =j(n/2)−1

(r√−Δa

)(Ju)(a)

j(n/2)−1

(r√−Δa

)(J)(a)

,

where j is the (modified) Bessel function

j(n/2)−1(x) =∞∑

m=0

(−1)mcm

x2m , cm = 2mm!n(n+ 2) · · · (n+ 2m− 2).

This ”generalized Pizzetti’s formula” follows easily from the corresponding formulain Euclidean space, since the Riemannian sphere S(a, r) is the image under Expaof the Euclidean sphere with center 0 and radius r. For isotropic Riemanniansymmetric spaces M = G/K the spheres are K-orbits and we may compare (3.10)with our Theorem 3.2: the Gray-Willmore expansion uses the same simple powerseries as in the Euclidean case, but in general their differential operator Δa is notG-invariant on M . Thus no simple expansion of the spherical functions seems tocome out of (3.10).Remark 2. The symmetry f(X,Y ) = f(Y,X) (Remark 3 of 3.1) together with(3.8) imply the commutativity of the algebra D(G/K). Thus Theorem 3.2 gives yetanother proof of MXMY = MY MX .

References

[1] Mogens Flensted-Jensen and Tom Koornwinder, The convolution structure for Jacobi functionexpansions, Ark. Mat. 11 (1973), 245–262. MR0340938 (49 #5688)

[2] A. Gray and T. J. Willmore, Mean-value theorems for Riemannian manifolds, Proc. Roy. Soc.Edinburgh Sect. A 92 (1982), no. 3-4, 343–364, DOI 10.1017/S0308210500032571. MR677493(84f:53038)

[3] Sigurdur Helgason, Differential operators on homogenous spaces, Acta Math. 102 (1959),239–299. MR0117681 (22 #8457)

[4] Sigurdur Helgason, Groups and geometric analysis, Pure and Applied Mathematics, vol. 113,Academic Press Inc., Orlando, FL, 1984. Integral geometry, invariant differential operators,and spherical functions. MR754767 (86c:22017)

[5] Sigurdur Helgason, Geometric analysis on symmetric spaces, 2nd ed., Mathematical Surveysand Monographs, vol. 39, American Mathematical Society, Providence, RI, 2008. MR2463854(2010h:22021)

[6] Sigurdur Helgason, Integral geometry and Radon transforms, Springer, New York, 2011.MR2743116 (2011m:53144)

[7] Fritz John, Plane waves and spherical means applied to partial differential equations, DoverPublications Inc., Mineola, NY, 2004. Reprint of the 1955 original. MR2098409

[8] Tom H. Koornwinder, Jacobi functions and analysis on noncompact semisimple Lie groups,Special functions: group theoretical aspects and applications, Math. Appl., Reidel, Dordrecht,

1984, pp. 1–85. MR774055 (86m:33018)

[9] ROUVIERE, F., Symmetric spaces and the Kashiwara-Vergne method, in preparation.


Laboratoire Dieudonne, Universite de Nice, Parc Valrose, 06108 Nice cedex 2,

France

E-mail address: [email protected]: http://math.unice.fr/~frou


Semyanistyi fractional integrals and Radon transforms

B. Rubin

Dedicated to Professor Sigurdur Helgason on his 85th birthday

Abstract. Many Radon-like transforms are members of suitable operatorfamilies indexed by a complex parameter. Semyanistyi’s fractional integralsassociated to the classical Radon transform on Rn are a typical example ofsuch a family. We obtain sharp inequalities for these integrals and the corre-sponding Radon transform acting on Lp spaces with a radial power weight.The operator norms are explicitly evaluated. Similar results are obtained forfractional integrals associated to the k-plane transform for any 1 ≤ k < n. Wealso give a brief survey of diverse Semyanistyi type integrals arising in integralgeometry and PDE.

1. Introduction

The classical Radon transform takes a function f(x) on Rn to a function (Rf)(τ ) =∫τf on the set Πn of hyperplanes in Rn. This transform plays the central role

in integral geometry and has numerous applications [6, 11, 20, 21]. In 1960 V.I.Semyanistyi [44] came up with an idea to regard Rf as a member of an analyticfamily of fractional integrals Rαf , so that for sufficiently good f ,

(1.1) Rαf∣∣α=0

= Rf.

This idea has proved to be very fruitful in subsequent developments; see Section 4below. The Semyanistyi fractional integrals are defined by the formula

(1.2) (Rαf)(τ ) =1

γ1(α)

∫Rn

f(x) [dist(x, τ )]α−1 dx,

(1.3) γ1(α) = 2Γ(α) cos(απ/2), Reα > 0; α �= 1, 3, . . . ,

where dist(x, τ ) is the Euclidean distance between the point x and the hyperplane τ .The normalizing coefficient 1/γ1(α) stems from the one-dimensional Riesz potential[16,29]. More general fractional integrals, when τ is a plane of arbitrary dimension1≤k≤n−1, were introduced in [36].

The present article consists of two parts. The first part (Sections 2,3) is devotedto mapping properties of Radon transforms on Lebesgue spaces. This topic was

2010 Mathematics Subject Classification. Primary 44A12; Secondary 47G10.Key words and phrases. Radon transforms, weighted norm estimates.The author is thankful to Tufts University for the hospitality and support during the Joint

AMS meeting and the Workshop on Geometric Analysis on Euclidean and Homogeneous Spacesin January 2012.

c©2013 American Mathematical Society221

222 B. RUBIN

studied in many publications from different points of view; see, e.g., [4,5,7–9,13,15,21,27,28,45,48], to mention a few. We suggest an alternative approach whichworks well in weighted Lp spaces with a radial power weight, yields sharp estimatesfor the parameters and explicit formulas for the operator norms. Moreover, itextends to fractional integrals (1.2). The same method is applicable to more generaloperators associated to the k-plane transform for any 1 ≤ k ≤ n− 1. Main resultsare presented by Theorems 2.1, 2.3, 3.2, and 3.3. Our approach was inspired by aseries of works on operators with homogeneous kernels; see, e.g., [12,40, 47,49].A common point for this class of operators and the afore-mentioned operators ofintegral geometry is a nice behavior with respect to rotations and dilations. Thesame approach is applicable to the dual Radon transforms and the correspondingdual Semyanistyi integrals; see, e.g., [38], where a different technique has been used.

In the second part of the paper (Sections 4) we give a brief survey of variousimportant Semyanistyi type fractional integrals associated to the correspondingRadon-like transforms.

The author thanks the referee for valuable suggestions.

2. Fractional integrals associated to the Radon transform

2.1. Preliminaries. In the following σn−1 = 2πn/2/Γ(n/2) is the area ofthe unit sphere Sn−1 in Rn; dθ stands for the normalized measure on Sn−1 sothat

∫Sn−1 dθ = 1; e1, . . . , en are coordinate unit vectors; O(n) is the group of

orthogonal transformations of Rn endowed with the invariant probability measure.The Lp spaces, 1 ≤ p ≤ ∞, are defined in a usual way, 1/p+ 1/p′ = 1; Πn denotesthe set of all hyperplanes τ in Rn. Every τ ∈ Πn can be parametrized as

(2.1) τ (θ, t) = {x ∈ Rn : x · θ = t}, (θ, t) ∈ Sn−1 × R.

Clearly,

(2.2) τ (θ, t) = τ (−θ,−t).We set

Lpμ(R

n) = {f : ||f ||p,μ ≡ || |x|μf ||Lp(Rn) <∞},Lpν(S

n−1 × R) = {ϕ : ||ϕ||∼p,ν ≡ || |t|νϕ||Lp(Sn−1×R) <∞},where μ and ν are real numbers and the Lp-norm on Sn−1×R is taken with respectto the measure dθdt. Passing to polar coordinates, we have

(2.3) ||f ||p,μ=

⎛⎜⎝σn−1

∞∫0

rn−1+μp dr

∫O(n)

|f(rγe1)|p dγ

⎞⎟⎠1/p

, 1 ≤ p <∞,

(2.4) ||f ||∞,μ = ess supr,γ

rμ |f(rγe1)|.

Similarly,

(2.5) ||ϕ||∼p,ν =

⎛⎜⎝ ∞∫−∞

|t|νp dt∫

O(n)

|ϕ(γe1, t)|p dγ

⎞⎟⎠1/p

, 1 ≤ p <∞,

(2.6) ||ϕ||∼∞,ν = ess supt,γ

|t|ν |ϕ(γe1, t)|.

SEMYANISTYI FRACTIONAL INTEGRALS 223

We write the Radon transform in terms of the parametrization (2.1) as

(2.7) (Rf)(τ ) ≡ (Rf)(θ, t) =

∫θ⊥

f(tθ + u) dθu,

where θ⊥={x : x · θ=0} and dθu denotes the Lebesgue measure on θ⊥. Similarly,

(2.8) (Rαf)(τ ) ≡ (Rαf)(θ, t) =1

γ1(α)

∫Rn

f(x)|t− x · θ|α−1 dx.

Theorem 2.1. Let 1 ≤ p ≤ ∞, 1/p+ 1/p′ = 1, α > 0. Suppose

(2.9) ν = μ− α− (n− 1)/p′,

(2.10) α− 1 + n/p′ < μ < n/p′.

Then

(2.11) ||Rαf ||∼p,ν ≤ cα ||f ||p,μ,

where

cα = ||Rα|| =21/p−α π(n−1)/2 Γ

(n/p′ − μ

2

)Γ

(1− α+ μ− n/p′

2

)σ1/pn−1 Γ

(μ+ n/p

2

)Γ

(α+ n/p′ − μ

2

) .

Remark 2.2. The necessity of (2.9) can be proved using the standard scalingargument; cf. [46, p. 118]. Let fλ(x) = f(λx), λ > 0. Then

||fλ||p,μ = λ−μ−n/p ||f ||p,μ, ||Rαfλ||∼p,ν = λ−ν−n−α+1/p′ ||Rαf ||∼p,ν .

Hence, whenever ||Rαf ||∼p,ν ≤ c ||f ||p,μ with c independent of f , we get ν = μ−α−(n− 1)/p′. As we shall see below, the condition (2.10) is also best possible.

2.2. Proof of Theorem 2.1.2.2.1. Step 1. Let us prove (2.11). For t > 0, changing variables θ = γe1,

γ ∈ O(n), and x = tγy, we get

(2.12) (Rαf)(γe1, t) =tα+n−1

γ1(α)

∫Rn

f(tγy) |1− y1|α−1 dy.

If 1 ≤ p < ∞, then, by Minkowski’s inequality, using (2.5) and (2.3), we obtainthat the norm ||Rαf ||∼p,ν does not exceed the following:

21/p

γ1(α)

∫Rn

|1− y1|α−1

⎛⎜⎝ ∞∫0

∫O(n)

t(α+n−1+ν)p |f(tγy)|p dγdt

⎞⎟⎠1/p

dy

=cα ||f ||p,μ, cα=1

γ1(α)

(2

σn−1

)1/p∫Rn

|1−y1|α−1|y|−μ−n/p dy.(2.13)

If p = ∞, then (2.6) and (2.4) give a similar estimate ||Rαf ||∼∞,ν ≤ cα ||f ||∞,μ inwhich cα has the same form with 1/p = n/p = 0.

224 B. RUBIN

To evaluate cα, denoting λ = μ+ n/p, we have

cα=1

γ1(α)

(2

σn−1

)1/p∞∫

−∞

|1−y1|α−1 dy1

∫Rn−1

(|y′|2+y21)−λ/2 dy′=c1 c2,

c1=

(2

σn−1

)1/p ∫Rn−1

(1+|z|2)−λ/2 dz=21/p π(n−1)/2 Γ((λ+ 1− n)/2)

σ1/pn−1 Γ(λ/2)

,

c2 =1

γ1(α)

∞∫−∞

|1− y1|α−1|y1|n−λ−1 dy1 =γ1(n− λ)

γ1(α+ n− λ);

see, e.g., [16], where convolutions of Riesz kernels are considered in any dimensions.Combining these formulas with (1.3), we obtain the desired expression. Note thatthe integral in (2.13) is finite if and only if α−1+n/p′ < μ < n/p′, which is (2.10).

2.2.2. Step 2. Let us show that cα = ||Rα||. By Step 1, ||Rα|| ≤ cα. Thus, itremains to prove that ||Rα|| ≥ cα. Since the operator R

α : Lpμ(Rn)→ Lp

ν(Sn−1×R)

is bounded, for any f ∈ Lpμ(Rn) and ϕ ∈ Lp′

−ν(Sn−1 ×R) by Holder’s inequality we

have

(2.14) I =

∣∣∣∣∣∫

Sn−1×R

(Rαf)(θ, t)ϕ(θ, t) dtdθ

∣∣∣∣∣ ≤ ||Rα|| ||f ||p,μ ||ϕ||∼p′,−ν .

Suppose f(x) ≡ f0(|x|) ≥ 0 and ϕ(θ, t) ≡ ϕ0(|t|) ≥ 0. Then

I =2

γ1(α)

∞∫0

ϕ0(t) dt

∫Rn

f0(|x|) |t− x · e1|α−1 dx

=2σn−1

γ1(α)

∞∫0

ϕ0(t) dt

∞∫0

rn−1f0(r) dr

∫Sn−1

|t− rη1|α−1 dη

=2σn−1

γ1(α)

∫Sn−1

dη

∞∫0

|1−sη1|α−1sn−1 ds

∞∫0

ϕ0(t)f0(ts) tn+α−1 dt.

Let 1 < p <∞. In this case

||f ||p,μ =

(σn−1

∞∫0

rn−1+μpfp0 (r) dr

)1/p

,(2.15)

||ϕ||∼p′,−ν =

(2

∞∫0

t−νp′ϕp′

0 (t) dt

)1/p′

.


Choose ϕ0(t) = tμp−αfp−10 (t) so that ||ϕ||∼p′,−ν = (2/σn−1)

1/p′ ||f ||p−1p,μ . Then (2.14)

yields

21/pσ1+1/p′

n−1

γ1(α)

∫Sn−1

dη

∞∫0

|1−sη1|α−1sn−1 ds

×∞∫0

tμp+n−1fp−10 (t)f0(ts) dt ≤ ||Rα|| ||f ||pp,μ.(2.16)

Finally we set f0(t) = 0 if t < 1 and f0(t) = t−μ−n/p−ε, ε > 0, if t > 1. Then||f ||pp,μ = σn−1/εp and (2.16) becomes

||Rα|| ≥21/pσ

1/p′

n−1

γ1(α)

∫Sn−1

dη

∞∫0

|1−sη1|α−1sn/p′−μ−1−ε

{sεp, s < 11, s > 1

}ds

=1

γ1(α)

(2

σn−1

)1/p∫Rn

|1− y1|α−1|y|−μ−n/p−ε

{|y|εp, |y|<11, |y|>1

}dy;

cf. (2.13). Passing to the limit as ε→ 0, we obtain ||Rα|| ≥ cα.If p = 1, then ν = μ − α. We choose ϕ0(t) = tμ−α and proceed as above. If

p =∞, we choose f0(r) = r−μ. Then ||f ||∞,μ = 1,

I =2σn−1

γ1(α)

∫Sn−1

dη

∞∫0

|1−sη1|α−1sn−μ−1 ds

∞∫0

ϕ0(t) tn+α−μ−1 dt,

and I ≤ ||Rα|| ||ϕ||∼1,−ν . We set ϕ0(t) = 0 if t < 1 and ϕ0(t) = t−δ if t > 1, where δis big enough. This gives

ν + δ − 1

(μ+ δ − n− α) γ1(α)

∫Rn

|1− y1|α−1|y|−μ dy ≤ ||Rα||.

Assuming δ →∞, we obtain cα ≤ ||Rα||, as desired; cf. (2.13). �

2.3. The case α = 0. This case corresponds to the Radon transform (2.7)and needs independent consideration.

Theorem 2.3. Let 1 ≤ p ≤ ∞, 1/p+ 1/p′ = 1,

(2.17) ν = μ− (n− 1)/p′, μ > n/p′ − 1.

Then ||Rf ||∼p,ν ≤ c ||f ||p,μ, where

(2.18) c = ||R|| =21/p π(n−1)/2 Γ

(1 + μ− n/p′

2

)σ1/pn−1 Γ

(μ+ n/p

2

) .

Proof. The necessity of (2.17) can be checked as in Theorem 2.1. To provethe norm inequality, setting θ = γe1, γ ∈ O(n), t > 0, we have

(Rf)(θ, t) =

∫θ⊥

f(tθ + u) dθu = tn−1

∫Rn−1

f(tγ(e1 + z)) dz.

226 B. RUBIN

Hence,

||Rf ||∼p,ν =

(2

∞∫0

∫O(n)

∣∣∣∣∣∫

Rn−1

t(n−1+ν)p |f(tγ(e1 + z))|p dz∣∣∣∣∣ dγdt

)1/p

≤ 21/p∫

Rn−1

( ∞∫0

∫O(n)

|f(tγ(e1 + z))|p t(n−1+ν)p dγdt

)1/p

dz

(set t|e1 + z| = t(1 + |z|2)1/2 = s)

= c

( ∞∫0

∫Sn−1

|f(sη)|p s(n−1+ν)p dηds

)1/p

= c ||f ||p,μ,

where

c=

(2

σn−1

)1/p ∫Rn−1

dz

(1+|z|2)(ν+n−1/p′)/2=

(2

σn−1

)1/p ∫Rn−1

dz

(1+|z|2)(μ+n/p)/2.

The last integral gives an expression in (2.18). The proof of the equality c = ||R||mimics that in Section 2.2.2. �

3. Fractional integrals associated to the k-plane transforms

We denote by Πn,k the set of all nonoriented k-dimensional planes in Rn, 1 ≤k ≤ n − 1. To parameterize these planes and define the corresponding analoguesof R and Rα, we invoke the Stiefel manifold Vn,n−k ∼ O(n)/O(k) of n × (n − k)real matrices, the columns of which are mutually orthogonal unit n-vectors. Forv ∈ Vn,n−k, dv stands for the left O(n)-invariant probability measure on Vn,n−k

which is also right O(n− k)-invariant. A plane τ ∈ Πn,k can be parameterized by

(3.1) τ (v, t) = {x ∈ Rn : vTx = t}, (v, t) ∈ Vn,n−k × Rn−k,

where vT stands for the transpose of the matrix v. The case k = n− 1 agrees with(2.1). Clearly,

(3.2) τ (v, t) = τ (vωT , ωt) ∀ω ∈ O(n− k).

This equality is a substitute for (2.2) for all 1 ≤ k ≤ n− 1.The k-plane transform takes a function f on Rn to a function (Rkf)(τ ) =

∫τf

on Πn,k. In terms of the parametrization (3.1) it has the form

(3.3) (Rkf)(v, t) =

∫v⊥

f(vt+ u) dvu,

where v⊥ denotes the k-dimensional linear subspace orthogonal to v and dvu standsthe usual Lebesgue measure on v⊥. Similarly,

(3.4) (Rαk f)(v, t) =

1

γn−k(α)

∫Rn

f(x) |vTx− t|α+k−n dx,

where |vTx− t| denotes the Euclidean norm of vTx− t in Rn−k and

(3.5) γn−k(α) =2απ(n−k)/2Γ(α/2)

Γ((n− k − α)/2).


The normalizing coefficient 1/γn−k(α) in (3.4) is the same as in the Riesz potentialon Rn−k [16,29].

Remark 3.1. One can regard Πn,k as a fiber bundle over the Grassmann mani-fold of all k-dimensional linear subspaces of Rn. The corresponding parametrizationof k-planes is different from (3.1); cf. [36,38]. In the present article we prefer towork with parametrization (3.1) because it reduces to (2.1) when k = n− 1.

Let

Lpν(Vn,n−k × Rn−k) = {ϕ : ||ϕ||∼p,ν ≡ || |t|νϕ||Lp(Vn,n−k×Rn−k) <∞},

where the Lp norm on Vn,n−k × Rn−k is taken with respect to the measure dvdt.In the following

v0 =

[In−k

0

]∈ Vn,n−k

and In−k is the identity (n− k)× (n− k) matrix. Then

(3.6) ||ϕ||∼p,ν =

⎛⎜⎝σn−k−1

∞∫0

rνp+n−k−1 dr

∫O(n)

dγ

∫O(n−k)

|ϕ(γv0, rωe1)|p dω

⎞⎟⎠1/p

,

if 1 ≤ p <∞, and

(3.7) ||ϕ||∼∞,ν = ess supr,γ,ω

|r|ν |ϕ(γv0, rωe1)|.

Theorem 3.2. Let 1 ≤ p ≤ ∞, 1/p+ 1/p′ = 1, α > 0. Suppose

(3.8) ν = μ− α− k/p′,

(3.9) α+ k − n/p < μ < n/p′.

Then

||Rαk f ||∼p,ν ≤ cα,k ||f ||p,μ,

where

cα,k= ||Rαk ||=2−απk/2

(σn−k−1

σn−1

)1/p Γ

(n/p′−μ

2

)Γ

(μ+n/p−k−α

2

)Γ

(μ+n/p

2

)Γ

(n/p′−μ+a

2

) .

3.1. Proof of Theorem 3.2.3.1.1. Step 1. The necessity of (3.8) is a consequence of the equalities

||fλ||p,μ = λ−μ−n/p ||f ||p,μ, ||Rαk fλ||∼p,ν = λ−ν−α−k/p′−n/p ||Rα

k f ||∼p,ν ,where fλ(x) = f(λx), λ > 0. Let γ ∈ O(n) and ω ∈ O(n−k) be such that v = γv0,t = |t|ωe1. Changing variables in (3.4) by setting

x = |t|γωy, ω =

[ω 00 Ik

],

we obtain

(3.10) (Rαk f)(γv0, |t|ωe1) =

|t|α+k

γn−k(α)

∫Rn

f(|t|γωy) |vT0 y − e1|α+k−n dy.

228 B. RUBIN

Suppose first 1 ≤ p < ∞. Then, by Minkowski’s inequality, owing to (3.6) and(3.8), the norm ||Rα

k f ||∼p,ν does not exceed the following:⎛⎜⎝σn−k−1

∞∫0

∫O(n)

∫O(n−k)

rνp+n−k−1|(Rαk f)(γv0, rωe1)|p dωdγdr

⎞⎟⎠1/p

≤σ1/pn−k−1

γn−k(α)

∫Rn

|vT0 y − e1|α+k−n

×

⎛⎜⎝ ∞∫0

∫O(n)

∫O(n−k)

|f(rγωy)|prμp+n−1 dωdγdr

⎞⎟⎠1/p

dy = cα,k ||f ||p,μ,

(3.11) cα,k =

(σn−k−1

σn−1

)1/p1

γn−k(α)

∫Rn

|vT0 y − e1|α+k−n|y|−μ−n/p dy.

If p =∞ we similarly have ||Rαk f ||∼∞,ν ≤ cα,k ||f ||∞,μ, where cα,k has the same form

as above with 1/p = n/p = 0.To compute cα,k, we set λ = μ+ n/p. Then

cα,k=σ1/pn−k−1

σ1/pn−1 γn−k(α)

∫Rn−k

|y′ − e1|α+k−n dy′∫Rk

(|y′|2 + |y′′|2)−λ/2 dy′′=c1 c2,

where

c1 =

(σn−k−1

σn−1

)1/p ∫Rk

(1 + |z|2)−λ/2 dz =

(σn−k−1

σn−1

)1/pπk/2 Γ((λ− k)/2)

Γ(λ/2),

c2 =1

γn−k(α)

∫Rn−k

|y′ − e1|α+k−n |y′|k−λ dy′ =γn−k(n− λ)

γn−k(n− λ+ α);

see also (3.5). It remains to put these formulas together and make obvious simpli-fications. Note that the repeated integral in the expression for cα,k is finite if andonly if α+ k − n/p < μ < n/p′, which is (3.9). Thus ||Rα

k || ≤ cα,k.3.1.2. Step 2. To prove that ||Rα

k || ≥ cα,k we follow the reasoning from Section

2.2.2. For any f ∈ Lpμ(Rn) and ϕ ∈ Lp′

−ν(Vn,n−k × Rn−k),

I =

∣∣∣∣∣∫

Vn,n−k×Rn−k

(Rαk f)(v, t)ϕ(v, t) dtdv

∣∣∣∣∣ ≤ ||Rαk || ||f ||p,μ ||ϕ||∼p′,−ν .

Let f(x) ≡ f0(|x|) ≥ 0, ϕ(v, t) ≡ ϕ0(|t|) ≥ 0. Then

I=1

γn−k(α)

∫Rn−k

ϕ0(|t|) |t|α+k dt

∫Rn

f0(|t||y|) |vT0 y − e1|α+k−n dy

=σn−k−1 σn−1

γn−k(α)

∫Sn−1

dη

∞∫0

|svT0 η−e1|α+k−nsn−1 ds

∞∫0

ϕ0(r)f0(rs) rn+α−1 dr.


Suppose 1 < p <∞. Then ||f ||p,μ can be computed by (2.15) and

||ϕ||∼p′,−ν =

⎛⎝σn−k−1

∞∫0

r−νp′+n−k−1 ϕp′

0 dr

⎞⎠1/p

.

cf. (3.6). Choose ϕ0(r) = rμp−αfp−10 (r). Then

||ϕ||∼p′,−ν =

(σn−k−1

σn−1

)1/p′

||f ||p−1p,μ

and we have

σn−k−1 σn−1

γn−k(α)

∫Sn−1

dη

∞∫0

|svT0 η − e1|α+k−nsn−1 ds

∞∫0

fp−10 (r)f0(rs) r

μp+n−1 dr

≤(σn−k−1

σn−1

)1/p′

||Rαk || ||f ||pp,μ.(3.12)

Setting f0(r) = 0 if r < 1 and f0(r) = r−μ−n/p−ε, ε > 0, if r > 1, we obtain||f ||pp,μ = σn−1/εp, and therefore,

||Rαk || ≥

(σn−k−1

σn−1

)1/pσn−1

γn−k(α)

∫Sn−1

dη

×∞∫0

|svT0 η − e1|α+k−nsn−1−μ−n/p−ε

{sεp, s < 11, s > 1

}ds

=

(σn−k−1

σn−1

)1/p1

γn−k(α)

∫Rn

|vT0 y−e1|α+k−n

|y|μ+n/p+ε

{|y|εp, |y| < 11, |y| > 1

}dy;

cf. (3.11) Passing to the limit as ε → 0, we obtain ||Rαk || ≥ cα,k. The cases p = 1

and p =∞ are treated as in Section 2.2.2.

3.2. Weighted norm estimates for the k-plane transform. The followingstatement deals with the k-plane transform (3.3) and formally corresponds to α = 0in Theorem 3.2.

Theorem 3.3. Let 1 ≤ p ≤ ∞, 1/p+ 1/p′ = 1. Suppose that

(3.13) ν = μ− k/p′, μ > k − n/p.

Then ||Rkf ||∼p,ν ≤ ck ||f ||p,μ, where

(3.14) ck = ||Rk|| =πk/2

(σn−k−1

σn−1

)1/p Γ

(μ+n/p−k

2

)Γ

(μ+n/p

2

) .

Proof. As before, the conditions (3.13) are sharp. To prove the norm inequal-ity, as in Section 3.1.1 we set v = γv0, t = rωe1, γ ∈ O(n), ω ∈ O(n − k), r > 0.This gives

(Rkf)(γv0, rωe1) = rk∫Rk

f(rγω(e1 + z)) dz, ω =

[ω 00 Ik

].

230 B. RUBIN

If 1 ≤ p < ∞, then combining (3.6) with Minkowski’s inequality, we majorize||Rkf ||∼p,ν by the following:⎛⎜⎝σn−k−1

∞∫0

∫O(n)

∫O(n−k)

rνp+n−k−1|(Rkf)(γv0, rωe1)|p dωdγdr

⎞⎟⎠1/p

≤ σ1/pn−k−1

∫Rk

⎛⎜⎝ ∞∫0

∫O(n)

∫O(n−k)

|f(rγω(e1+z))|prμp+n−1 dωdγdr

⎞⎟⎠1/p

dz

= ck ||f ||p,μ,

ck =

(σn−k−1

σn−1

)1/p ∫Rk

(1 + |z|2)−(μ+n/p)/2 dz.

The last integral was computed in the previous section. In the case p = ∞ wesimilarly have ||Rkf ||∼∞,ν ≤ ck ||f ||∞,μ with 1/p = n/p = 0. The proof of theequality ck = ||Rk|| mimics that in Theorem 2.1. �

4. Some generalizations and modifications

Following Semyanistyi’s idea, one can construct many integral operators whosekernel behaves like a power function of the geodesic distance between the point andthe respective manifold. These operators can be regarded as fractional analoguesof the corresponding Radon-like transforms. Below we review some examples.

4.1. Fractional integrals associated to hyperplanes in Rn. The follow-ing convolution operators are typical objects in the one-dimensional fractional cal-culus [29,43]:

(4.1) Jα±ω = hα

± ∗ ω, Jαω = hα ∗ ω, Jαs ω = hα

s ∗ ω.

Here

hα±(t) =

tα−1±Γ(α)

=1

Γ(α)

{|t|α−1 if ± t > 0,0 otherwise;

hα(t) =1

γ1(α)

{|t|α−1 if α �= 1, 3, 5, . . . ,tα−1 log |t| if α = 1, 3, 5, . . . ;

hαs (t) =

1

γ′1(α)

{|t|α−1 sgn t if α �= 2, 4, 6, . . . ,tα−1 log |t| sgn t if α = 2, 4, 6, . . . ;

γ1(α) =

⎧⎨⎩2Γ(α) cos(απ/2) if α �= 1, 3, 5, . . . ,

(−1)k+122kπ1/2k!Γ(k + 1/2) if α = 2k + 1 = 1, 3, 5, . . . ;

γ′1(α) =

⎧⎨⎩2iΓ(α) sin(απ/2) if α �= 2, 4, 6, . . . ,

(−1)k−122k−1iπ1/2(k − 1)!Γ(k + 1/2) if α = 2k = 2, 4, 6, . . . ;


For (θ, t) ∈ Sn−1×R, Reα > 0, the corresponding “fractional Radon transforms”were defined in [30] by

(Rα±f)(θ, t) =

∫Rn

f(x)hα±(t− x · θ) dx,(4.2)

(Rαf)(θ, t) =

∫Rn

f(x)hα(t− x · θ) dx,(4.3)

(Rαs f)(θ, t) =

∫Rn

f(x)hαs (t− x · θ) dx.(4.4)

In particular,

(4.5) (R1+f)(θ, t) =

∫x·θ<t

f(x) dx, (R1−f)(θ, t) =

∫x·θ>t

f(x) dx.

The formula (4.3) gives the original Semyanistyi integral (1.2). Setting (Rθf)(t) =(Rf)(θ, t), we get

(4.6) (Rα±f)(θ, t) = (Jα

±Rθf)(t),

(4.7) (Rαf)(θ, t) = (JαRθf)(t), (Rαs f)(θ, t) = (Jα

s Rθf)(t).

The corresponding dual transforms are defined by

(∗Rα

±ϕ)(x) =

∫Sn−1

∫R

ϕ(θ, t)hα±(t−x · θ) dtdθ=R∗Jα

∓ϕ,(4.8)

(∗Rαϕ)(x) =

∫Sn−1

∫R

ϕ(θ, t)hα(t−x · θ) dtdθ=R∗Jαϕ,(4.9)

(∗R

αsϕ)(x) =

∫Sn−1

∫R

ϕ(θ, t)hαs (t−x · θ) dtdθ=−R∗Jα

s ϕ.(4.10)

Here Jα±, Jα, and Jα

s act in the second argument of ϕ and R∗ denotes the dualRadon transform

(R∗ψ)(x) =

∫Sn−1

ψ(θ, x · θ) dθ.

It is instructive to consider fractional Radon transforms and their duals onfunctions belonging to the Semyanistyi spaces Φ(Rn) and Φ(Sn−1×R) [44].1 Werecall the definition of these spaces.

Let S(Rn) be the Schwartz space of rapidly decreasing smooth functions on Rn

and let S(Sn−1×R) be a similar space of smooth functions g(θ, t) on Sn−1×R withthe topology defined by the sequence of norms

‖g‖m = sup|γ|+j≤m

supθ,t

(1 + |t|)m|∂γθ ∂

jt g(θ, t)|, m ∈ Z+ = {0, 1, 2, . . . },

∂γθ g(θ, t) =

[∂|γ|g(x/|x|, t)∂xγ1

1 · · · ∂xγnn

]x=θ

, γ = (γ1, · · · , γn) ∈ Zn+.

1See also [11] for different notation.

232 B. RUBIN

Let f(y) ≡ (Ff)(y) =∫Rn f(x) eix·ydx be the Fourier transform of f ; f∨(x) =

(F−1f)(x). Following [44], we denote

Ψ(Rn) = {ω(x) ∈ S(Rn) : (∂γω)(0) = 0 ∀γ ∈ Zn+};

Ψ(Sn−1×R) = {ψ(θ, t) ∈ S(Sn−1×R) : (∂γθ ∂

jtψ)(θ, 0) = 0

∀γ ∈ Zn+, j ∈ Z+, θ ∈ Sn−1};

Φ(Rn) = F [Ψ(Rn)], Φ(Sn−1×R) = F [Ψ(Sn−1×R)](in the last equality F acts in the t-variable). The spaces Φ(Rn) and Φ(Sn−1×R) areclosed subspaces of S(Rn) and S(Sn−1×R), respectively, with the induced topology.We denote

Φeven(Sn−1×R) = {ϕ(θ, t) ∈ Φ(Sn−1×R) : ϕ(θ, t) = ϕ(−θ,−t)}.

The operators

R : Φ(Rn)→Φeven(Sn−1×R), R∗ : Φeven(S

n−1×R)→Φ(Rn)

are isomorphisms [11,44]. For α ∈ C and η ∈ R, we denote

(∓iη)−α=exp(−α log |η|±απi

2sgn η)= |η|−α

(cos

απ

2±i sin απ

2sgn η

).

We also recall that the Riesz potential operator Iα, which can be defined on func-

tions f ∈ Φ(Rn) by the formula (Iαf)∧(y) = |y|−αf(y), is an automorphism ofΦ(Rn) for any α ∈ C.

Lemma 4.1. [30] If f ∈ Φ(Rn), ϕ ∈ Φ(Sn−1×R), then integrals ( 4.2)-( 4.4)and ( 4.8)-( 4.10) extend as entire functions of α by the formulas:

(4.11) [(Rα±f)(θ, ·)]∧(η) = f(ηθ)(∓iη)−α,

(4.12) [(Rαf)(θ, ·)]∧(η) = f(ηθ)|η|−α,

(4.13) [(Rαs f)(θ, ·)]∧(η) = f(ηθ)|η|−α sgn η;

(4.14) (∗Rα

±ϕ)(x) = (R∗[ϕ(θ, η)(±iη)−α]∨)(x),

(4.15) (∗Rαϕ)(x) = (R∗[ϕ(θ, η)|η|−α]∨)(x),

(4.16) (∗R

αsϕ)(x) = −(R∗[ϕ(θ, η)|η|−α sgn η]∨)(x).

Lemma 4.1 yields the following series of composition formulas which agree withthe classical equality R∗Rf = cnI

n−1f from [11,21].

Theorem 4.2. [30] Let α, β∈C; f ∈Φ(Rn), cn=2n−1πn/2−1Γ(n/2). Then

(4.17)∗R

β±R

α∓f = cα,βI

α+β+n−1f, cα,β = cn cos((α+ β)π/2));

(4.18)∗R

β±R

α±f = c′α,βI

α+β+n−1f, c′α,β = cn cos((±α∓ β)π/2);

(4.19)∗RβRα

±f =∗Rα

±Rβf = cαI

α+β+n−1f, cα = cn cos(απ/2);

(4.20)∗Rβ

sRα±f =

∗Rα

±Rβs f = c′αI

α+β+n−1f, c′α = ∓icn sin(απ/2);


(4.21)∗RβRαf =

∗Rβ

sRαs f = cn I

α+β+n−1f ;

(4.22)∗R

βsR

αf =∗R

βRαs f = 0.

Equalities (4.17)–(4.22) yield a variety of inversion formulas in which cn =2n−1πn/2−1Γ(n/2):

(4.23)∗R

1−n−α± Rα

∓f =

{0 for n even,

cn(−1)(n−1)/2f for n odd,

(4.24)∗R

1−n−α± Rα

±f = cn cos ((2α− 1 + n)π/2) f,

(4.25)∗R1−n−αRα

±f = cn cos(απ/2) f,

(4.26)∗R

1−n−αs Rα

±f = ∓icn sin(απ/2) f,

(4.27)∗R

1−n−α± Rαf = cn cos ((n+ α− 1)π/2) f,

(4.28)∗R

1−n−α± Rα

s = ±icn sin((n+ α− 1)π/2) f,

(4.29)∗R 1−n−αRαf =

∗R 1−n−α

s Rαs = cnf.

In particular, the Radon transform R (= R0± = R0) can be inverted by

(4.30)∗R

1−n± Rf = cn(−1)(n−1)/2f (for n odd),

(4.31)∗R

1−nRf = cnf (for any n ≥ 2).

By (4.8) and (4.9), the last two formulas can be written as

R∗[∂n−1t (Rf)(θ, t)] = cn(−1)(n−1)/2f and R∗[J1−nRf ] = cnf,

respectively. For the half-space transforms R1± we have

(4.32)∗R

−n∓ R1

±f = cn(−1)(n−1)/2f (n odd),

(4.33)∗R

−ns R1

±f = ∓icnf (any n ≥ 2),

or

(4.34) R∗[∂nτ (R

1±f)(θ, τ )]=∓cn(−1)(n−1)/2f, R∗[J−n

s R1±f ]=±icnf.

Another important special case is the operator Rs ≡ R0s defined by

(4.35) (Rsf)(θ, t) = p.v.1

πi

∫Rn

f(x) dx

t− x · θ = limε→0

1

πi

∫|t−x·θ|>ε

f(x) dx

t− x · θ

or, equivalently,

(4.36) (Rsf)(θ, t) = [f(ηθ) sgn η]∨(t)

234 B. RUBIN

(at least for f ∈ Φ(Rn); cf. (4.13)). We call this operator the Radon-Hilberttransform in view of the obvious equality

(4.37) (Rsf)(θ, t) = p.v.1

πi

∞∫−∞

(Rθf)(y)

t− ydy.

According to (4.27) and (4.29), Rs can be inverted by the formulas

(4.38)∗R

1−n± Rsf=±cn(−1)n/2f (for n even),

(4.39)∗R 1−n

s Rsf = cnf (for any n≥2).

These formulas correspond to the following:

(4.40) R∗[∂n−1t (Rsf)(θ, t)] = cn(−1)n/2f, R∗[J1−n

s Rsf ] = cnf.

Theorem 4.2 is a source of another series of inversion formulas, which can beobtained from (4.17)–(4.21) by setting β = 0 and applying I1−n−α from the left.Namely, for any α∈C and f ∈Φ(Rn),

(4.41) I1−n−αR∗Rα±f = cn cos(απ/2) f,

(4.42) I1−n−α∗R0

sRα±f = ±cn sin(απ/2) f,

(4.43) I1−n−αR∗Rαf = cnf, I1−n−α∗R

0sR

αs f = cnf,

cn = 2n−1πn/2−1Γ(n/2). The first formula in (4.43) is well known [21, SectionII.2]. Moreover, for α = 1 (4.42) gives

(4.44) I−n∗R 0

sR1±f = ±cnf.

This inversion formula for the half-space transforms is alternative to (4.32)–(4.34).

Remark 4.3. An advantage of the Semyanistyi spaces Φ(Rn) and Φ(Sn−1×R)is that in the framework of these spaces all formulas in this section can be easilyjustified and are available for all complex α. Many of them extend to arbitrarySchwartz functions or even to Lp functions. However, this extension leads to in-evitable restrictions on the parameters and requires special analytic tools. Thetheory of Semyanistyi spaces was substantially extended by Lizorkin [17]-[19] andSamko [42] for needs of function theory and multidimensional fractional calculus.

4.2. Some other modifications of Semyanistyi’s integrals. Below wegive more examples of Semyanistyi type integrals arising in integral geometry andrelated areas. The references below are far from being complete. More informationcan be found in cited papers.

Fractional integrals (3.4) associated to the k-plane transforms and their dualswere introduced in [36]. More general operators on functions of matrix argumentwere defined in [26] and applied to inversion of the corresponding Radon transforms.An analogue of (3.4) for the unit sphere Sn−1 in Rn has the form

(4.45) (Cλf)(ξ) = γn,k(λ)

∫Sn−1

f(θ) (sin [d(θ, ξ)])λ dθ,

where γn,k(λ) is a normalizing coefficient and d(θ, ξ) stands for the geodesic distancebetween the point θ ∈ Sn−1 and the k-dimensional totally geodesic submanifold ξ


of Sn−1, 1 ≤ k ≤ n−2; see [34]. For k = n−2 this integral operator can be writtenas

(4.46) (Cλf)(η) = αn(λ)

∫Sn−1

f(θ) |η · θ|λ dθ, η ∈ Sn−1,

with the relevant coefficient αn(λ). The operator (4.46) belongs to the family ofthe so-called cosine transforms, playing an important role in convex geometry andmany other areas [22,37]. More general analytic families of cosine transforms onStiefel and Grassmann manifolds were studied in [1,22,39]. Analogues of (4.45)and (4.46) for the n-dimensional real hyperbolic space were studied in [3,35]. An“odd version” of (4.46) having the form

(4.47) (Cλf)(η)= αn(λ)

∫Sn−1

f(θ)|η · θ|λ sgn(η · θ) dθ, η∈Sn−1,

and associated to the hemispherical Funk transform on Sn−1 was studied in [31];see also [41].

One should also mention analytic families of mean value operators arising inthe theory of the Euler-Poisson-Darboux equation on the constant curvature spaces[23]-[25] and associated to the relevant spherical mean Radon transforms. Forexample, in the Euclidean case these operators have the form

(4.48) (Mλf)(x, t) = cn,λ

∫|y|<1

(1−|y|2)λf(x−ty) dy, t > 0.

An analogue of (4.48) for functions on the unit sphere Sn in Rn+1 is defined by

(MλSf)(η, t)=

cn,λ(1−t2)λ+n/2

∫η·θ>t

(η · θ−t)λf(θ) dθ, t∈(−1, 1),

with the corresponding normalizing coefficient. These operators are intimatelyrelated to the inverse problems for the corresponding PDE’s and play an importantrole in thermoacoustic tomography [2]. Injectivity of Mλ

S for fixed t ∈ (0, 1) is adifficult problem leading to Diophantine approximations and small denominatorsfor spherical harmonic expansions [32,33].

This list of examples can be continued.

References

[1] S. Alesker, The α-cosine transform and intertwining integrals, Preprint, 2003.[2] Y. A. Antipov, R. Estrada, and B. Rubin, Method of analytic continuation for the inverse

spherical mean transform in constant curvature spaces, J. Anal. Math. 118 (2012), 623–656,DOI 10.1007/s11854-012-0046-y. MR3000693

[3] C. A. Berenstein and B. Rubin, Totally geodesic Radon transform of Lp-functions on realhyperbolic space, Fourier analysis and convexity, Appl. Numer. Harmon. Anal., BirkhauserBoston, Boston, MA, 2004, pp. 37–58. MR2087237 (2005f:44003)

[4] A. P. Calderon, On the Radon transform and some of its generalizations, (Chicago, Ill.,1981), Wadsworth Math. Ser., Wadsworth, Belmont, CA, 1983, pp. 673–689. MR730101(86h:44002)

[5] M. Christ, Estimates for the k-plane transform, Indiana Univ. Math. J. 33 (1984), no. 6,891–910, DOI 10.1512/iumj.1984.33.33048. MR763948 (86k:44004)

[6] S. R. Deans, The Radon transform and some of its applications, Robert E. Krieger PublishingCo. Inc., Malabar, FL, 1993. Revised reprint of the 1983 original. MR1274701 (95a:44003)

236 B. RUBIN

[7] S. W. Drury, A survey of k-plane transform estimates, Commutative harmonic analysis(Canton, NY, 1987), Contemp. Math., vol. 91, Amer. Math. Soc., Providence, RI, 1989,pp. 43–55, DOI 10.1090/conm/091/1002587. MR1002587 (92b:44002)

[8] J. Duoandikoetxea, V. Naibo, and O. Oruetxebarria, k-plane transforms and relatedoperators on radial functions, Michigan Math. J. 49 (2001), no. 2, 265–276, DOI10.1307/mmj/1008719773. MR1852303 (2002g:42017)

[9] K. J. Falconer, Continuity properties of k-plane integrals and Besicovitch sets, Math.

Proc. Cambridge Philos. Soc. 87 (1980), no. 2, 221–226, DOI 10.1017/S0305004100056681.MR553579 (81c:53067)

[10] I. M. Gel′fand and G. E. Shilov, Generalized functions. Vol. 1, Academic Press [HarcourtBrace Jovanovich Publishers], New York, 1964 [1977]. Properties and operations; Translatedfrom the Russian by Eugene Saletan. MR0435831 (55 #8786a)

[11] S. Helgason, Integral geometry and Radon transforms, Springer, New York, 2011. MR2743116(2011m:53144)

[12] N. K. Karapetyants, Integral operators with homogeneous kernels, Reports of the extendedsessions of a seminar of the I. N. Vekua Institute of Applied Mathematics, Vol. I, no. 1 (Rus-sian) (Tbilisi, 1985), Tbilis. Gos. Univ., Tbilisi, 1985, pp. 98–101, 246 (Russian). MR861550

[13] A. Kumar and S. K. Ray, Mixed norm estimate for Radon transform on weighted Lp spaces,Proc. Indian Acad. Sci. Math. Sci. 120 (2010), no. 4, 441–456, DOI 10.1007/s12044-010-0043-y. MR2761772 (2011k:44005)

[14] A. Kumar and S. K. Ray, Weighted estimates for the k-plane transform of radial functionson Euclidean spaces, Israel J. Math. 188 (2012), 25–56, DOI 10.1007/s11856-011-0091-8.MR2897722

[15] I. �Laba and T. Tao, An x-ray transform estimate in Rn, Rev. Mat. Iberoamericana 17 (2001),no. 2, 375–407, DOI 10.4171/RMI/298. MR1891202 (2003a:44003)

[16] N. S. Landkof, Foundations of modern potential theory, Springer-Verlag, New York, 1972.Translated from the Russian by A. P. Doohovskoy; Die Grundlehren der mathematischenWissenschaften, Band 180. MR0350027 (50 #2520)

[17] P. I. Lizorkin,Generalized Liouville differentiation and the functional spaces Lpr(En). Imbed-

ding theorems, Mat. Sb. (N.S.) 60 (102) (1963), 325–353 (Russian). MR0150615 (27 #610)

[18] P. I. Lizorkin, Generalized Liouville differentiation and the method of multipliers in thetheory of imbeddings of classes of differentiable functions, Proc. Steklov Inst. Math., 105(1969), 105–202.

[19] P. I. Lizorkin, Operators connected with fractional differentiation, and classes of differentiablefunctions, Trudy Mat. Inst. Steklov. 117 (1972), 212–243, 345 (Russian). Studies in thetheory of differentiable functions of several variables and its applications, IV. MR0370166(51 #6395)

[20] A. Markoe, Analytic tomography, Encyclopedia of Mathematics and its Applications, vol. 106,Cambridge University Press, Cambridge, 2006. MR2220852 (2007c:44001)

[21] F. Natterer, The mathematics of computerized tomography, Classics in Applied Mathematics,vol. 32, Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 2001.Reprint of the 1986 original. MR1847845 (2002e:00008)

[22] G.Olafsson, A. Pasquale, and B. Rubin, Analytic and Group-Theoretic Aspects of the CosineTransform arXiv:1209.1822v1; see this volume.

[23] M. Olevsky, Quelques theoremes de la moyenne dans les espaces a courbure constante, C.R. (Doklady) Acad. Sci. URSS (N.S.) 45 (1944), 95–98 (French). MR0011886 (6,230c)

[24] M. N. Olevskiı, On the equation Apu(P, t) = (∂2/∂t2 + p(t)∂/∂t + q(t))u(P, t) (Ap a linearoperator) and the solution of Cauchy’s problem for a generalized Euler-Darboux equation,Doklady Akad. Nauk SSSR (N.S.) 93 (1953), 975–978 (Russian). MR0061743 (15,875e)

[25] M. N. Olevskiı, On a generalization of the Pizetti formula in spaces of constant curvature andsome mean-value theorems, Selecta Math. 13 (1994), no. 3, 247–253. Selected translations.MR1306765 (96g:31008)

[26] E. Ournycheva and B. Rubin, Semyanistyi’s integrals and Radon transforms on matrix spaces,J. Fourier Anal. Appl. 14 (2008), no. 1, 60–88, DOI 10.1007/s00041-007-9002-0. MR2379753(2009b:42014)

[27] D. M. Oberlin and E. M. Stein, Mapping properties of the Radon transform, Indiana Univ.Math. J. 31 (1982), no. 5, 641–650, DOI 10.1512/iumj.1982.31.31046. MR667786 (84a:44002)


[28] E. T. Quinto, Null spaces and ranges for the classical and spherical Radon transforms, J.Math. Anal. Appl. 90 (1982), no. 2, 408–420, DOI 10.1016/0022-247X(82)90069-5. MR680167(85e:44004)

[29] B. Rubin, Fractional integrals and potentials, Pitman Monographs and Surveys in Pure andApplied Mathematics, vol. 82, Longman, Harlow, 1996. MR1428214 (98h:42018)

[30] B. Rubin, Fractional calculus and wavelet transforms in integral geometry, Fract. Calc. Appl.Anal. 1 (1998), no. 2, 193–219. MR1656315 (99i:42054)

[31] B. Rubin, Inversion and characterization of the hemispherical transform, J. Anal. Math. 77(1999), 105–128, DOI 10.1007/BF02791259. MR1753484 (2001m:44004)

[32] B. Rubin, Generalized Minkowski-Funk transforms and small denominators on the sphere,Fract. Calc. Appl. Anal. 3 (2000), no. 2, 177–203. MR1757273 (2002f:42028)

[33] B. Rubin, Arithmetical properties of Gegenbauer polynomials and small denominators on thesphere (open problem), Fract. Calc. Appl. Anal. 3 (2000), no. 3, 315–316. MR1788168

[34] B. Rubin, Inversion formulas for the spherical Radon transform and the generalized cosinetransform, Adv. in Appl. Math. 29 (2002), no. 3, 471–497, DOI 10.1016/S0196-8858(02)00028-3. MR1942635 (2004c:44006)

[35] B. Rubin, Radon, cosine and sine transforms on real hyperbolic space, Adv. Math. 170(2002), no. 2, 206–223, DOI 10.1006/aima.2002.2074. MR1932329 (2004b:43007)

[36] B. Rubin, Reconstruction of functions from their integrals over k-planes, Israel J. Math. 141(2004), 93–117, DOI 10.1007/BF02772213. MR2063027 (2005b:44004)

[37] B. Rubin, Intersection bodies and generalized cosine transforms, Adv. Math. 218 (2008),no. 3, 696–727, DOI 10.1016/j.aim.2008.01.011. MR2414319 (2009m:44010)

[38] B. Rubin, Weighted norm inequalities for k-plane transforms, arXiv:1207.5180v1, (to appearin Proceedings of the AMS).

[39] B. Rubin, Funk, Cosine, and Sine transforms on Stiefel and Grassmann manifolds, J. ofGeom. Anal. (to appear).

[40] S. G. Samko, Proof of the Babenko-Stein theorem, Izv. Vyss. Ucebn. Zaved. Matematika5(156) (1975), 47–51 (Russian). MR0387979 (52 #8816)

[41] S. G. Samko, Generalized Riesz potentials and hypersingular integrals with homogeneouscharacteristics; their symbols and inversion, Trudy Mat. Inst. Steklov. 156 (1980), 157–222,

263 (Russian). Studies in the theory of differentiable functions of several variables and itsapplications, VIII. MR622233 (83a:45004)

[42] S. G. Samko, Hypersingular integrals and their applications, Analytical Methods and SpecialFunctions, vol. 5, Taylor & Francis Ltd., London, 2002. MR1918790 (2004a:47057)

[43] S. G. Samko, A. A. Kilbas, and O. I. Marichev, Fractional integrals and derivatives, Gordonand Breach Science Publishers, Yverdon, 1993. Theory and applications; Edited and witha foreword by S. M. Nikol′skiı; Translated from the 1987 Russian original; Revised by theauthors. MR1347689 (96d:26012)

[44] V. I. Semjanistyı, On some integral transformations in Euclidean space, Dokl. Akad. NaukSSSR 134 (1960), 536–539 (Russian). MR0162136 (28 #5335)

[45] D. C. Solmon, A note on k-plane integral transforms, J. Math. Anal. Appl. 71 (1979), no. 2,351–358, DOI 10.1016/0022-247X(79)90196-3. MR548770 (80m:44010)

[46] E. M. Stein, Singular integrals and differentiability properties of functions, Princeton Math-ematical Series, No. 30, Princeton University Press, Princeton, N.J., 1970. MR0290095(44 #7280)

[47] R. S. Strichartz, Lp estimates for integral transforms, Trans. Amer. Math. Soc. 136 (1969),33–50. MR0234321 (38 #2638)

[48] R. S. Strichartz, Lp estimates for Radon transforms in Euclidean and non-Euclidean spaces,Duke Math. J. 48 (1981), no. 4, 699–727, DOI 10.1215/S0012-7094-81-04839-0. MR782573(86k:43008)

[49] T. Walsh, On Lp estimates for integral transforms, Trans. Amer. Math. Soc. 155 (1971),195–215. MR0284880 (44 #2104)

Department of Mathematics, Louisiana State University, Baton Rouge, Louisiana,

70803



Radon–Penrose transform between symmetric spaces

Hideko Sekiguchi

Dedicated to Professor Helgason on the occasion of his 85th birthday.

Abstract. We consider the Penrose transform for Dolbeault cohomologiesthat correspond to Zuckerman’s derived functor modules Aq(λ) with focuson singular parameter λ. We clarify delicate features of these modules whenλ wanders outside the good range in the sense of Vogan. We then discussan example that the Penrose transform is not injective with large kernel inthe sense that its Gelfand–Kirillov dimension is the same with that of theinitial Dolbeault cohomology. We also discuss an example that two differentopen complex manifolds give an isomorphic representation on the Dolbeaultcohomologies.

1. Dolbeault cohomologies and Zuckerman modules Aq(λ)

In this section we discuss some subtle questions on Zuckerman derived functormodules Aq(λ) for singular parameter λ, and translate them in terms of Dolbeaultcohomology spaces over certain complex homogeneous spaces. We refer to [19]for an excellent exposition, and to [15] for detailed algebraic theory. These subtlequestions on Zuckerman’s modules are closely related to an active area of infinitedimensional representations of semisimple Lie groups, in particular, to the long-standing unsolved problem of the classification of the unitary dual (e.g. [21,28,40]).In turn, we shall observe that they serve as a representation theoretic backgroundof the ‘large kernel’ of the Penrose transform which we discuss in Section 3 and the‘twistor transform’ (see [3]) in Section 4.

Let us fix some notation. Suppose G is a real reductive linear group, θ a Cartaninvolution of G, and K the corresponding maximal compact subgroup of G. Wewrite g = k+ p for the complexification of the Cartan decomposition g0 = k0 + p0.Choose a maximal torus T in K, and write t0 for its Lie algebra. We fix a positivesystem Δ+(k, t) once and for all.

Let q = l + u be a θ-stable parabolic subalgebra of g given as follows. Firstof all we observe that if X ∈

√−1t0 then

√−1X is an elliptic element in g0

(or X is an elliptic element by a little abuse of terminology) in the sense that

2010 Mathematics Subject Classification. Primary 22E46; Secondary 43A85, 33C70, 32L25.Key words and phrases. Riemannian symmetric space, reductive group, symmetric pair,

Zuckerman derived functor module, Penrose transform, Dolbeault cohomology, singular unitaryrepresentation, integral geometry.

The author was supported in part by Grant-in-Aid for Scientific Research (C) 23540073,Japan Society for the Promotion of Science.

c©2013 by the author

239

240 HIDEKO SEKIGUCHI

ad(√−1X) ∈ End(g) is semisimple and has only purely imaginary eigenvalues.

Fix such X which is Δ+(k, t)-dominant, and we define a reductive subgroup L :=ZG(X) ≡ {g ∈ G : Ad(g)X = X}, and a nilpotent subalgebra u of g as the sum ofall eigenspaces of ad(X) ∈ End(g) with positive eigenvalues. Then

q := l+ u ≡ Lie(L)⊗R C+ u

is a θ-stable parabolic subalgebra of g, and the elliptic adjoint orbit

Ad(G)X G/L

becomes a complex manifold with holomorphic cotangent bundle T ∗(G/L) G×L

u. In fact, let GC be a complexification of G, and Q a parabolic subgroup withLie subalgebra q = l+ u. Then, Q ∩G = L holds, and we have a generalized Borelembedding:

(1.1) G/L ↪→ GC/Q,

from which we induce a complex structure on G/L as an open subset of the complexgeneralized flag manifold GC/Q, see [9,16].

The above formulation fits with the ‘geometric quantization’ of elliptic coadjointorbits. For this we fix a non-degenerate, invariant bilinear form on g0, and identifyg0 with the dual space g∗0. We say μ ∈

√−1g∗0 is elliptic if the corresponding

element Xμ ∈√−1g0 is elliptic. Then the elliptic coadjoint orbit Oμ := Ad∗(G)μ

carries a complex manifold structure on which G acts biholomorphically, becauseAd∗(G)μ is isomorphic to G/L as a homogeneous space. Further the restrictionof the complex linear form μ to l gives a one-dimensional representation of theLie algebra l, so that an elliptic element μ gives rise to a holomorphic line bundleLμ → Oμ when μ lifts to Q.

Remark 1.1. Concerning the geometry of the double coset G\GC/Q, the finite-ness of G-orbits on the flag variety GC/Q was proved by K. Aomoto [2] and alsoby Wolf [44], and the complete classification of G-orbits on GC/Q was given by T.Matsuki [26]. In the case where G/L is a symmetric space, such a complex manifoldG/L is called 1

2 -Kahler by M. Berger in his (infinitesimal) classification of semisim-ple symmetric pairs. In the above general setting (1.1), the complex homogeneousspace G/L is isomorphic to an elliptic coadjoint orbit, and conversely, any ellipticcoadjoint orbit is obtained in this manner. This viewpoint is important for theconstruction of irreducible unitary representations via the Kirillov–Kostant–Duflo–Vogan orbit philosophy [19,41]. The monograph [9] treated a different aspect ofthe geometry of G/L, especially, in connection with the Akhiezer–Gindikin domain[1], also referred to as the crown domain, and the authors call G/L a flag domain.Our treatment here emphasizes both aspects, namely, the geometric quantizationof an elliptic coadjoint orbit Oμ G/L [19] and an open G-orbit in GC/Q whichis the Matsuki dual of the closed KC-orbit GC/Q [25] leading us to the Hecht–Milicic–Schmid–Wolf duality [11].

In order to describe the condition on the line bundle precisely, we fix a Cartansubalgebra h of l, and denoted by Δ(u, h) the set of weights for u. For a characterλ of the Lie algebra l, following the terminology of Vogan [42], we say λ is in thegood range if

Re〈λ+ ρl, α〉 > 0 for any α ∈ Δ(u, h),

RADON–PENROSE TRANSFORM BETWEEN SYMMETRIC SPACES 241

in the fair range (respectively, weakly fair range) if

(1.2) Re〈λ, α〉 > 0 (respectively, ≥ 0) for any α ∈ Δ(u, h).

Then the G-translates

lg : G/L→ G/L, xL �→ gxL

are biholomorphic for all g ∈ G. The canonical bundle of G/L is given by∧dimG/LT ∗(G/L) G×L

∧topu G×L C2ρ(u),

where 2ρ(u) ∈ l∗ is the differential of the character of L acting on∧top

u.In what follows, we adopt the normalization of the ‘ρ-shift’ [19] for the line bun-

dle that fits the Kirillov–Kostant–Duflo–Vogan orbit philosophy when we considerthe ‘correspondence’

{Coadjoint orbits} - - -→ {Irreducible unitary representations},(see [19,41]) for details. Assume that the character λ + ρ(u) of the Lie algebra l

lifts to L, to be denoted by Cλ+ρ(u). We define a holomorphic line bundle over G/Lby

(1.3) Lλ+ρ(u) := G×L Cλ+ρ(u).

We say that the line bundle Lλ+ρ(u) → G/L is in the good range or the (weakly)fair range, if λ is in the good range or in the (weakly) fair range, respectively. Wedenote the space of ∂-closed j-forms by

Zj := Ker(∂ : E0,j(G/L,Lλ+ρ(u))→ E0,j+1(G/L,Lλ+ρ(u))),

and the space of ∂-exact j-forms by

Bj := Image(∂ : E0,j−1(G/L,Lλ+ρ(u))→ E0,j(G/L,Lλ+ρ(u))).

Then the group G acts naturally on the Dolbeault cohomology spaceHj

∂(G/L,Lλ+ρ(u)) := Zj/Bj . However, there are some difficult problems:

Problem 1. Is Bj closed in E0,j(G/L,Lλ+ρ(u)) in the usual Frechet topology?

Problem 1 was quoted as “formidable” in the early 1980s in the ‘Green Book’[39], and was later referred to as the maximal globalization conjecture (e.g. [46]).We note that without an affirmative solution to Problem 1, we cannot define a rea-sonable Hausdorff topology on the Dolbeault cohomology space, and thus cannotapply any general theory of infinite dimensional continuous representations on com-plete locally convex topological vector spaces. Problem 1 was settled affirmativelyby W. Schmid in the 1960s [30] in a special case, and by his student H. Wong atHarvard in the general case in the early 1990s [45]. In the meantime, algebraicrepresentation theory has developed largely since the late 1970s, particularly, inconnection with Zuckerman’s derived functor in the category of (g,K)-modules,which was introduced by Zuckerman as an algebraic analogue of Dolbeault coho-mologies, see [15]. The point here was that no topology is specified in the theoryof (g,K)-modules. Thanks to the closed range theorem of the ∂-operator by Wong[45], the Dolbeault cohomology space carries the Frechet topology induced by the

quotient map Zj → Hj

∂(G/L,Lλ+ρ(u)), ω �→ [ω]. If we adopt the normalization of

the ‘ρ-shift’ of Vogan–Zuckerman [43] for Aq(λ) and Vogan [41] for RSq (Cλ) with

S := dimC(u ∩ k), we have isomorphisms of (g,K)-modules ([46], see also [19]):

(1.4) Aq(λ− ρ(u)) RSq (Cλ) HS

∂ (G/L,Lλ+ρ(u))K .


The continuous representation π of G on Hj

∂(G/L,Lλ+ρ(u)) defined by [ω] �→

π(g)[ω] := [l∗g−1ω] gives a maximal globalization of its underlying (g,K)-module

in the sense of Schmid [31] of the Zuckerman derived functor module Rjq(Cλ). The

three different parameter ‘λ − ρ(u)’, λ, and ‘λ + ρ(u)’ in (1.4) indicate delicatefeatures of these modules when λ is singular (We remark that even the standardtextbooks [15,39,41] use slightly different ρ-shifts and notations.).

Assume now that λ is in the weakly fair range. This assumption is the mostnatural from a viewpoint of unitary representations. We notice that λ is automati-cally in the fair range if q is recovered from ad(Xλ+ρ(u)) with the notation as before.In this case the Dolbeault cohomologies vanish for all j but for S = dimC(u ∩ k)([39]). The remaining cohomology of degree S is unitarizable by [40]. However, weencounter the following difficult problems:

Problem 2. Is HS∂(G/L,Lλ+ρ(u)) irreducible as a continuous representation of G?

Problem 3. Is HS∂(G/L,Lλ+ρ(u)) non-zero?

Owing to the isomorphism (1.4), Problems 2 and 3 can be restated in terms ofZuckerman derived functor modules RS

q (Cλ). A negative example could be easilyconstructed outside the weakly fair range. Because of the importance of theseproblems in the weakly fair range for unitary representation theory [28,40], severaleffective techniques have been developed to study Problems 2 and 3 over the lastthree decades. Nevertheless, neither Problem 2 nor Problem 3 has been completelysolved as of now.

Concerning Problem 2, we recall an approach based on the theory of Beilinson–Bernstein and Brylinsky–Kashiwara on the localization of g-modules. Let Dλ bethe ring of twisted differential operators on the generalized flag variety GC/Q (seeKashiwara [14]). Then we have a natural ring homomorphism

(1.5) Ψ : U(g)→ Dλ.

The irreducibility result due to J. Bernstein together with the isomorphism (1.4)shows the following (see [42, Proposition 5.7], [17, Fact 6.2.4]):

Proposition 1.2. Suppose λ is in the weakly fair range such that λ+ρ(u) liftsto L. Then we have:

1) HS∂(G/L,Lλ+ρ(u))K is irreducible or zero as a Dλ-module.

2) If λ is in the good range, then Ψ is surjective.3) If the moment map T ∗(GC/Q)→ g∗ is birational and has a normal image,

then Ψ is surjective.

Corollary 1.3. Suppose λ is in the weakly fair range such that λ+ ρ(u) liftsto L. Then we have:

1) If λ is in the good range, then HS∂(G/L,Lλ+ρ(u)) is non-zero and irre-

ducible as a G-module.2) If g = gl(n,C) (or sl(n,C)), then HS

∂(G/L,Lλ+ρ(u)) is irreducible (or

zero) as a G-module.

Proof of Corollary. 1) The first statement is immediate from Proposition1.2 (1). See [15, Theorem 8.2] for purely algebraic proof.2) By Kraft–Procesi [22] the moment map T ∗(GC/Q)→ g∗ is birational and has anormal image for all parabolic subalgebra q if g = gl(n,C). The second statementnow follows from Proposition 1.2. �


In contrast to Corollary 1.3 (2), the birationality of the moment map fails formany parabolic subalgebras q if g = sp(n,C). This does not imply that the irre-ducibility fails immediately after λ goes outside the good range. There is a detailedstudy by Vogan [42] and Kobayashi [17] on the condition of the triple (g, q, λ)that assures Zuckerman’s derived functor module RS

q (Cλ) (or equivalently the Dol-

beault cohomology HS∂(G/L,Lλ+ρ(u))) stays irreducible (allowing to be zero) where

λ wanders outside of the good range but lies in the weakly fair range (i.e. the mostinteresting range of parameters). The following irreducible result is a special caseof [17, Corollary 6.4.1] applied to n2 = · · · = nk = 0 and dimW = 1.

Example 1.4. Suppose g = sp(n,C) and l = gl(k,C) + sp(n − k,C). In thestandard coordinates of the Cartan subalgebra of l, we have

ρ(u) = (n− k − 1

2)1k ⊕ 01n−k.

By a little abuse of notation, we write Cλ for the one-dimensional representation ofthe Lie algebra l given by λ1k⊕01n−k. We note that Cλ is in the weakly fair rangeif and only if λ ≥ 0; Cλ is in the good range if and only if λ > n− k+1

2 . Thus the

general theory (e.g. [15]) does not say about the irreducibility for 0 ≤ λ < n− k+12 .

Let G be any real form of Sp(n,C) such that q = l + u is a θ-stable parabolicsubalgebra of g. Then HS

∂(G/L,Lλ+ρ(u)) is irreducible or zero if λ satisfies

(1.6) λ >k − 1

2.

We note that this irreducibility condition depends neither on real forms of g =sp(n,C) nor on the choice of θ-stable parabolic subalgebra q. The latter meansthat once we fix a parabolic subgroup Q of GC, we can apply the irreducibilitycondition for Dolbeault cohomology spaces for all open G-orbits in GC/Q (flagdomains for both holomorphic type and non-holomorphic type in the sense of [9]).We shall see in Section 3 that the irreducibility fails at the critical parameter λ,namely, when λ = k−1

2 with a specific choice of real forms of g and a specific choiceof flag domains. In Theorem 3.1, we observe the Penrose transform detects thereducibility.

Concerning Problem 3, there are two simple cases:

• If μλ (see (2.2)) is Δ+(k, t)-dominant then the answer is affirmative.• In the case when q is a Borel subalgebra, the answer to Problem 3 is alsosimple.

For more general case, there are several families of (g, q) for which a completeanswer to Problem 3 is known. However, these preceding results indicate that thecondition on λ for the non-vanishing is combinatorially complicated (see Kobayashi[17, Chapters 3,4], Trapa [38]). We provide this picture in a special case:

Example 1.5. Suppose G = U(6, 1), L = T2 × U(4, 1). We may identify G/Lwith the following set{

(l1, l2) :(1) l1 ⊂ l2 ⊂ C7,(2) lj is positive j-plane in C6,1

}.

See Section 4 for the notation Cp,q in general. Using this realization we define acomplex structure on G/L as an open subset of the partial flag variety. Identifying


the character group T2 of T2 with Z2 in a standard way, we see that the set ofweakly fair (respectively, fair) parameters is given by

{(λ1, λ2) ∈ Z2 : λ1 ≥ λ2 ≥ 0}, (respectively, {(λ1, λ2) ∈ Z2 : λ1 > λ2 > 0}).

For instance, (λ1, λ2) = (2, 1) lies in the fair range, however, there does not exist anon-zero G-homomorphism

Hj

∂(G/L,L(2,1)+ρ(u))→ E(G/K, τ )

for any τ ∈ K and any j ∈ N. In particular, we cannot hope to construct a non-zero‘Penrose transform’ (or its variant) in this case. In fact, Hj

∂(G/L,Lλ+ρ(u)) = 0 for

all j ∈ N if and only if (λ1, λ2) = (2, 0), (1, 0), (2, 1) or λ1 = λ2.

2. Penrose transform for Dolbeault cohomologies

In this section we give a quick review of the general construction of the Radon–Penrose transform on the Dolbeault cohomologies that correspond exactly to Zuck-erman derived functor modules Aq(λ), see [30,33,45].

We write the natural embedding of the compact complex manifold K/(L∩K) KC/(Q ∩KC) as

i : K/(L ∩K)→ G/L.

Its translate Cg := lg ◦ i(K/(L∩K)) is also a compact complex submanifold in G/Lfor every g ∈ G. We say Cg is a cycle following Gindikin [10]. Since Cg = Cg′ ifg′ ∈ gK, we may regard that the Riemannian symmetric space G/K parametrizesthe space of cycles

C := {Cg : gK ∈ G/K}.Thus we have a double fibration in the sense of S.-S. Chern [5]:

G/(L ∩K)

↙ ↘(2.1)

GC/Q ⊃open

G/L G/K C

which is G-equivariant. Let us consider the Penrose transform in this generality.Suppose λ ∈ l∗ is a one-dimensional representation of the Lie algebra l. Thenλ|[l,l] ≡ 0. We define μλ ∈ (l ∩ k)∗ by

(2.2) μλ := λ|l∩k + ρ(u)|l∩k − 2ρ(u ∩ k).

Here we note that the canonical bundle of the compact complex submanifoldK/(L∩K) is given by K ×L∩K C2ρ(u∩k) with 2ρ(u ∩ k) ∈ (l ∩ k)∗.

Assume that μλ|t is Δ+(k, t)-dominant and lifts to T . This implies that thecharacter λ+ ρ(u) of the Lie algebra l lifts to L, denoted by Cλ+ρ(u) as in Section1.

We note that Rjq(Cλ) = 0 if j �= S and λ is in the weakly fair range of the

parameter, and RSq (Cλ) is non-zero if μλ|t is Δ+(k, t)-dominant and lifts to T .

It follows from the Borel–Weil–Bott theorem for compact Lie groups thatHS

∂(K/(L∩K), i∗Lλ+ρ(u)) is an irreducible representation of K with highest weight

μλ, which we denote by Vμλ. Let Vμλ

:= G ×K Vμλbe the homogeneous vector

bundle over G/K C associated to Vμλ.


If ω ∈ ZS , then i∗l∗gω ∈ E0,S(K/(L ∩K), i∗Lλ+ρ(u)) is also a ∂-closed form onK/(L ∩K), giving rise to a cohomology class

[i∗l∗gω] ∈ HS∂ (K/(L ∩K), i∗Lλ+ρ(u)) Vμλ

.

Thus, we have defined a map

(2.3) R : ZS ×G→ Vμλ, (ω, g) �→ [i∗l∗gω].

If ω is a ∂-exact form on G/L, then i∗l∗gω is also a ∂-exact form on K/(L ∩ K).Therefore the map (2.3) is well-defined on the level of cohomology:

(2.4) R : HS∂ (G/L,Lλ+ρ(u))×G→ Vμλ

, ([ω], g) �→ [i∗l∗gω].

It follows from the definition that the map R in (2.4) satisfies:

R(π(g0)[ω], g) = R([l∗g−10

ω], g) = [i∗l∗gl∗g−10ω] = R([ω], g−1

0 g),

R([ω], gh) = [i∗l∗hl∗gω] = [l∗hi

∗l∗gω] = h−1R([ω], g),

for any g, g0 ∈ G, h ∈ K. These two relations imply that the map R (2.4) inducesa G-intertwining operator between representations of G:

R : HS∂ (G/L,Lλ+ρ(u))→ E(G/K,Vμλ

), [ω] �→ R([ω], ·).This is a brief explanation of the following ([33, Theorem 2.4]):

Theorem 2.1. Let q = l+ u be a θ-stable parabolic subalgebra of g and λ ∈ l∗.We assume that μλ|t is Δ+(k, t)-dominant and lifts to the torus T . Then R :

HS∂(G/L,Lλ+ρ(u))→ E(G/K,Vμλ

), [ω] �→ R([ω], ·) is a continuous G-intertwiningoperator between Frechet G-modules.

We note that the transform R makes sense even if λ does not satisfy the pos-itivity condition such as the weakly fair range property (1.2). We say that R isthe Penrose transform for the Dolbeault cohomology that corresponds to Zuck-erman’s derived functor module. The transform R is compatible with discretelydecomposable restrictions to subgroups in the sense of [18].

We recall from [9, Definition 5.1.4] that the ‘cycle space’ is given by

C := {Cg : g ∈ GC, Cg ⊂ G/L}.Clearly, the definition (2.4) makes sense for g ∈ GC such that Cg ⊂ G/L. Weremark that the space of cycles a la Gindikin is contained in the cycle space in

general, namely, C ⊂ C holds. However,

C = Cwhen G/L is the ‘Hermitian holomorphic case’ ([9, §5.4]). In particular, for all the

examples G/L ⊂ GC/Q which we shall treat in Sections 3 and 4, we have C = C.The Radon–Penrose transform R is injective if λ is in the good range of pa-

rameter. However, from representation theoretic viewpoints, as we saw in Section1 Zuckerman derived functor modules with singular infinitesimal characters aremore involved and particularly interesting. The behavior of the Penrose transformR becomes more delicate when the parameter λ of the line bundle tends to besingular.

In the summer seminar 1994, a general scheme interacting

1) a characterization of singular irreducible infinite dimensional representa-tions by means of differential equations,


2) a generalization of the Gauss–Aomoto–Gelfand hypergeometric differen-tial equations to higher order,

3) integral geometry, arising from (non-minimal) parabolic subalgebras,4) invariant theory (prehomogeneous vector spaces, b-functions, Capelli iden-

tities)

was posed by T. Kobayashi, especially a suggestion of an effective application of(4) to integral geometry.

This scheme has enabled us to study the Penrose transform and the behaviorof K-finite vectors under the transform R in details even when the parameters aresingular. In particular, we have discovered an interesting phenomenon:

(A) The injectivity of the Penrose transform R may fail, and both the kerneland the image of R may be as large as the initial Dolbeault cohomologyspaces in the sense of the Gelfand–Kirillov dimension.

We also extend a result of Eastwood–Penrose–Wells [7] for the twistor transformand give an example of the following:

(B) Two different complex geometry may give rise to an isomorphic represen-tation on the Dolbeault cohomologies.

By a general theory of Zuckerman and Vogan, we see that neither (A) nor (B)occurs in the good range of parameters (see Corollary 1.3, Theorem 4.1). Thelatter part of this article is to discuss examples of (A) or (B) from the generaltheory of Zuckerman derived functor modules, see Sections 3 and 4, respectively.

3. Large kernel of the Penrose transform

In this section we discuss the kernel of the Penrose transform. We shall give anexample for (A) based on [33], but with some additional argument on the associatedvarieties. We begin by recalling the setting of [33] in a more geometric way thanwhat was given originally in the group language.

3.1. Homogeneous complex manifold of Sp(n,R). Let (R2n, ω) be a sym-plectic vector space with a fixed symplectic form ω . For each 1 ≤ k ≤ n, we define:

(3.1) Xk :=

⎧⎪⎪⎨⎪⎪⎩(V, J) :

(1) V is a 2k-dimensional subspace of R2n

(2) ω|V is non-degenerate,(3) J ∈ GL(V ), J2 = − IdV ,(4) ω(·, J ·) is positive definite on V

⎫⎪⎪⎬⎪⎪⎭ .

For general k, the real symplectic group of rank n acts transitively on Xk. ThenXk is given as a homogeneous space:

Xk Sp(n,R)/(U(k)× Sp(n− k,R)) ≡ G/Lk,

and in particular, Xk is a non-compact complex manifold as an open subset of theisotropic Grassmannian manifold Sp(n)/(U(k)×Sp(n−k)) for every k (1 ≤ k ≤ n).

For k = n, the second condition in (3.1) is trivial, and we have

Xn {J ∈ GL(2n,R) : J2 = − Id, ω(·, J ·) is positive definite

}.

We define a one-dimensional representation of Lk:

χ(k)l : Lk → C×, (A,D) �→ (detA)l,


and a G-equivariant holomorphic line bundle over Xk G/Lk:

Ll ≡ L(k)l := G×Lk

(χ(k)l ,C).

Fix k (1 ≤ k ≤ n). Next, we introduce a family of cycles in the complexmanifold Xk.

Take J ∈ Xn, i.e. a complex structure J on R2n such that ω(·, J ·) is positivedefinite on R2n. Then, for any J-invariant 2k-dimensional vector spaces W , thepair (W,J |W ) belongs to Xk. We collect all such W and define a submanifold CJ

(cycle) in Xk by

CJ := {(W,J |W ) : W ⊂ R2n such that JW = W, dimR W = 2k}.Then CJ is naturally isomorphic to the complex Grassmannian manifoldGrk(Cn),

and the parameter J is regarded as an element of Xn Sp(n,R)/U(n) ≡ G/Ln =G/K, which we shall realize as the Siegel upper half space,

{Z ∈ Sym(n,C) : ImZ + 0}.

3.2. The Penrose transform for Sp(n,R). Let F (U(n), ν) denote the irre-ducible representation of U(n) with highest weight ν. We define W (n, k)± to be themaximal globalization of irreducible highest weight (g,K)-modules (W (n, k)±)Kthat are uniquely determined by the following K-type formulas:

(W (n, k)+)K ⊕

x1≥···≥x2k≥0,xj∈2N

F (U(n), (x1 + k, · · · , x2k + k, k, · · · , k)),(3.2)

(W (n, k)−)K ⊕

x1≥···≥x2k≥0,xj∈2N+1

F (U(n), (x1 + k, · · · , x2k + k, k, · · · , k)).(3.3)

Alternatively, (W (n, k)+)K , (W (n, k)−)K corresponds to the trivial, signature, one-dimensional representation of O(2k) by the theta correspondence.

Next, we take global coordinates zij (1 ≤ i ≤ j ≤ n) of Sym(n,C). Let

∂

∂Z:=

⎛⎜⎜⎜⎝∂

∂z1112

∂∂z12

· · · 12

∂∂z1n

12

∂∂z12

∂∂z22

.... . .

...12

∂∂z1n

· · · ∂∂znn

⎞⎟⎟⎟⎠ for Z =

⎛⎜⎝z11 · · · z1n...

. . ....

z1n · · · znn

⎞⎟⎠ .

For subsets I, J ⊂ {1, 2, · · · , n} with |I| = |J |, we set

P (I, J) := det(∂

∂Z)i∈I,j∈J .

Fix l (1 ≤ l ≤ n), and we define the system (Nl) of differential equations for each l:

(Nl) P (I, J)F (Z) = 0 for any I, J with |I| = |J | = l.

The space of global holomorphic solutions on Xn is denoted by :

Sol(Nl) = {F ∈ O(Xn) : F satisfies (Nl)}.The following theorem determines the image and the kernel of the Penrose

transform:

Theorem 3.1 ([33]). Let n, k ∈ Z satisfy 1 ≤ k ≤ [n2 ] and G = Sp(n,R).


1) The Penrose transform

R : Hk(n−k)

∂(Xk,L(k)

n )→ C∞(Xn,L(n)k )

is a non-zero G-intertwining operator.2) KerR = W (n, k)−.3) ImageR = Sol(N2k+1).4) The Dolbeault cohomology space splits into a direct sum of irreducible G-

modules:

Hk(n−k)

∂(Xk,L(k)

n ) W (n, k)+ ⊕W (n, k)−.

The third statement of Theorem 3.1 is a characterization of the image of thePenrose transform R. It is exactly the space of global solutions to a system (N2k+1)of certain partial differential equations. The second statement asserts more thanKerR �= {0}. It gives a precise description of KerR by (3.3).

Remark 3.2. The proof of [33] uses Bernstein–Sato’s b-functions for prehomo-geneous vector spaces and Kobayashi’s theory on discrete decomposable branchinglaws [18]. The advantage is that we get a precise information of all K-types underthe Penrose transforms (see (3.2) and (3.3) for the K-type formulas of KerR andImageR).

Remark 3.3. In connection with Problem 2 in Section 1 and also with (A) inSection 2, it might be interesting to analyze the above example from a differentapproach by using an idea of the spectral sequence given in Baston–Eastwood [3].

It is noteworthy that both the kernel and the image of R are ‘large’ in the senseof the Gelfand–Kirillov dimension as follows:

Theorem 3.4. In the setting of Theorem 3.1, we have the following equalitiesamong the Gelfand–Kirillov dimensions:

DIM(KerR) = DIM(ImageR) = DIM(Hk(n−k)

∂(Xk,L(k)

n )).

Proof of Theorem 3.4. The statement follows from Theorem 3.1 and Propo-sition 3.7 below. �

Lemma 3.5. The line bundle L(k)n over G/Lk lies in the weakly fair range.

Proof. We can decompose dχ(k)n as

dχ(k)n = λ+ ρ(u)

where

(3.4) λ :=k − 1

21k ⊕ 01n−k,

and ρ(u) = (n− k−12 )1k⊕01n−k. Since

k−12 ≥ 0, λ lies in the weakly fair range. �

Remark 3.6. As we examined in Example 1.4, the parameter (3.4) lies in theboundary of the criterion of irreducibility (see (1.6)).

Proposition 3.7. The associated varieties of the (g,K)-modules of W (n, k)+,

W (n, k)−, and Hk(n−k)

∂(Xk,L(k)

n ) are all the same, and are given by Ad(KC)(u∩p).


Proof. The associated variety of RSq (Cλ) is Ad(KC)(u∩ p) if λ is in the good

range by a theory of Borho–Brylinski [4], and this remains true in the weaklyfair range as far as RS

q (Cλ) �= 0 (see [18, Part III, Lemma 2.7]). Hence the lastassertion hods by Lemma 3.5 and the isomorphism (1.4). Then applying the theoryof discretely decomposable restriction [18, Part III] to the theta correspondenceO(2k)↔ Sp(n,R), or alternatively by [8], we conclude that the associated varietyof W (n, k)+ is the same with that of W (n, k)−. Therefore it also coincides withthat of W (n, k)+ ⊕W (n, k)−. �

We notice that the Dolbeault cohomology space HS∂(G/L,Lλ+ρ(u)) tends to

be reducible with a proper submodule whose Gelfand–Kirillov dimension is strictlysmaller than that of HS

∂(G/L,Lλ+ρ(u)) when λ wanders outside the weakly fair

range. This follows from the equivalent assertion for the Zuckerman derived functormodule RS

q (Cλ) through the isomorphism (1.4). An elementary example is:

Example 3.8. Let G/L = SL(2,R)/SO(2) with λ = −ρ(u) and S = 0. ThenHS

∂(G/L,Lλ+ρ(u)) O(G/K) contains the trivial one-dimensional representation

1 as a submodule and its quotient is a holomorphic discrete series representation πof G of which the underlying (g,K)-module has a non-zero (g,K)-cohomology. Thesize of the two representations may be measured by the Gelfand–Kirillov dimension,denoted by DIM, namely,

DIM1 = 0, DIMπ = DIMO(G/K) = 1.

The same phenomenon happens for vector bundle cases instead of line bundlecases. Accordingly, it would not be surprising to get an example where the Penrosetransform R fails to be injective if we allow the parameter to be outside the weaklyfair range or if we allow the vector bundle case, however, it is unlikely to have ananalogous result to Theorem 3.4 in such a case, as is suggested by Example 3.8 atthe level of the module structure. In this sense, Theorem 3.4 is in good contrast tothis usual phenomenon.

The Laplace expansion formula of the determinant of matrices implies the fol-lowing inclusion of subspaces (not as G-modules):

C Sol(N1) ⊂ Sol(N2) ⊂ · · · ⊂ Sol(Nn) ⊂ Sol(Nn+1) O(Xn).

Theorem 3.1(3) asserts that Sol(Nj) for odd j appears as the image of thePenrose transform of certain Dolbeault cohomologies. As an example for even j,consider the case j = 2. Then an analogous result to Theorem 3.4 does not occur:

Proposition 3.9. There is no geometric setting (G,L,Q,Lλ+ρ(u)) in the weaklyfair range (see (1.3)) that gives rise to the equality

DIM(HS∂ (G/L,Lλ+ρ(u))) = DIM(Sol(N2)).

In particular, there does note exist an isomorphism of (g,K)-modules:

HS∂ (G/L,Lλ+ρ(u))K Sol(N2)K .

Sketch of Proof. The Gelfand–Kirillov dimension of the Dolbeault coho-mology space in the weakly fair range is given by DIM(HS

∂(G/L,Lλ+ρ(u))) =

dimAd(KC)(u∩ p). A simple computation shows that this is strictly larger than nfor any θ-stable maximal parabolic subalgebra q = l + u of g = sp(n,C). On theother hand, Sol(N2) arises as the representation space of the Weil representation.Hence, DIM(Sol(N2)) = n. �


4. Twistor transform and Penrose transform

This section discusses (B) in Section 2, namely, that a non-trivial isomorphismbetween two geometrically distinct Dolbeault cohomology spaces give rise to thesame representation space. This is given by the twistor transform which was in-troduced by Eastwood–Penrose–Wells [3,7]. To the best of the knowledge of theauthor, there is no literature that clarifies the relationship between the twistortransform and delicate behaviors of Zuckerman’s derived functor modules with sin-gular parameters as we discussed in Section 1. By this reason, it might be usefulto compare our geometric setting here with the results in Section 1 on Zuckerman’sderived functor modules Rj

q(Cλ).We begin with an observation that the geometric setting for the existence of

the twistor transform is ‘quite rare’. To be more precise, let G be a real reductiveLie group. Suppose that G/L(j) ⊂ GC/Q

(j) are two G-open orbits defined by the θ-stable parabolic subalgebras q(j) = l(j)+u(j) for j = 1, 2, respectively. Without lossof generality, we may and do assume that q(j) = l(j)+u(j) is compatible with a fixedpositive system Δ+(k, t) as in Section 1. In particular, l(1) and l(2) have a commonCartan subalgebra h such that h∩k = t. Suppose that the characters Cλ(j)+ρ(u(j)) lift

to L(j) for j = 1, 2, respectively. We then discuss when HS(1)

∂(G/L(1),Lλ(1)+ρ(u(1)))

and HS(2)

∂(G/L(2),Lλ(2)+ρ(u(2))) are isomorphic to each other.

By inspecting the action of the center of the enveloping algebra, one sees easilythat this happens only if

(4.1) λ(1) + ρl(1) and λ(2) + ρl(2) must be conjugate by W (g, h).

Here W (g, h) denotes the Weyl group for Δ(g, h). The necessary condition (4.1)depends only on the complex Lie algebras (g, q(1)), and (g, q(2)). On the otherhand, the following result depends on the real forms G of GC, and shows that thenon-trivial twistor isomorphism exists only for outside good range of parameters.Namely, we point out:

Theorem 4.1. Assume that λ(j) are in the good range such that λ(j) + ρ(u(j))lifts to L(j) (j = 1, 2), respectively. If there exists a G-isomorphism

HS(1)

∂ (G/L(1),Lλ(1)+ρ(u(1)))∼→ HS(2)

∂ (G/L(2),Lλ(2)+ρ(u(2))),

then

(4.2) both L(1)/(L(1) ∩ L(2)) and L(2)/(L(1) ∩ L(2)) are compact.

Sketch of Proof. Combine (1.4) with the general theory of Zuckerman’sderived functor modules [43]. �

Remark 4.2. If (4.2) holds then such an isomorphism holds by a suitablechoice of λ(1) and λ(2) by using the Borel–Weil–Bott theorem.

Remark 4.3. Any discrete series representations of G can be realized naturallyin the Dolbeault cohomology spaces by taking Q to be a Borel subgroup of GC andλ in the good range [15,31]. Then by Theorem 4.1, we do not have a non-trivialtwistor transform for discrete series representations (cf. [3, §10.6]).

Remark 4.4. We cannot relax the assumption on λ for “good” range to “weaklyfair range”, as we shall see in Theorem 4.5 with p = q (see Proposition 4.6(3)).


We give an example of a twistor transform, which is an extension of the exam-ples of [3, §10.4]. Let Cp,q be the complex vector space Cp+q equipped with thestandard indefinite Hermitian form of signature (p, q):

(z, w) :=

p∑j=1

zjwj −p+q∑

j=p+1

zjwj for z, w ∈ Cp+q.

We say a k-plane in Cp,q is maximally positive if the restriction of the form ispositive definite (0 < k ≤ p), or non-degenerate with signature (p, k−p) (p+1 ≤ k <p+q). The set Gr+k (C

p,q) of maximally positive k-planes in Cp,q becomes a complexmanifold as an open subset of the complex Grassmannian manifold Grk(Cp+q). Wecall Gr+k (C

p,q) an indefinite Grassmannian manifold. As homogeneous spaces, wehave

Gr+k (Cp,q) U(p, q)/(U(k)× U(p− k, q)) ≡ G/L+

k for k ≤ p,

Gr+p+q−k(Cp,q) U(p, q)/(U(p, q − k)× U(k)) ≡ G/L−

k for k ≤ q.

In particular, G/L±k are reductive symmetric spaces and (4.2) fails.

The general linear group GC := GL(p+ q,C) acts holomorphically and transi-tively on Grk(Cp+q). Let Qk be the isotropy subgroup at the k-plane spanned by�e1, · · · , �ek where {�ej : 1 ≤ j ≤ p+ q} is the standard basis of Cp+q. Then Qk is ofthe following matrix form:

(4.3) Qk =

{(A BO D

):A ∈ GL(k,C), D ∈ GL(p+ q − k,C),

B ∈M(k, p+ q − k;C)

}.

The natural open embedding

Gr+k (Cp,q) ⊂ Grk(Cp+q) GC/Qk

is the generalized Borel embedding as explained in (1.1).Given a pair of integers (m,n) ∈ Z2, we define a one-dimensional representation

χm,n : Qk → C×,

(A BO D

)�→ (detA)m(detD)n.

Then, we get a GC-equivariant holomorphic line bundle

(4.4) Lm,n := GC ×Qk(χm,n,C)

over Grk(Cp+q) GC/Qk. In our notation, the canonical bundle of Grk(Cp+q) isisomorphic to Lp+q−k,−k.

For simplicity, the restriction of a line bundle to a submanifold of the basespace will be denoted by the same letter. In particular, restricting to an open sub-manifold Gr+k (C

p,q), we see that the U(p, q)-equivariant bundle Lm,n → Gr+k (Cp,q)

induces naturally continuous representations of the same group on the Dolbeaultcohomology spaces Hj

∂(Gr+k (C

p,q),Lm,n) endowed with Frechet topology.

Theorem 4.5 (twistor transform). For any k ≤ min(p, q), there is a canon-ical U(p, q)-equivariant topological isomorphism between the following two infinitedimensional Frechet spaces:

Tp,q : Hk(p−k)

∂(Gr+k (C

p,q),Lp+q,q)∼−→ H

k(q−k)

∂(Gr+p+q−k(C

p,q),Lk+q,k).

It should be noted that the base spaces Gr+k (Cp,q) and Gr+p+q−k(C

p,q) are notbiholomorphic to each other.

Back to the concrete example of this section, we have the following:


Proposition 4.6. (1) None of the line bundles in Theorem 4.5 is in the goodrange.(2) Suppose p �= q. Then one of the following two bundles

Lp+q,q → Gr+k (Cp,q)

Lk+q,k → Gr+p+q−k(Cp,q)

is outside of the fair range, and the other is in the weakly fair range.(3) Suppose p = q. Then both of the line bundles are in the weakly fair range, butnone of them are in the fair range.

Proof. We already know the first statement by the general result, Theorem4.1, however, we shall see it directly from the proof of the second and third state-ments.Case I. The holomorphic line bundle Lp+q,q → Gr+k (C

p,q).We write

λ+ ρ(u) = dχp+q,q

where ρ(u) = p+q−k2 1k ⊕ −k

2 1p+q−k. Then we have

(4.5) λ =p+ q + k

21k ⊕ (q +

k

2)1p+q−k.

In this case, for a, b ∈ R, λ = a1k ⊕ b1p+q−k is in the good range if and only if

a− b > p+q2 − 1, the parameter (4.5) does not lie in the good range because q ≥ 1.

On the other hand, λ = a1k ⊕ b1p+q−k is in the fair range if and only if a− b > 0,the parameter (4.5) lies in the fair range if and only if p > q.Case II. The holomorphic line bundle Lk+q,k → Gr+p+q−k(C

p,q).We write

λ+ ρ(u) = dχk+q,k

where ρ(u) = k21p+q−k ⊕ −p−q+k

2 1k. Then we have

(4.6) λ = (k

2+ q)1p+q−k ⊕

p+ q + k

21k.

Since λ = a1p+q−k ⊕ b1k is in the good range if and only if a − b > p+q2 − 1, the

parameter (4.6) does not lie in the good range because p ≥ 1. On the other hand,the fair range condition for λ = a1p+q−k ⊕ b1k amounts to a > b, the parameter(4.6) is in the fair range if and only if q > p. Hence Proposition is proved. �

As in [3,5], the key machinery for Theorem 4.5 is a Radon–Penrose transformconstructed by using the cycle spaces for the embeddings

(4.7) Grk(Cp) ↪→ Gr+k (Cp,q) and Grk(Cq)∨ ↪→ Gr+p+q−k(C

p,q),

respectively. Then the proof of Theorem 4.5 boils down to the basic properties ofthe Penrose transforms R+

k and R−p+q−k summarized as follows:

Theorem 4.7 (Penrose transform). Suppose k ≤ min(p, q). Associated to thecycles (4.7), we have U(p, q)-equivariant topological isomorphisms:

R+k :H

k(p−k)

∂(Gr+k (C

p,q),Lp+q,q)∼−→ Sol(Ω,Mk),

R−p+q−k :H

k(q−k)

∂(Gr+p+q−k(C

p,q),Lk+q,k)∼−→ Sol(Ω,Mk).


Here, Ω U(p, q)/(U(p) × U(q)) is a bounded symmetric domain, (Mk) is asystem of differential equations of order k + 1 on Ω of the determinant type as in[32,34], and Sol(Ω,Mk) is the space of global holomorphic solutions to (Mk). Tobe more precise, we define Ω as

Ω := {Z ∈M(q, p;C) : Ip − Z∗Z + 0}.

Let {zij : 1 ≤ i ≤ q, 1 ≤ j ≤ p} be the standard coordinates of Ω. For I ⊂{1, 2, · · · , q}, J ⊂ {1, 2, · · · , p} such that |I| = |J | = k+1, we define a holomorphicdifferential operator of order k + 1 on Ω:

P (I, J) = det(∂

∂zij)i∈I,j∈J .

Then the system (Mk) of partial differential equations on Ω is defined as

(Mk) P (I, J)F (Z) = 0 for any I, J such that |I| = |J | = k + 1.

The twistor transform Tp,q in Theorem 4.5 is characterized by the followingcommutative diagram:

Hk(p−k)

∂(Gr+k (C

p,q),Lp+q,q)∼−→

Tp,q

Hk(q−k)

∂(Gr+p+q−k(C

p,q),Lk+q,k)

R+k ↘ ↙R−

p+q−k

Sol(Ω,Mk)

The geometry of flag varieties for k = 1 appears in Baston–Eastwood [3, §10.4].Our theorems in a special case (k = 1, p = q = 2) corresponds to an original resultof Eastwood–Penrose–Wells [7]. We note that the system (Mk) reduces to a singledifferential equation of order 1 + 1 = 2 if k = 1 and p = q = 2. See also [6] forthe D-module approach. Our approach based on the Bernstein–Sato b-functionsof prehomogeneous vector spaces [29] is different from their proof. The first partof Theorem 4.7 generalizes the main theorems of [32] (p = q case), and of [34](p ≥ q case). The case p < q is a little more involved. In this case, the vanishingof Dolbeault cohomologies in other degrees does not follow from the general theory[39] of Zuckerman’s derived functor modules because the parameter is outside theweakly fair range as we saw in the proof of Proposition 4.6. However, the vanishingstatement still holds in the above specific setting.

〈Acknowledgement〉 This article is based on a talk delivered at 2012 “AMSSpecial Session on Radon Transforms and Geometric Analysis in Honor of SigurdurHelgason” on January 6–7, 2012. The author would like to thank Professor Helgasonfor his warm comments on that occasion. Thanks are also due to the organizers,Professors Jens Christensen, Fulton Gonzalez and Todd Quinto for their warmhospitality. The author owes much to the anonymous referee for reading verycarefully and giving useful suggestions to the original manuscript.

References

[1] D. N. Akhiezer and S. G. Gindikin, On Stein extensions of real symmetric spaces, Math.Ann. 286 (1990), no. 1-3, 1–12, DOI 10.1007/BF01453562. MR1032920 (91a:32047)

[2] Kazuhiko Aomoto, On some double coset decompositions of complex semisimple Lie groups,J. Math. Soc. Japan 18 (1966), 1–44. MR0191994 (33 #221)


[3] Robert J. Baston and Michael G. Eastwood, The Penrose transform, Oxford MathematicalMonographs, The Clarendon Press Oxford University Press, New York, 1989. Its interactionwith representation theory; Oxford Science Publications. MR1038279 (92j:32112)

[4] W. Borho, J. Brylinski, Differential operators on homogeneous spaces I, Irreducibility of theassociated variety for annihilators of induced modules, Invent. Math. 69 (1982), no. 3, 437–476; MR0679767 (84b:17007); Part III, Characteristic varieties of Harish-Chandra modulesand of primitive ideals 80 (1985), 1–68. MR784528 (87i:22045)

[5] Shiing-shen Chern, On integral geometry in Klein spaces, Ann. of Math. (2) 43 (1942), 178–189. MR0006075 (3,253h)

[6] Andrea D’Agnolo and Corrado Marastoni, Real forms of the Radon-Penrose transform, Publ.Res. Inst. Math. Sci. 36 (2000), no. 3, 337–383, DOI 10.2977/prims/1195142951. MR1781434(2002g:32025)

[7] Michael G. Eastwood, Roger Penrose, and R. O. Wells Jr., Cohomology and massless fields,Comm. Math. Phys. 78 (1980/81), no. 3, 305–351. MR603497 (83d:81052)

[8] Thomas J. Enright and Jeb F. Willenbring, Hilbert series, Howe duality and branch-ing for classical groups, Ann. of Math. (2) 159 (2004), no. 1, 337–375, DOI 10.4007/an-nals.2004.159.337. MR2052357 (2005d:22013)

[9] Gregor Fels, Alan Huckleberry, and Joseph A. Wolf, Cycle spaces of flag domains, Progressin Mathematics, vol. 245, Birkhauser Boston Inc., Boston, MA, 2006. A complex geometricviewpoint. MR2188135 (2006h:32018)

[10] S. Gindikin, Lectures on integral geometry and the Penrose transform (at University of Tokyo)1994.

[11] Henryk Hecht, Dragan Milicic, Wilfried Schmid, and Joseph A. Wolf, Localization and stan-dard modules for real semisimple Lie groups. I. The duality theorem, Invent. Math. 90 (1987),no. 2, 297–332, DOI 10.1007/BF01388707. MR910203 (89e:22025)

[12] Sigurdur Helgason, Geometric analysis on symmetric spaces, Mathematical Surveys andMonographs, vol. 39, American Mathematical Society, Providence, RI, 1994. MR1280714(96h:43009)

[13] Fritz John, The ultrahyperbolic differential equation with four independent variables, DukeMath. J. 4 (1938), no. 2, 300–322, DOI 10.1215/S0012-7094-38-00423-5. MR1546052

[14] Masaki Kashiwara, Representation theory and D-modules on flag varieties, Asterisque 173-174 (1989), 9, 55–109. Orbites unipotentes et representations, III. MR1021510 (90k:17029)

[15] Anthony W. Knapp and David A. Vogan Jr., Cohomological induction and unitary represen-tations, Princeton Mathematical Series, vol. 45, Princeton University Press, Princeton, NJ,1995. MR1330919 (96c:22023)

[16] Toshiyuki Kobayashi and Kaoru Ono, Note on Hirzebruch’s proportionality principle, J. Fac.Sci. Univ. Tokyo Sect. IA Math. 37 (1990), no. 1, 71–87. MR1049019 (91f:32037)

[17] Toshiyuki Kobayashi, Singular unitary representations and discrete series for indefiniteStiefel manifolds U(p, q;F)/U(p − m, q;F), Mem. Amer. Math. Soc. 95 (1992), no. 462,vi+106. MR1098380 (92f:22023)

[18] T. Kobayashi, Discrete decomposability of the restriction of Aq(λ) with respect to reductivesubgroups and its applications, Invent. Math. 117 (1994), 181–205, MR1273263 (95b:22027);Part II, Micro-local analysis and asymptotic K-support, Ann. of Math. (2) 147 (1998), no.3, 709–729, MR1637667 (99k:22020); Part III, Restriction of Harish-Chandra modules andassociated varieties, Invent. Math. 131 (1998), 229–256. MR1608642 (99k:22021)

[19] T. Kobayashi, Harmonic analysis on reduced homogeneous manifolds and unitary represen-tation theory, Sugaku 46 (1994), no. 2, 124–143 (Japanese), Math. Soc. Japan, MR1303773(95k:22011); English translation, Harmonic analysis on reduced homogeneous manifolds of re-ductive type and unitary representation theory, Selected papers on harmonic anaylsis, groups,and invariants, Amer. Math. Soc. Transl. Ser. 2, 183, Amer. Math. Soc., Providence, RI,(1998), 1 – 31. MR1615135

[20] Toshiyuki Kobayashi, Multiplicity-free representations and visible actions on complex mani-

folds, Publ. Res. Inst. Math. Sci. 41 (2005), no. 3, 497–549. MR2153533 (2006e:22017)[21] Toshiyuki Kobayashi, Branching problems of Zuckerman derived functor modules, Repre-

sentation theory and mathematical physics, Contemp. Math., vol. 557, Amer. Math. Soc.,Providence, RI, 2011, pp. 23–40, DOI 10.1090/conm/557/11024. MR2848919

[22] Hanspeter Kraft and Claudio Procesi, Closures of conjugacy classes of matrices are normal,Invent. Math. 53 (1979), no. 3, 227–247, DOI 10.1007/BF01389764. MR549399 (80m:14037)


[23] Lisa A. Mantini, An integral transform in L2-cohomology for the ladder representationsof U(p, q), J. Funct. Anal. 60 (1985), no. 2, 211–242, DOI 10.1016/0022-1236(85)90051-5.MR777237 (87a:22029)

[24] Lisa A. Mantini, An L2-cohomology construction of unitary highest weight modules forU(p, q), Trans. Amer. Math. Soc. 323 (1991), no. 2, 583–603, DOI 10.2307/2001546.MR1020992 (91e:22018)

[25] Toshihiko Matsuki, The orbits of affine symmetric spaces under the action of minimal para-

bolic subgroups, J. Math. Soc. Japan 31 (1979), no. 2, 331–357, DOI 10.2969/jmsj/03120331.MR527548 (81a:53049)

[26] Toshihiko Matsuki,Orbits on affine symmetric spaces under the action of parabolic subgroups,Hiroshima Math. J. 12 (1982), no. 2, 307–320. MR665498 (83k:53072)

[27] R. Penrose, Twistor algebra, J. Mathematical Phys. 8 (1967), 345–366. MR0216828(35 #7657)

[28] Susana A. Salamanca-Riba and David A. Vogan Jr., On the classification of unitary rep-resentations of reductive Lie groups, Ann. of Math. (2) 148 (1998), no. 3, 1067–1133, DOI10.2307/121036. MR1670073 (2000d:22017)

[29] M. Sato, Theory of prehomogeneous vector spaces, Sugaku no Ayumi 15-1 (1970), 85–156.[30] Wilfried Schmid, HOMOGENEOUS COMPLEX MANIFOLDS AND REPRESENTA-

TIONS OF SEMISIMPLE LIE GROUPS, ProQuest LLC, Ann Arbor, MI, 1967. Thesis(Ph.D.)–University of California, Berkeley. MR2617034

[31] Wilfried Schmid, Boundary value problems for group invariant differential equations,

Asterisque Numero Hors Serie (1985), 311–321. The mathematical heritage of Elie Cartan(Lyon, 1984). MR837206 (87h:22018)

[32] Hideko Sekiguchi, The Penrose transform for certain non-compact homogeneous manifoldsof U(n, n), J. Math. Sci. Univ. Tokyo 3 (1996), no. 3, 655–697. MR1432112 (98d:22011)

[33] Hideko Sekiguchi, The Penrose transform for Sp(n,R) and singular unitary representations,J. Math. Soc. Japan 54 (2002), no. 1, 215–253, DOI 10.2969/jmsj/1191593961. MR1864934(2002j:32019)

[34] Hideko Sekiguchi, Penrose transform for indefinite Grassmann manifolds, Internat. J. Math.22 (2011), no. 1, 47–65, DOI 10.1142/S0129167X11006738. MR2765442 (2012e:32037)

[35] Goro Shimura, On differential operators attached to certain representations of classicalgroups, Invent. Math. 77 (1984), no. 3, 463–488, DOI 10.1007/BF01388834. MR759261(86c:11034)

[36] Toshiyuki Tanisaki, Hypergeometric systems and Radon transforms for Hermitian symmetricspaces, Analysis on homogeneous spaces and representation theory of Lie groups, Okayama–Kyoto (1997), Adv. Stud. Pure Math., vol. 26, Math. Soc. Japan, Tokyo, 2000, pp. 235–263.MR1770723 (2001m:22020)

[37] Corrado Marastoni and Toshiyuki Tanisaki, Radon transforms for quasi-equivariant D-modules on generalized flag manifolds, Differential Geom. Appl. 18 (2003), no. 2, 147–176,

DOI 10.1016/S0926-2245(02)00145-6. MR1958154 (2004c:32045)[38] Peter E. Trapa, Annihilators and associated varieties of Aq(λ) modules for U(p, q), Composi-

tio Math. 129 (2001), no. 1, 1–45, DOI 10.1023/A:1013115223377. MR1856021 (2002g:22031)[39] David A. Vogan Jr., Representations of real reductive Lie groups, Progress in Mathematics,

vol. 15, Birkhauser Boston, Mass., 1981. MR632407 (83c:22022)[40] David A. Vogan Jr., Unitarizability of certain series of representations, Ann. of Math. (2)

120 (1984), no. 1, 141–187, DOI 10.2307/2007074. MR750719 (86h:22028)[41] David A. Vogan Jr., Unitary representations of reductive Lie groups, Annals of Mathematics

Studies, vol. 118, Princeton University Press, Princeton, NJ, 1987. MR908078 (89g:22024)[42] David A. Vogan Jr., Irreducibility of discrete series representations for semisimple symmet-

ric spaces, Representations of Lie groups, Kyoto, Hiroshima, 1986, Adv. Stud. Pure Math.,vol. 14, Academic Press, Boston, MA, 1988, pp. 191–221. MR1039838 (91b:22023)

[43] David A. Vogan Jr. and Gregg J. Zuckerman, Unitary representations with nonzero coho-mology, Compositio Math. 53 (1984), no. 1, 51–90. MR762307 (86k:22040)

[44] Joseph A. Wolf, The action of a real semisimple group on a complex flag manifold. I. Orbitstructure and holomorphic arc components, Bull. Amer. Math. Soc. 75 (1969), 1121–1237.MR0251246 (40 #4477)


[45] Hon-Wai Wong, Dolbeault cohomological realization of Zuckerman modules associatedwith finite rank representations, J. Funct. Anal. 129 (1995), no. 2, 428–454, DOI10.1006/jfan.1995.1058. MR1327186 (96c:22024)

[46] Hon-Wai Wong, Cohomological induction in various categories and the maximal globaliza-tion conjecture, Duke Math. J. 96 (1999), no. 1, 1–27, DOI 10.1215/S0012-7094-99-09601-1.MR1663911 (2001f:22051)

Graduate School of Mathematical Sciences, the University of Tokyo, 3-8-1 Komaba,

Meguro, Tokyo, 153-8914, Japan



Principal series representations of infinite dimensional Liegroups, II: Construction of induced representations

Joseph A. Wolf

Abstract. We study representations of the classical infinite dimensional realsimple Lie groups G induced from factor representations of minimal parabolicsubgroups P . This makes strong use of the recently developed structure the-ory for those parabolic subgroups and subalgebras. In general parabolics inthe infinite dimensional classical Lie groups are somewhat more complicatedthan in the finite dimensional case, and are not direct limits of finite dimen-sional parabolics. We extend their structure theory and use it for the infinitedimensional analog of the classical principal series representations. In order

to do this we examine two types of conditions on P : the flag-closed conditionand minimality. We use some riemannian symmetric space theory to provethat if P is flag-closed then any maximal lim-compact subgroup K of G istransitive on G/P . When P is minimal we prove that it is amenable, andwe use properties of amenable groups to induce unitary representations τ ofP up to continuous representations IndGP (τ) of G on complete locally convextopological vector spaces. When P is both minimal and flag-closed we have adecomposition P = MAN similar to that of the finite dimensional case, andwe show how this gives K–spectrum information IndGP (τ)|K = IndK

M (τ |M ).

1. Introduction

This paper continues a program of extending aspects of representation the-ory from finite dimensional real semisimple groups to infinite dimensional real Liegroups. The finite dimensional theory depends on the structure of parabolic sub-groups. That structure was recently been worked out for the classical real directlimit Lie algebras such as sl(∞,R) and sp(∞;R) [7] and then developed for min-imal parabolic subgroups ([25], [27]). Here we refine that structure theory, andinvestigate it in detail when the flags defining the parabolic consist of closed (in theMackey topology) subspaces. Then we develop a notion of induced representationthat makes use of the structure of minimal parabolics, and we use it to constructan infinite dimensional counterpart of the principal series representations of finitedimensional real reductive Lie groups.

The representation theory of finite dimensional real reductive Lie groups isbased on the now–classical constructions and Plancherel Formula of Harish-Chandra.Let G be a real reductive Lie group of Harish-Chandra class, e.g. SL(n;R), U(p, q),

2010 Mathematics Subject Classification. Primary 32L25; Secondary 22E46, 32L10.Research partially supported by the Simons Foundation.


257

258 JOSEPH A. WOLF

SO(p, q), . . . . Then one associates a series of representations to each conjugacy classof Cartan subgroups. Roughly speaking this goes as follows. Let Car(G) denotethe set of conjugacy classes [H] of Cartan subgroups H of G. Choose [H] ∈ Car(G),H ∈ [H], and an irreducible unitary representation χ of H. Then we have a “cusp-idal” parabolic subgroup P of G constructed from H, and a unitary representationπχ of G constructed from χ and P . Let Θπχ

denote the distribution character ofπχ . The Plancherel Formula: if f ∈ C(G), the Harish-Chandra Schwartz space,then

(1.1) f(x) =∑

[H]∈Car(G)

∫H

Θπχ(rxf)dμ[H](χ)

where rx is right translation and μ[H] is Plancherel measure on the unitary dual H.

In order to extend elements of this theory to real semisimple direct limit groups,we have to look more closely at the construction of the Harish–Chandra series thatenter into (1.1).

Let H be a Cartan subgroup of G. It is stable under a Cartan involution θ, aninvolutive automorphism ofG whose fixed point setK = Gθ is a maximal compactlyembedded1 subgroup. ThenH has a θ–stable decomposition T×A where T = H∩Kis the compactly embedded part and (using lower case gothic letters for Lie algebras)exp : a → A is a bijection. Then a is commutative and acts diagonalizably on g.Any choice of positive a–root system defines a parabolic subalgebra p = m+ a+ n

in g and thus defines a parabolic subgroup P = MAN in G. If τ is an irreducibleunitary representation of M and σ ∈ a∗ then ητ,σ : man �→ eiσ(log a)τ (m) is awell defined irreducible unitary representation of P . The equivalence class of theunitarily induced representation πτ,σ = IndG

P (ητ,σ) is independent of the choice ofpositive a–root system. The group M has (relative) discrete series representations,and {πτ,σ | τ is a discrete series rep of M} is the series of unitary representationsassociated to {Ad(g)H | g ∈ G}.

Here we work with the simplest of these series, the case where P is a minimalparabolic subgroup of G, for the classical infinite dimensional real simple Lie groupsG. In [27] we worked out the basic structure of those minimal parabolic subgroups.Recall [21] that lim–compact group means a direct limit of compact groups. As inthe finite dimensional case, a minimal parabolic has structure P = MAN whereM = P ∩ K is a (possibly infinite) direct sum of torus groups, compact classicalgroups such as Spin(n), SU(n), U(n) and Sp(n), and their classical direct limitsSpin(∞), SU(∞), U(∞) and Sp(∞) (modulo intersections and discrete centralsubgroups). In particular M is lim–compact. There in [27] we also discussedvarious classes of representations of the lim-compact group M and the parabolicP . Here we discuss the unitary induction procedure IndG

MAN (τ ⊗ eiσ) where τ isa unitary representation of M and σ ∈ a∗. The complication, of course, is that wecan no longer integrate over G/P .

There are several new ideas in this note. One is to define a new class ofparabolics, the flag-closed parabolics, and apply some riemannian geometry to provea transitivity theorem, Theorem 3.5. Another is to extend the standard finite

1A subgroup of G is compactly embedded if it has compact image under the adjoint repre-sentation of G.

PRINCIPAL SERIES REPRESENTATIONS OF INFINITE DIMENSIONAL LIE GROUPS 259

dimensional decomposition P = MAN to minimal parabolics; that is Theorem4.4. A third is to put these together with amenable group theory to construct ananalog of induced representations in which integration over G/P is replaced by aright P–invariant mean on G. That produces continuous representations of G oncomplete locally convex topological vector spaces, which are the analog of principalseries representations. Finally, if P is flag-closed and minimal, a close look at thisamenable induction process gives the K-spectrum of our representations.

We sketch the nonstandard part of the necessary background in Section 2.First, we recall the classical simple real direct limit Lie algebras and Lie groups.There are no surprises. Then we sketch the theory of complex and real parabolicsubalgebras. Finally we indicate structural aspects such as Levi components andthe Chevalley decomposition. That completes the background.

In Section 3 we specialize to parabolics whose defining flags consist of closedsubspaces in the Mackey topology, that is F = F⊥⊥. The main result, Theorem3.5, is that a maximal lim–compact subgroup K ⊂ G is transitive on G/P . Thisinvolves the geometry of the (infinite dimensional) riemannian symmetric spaceG/K. Without the flag–closed property it would not even be clear whether K hasan open orbit on G/P .

In Section 4 we work out the basic properties of minimal self–normalizing par-abolic subgroups of G, refining results of [25] and [27]. The Levi components arelocally isomorphic to direct sums in an explicit way of subgroups that are either thecompact classical groups SU(n), SO(n) or Sp(n), or their limits SU(∞), SO(∞)or Sp(∞). The Chevalley (maximal reductive part) components are slightly morecomplicated, for example involving extensions 1 → SU(∗) → U(∗) → T 1 → 1 aswell as direct products with tori and vector groups. The main result, Theorem 4.4,is the minimal parabolic analog of standard structure theory for real parabolics infinite dimensional real reductive Lie groups. Proposition 4.14 then gives an explicitconstruction for a self-normalizing flag-closed minimal parabolic with a given Levifactor.

In Section 5 we put all this together with amenable group theory. Since strictdirect limits of amenable groups are amenable, our maximal lim-compact group Kand minimal parabolic subgroups P are amenable. In particular there are means onG/P , and we consider the set M(G/P ) of all such means. Given a homogeneoushermitian vector bundle Eτ → G/P , we construct a continuous representation

IndGP (τ ) of G. The representation space is a complete locally convex topological

vector space, completion of the space of all right uniformly continuous boundedsections of Eτ → G/P . These representations form the principal series for ourreal group G and choice of parabolic P . In the flag-closed case we also obtain theK-spectrum.

In fact we carry out this “amenably induced representation” construction some-what more generally: whenever we have a topological group G, a closed amenablesubgroup H and a G–invariant subset ofM(G/H).

We have been somewhat vague about the unitary representation τ of P . Thisis discussed, with references, in [27]. We go into it in more detail in an Appendix.

I thank Elizabeth Dan-Cohen for pointing out the result indicated below asProposition 3.1. I also thank Gestur Olafsson for fruitful discussions on invariantmeans which led to a technical result, JM(G/H)(G/H;Eτ ) = 0, in Section 5B. That

260 JOSEPH A. WOLF

technical result led to an improvement, Corollary 5.16, in the general constructionof amenably induced representations.

2. Parabolics in Finitary Simple Real Lie Groups

In this section we sketch the real simple countably infinite dimensional locallyfinite (“finitary”) Lie algebras and the corresponding Lie groups, following resultsfrom [1], [2] and [7]. Then we recall the structure of parabolic subalgebras ofthe complex Lie algebras gC = gl(∞;C), sl(∞);C), so(∞;C) and sp(∞;C). Next,we indicate the structure of real parabolic subalgebras, in other words parabolicsubalgebras of real forms of those algebra gC. This summarizes results from [4], [5]and [7].

2A. Finitary Simple Real Lie Groups. The three classical simple locallyfinite countable–dimensional complex Lie algebras are the classical direct limitsgC = lim−→ gn,C given by

(2.1)

sl(∞,C) = lim−→ sl(n;C),so(∞,C) = lim−→ so(2n;C) = lim−→ so(2n+ 1;C),sp(∞,C) = lim−→ sp(n;C),

where the direct systems are given by the inclusions of the form A �→ (A 00 0 ). We

will also consider the locally reductive algebra gl(∞;C) = lim−→ gl(n;C) along with

sl(∞;C). The direct limit process of (2.1) defines the universal enveloping algebras

(2.2)

U(sl(∞,C)) = lim−→U(sl(n;C)) and U(gl(∞,C)) = lim−→U(gl(n;C)),U(so(∞,C)) = lim−→U(so(2n;C)) = lim−→U(so(2n+ 1;C)), and

U(sp(∞,C)) = lim−→U(sp(n;C)),

Of course each of these Lie algebras gC has the underlying structure of a realLie algebra. Besides that, their real forms are as follows ([1], [2], [7]).

If gC = sl(∞;C), then g is one of sl(∞;R) = lim−→ sl(n;R), the real special linearLie algebra; sl(∞;H) = lim−→ sl(n;H), the quaternionic special linear Lie algebra,

given by sl(n;H) := gl(n;H)∩sl(2n;C); su(p,∞) = lim−→ su(p, n), the complex special

unitary Lie algebra of real rank p; or su(∞,∞) = lim−→ su(p, q), complex specialunitary algebra of infinite real rank.

If gC = so(∞;C), then g is one of so(p,∞) = lim−→ so(p, n), the real orthogonal

Lie algebra of finite real rank p; so(∞,∞) = lim−→ so(p, q), the real orthogonal Lie

algebra of infinite real rank; or so∗(2∞) = lim−→ so∗(2n)

If gC = sp(∞;C), then g is one of sp(∞;R) = lim−→ sp(n;R), the real symplectic

Lie algebra; sp(p,∞) = lim−→ sp(p, n), the quaternionic unitary Lie algebra of real

rank p; or sp(∞,∞) = lim−→ sp(p, q), quaternionic unitary Lie algebra of infinite realrank.

If gC = gl(∞;C), then g is one gl(∞;R) = lim−→ gl(n;R), the real general linear

Lie algebra; gl(∞;H) = lim−→ gl(n;H), the quaternionic general linear Lie algebra;

u(p,∞) = lim−→ u(p, n), the complex unitary Lie algebra of finite real rank p; or

u(∞,∞) = lim−→ u(p, q), the complex unitary Lie algebra of infinite real rank.


As in (2.2), given one of these Lie algebras g = lim−→ gn we have the universalenveloping algebra. Just as in the finite dimensional case, we use the universal en-veloping algebra of the complexification. Thus when we write U(g) it is understoodthat we mean U(gC).

The corresponding Lie groups are exactly what one expects. First the complexgroups, viewed either as complex groups or as real groups,

(2.3)

SL(∞;C) = lim−→SL(n;C) and GL(∞;C) = lim−→GL(n;C),SO(∞;C) = lim−→SO(n;C) = lim−→SO(2n;C) = lim−→SO(2n+ 1;C),Sp(∞;C) = lim−→Sp(n;C).

The real forms of the complex special and general linear groups SL(∞;C) andGL(∞;C) are

(2.4)

SL(∞;R) and GL(∞;R) : real special/general linear groups,

SL(∞;H) : quaternionic special linear group,

SU(p,∞) : special unitary groups of real rank p <∞,

SU(∞,∞) : unitary groups of infinite real rank,

U(p,∞) : unitary groups of real rank p <∞,

U(∞,∞) : unitary groups of infinite real rank.

The real forms of the complex orthogonal and spin groups SO(∞;C) and Spin(∞;C)are

(2.5)

SO(p,∞), Spin(p;∞) : orthogonal/spin groups of real rank p <∞,

SO(∞,∞), Spin(∞,∞) : orthogonal/spin groups of real rank ∞,

SO∗(2∞) = lim−→SO∗(2n), which doesn’t have a standard name

Here SO∗(2n)=SO(2n;C)∩U(n, n) where SO∗(2n) is defined by the form κ(x, y) :=∑x�iy� = txiy and SO(2n;C) is defined by (u, v) =

∑(ujvn+j + un+jvj). Finally,

the real forms of the complex symplectic group Sp(∞;C) are

(2.6)

Sp(∞;R) : real symplectic group,

Sp(p,∞) : quaternion unitary group of real rank p <∞, and

Sp(∞,∞) : quaternion unitary group of infinite real rank.

2B. Parabolic Subalgebras. For the structure of parabolic subalgebras wemust describe gC in terms of linear spaces. Let VC and WC be nondegeneratelypaired countably infinite dimensional complex vector spaces. Then gl(∞,C) =gl(VC,WC) := VC ⊗ WC consists of all finite linear combinations of the rank 1operators v⊗w : x �→ 〈w, x〉v. In the usual ordered basis of VC = C∞, parameterizedby the positive integers, and with the dual basis of WC = V ∗

C= (C∞)∗, we can view

gl(∞,C) as infinite matrices with only finitely many nonzero entries. However VC

has more exotic ordered bases, for example parameterized by the rational numbers,where the matrix picture is not intuitive.

The rank 1 operator v ⊗ w has a well defined trace, so trace is well defined ongl(∞,C). Then sl(∞,C) is the traceless part, {g ∈ gl(∞;C) | trace g = 0}.

In the orthogonal case we can take VC = WC using the symmetric bilinear formthat defines so(∞;C). Then

so(∞;C) = so(V, V ) = Λgl(∞;C) where Λ(v ⊗ v′) = v ⊗ v′ − v′ ⊗ v.

262 JOSEPH A. WOLF

In other words, in an ordered orthonormal basis of VC = C∞ parameterized by thepositive integers, so(∞;C) can be viewed as the infinite antisymmetric matriceswith only finitely many nonzero entries.

Similarly, in the symplectic case we can take VC = WC using the antisymmetricbilinear form that defines sp(∞;C), and then

sp(∞;C) = sp(V, V ) = Sgl(∞;C) where S(v ⊗ v′) = v ⊗ v′ + v′ ⊗ v.

In an appropriate ordered basis of VC = C∞ parameterized by the positive integers,sp(∞;C) can be viewed as the infinite symmetric matrices with only finitely manynonzero entries.

In the finite dimensional complex setting, Borel subalgebra means a maximalsolvable subalgebra, and parabolic subalgebra means one that contains a Borel. Itis the same here except that one must use locally solvable to avoid the prospect ofan infinite derived series.

Definition 2.7. A maximal locally solvable subalgebra of gC is called a Borelsubalgebra of gC . A parabolic subalgebra of gC is a subalgebra that contains a Borelsubalgebra. ♦

In the finite dimensional setting a parabolic subalgebra is the stabilizer of anappropriate nested sequence of subspaces (possibly with an orientation conditionin the orthogonal group case). In the infinite dimensional setting here, one mustbe very careful as to which nested sequences of subspaces are appropriate. If F is asubspace of VC then F⊥ denotes its annihilator in WC. Similarly if ′F is a subspace

of WC the ′F⊥

denotes its annihilator in VC. We say that F (resp. ′F ) is closed if

F = F⊥⊥ (resp. ′F = ′F⊥⊥

). This is the closure relation in the Mackey topology[12], i.e. the weak topology for the functionals on VC defined by the elements ofWC and on WC defined by the elements of VC.

In order to avoid repeating the following definitions later on, we make them insomewhat greater generality than we need just now.

Definition 2.8. Let V and W be countable dimensional vector spaces overa real division ring D = R,C or H, with a nondegenerate bilinear pairing 〈·, ·〉 :V ×W → D. A chain or D–chain in V (resp. W ) is a set of D–subspaces totallyordered by inclusion. A generalized D–flag in V (resp. W ) is a D–chain suchthat each subspace has an immediate predecessor or an immediate successor in theinclusion ordering, and every nonzero vector of V (or W ) is caught between animmediate predecessor successor (IPS) pair. A generalized D–flag F in V (resp. ′Fin W ) is semiclosed if F ∈ F with F �= F⊥⊥ implies {F, F⊥⊥} is an IPS pair (resp.′F ∈ ′F with ′F �= ′F

⊥⊥implies {′F, ′F⊥⊥} is an IPS pair). ♦

Definition 2.9. Let D, V and W be as above. Generalized D–flags F in Vand ′F in W form a taut couple when (i) if F ∈ F then F⊥ is invariant by thegl–stabilizer of ′F and (ii) if ′F ∈ ′F then its annihilator ′F⊥ is invariant by thegl–stabilizer of F . ♦

In the so and sp cases one can use the associated bilinear form to identify VC

with WC and F with ′F . Then we speak of a generalized flag F in V as self–taut.If F is a self–taut generalized flag in V then Remark 2.3 and Lemma 2.4 of [7] showthat every F ∈ F is either isotropic or co–isotropic.


Theorem 2.10. The self–normalizing parabolic subalgebras of the Lie algebrassl(V,W ) and gl(V,W ) are the normalizers of taut couples of semiclosed generalizedflags in V and W , and this is a one to one correspondence. The self–normalizingparabolic subalgebras of sp(V ) are the normalizers of self–taut semiclosed generalizedflags in V , and this too is a one to one correspondence.

Theorem 2.11. The self–normalizing parabolic subalgebras of so(V ) are thenormalizers of self–taut semiclosed generalized flags F in V , and there are twopossibilities:

(1) the flag F is uniquely determined by the parabolic, or(2) there are exactly three self–taut generalized flags with the same stabilizer

as F .The latter case occurs precisely when there exists an isotropic subspace L ∈ F withdimC L⊥/L = 2. The three flags with the same stabilizer are then

{F ∈ F | F ⊂ L or L⊥ ⊂ F}{F ∈ F | F ⊂ L or L⊥ ⊂ F} ∪M1

{F ∈ F | F ⊂ L or L⊥ ⊂ F} ∪M2

where M1 and M2 are the two maximal isotropic subspaces containing L.

If p is a (real or complex) subalgebra of gC and q is a quotient algebra isomorphicto gl(∞;C), say with quotient map f : p → q, then we refer to the compositiontrace ◦ f : p → C as an infinite trace on gC. If {fi} is a finite set of infinite traceson gC and {ci} are complex numbers, then we refer to the condition

∑cifi = 0 as

an infinite trace condition on p.

Theorem 2.12. The parabolic subalgebras p in gC are the algebras obtainedfrom self normalizing parabolics p by imposing infinite trace conditions.

As a general principle one tries to be explicit by constructing representationsthat are as close to irreducible as feasible. For this reason we will be construct-ing principal series representations by inducing from parabolic subgroups that areminimal among the self–normalizing parabolic subgroups.

Now we discuss the structure of parabolic subalgebras of real forms of theclassical sl(∞,C), so(∞,C), sp(∞,C) and gl(∞,C). In this section gC will alwaysbe one of them and GC will be the corresponding connected complex Lie group.Also, g will be a real form of gC, and G will be the corresponding connected realsubgroup of GC.

Definition 2.13. Let g be a real form of gC. Then a subalgebra p ⊂ g is aparabolic subalgebra if its complexification pC is a parabolic subalgebra of gC. ♦

When g has two inequivalent defining representations, in other words when

g = sl(∞;R), gl(∞;R), su(∗,∞), u(∗,∞), or sl(∞;H)

we denote them by V and W , and when g has only one defining representation, inother words when

g = so(∗,∞), sp(∗,∞), sp(∞;R), or so∗(2∞) as quaternion matrices,

we denote it by V . The commuting algebra of g on V is a real division algebra D.The main result of [7] is

264 JOSEPH A. WOLF

Theorem 2.14. Suppose that g has two inequivalent defining representations.Then a subalgebra of g (resp. subgroup of G) is parabolic if and only if it is defined byinfinite trace conditions (resp. infinite determinant conditions) on the g–stabilizer(resp. G–stabilizer) of a taut couple of generalized D–flags F in V and ′F in W .

Suppose that g has only one defining representation. A subalgebra of g (resp.subgroup of G) is parabolic if and only if it is defined by infinite trace conditions(resp. infinite determinant conditions) on the g–stabilizer (resp. G–stabilizer) of aself–taut generalized D–flag F in V .

2C. Levi Components and Chevalley Decompositions. Now we turn toLevi components of complex parabolic subalgebras, recalling results from [8], [9],[5], [10], [6] and [25]. We start with the definition.

Definition 2.15. Let pC be a locally finite Lie algebra and rC its locally solvableradical. A subalgebra lC ⊂ pC is a Levi component if [pC, pC] is the semidirect sum(rC ∩ [pC, pC]) � lC. ♦

Every finitary Lie algebra has a Levi component. Evidently, Levi componentsare maximal semisimple subalgebras, but the converse fails for finitary Lie algebras.In any case, parabolic subalgebras of our classical Lie algebras gC have maximalsemisimple subalgebras, and those are their Levi components.

Definition 2.16. Let XC ⊂ VC and YC ⊂ WC be paired subspaces, isotropicin the orthogonal and symplectic cases. The subalgebras

gl(XC, YC) ⊂ gl(VC,WC) and sl(XC, YC) ⊂ sl(VC,WC),

Λgl(XC, YC) ⊂ Λgl(VC, VC) and Sgl(XC, YC) ⊂ Sgl(VC, VC)

are called standard. ♦

Proposition 2.17. A subalgebra lC ⊂ gC is the Levi component of a parabolicsubalgebra of gC if and only if it is the direct sum of standard special linear subal-gebras and at most one subalgebra Λgl(XC, YC) in the orthogonal case, at most onesubalgebra Sgl(XC, YC) in the symplectic case.

The occurrence of “at most one subalgebra” in Proposition 2.17 is analogousto the finite dimensional case, where it is seen by deleting some simple root nodesfrom a Dynkin diagram.

Let pC be the parabolic subalgebra of sl(VC,WC) or gl(VC,WC) defined by thetaut couple (F , ′F) of semiclosed generalized flags. Denote

(2.18)J = {(F ′, F ′′) IPS pair in F | F ′ = (F ′)⊥⊥ and dimF ′′/F ′ > 1},′J = {(′F ′, ′F ′′) IPS pair in ′F | ′F ′

= (′F ′)⊥⊥, dim ′F ′′/′F ′ > 1}.Since VC ×WC → C is nondegenerate the sets J and ′J are in one to one corre-spondence by: (F ′′/F ′)× (′F ′′/′F ′)→ C is nondegenerate. We use this to identifyJ with J ′, and we write (F ′

j , F′′j ) and (′F ′

j ,′F ′′

j ) treating J as an index set.

Theorem 2.19. Let pC be the parabolic subalgebra of sl(VC,WC) or gl(VC,WC)defined by the taut couple F and ′F of semiclosed generalized flags. For each j ∈ Jchoose a subspace Xj,C ⊂ VC and a subspace Yj,C ⊂ WC such that F ′′

j = Xj,C + F ′j

and ′F ′′j = Yj,C + ′Fj

′ Then⊕

j∈J sl(Xj,C, Yj,C) is a Levi component of pC. The

inclusion relations of F and ′F induce a total order on J .


Conversely, if lC is a Levi component of pC then there exist subspaces Xj,C ⊂ VC

and Yj,C ⊂WC such that l =⊕

j∈J sl(Xj,C, Yj,C).

Now the idea of finite matrices with blocks down the diagonal suggests theconstruction of pC from the totally ordered set J and the Lie algebra direct sumlC =

⊕j∈J sl(Xj,C, Yj,C) of standard special linear algebras. We outline the idea

of the construction; see [6]. First, 〈Xj,C, Yj′,C〉 = 0 for j �= j′ because the lj =sl(Xj,C, Yj,C) commute with each other. Define Uj,C := ((

⊕k�j Xk,C)

⊥ ⊕ Yj,C)⊥.

Then one proves Uj,C = ((Uj,C⊕Xj,C)⊥⊕ Yj,C)

⊥. From that, one shows that thereis a unique semiclosed generalized flag Fmin in VC with the same stabilizer as theset {Uj,C, Uj,C ⊕ Xj,C | j ∈ J}. One constructs similar subspaces ′Uj,C ⊂ WC andshows that there is a unique semiclosed generalized flag ′Fmin in WC with the samestabilizer as the set {′Uj,C,

′U j,C⊕Yj,C | j ∈ J}. In fact (Fmin,′Fmin) is the minimal

taut couple with IPS pairs Uj,C ⊂ (Uj,C ⊕ Xj,C) in Fmin and (Uj,C ⊕ Xj,C)⊥ ⊂

((Uj,C⊕Xj,C)⊥⊕Yj,C) in

′Fmin for j ∈ J . If (Fmax,′Fmax) is maximal among the

taut couples of semiclosed generalized flags with IPS pairs Uj,C ⊂ (Uj,C ⊕Xj,C) inFmax and (Uj,C⊕Xj,C)

⊥ ⊂ ((Uj,C⊕Xj,C)⊥⊕Yj,C) in

′Fmax then the correspondingparabolic pC has Levi component lC.

The situation is essentially the same for Levi components of parabolic subalge-bras of gC = so(∞;C) or sp(∞;C), except that we modify the definition (2.18) ofJ to add the condition that F ′′ be isotropic, and we add the orientation aspect ofthe so case.

Theorem 2.20. Let pC be the parabolic subalgebra of gC = so(VC) or sp(VC),

defined by the self–taut semiclosed generalized flag F . Let F be the union of all

subspaces F ′′ in IPS pairs (F ′, F ′′) of F for which F ′′ is isotropic. Let ′F be theintersection of all subspaces F ′ in IPS pairs for which F ′ is closed (F ′ = (F ′)⊥⊥)and coisotropic. Then lC is a Levi component of pC if and only if there are isotropicsubspaces Xj,C, Yj,C in VC such that

F ′′j = F ′

j +Xj,C and ′F ′′j = ′Fj + Yj,C for every j ∈ J

and a subspace ZC in VC such that F = ZC+ ′F , where ZC = 0 in case gC = so(VC)

and dim F /′F � 2, such that

lC = sp(ZC)⊕⊕

j∈Jsl(Xj,C, Yj,C) if gC = sp(VC),

lC = so(ZC)⊕⊕

j∈Jsl(Xj,C, Yj,C) if gC = so(VC).

Further, the inclusion relations of F induce a total order on J which leads to aconstruction of pC from lC.

Next we describe the Chevalley decomposition for parabolic subalgebras, fol-lowing [5].

Let pC be a locally finite linear Lie algebra, in our case a subalgebra of gl(∞,C).Every element ξ ∈ pC has a Jordan canonical form, yielding a decomposition ξ =ξss + ξnil into semisimple and nilpotent parts. The algebra pC is splittable if itcontains the semisimple and the nilpotent parts of each of its elements. Note thatξss and ξnil are polynomials in ξ; this follows from the finite dimensional fact. Inparticular, if XC is any ξ–invariant subspace of VC then it is invariant under bothξss and ξnil.

266 JOSEPH A. WOLF

Conversely, parabolic subalgebras (and many others) of our classical Lie alge-bras gC are splittable.

The linear nilradical of a subalgebra pC ⊂ gC is the set pnil,C of all nilpotentelements of the locally solvable radical rC of pC. It is a locally nilpotent ideal in pC

and satisfies pnil,C ∩ [pC, pC] = rC ∩ [pC, pC].

If pC is splittable then it has a well defined maximal locally reductive subalgebrapred,C. This means that pred,C is an increasing union of finite dimensional reductiveLie algebras, each reductive in the next. In particular pred,C maps isomorphicallyunder the projection pC → pC/pnil,C. That gives a semidirect sum decompositionpC = pnil,C � pred,C analogous to the Chevalley decomposition for finite dimensionalalgebraic Lie algebras. Also, here,

(2.21) pred,C = lC � tC and [pred,C, pred,C] = lC

where tC is a toral subalgebra and lC is the Levi component of pC. A glance atu(∞) or gl(∞;C) shows that the semidirect sum decomposition of pred,C need notbe direct.

Now we turn to Levi components and Chevalley decompositions for real para-bolic subalgebras in the real classical Lie algebras.

Let g be a real form of a classical locally finite complex simple Lie algebragC. Consider a real parabolic subalgebra p. It has form p = pC ∩ g where itscomplexification pC is parabolic in gC. Let τ denote complex conjugation of gCover g. Then the locally solvable radical rC of pC is τ–stable because rC + τ rC isa locally solvable ideal, so the locally solvable radical r of p is a real form of rC.Similarly the linear nilradical n of p is a real form of the linear nilradical nC of gC.

Let l be a maximal semisimple subalgebra of p. Its complexification lC is amaximal semisimple subalgebra, hence a Levi component, of pC. Thus [pC, pC] isthe semidirect sum (rC∩[pC, pC]) � lC. The elements of this formula all are τ–stable,so we have proved

Proposition 2.22. The Levi components of p are real forms of the Levi com-ponents of pC.

Remark 2.23. If gC is sl(VC,WC) or gl(VC,WC) as in Theorem 2.19 then wehave lC =

⊕j∈J sl(Xj,C, Yj,C). Initially the possibilities for the action of τ are

• τ preserves sl(Xj,C, Yj,C) with fixed point set sl(Xj , Yj) ∼= sl(∗;R),• τ preserves sl(Xj,C, Yj,C) with fixed point set sl(Xj , Yj) ∼= sl(∗;H),• τ preserves sl(Xj,C, Yj,C) with fixed point set su(X ′

j , X′′j )∼= su(∗, ∗) where

Xj = X ′j +X ′′

j , and• τ interchanges two summands sl(Xj,C, Yj,C) and sl(Xj′,C, Yj′,C) of lC, withfixed point set the diagonal (∼= sl(Xj,C, Yj,C)) of their direct sum.

If gC = so(VC) as in Theorem 2.20, lC can also have a summand so(ZC), or ifgC = sp(VC) it can also have a summand sp(ZC). Except when A3 = D3 occurs,these additional summands must be τ–stable, resulting in fixed point sets

• when gC = so(VC): so(ZC)τ is so(∗, ∗) or so∗(2∞),

• when gC = sp(VC): sp(ZC)τ is sp(∗, ∗) or sp(∗;R).

And A3 = D3 cases will not cause problems. ♦


3. Parabolics Defined by Closed Flags

A semiclosed generalized flag F = {Fα}α∈A is closed if all successors in thegeneralized flag are closed i.e. if F ′′

α = (F ′′α )

⊥⊥ for each immediate predecessorsuccessor (IPS) pair (F ′

α, F′′α ) in F . If a complex parabolic pC is defined by a taut

couple of closed generalized flags, or by a self dual closed generalized flag, thenwe say that pC is flag-closed. We say that a real parabolic subalgebra p ⊂ g isflag-closed if it is a real form of a flag-closed parabolic subalgebra pC ⊂ gC. Wesay “flag-closed” for parabolics in order to avoid confusion later with topologicalclosure. Theorems 5.6 and 6.6 in the paper [5] of E. Dan-Cohen and I. Penkov tellus

Proposition 3.1. Let p be a parabolic subalgebra of g and let n denote itslinear nilradical. If p is flag-closed, then p = n⊥ relative to the bilinear form(x, y) = trace (xy) on g.

Given G = lim−→Gn acting on V = lim−→Vn where the dn = dimVn are increasing

and finite, we have Cartan involutions θn of the groups Gn such that θn+1|Gn= θn,

and their limit θ = lim−→ θn (in other words θ|Gn= θn) is the corresponding Cartan

involution of G. It has fixed point set

K = Gθ = lim−→Kn

whereKn = Gθnn is a maximal compact subgroup of Gn. We refer toK as amaximal

lim-compact subgroup of G, and to k = gθ as a maximal lim-compact subalgebra ofg . Here, for brevity, we write θ instead of dθ for the Lie algebra automorphisminduced by θ.

Lemma 3.2. Any two maximal lim-compact subgroups of G are Aut(G)-conjugate.

Proof. Given two expressions lim−→Gn=G=lim−→G′n, corresponding to lim−→Vn =

V = lim−→Vn, we have an increasing function f : N → N such that V ′n ⊂ Vf(n).

Thus the two direct limit systems have a common refinement, and we may assumeV ′n = Vn and G′

n = Gn. It suffices now to show that the Cartan involutionsθ = lim−→ θn and θ′ = lim−→ θ′n are conjugate in Aut(G).

Recursively we assume that θn and θ′n are conjugate in Aut(Gn), say θ′n =γn · θn · γ−1

n for n > 0. This gives an isomorphism between the direct systems{(Gn, θn)} and {(Gn, θ

′n)}. As in [14, Appendix A] and [26] this results in an

automorphism of G that conjugates θ to θ′ in Aut(G) and sends K to K ′. �The Lie algebra g = k + s where k is the (+1)–eigenspace of θ and s is the

(−1)–eigenspace. We have just seen that any two choices of K are conjugate by anautomorphism of G, so we have considerable freedom in selecting k. Also as in thefinite dimensional case (and using the same proof), [k, k] ⊂ k, [k, s] ⊂ s and [s, s] ⊂ k.

Proposition 3.3. Let p be a flag-closed parabolic subalgebra of g , let θ be aCartan involution, and let g = k + s be the corresponding Cartan decomposition.Then g = k+ p.

Proof. Our bilinear form (x, y) �→ trace (xy) is nondegenerate on the θ–stablesubspace space k + p + θp of g. If k + p + θp �= g then g has nonzero elementsx ∈ (k+ p+ θp)⊥. Any such satisfies x ⊥ n so x ∈ p, and x ⊥ θn so x ∈ θp. Now xbelongs to the ( , )–nondegenerate subspace p∩ θp, contradicting x ∈ (k+ p+ θp)⊥.We have shown that g = k+ p+ θp .

268 JOSEPH A. WOLF

Let x ∈ g. We want to show x = 0 modulo k+p. Modulo k we express x = y+θzwhere y, z ∈ p . Then x − (y − z) = θz + z ∈ k, so x ∈ k modulo p. Now x = 0modulo k+ p. �

Lemma 3.4. If p is a flag-closed parabolic subalgebra of g , and pred,R is areductive part, then pred,R is stable under some Cartan involution θ of g, and forthat choice of θ we have p = (p ∩ k) + (p ∩ s) + n.

The global version of Proposition 3.3 is the main result of this section:

Theorem 3.5. Let P be a flag-closed parabolic subgroup of G and let K be amaximal lim–compact subgroup of G . Then G = KP .

The proof of Theorem 3.5 requires some riemannian geometry. We collect anumber of relevant semi–obvious (given the statement, the proof is obvious) results.The key point here is that the real analytic structure on G defined in [13] is theone for which exp : g → G restricts to a diffeomorphism of an open neighborhoodof 0 ∈ g onto an open neighborhood of 1 ∈ G , and that this induces a G–invariantanalytic structure on G/K .

Lemma 3.6. Define X = G/K, with the real analytic structure defined in [13]and the G–invariant riemannian metric defined by the positive definite Ad(K)–invariant bilinear form 〈ξ, η〉 = −trace (ξ · θη). Let x0 ∈ X denote the base point1K . Then X is a riemannian symmetric space, direct limit of the finite dimensionalriemannian symmetric spaces Xn = Gn(x0) = Gn/Kn, and each Xn is a totallygeodesic submanifold of X.

The proof of Theorem 3.5 will come down to an examination of the boundaryof P (x0) in X, and that will come down to an estimate based on

Lemma 3.7. Let π : g → s be the 〈·, ·〉–orthogonal projection, given by π(ξ) =12 (ξ − θξ). If ξ ∈ n then ||π(ξ)||2 = 1

2 ||ξ||2. If p is a flag-closed parabolic then π :

(p∩s)+n→ s is a linear isomorphism, and if ξ ∈ (p∩s)+n then ||π(ξ)||2 12 ||ξ||2.

Proof. Whether p is flag-closed or not, it is orthogonal to n relative to thetrace form, so if ξ ∈ n then 〈ξ, θξ〉 = −trace (ξ · θ2ξ) = −trace (ξ · ξ) = 0. Now||π(ξ)||2 = 1

4 (||ξ||2 + ||θξ||2) =12 ||ξ||2.

Now suppose that p is flag-closed. Then π : (p ∩ s) + n→ s is a linear isomor-phism by Lemma 3.4. The summands p∩s and n are orthogonal relative to the traceform so they are also orthogonal relative to 〈·, ·〉 because 〈ξ, η〉 = −trace (ξ · η) = 0for ξ ∈ n and η ∈ p ∩ s. Note that their π–images are also orthogonal because〈π(θξ), π(θη)〉 = 〈π(θξ), η〉 vanishes using the opposite parabolic θn+ pred,R. Now||π(ξ + η)||2 = ||π(ξ)||2 + ||η||2 1

2 ||ξ||2 + ||η||2 12 ||ξ + η||2. �

Given η ∈ sR, the riemannian distance dist(x0, exp(η)x0) from the base pointx0 to exp(η)x0 is ||η||. This can be seen directly, or it follows by choosing n suchthat η ∈ gn and looking in the symmetric space Xn. Now Lemma 3.7 implies

Lemma 3.8. If p is a flag-closed parabolic then exp((p ∩ s) + n)x0 = X. Inparticular, if r > 0 then the geodesic ball BX(r) = {x ∈ X | dist(x0, x) < r} iscontained in exp((p ∩ s) + n)x0.

Finally we are in a position to prove the main result of this section.


Proof of Theorem 3.5. Let η ∈ sR with ||η|| = 1 and consider the geodesicγ(t) = exp(tη)x0 in X. Here t is arc length and γ is defined on a maximal intervala < t < b where −∞ � a < 0 and 0 < b � ∞. If b < ∞ choose r > 0 with r < band ξ ∈ (p ∩ s) + n such that exp(ξ)x0 = γ(b − r). Then γ can be extended pastγ(b) inside the geodesic ball exp(ξ)BX(2r) of radius 2r with center exp(ξ)x0 . Thatdone, t �→ γ(t) is defined on the interval a < t < b + r. Thus b = ∞. Similarlya = −∞. We have proved that if p is a flag-closed parabolic and η ∈ s thenexp(tη)x0 ∈ P (x0) for every t ∈ R. In other words X = exp(s)x0 is equal to P (x0).That transitivity of P on X = G/K is equivalent to G = PK . Under x �→ x−1

that is the same as G = KP . �

4. Minimal Parabolic Subgroups

In this section we study the subgroups of G from which our principal seriesrepresentations are constructed.

4A. Structure. We specialize to the structure of minimal parabolic subgroupsof the classical real simple Lie groups G, extending structural results from [27].

Proposition 4.1. Let p be a parabolic subalgebra of g and l a Levi componentof p. If p is a minimal parabolic subalgebra then l is a direct sum of finite di-mensional compact algebras su(p), so(p) and sp(p), and their infinite dimensionallimits su(∞), so(∞) and sp(∞). If l is a direct sum of finite dimensional compactalgebras su(p), so(p) and sp(p) and their limits su(∞), so(∞) and sp(∞), then p

contains a minimal parabolic subalgebra of g with the same Levi component l.

Proof. Suppose that p is a minimal parabolic subalgebra of g. If a directsummand l′ of l has a proper parabolic subalgebra q, we replace l′ by q in l andp. In other words we refine the flag(s) that define p. The refined flag defines aparabolic q � p. This contradicts minimality. Thus no summand of l has a properparabolic subalgebra. Theorems 2.19 and 2.20 show that su(p), so(p) and sp(p),and their limits su(∞), so(∞) and sp(∞), are the only possibilities for the simplesummands of l.

Conversely suppose that the summands of l are su(p), so(p) and sp(p) or theirlimits su(∞), so(∞) and sp(∞). Let (F , ′F) or F be the flag(s) that define p. Inthe discussion between Theorems 2.19 and 2.20 we described a minimal taut couple(Fmin,

′Fmin) and a maximal taut couple (Fmax,′Fmax) (in the sl and gl cases) of

semiclosed generalized flags which define parabolics that have the same Levi com-ponent lC as pC. By construction (F , ′F) refines (Fmin,

′Fmin) and (Fmax,′Fmax)

refines (F , ′F). As (Fmin,′Fmin) is uniquely defined from (F , ′F) it is τ–stable.

Now the maximal τ–stable taut couple (F∗max,

′F∗max) of semiclosed generalized

flags defines a τ–stable parabolic qC with the same Levi component lC as pC, andq := qC ∩ g is a minimal parabolic subalgebra of g with Levi component l.

The argument is the same when gC is so or sp. �Proposition 4.1 says that the Levi components of the minimal parabolics are

countable sums of compact real forms, in the sense of [21], of complex Lie algebrasof types sl, so and sp. On the group level, every element of M is elliptic, andpred = l � t where t is toral, so every element of pred is semisimple. This iswhere we use minimality of the parabolic p. Thus pred ∩ gn is reductive in gm,R forevery m n. Consequently we have Cartan involutions θn of the groups Gn such

270 JOSEPH A. WOLF

that θn+1|Gn= θn and θn(M ∩ Gn) = M ∩ Gn. Now θ = lim−→ θn (in other words

θ|Gn= θn) is a Cartan involution of G whose fixed point set contains M . We have

just extended the argument of Lemma 3.2 to show that

Lemma 4.2. M is contained in a maximal lim-compact subgroup K of G.

We fix a Cartan involution θ corresponding to the group K of Lemma 4.2.

Lemma 4.3. Decompose pred = m + a where m = pred ∩ k and a = pred ∩ s.Then m and a are ideals in pred with a commutative (in fact diagonalizable overR). In particular pred = m⊕ a, direct sum of ideals.

Proof. Since l = [pred, pred] we compute [m, a] ⊂ l ∩ a = 0. In particular[[a, a], a] = 0. So [a, a] is a commutative ideal in the semisimple algebra l, in otherwords a is commutative. �

The main result of this subsection is the following generalization of the standarddecomposition of a finite dimensional real parabolic. We have formulated it toemphasize the parallel with the finite dimensional case. However some details ofthe construction are rather different; see Proposition 4.14 and the discussion leadingup to it.

Theorem 4.4. The minimal parabolic subalgebra p of g decomposes as p =m+a+n = n � (m⊕a), where a is commutative, the Levi component l is an ideal inm , and n is the linear nilradical pnil. On the group level, P = MAN = N�(M×A)where N = exp(n) is the linear unipotent radical of P , A = exp(a) is diagonalizableover R and isomorphic to a vector group, and M = P ∩K is limit–compact withLie algebra m .

Proof. The algebra level statements come out of Lemma 4.3 and the semidi-rect sum decomposition p = pnil � pred.

For the group level statements, we need only check that K meets every topolog-ical component of P . Even though P ∩Gn need not be parabolic in Gn, the groupP ∩ θP ∩Gn is reductive in Gn and θn–stable, so Kn meets each of its components.Now K meets every component of P ∩θP . The linear unipotent radical of P has Liealgebra n and thus must be equal to exp(n), so it does not effect components. Thusevery component of Pred is represented by an element of K∩P ∩θP = K∩P = M .That derives P = MAN = N � (M ×A) from p = m+ a+ n = n � (m⊕ a). �

4B. Construction. Given a subalgebra l ⊂ g that is the Levi component of aminimal parabolic subalgebra p , we will extend the notion of standard of Definition2.16 from simple ideals of l to minimal parabolics and their reductive parts. Theconstruction of the standard flag-closed minimal parabolic p† = m + a† + n† withthe same Levi component as p = m+a+n will tell us that K is transitive on G/P †,and this will play an important role in construction of Harish–Chandra modules ofprincipal series representations.

We carry out the construction in detail for the cases where g is defined by ahermitian form f : VF × VF → F , where F is R, C or H. The idea is the same forthe other cases. See Proposition 4.14 below.

Write VF for V as a real, complex or quaternionic vector space, as appropri-ate, and similarly for WF. We use f for an F–conjugate–linear identification of


VF and WF. We are dealing with the Levi component l =⊕

j∈J lj,R of a min-imal self–normalizing parabolic p, where the lj,R are simple and standard in thesense of Definition 2.16. Let X levi

Fdenote the sum of the corresponding subspaces

(Xj)F ⊂ VF and Y leviF

the analogous sum of the (Yj)F ⊂ WF. Then XF and YF arenondegenerately paired. Of course they may be small, even zero. In any case,

(4.5)VF = X levi

F ⊕ (Y leviF )⊥ ,WF = Y levi

F ⊕ (X leviF )⊥, and

(X leviF )⊥ and (Y levi

F )⊥ are nondegenerately paired.

These direct sum decompositions (4.5) now become

(4.6) VF = X leviF ⊕ (X levi

F )⊥ and f is nondegenerate on each summand.

Let X ′ and X ′′ be paired maximal isotropic subspaces of (X leviF

)⊥. Then

(4.7) VF = X leviF ⊕ (X ′

F ⊕X ′′F )⊕QF where QF := (X levi

F ⊕ (X ′F ⊕X ′′

F ))⊥.

The subalgebra {ξ ∈ g | ξ(XF ⊕ QF) = 0} of g has maximal toral subalgebrascontained in s, in which every element has all eigenvalues real. The one we will useis

(4.8)

a† =⊕

�∈Cgl(x′

�R, x′′�R) where

{x′� | � ∈ C} is a basis of X ′

F and

{x′′� | � ∈ C} is the dual basis of X ′′

F .

It depends on the choice of basis of X ′F. Note that a† is abelian, in fact diagonal

over R as defined.

As noted in another argument, in the discussion between Theorems 2.19 and2.20 we described a minimal taut couple (Fmin,

′Fmin) and a maximal taut couple(Fmax,

′Fmax) (in the sl and gl cases) of semiclosed generalized flags which defineparabolics that have the same Levi component lC as pC. That argument of [6] doesnot require simplicity of the lj . It works with {lj}j∈J∪{gl(x′

�R, x′′�R)}�∈C and a total

ordering on J† := J∪C that restricts to the given total ordering on J . Any suchinterpolation of the index C of (4.8) into the totally ordered index set J of X levi

F=⊕

j∈J (Xj)F (and usually there will be infinitely many) gives a self–taut semiclosed

generalized flag F† and defines a minimal self–normalizing parabolic subalgebra p†

of g with the same Levi component as p. The decompositions corresponding to(4.5), (4.6) and (4.7) are given by

(4.9) X†F=⊕

d∈J†(Xd)F = X levi

F ⊕ (X ′F ⊕X ′′

F ) and Q†F= QF.

In the discussion just above, p† is the stabilizer of the flag F†. The nilradical

of p† is defined by ξXd ⊂⊕

d′<d Xd′ and ξQ†F= 0.

In addition, the subalgebra {ξ ∈ p | ξ(X leviF

⊕ (X ′F⊕X ′′

F)) = 0} has a maximal

toral subalgebra t′ in which every eigenvalue is pure imaginary, because f is definiteon QF. It is unique because it has derived algebra zero and is given by the actionof the p–stabilizer of QF on the definite subspace QF. This uniqueness tell us thatt′ is the same for p and p†.

Let t′′ denote the maximal toral subalgebra in {ξ ∈ p | ξ(XF ⊕ QF) = 0}. Itstabilizes each Span(x′

�, x′′� ) in (4.8) and centralizes a†, so it vanishes if F �= C. The

272 JOSEPH A. WOLF

p† analog of t′′ is 0 because X†F⊕QF = VF. In any case we have

(4.10) t = t† := t

′ ⊕ t′′ .

For each j ∈ J we define an algebra that contains lj,R and acts on (Xj)F by: if

lj,R = su(∗) then lj,R = u(∗) (acting on (Xj)C); otherwise lj,R = lj,R. Define

(4.11) l =⊕j∈J

lj,R and m† = l+ t .

Then, by construction, m† = m. Thus p† satisfies

(4.12) p† := m+ a

† + n† = n

†� (m⊕ a

†).

Let z denote the centralizer of m⊕ a in g and let z† denote the centralizer of m⊕ a†

in g. We claim

(4.13) m+ a = l+ z and m+ a† = l+ z† .

For by construction m ⊕ a = l + t + a ⊂ l + z. Conversely if ξ ∈ z it preserveseach Xj,F, each joint eigenspace of a on X ′

F⊕X ′′

F, and each joint eigenspace of t, so

ξ ⊂ l+ t+ a. Thus m+ a = l+ z. The same argument shows that m+ a† = l+ z†.

If a is diagonalizable as in the definition (4.8) of a†, in other words if it is asum of standard gl(1;R)’s, then we could choose a† = a, hence could construct F†

equal to F , resulting in p = p†. In summary:

Proposition 4.14. Let g be defined by a hermitian form and let p be a minimalself–normalizing parabolic subalgebra. In the notation above, the standard parabolicp† is a minimal self–normalizing parabolic subalgebra of g with m† = m. In partic-ular p† and p have the same Levi component. Further we can take p† = p if andonly if a is the sum of commuting standard gl(1;R)’s.

Similar arguments give the construction behind Proposition 4.14 for the otherreal simple direct limit Lie algebras.

Note also from the construction of p† we have

Proposition 4.15. The standard parabolic p† constructed above, is flag-closed.In particular, by Theorem 3.5, the maximal lim-compact subgroup K of G is tran-sitive on G/P † , and so G/P † ∼= K/M† as real analytic manifolds.

P and P † are minimal self normalizing parabolic subgroups of G. We willdiscuss representations of P and P †, and the induced representations of G. Thelatter are the principal series representations of G associated to p and p†, or moreprecisely to the pair (l, J) where l is the Levi component and J is the ordered indexset for the simple summands of l.

5. Amenable Induction

In this section we construct induced representations IndGH(τ ) where G is a

(possibly infinite dimensional) topological group, H is a closed amenable subgroup,and τ is a unitary representation of H. This requires construction of right–H–invariant means on G, in other words means on G/H. Those means allow us toconstruct induced representations without local compactness or invariant measures.The principal application is, of course, to the case whereG is a finitary real reductiveLie group and the subgroup is a self–normalizing minimal parabolic subgroup.


The induced representation construction goes through without change when τ isonly required to be a Banach representation, and goes through with minor changeswhen τ is a continuous representation on a complete locally convex topologicalvector space. It is conceivable that the Banach space representation case will beuseful in studying some analog of admissible representation appropriate for directlimit real reductive Lie groups.

5A. Amenable Groups. We consider a topological group G which is notassumed to be locally compact, and a closed subgroup H of G. We follow D.Beltita [3, Section 3] for amenability on H. Consider the commutative C∗ algebra

L∞(G/H) = {f : G/H → C continuous | supx∈G/H |f(x)| <∞}.

It has pointwise multiplication, norm ||f || = supx∈G/H |f(x)| and unit given by

1(x) = 1. We denote the usual left and right actions of G on L∞(G) by (�(g)f)(k) =f(g−1k) and (r(g)f)(k) = f(kg). We often identify L∞(G/H) with the closedsubalgebra of L∞(G) consisting of r(H)–invariant functions.

The space of right uniformly continuous bounded functions on G/H is

(5.1) RUCb(G/H) = {f ∈ L∞(G/H) | x �→ �(x)f continuous G→ L∞(G/H)}.In other words,

(5.2) if ε > 0, ∃ nbhd U of 1 in G s.t. |f(ux)− f(x)| < ε for x ∈ G/H, u ∈ U.

Similarly, the space LUCb(G) of left uniformly continuous bounded functions on Gis {f ∈ L∞(G) | x �→ r(x)f is a continuous map G→ L∞(G)}.

Lemma 5.3. The left action of G on RUCb(G/H) is a continuous representa-tion.

Proof. (5.1) and (5.2) give ||�(u)f − f ′||∞ � ||�(u)f − f ||∞ + ||f − f ′||∞. �

Example 5.4. Let ϕ be a unitary representation of G. This means a weaklycontinuous homomorphism into the unitary operators on a separable Hilbert spaceEϕ . If u, v ∈ Eϕ the coefficient function fu,v : G→ C is fu,v(x) = 〈u, ϕ(x)v〉. Letε > 0 and choose a neighborhood B of 1 in G such that ||u|| · ||v − ϕ(y)v|| < ε fory ∈ B. Then |fu,v(x)− fu,v(xy)| < ε for all x ∈ G and y ∈ B, so fu,v ∈ LUCb(G).Similarly, choose a neighborhood B′ of 1 such that ||u−ϕ(y)u|| · ||v|| < ε for y ∈ B′.Then |fu,v(x)− fu,v(y

−1x)| < ε for all x ∈ G and y ∈ B′, so fu,v ∈ RUCb(G). ♦

A mean on G/H is a linear functional μ : RUCb(G/H)→ C such that

(5.5)(i) μ(1) = 1 and

(ii) if f(x) 0 for all x ∈ G/H then μ(f) 0.

Any left invariant mean μ on G/H is a continuous functional on RUCb(G/H)and satisfies ||μ|| = 1.

The topological group H is amenable if it has a left invariant mean, or equiva-lently (using h �→ h−1) if it has a right invariant mean.

Proposition 5.6. (See (Beltita [3, Example 3.4]) Let {Hα} be a strict directsystem of amenable topological groups. Let H be a topological group in which thealgebraic direct limit lim−→Hα is dense. Then H is amenable.

274 JOSEPH A. WOLF

When we specialize this to our Lie group setting it will be useful to denote

(5.7) M(G/H) : all means on G/H with the action (�(g)μ)(f) = μ(�(g−1)f).

Let τ ∈ H with representation space Eτ and let Eτ → G/H be the associatedG–homogeneous hermitian vector bundle. Then we have the space

(5.8) RUCb(G/H;Eτ ) : right uniformly cont. bounded sections of Eτ → G/H.

If ω ∈ RUCb(G/H;Eτ ) then the pointwise norm function ||ω|| : gH �→ ||ω(gH)||belongs to RUCb(G/H). G acts on RUCb(G/H;Eτ ) by �(g)f(k) = f(g−1k). Everymean μ ∈M(G/H) defines a seminorm νμ on RUCb(G/H;Eτ ), by

(5.9) νμ(ω) = μ(||ω||).

Lemma 5.10. Let G be a topological group and H a closed amenable subgroup. If0 �= f ∈ RUCb(G/H;Eτ ) then there exists μ ∈M =M(G/H) such that νμ(f) > 0.

Note: Lemma 5.10 and its proof sharpen my original treatment. They weredeveloped in discussions with G. Olafsson. See [15].

Proof. Let ω ∈ RUCb(G/H;Eτ ) be annihilated by all the seminorms νμ,μ ∈M. Suppose that ω is not identically zero and choose x ∈ G/H with ω(x) �= 0.We can scale and assume ||ω(x)|| = 1. Evaluation δx(ϕ) = ϕ(x) is a mean on Gand δx(||ω||) = 1. Now the compact convex set S = {σ ∈ M(G) | σ(||f ||) = 1}(weak∗ topology) is nonempty. Since H is amenable it has a fixed point μω on S.Now μω is a mean on G/H and the seminorm νμω

(ω) = 1. �

A similar argument gives the following, which is well known in the locallycompact case and probably known in general:

Lemma 5.11. If H1 is a closed normal amenable subgroup of H and H/H1 isamenable then H is amenable.

Proof. Let μ be a left invariant mean on H1 and ν a left invariant mean onH/H1. Given f ∈ RUCb(H) and h ∈ H define fh = (�(h−1)(f))|H1

∈ RUCb(H1),so fh(y) = f(hy) for y ∈ H1. If y′ ∈ yH1 then μ(fy′) = μ(�(y′−1y)fy) = μ(fy),so we have gf ∈ RUCb(H/H1) defined by gf (hH1) = μ(fh). That defines a meanβ on G by β(f) = ν(gf ), and β is left invariant because β(�(a)f) = ν(g�(a)f ) =

ν(�(a−1)g�(a)f ) = β(f). �

Theorem 5.12. The maximal lim–compact subgroups K = lim−→Kn of G areamenable. Further, the minimal parabolic subgroups of G are amenable. Finally, if

P is a minimal parabolic subgroup of G and τ ∈ P then M(G/P ) separates pointson RUCb(G/P ;Eτ ).

Proof. By constructionK is a direct limit of compact (thus amenable) groups,so it is amenable by Proposition 5.6. In Theorem 4.4 we saw the decompositionP = MAN of the minimal parabolic subgroup. M is amenable because it is aclosed subgroup of the amenable group K. AN is a direct limit of finite dimensionalconnected solvable Lie groups, hence is amenable. And now the semidirect productP = (AN) � M is amenable by Lemma 5.11. Finally, Lemma 5.10 says thatM(G/P ) separates points on RUCb(G/P ;Eτ ). �


5B. Induced Representations: General Construction. Here is the gen-eral construction for amenable induction. Let G be a topological group and H a

closed amenable subgroup. We have seen that a unitary representation τ ∈ H, sayon Eτ , defines an G–homogeneous Hilbert space bundle Eτ → G/H. Using the setM(G/H) of right H–invariant means on G, we are going to apply Theorem 5.12

to define an induced representations IndGH(τ ) of G. The representation space will

be a complete locally convex topological vector space.

Consider a section ω ∈ RUCb(G/H;Eτ ), the bounded, right uniformly con-tinuous sections of Eτ → G/H. We mentioned in the discussion leading to (5.9)that its pointwise norm is a function gH �→ ||ω(gH)|| on G/H, and that eachright H–invariant mean μ ∈ M(G/H) defines a seminorm νμ : ω �→ μ(||ω||) onRUCb(G/H;Eτ ).

Given any left G–invariant subset M′ of M(G/H) we define

(5.13) JM′(G/H;Eτ ) = {ω ∈ RUCb(G/H;Eτ ) | νμ(ω) = 0 for all μ ∈M′}.

The seminorms νμ, μ ∈ M′, descend to RUCb(G/H;Eτ )/JM′(G/H;Eτ ). Thatfamily of seminorms defines the complete locally convex topological vector space

(5.14) ΓM′(G/H;Eτ ) : completion of RUCb(G/H;Eτ )JM′ (G/H;Eτ )

relative to {νμ | μ ∈M′}.

Lemma 5.3 now gives us

Proposition 5.15. The natural action of G on the complete locally convextopological vector space ΓM′(G/H;Eτ ) is a continuous representation of G.

Lemma 5.11 says that JM(G/H)(G/H;Eτ ) = 0, and writing

Γ(G/H;Eτ ) := ΓM(G/H)(G/H;Eτ ),

we have the special case

Corollary 5.16. The natural action of G on the complete locally convex topo-logical vector space Γ(G/H;Eτ ) is a continuous representation of G.

5C. Principal Series Representations. We specialize the construction ofProposition 5.15 to our setting where G is a real Lie group with complexificationGL(∞;C), SL(∞;C), SO(∞;C) or Sp(∞;C), and where P is a minimal self–normalizing parabolic subgroup. Theorem 5.12 says that M(G/P ) separates ele-ments of RUCb(G/P ;Eτ ). Given a unitary representation τ of P we then have

• the G–homogeneous hermitian vector bundle Eτ → G/P ,• the seminorms νμ, μ ∈M(G/P ;Eτ ), on RUCb(G/P ;Eτ ), and• the completion Γ(G/P ;Eτ ) of RUCb(G/P ;Eτ ) relative to that collectionof seminorms, which is a complete locally convex topological vector space.

Definition 5.17. The representation πτ of G on Γ(X;Eτ ) is amenably induced

from (P, τ ) to G. We denote it IndGP (τ ). The family of all such representations

forms the general principal series of representations of G. ♦

Proposition 5.18. If the minimal self–normalizing parabolic P is flag-closed,and τ is a unitary representation of P , then IndG

P (τ )|K = IndKM (τ |M ).

276 JOSEPH A. WOLF

Proof. Since P is flag closed, Theorem 3.5 says that K is transitive on X =G/P , so X = K/M as well. Thus Eτ → X can be viewed as the K–homogeneousHilbert space bundle Eτ |K → X defined by τ |K . Evidently RUCb(X;Eτ ) =RUCb(X;Eτ |K ). Now we have a K–equivariant identification M(K/M ;Eτ |K ) =M(G/P ;Eτ ), resulting in a K–equivariant isomorphism of Γ(K/M ;Eτ |K ) onto

Γ(G/P ;Eτ ), which in turn gives a topological equivalence of IndKM (τ |M ) with

IndGP (τ )|K . �

In the current state of the art, this construction provides more questions thananswers. Some of the obvious questions are

1. When does Γ(X;Eτ ) have a G–invariant Frechet space structure? Whenit exists, is it nuclear?

2. When does Γ(X;Eτ ) have a G–invariant Hilbert space structure? In other

words, when is IndGP (τ ) unitarizable?

3. What is the precise K–spectrum of πτ?4. When is the space of smooth vectors dense in Γ(X;Eτ )? In other words,

when (or to what extent) does the universal enveloping algebra U(g) act?5. If τ |M is a factor representation of type II1, and P is flag closed, does

the character of τ |M lead to an analog of character for IndGP (τ ), or for

IndKM (τ |M )?

The answers to (1.) and (2.) are well known in the finite dimensional case. Theyare also settled ([24]) when G = lim−→Gn restricts to P = lim−→Pn with Pn minimal

parabolic in Gn. However that is a very special situation. The answer to (3.) isonly known in special finite dimensional situations. Again, (4.) is classical in thefinite dimensional case, and also clear in the cases studied in [24], but in general oneexpects that the answer will depend on better understanding of the possibilities forτ and the structure ofM(G/P ). For that we append to this paper a short discussionof unitary representations of self normalizing minimal parabolic subgroups.

Appendix: Unitary Representations of Minimal Parabolics.

In order to describe the unitary representations τ of P that are basic to theconstruction of the principal series in Section 5, we must first choose a class ofrepresentations. The best choice is not clear, so we indicate some of the simplestchoices.

Reductions. First, we limit complications by looking only at unitary represen-tations τ of P = MAN that annihilate the linear nilradical N . Since the structureof N is not explicit, especially since we do not necessarily have a restricted rootdecomposition of n, the unitary representation theory of N and the correspond-ing extension with representations of MA present serious difficulties, which we willavoid. This is in accord with the finite dimensional setting.

Second, we limit surprises by assuming that τ |A is a unitary character. Thistoo is in accord with the finite dimensional setting. Thus we are looking at repre-sentations of the form τ (man)v = eiλ(log a)τ (m)v, v ∈ Eτ , where λ ∈ a∗ is a linearfunctional on a and τ |M is a unitary representation of M .

We know the structure of l from Proposition 4.1, and the construction of m fromthat of l combined with (2.21) and Lemma 4.3. Thus we are then in a position to


take advantage of known results on unitary representations of lim-compact groupsto obtain the factor representations of the identity component M0. Lemma 6.1below, shows how the unitary representations ofM are constructed from the unitaryrepresentations of M0.

Lemma 6.1. M = M0 × (AC ∩K) and every element of AC ∩K has square 1.In other words, M is the direct product of its identity component with a direct limitof elementary abelian 2–groups.

Proof. The parabolic PC is self–normalizing, and self-normalizing complexparabolics are connected. Thus MC and AC are connected. As M ⊂ K we haveMC = (M ∩Gu) exp(im) and M ∩Gu = M ∩G where Gu is the lim-compact realform of GC with Lie algebra gu = k + is. Now MC ∩ G is connected and equalto exp(mC ∩ g) = exp(m). First, this tells us that M0 = MC ∩ G. Second, itshows that MA = MCAC ∩ G = (MC ∩ G)(AC ∩ G). From the finite dimensionalcase, the topological components of M are given by AC ∩K. If x ∈ AC ∩K thenx = θx = x−1, so AC ∩K is a direct limit of elementary abelian 2–groups. �

Third, we further limit surprises by assuming that τ |AC∩K is a unitary characterχ. In other words, there is a unitary character eiλ⊗χ on (AC∩G) = A× (AC∩K)such that τ (m0maan)v = eiλ(log a)χ(ma)τ (m0)v for m0 ∈M0, ma ∈ AC∩K, a ∈ Aand n ∈ N .

Using (2.21) and Lemma 4.3 we have m = l � t and [m,m] = l where t is toral.So M0 is the semidirect product L�T where T is a direct limit of finite dimensional

torus groups. Let L be the group obtained from L by replacing each special unitaryfactor SU(∗) by the slightly larger unitary group U(∗). This absorbs a factor fromT and the result is a direct product decomposition

(6.2) M0 = L× T where T is toral.

Our fourth restriction, similar to the second and third, is that τ |˜T be a unitary

character.

In summary, we are looking at unitary representations τ of P whose kernelcontains N and which restrict to unitary characters on the commutative groups A,

AC ∩K and T . Those unitary characters, together with the unitary representationτ |

˜L, determine τ .

Representations. We discuss some possibilities for an appropriate class C(L)of representations of L. The standard group L is a product of standard groupsU(∗), and possibly one factor SO(∗) or Sp(∗). The representation theory of thefinite dimensional groups U(n), SO(n) and Sp(n) is completely understood, so weneed only consider the cases of U(∞), SO(∞) and Sp(∞). We will indicate somepossibilities for C(U(∞)). The situation is essentially the same for SO(∞) andSp(∞).

Tensor Representations of U(∞). In the classical setting, the symmetric group

Sn permutes factors of⊗n

(Cp). The resulting representation of U(p)×Sn specifiesrepresentations of U(p) on the various irreducible summands for that action of Sn.These summands occur with multiplicity 1. See Weyl’s book [23]. Segal [17],Kirillov [11], and Stratila & Voiculescu [18] developed and proved an analog ofthis for U(∞). It uses the infinite symmetric group S∞ := lim−→Sn and the infinite

tensor product⊗n(C∞) in place of the finite ones. These “tensor representations”

278 JOSEPH A. WOLF

are factor representations of type II∞, but they do not extend by continuity to theclass of unitary operators of the form identity + compact. See [19, Section 2] fora treatment of this topic. Because of this limitation one should also consider otherclasses of factor representations of U(∞).

Type II1 Representations of U(∞). If π is a continuous unitary finite factor

representation of U(∞), then it has a well defined character χπ(x) = trace π(x),the normalized trace. Voiculescu [22] worked out the parameter space for thesefinite factor representations. It consists of all bilateral sequences {cn}−∞<n<∞such that (i) det(cmi+j−i)1�i,j�N 0 for mi ∈ Z and N 0 and (ii)

∑cn = 1.

The character corresponding to {cn} and π is χπ(x) =∏

i p(zi) where {zi} is themultiset of eigenvalues of x and p(z) =

∑cnz

n. Here π extends to the group ofall unitary operators X on the Hilbert space completion of C∞ such that X − 1is of trace class. See [19, Section 3] for a more detailed summary. This is a veryconvenient choice of class CU(∞), and it is closely tied to the Olshanskii–Vershiknotion (see [16]) of tame representation.

Other Factor Representations of U(∞). Let H be the Hilbert space completion

of lim−→Hn where Hn is the natural representation space of U(n). Fix a bounded

hermitian operator B on H with 0 � B � I. Then

ψB : U(∞)→ C , defined by ψB(x) = det((1−B) +Bx)

is a continuous function of positive type on U(∞). Let πB denote the associatedcyclic representation of U(∞). Then ([20, Theorem 3.1], or see [19, Theorem 7.2]),

(1) ψB is of type I if and only if B(I −B) is of trace class. In that case πB

is a direct sum of irreducible representations.(2) ψB is factorial and type I if and only if B is a projection. In that case

πB is irreducible.(3) ψB is factorial but not of type I if and only if B(I − B) is not of trace

class. In that case(i) ψB is of type II1 if and only if B − tI is Hilbert–Schmidt where

0 < t < 1; then πB is a factor representation of type II1.(ii) ψB is of type II∞ if and only if (a) B(I −B)(B− tI)2 is trace class

where 0 < t < 1 and (b) the essential spectrum of B contains 0 or 1;then πB is a factor representation of type II∞.

(iii) ψB is of type III if and only if B(I − B)(B − tI)2 is not of traceclass whenever 0 < t < 1; then πB is a factor representation of typeIII.

Similar considerations hold for SU(∞), SO(∞) and Sp(∞).

In [28] we will examine the case where the inducing representation τ is a unitarycharacter on P . In the finite dimensional case that leads to a K–fixed vector,spherical functions on G and functions on the symmetric space G/K.

References

[1] A. A. Baranov, Finitary simple Lie algebras, J. Algebra 219 (1999), no. 1, 299–329, DOI10.1006/jabr.1999.7856. MR1707673 (2000f:17011)

[2] A. A. Baranov and H. Strade, Finitary Lie algebras, J. Algebra 254 (2002), no. 1, 173–211,DOI 10.1016/S0021-8693(02)00079-0. MR1927437 (2003g:17012)


[3] Daniel Beltita, Functional analytic background for a theory of infinite-dimensional reduc-tive Lie groups, Developments and trends in infinite-dimensional Lie theory, Progr. Math.,vol. 288, Birkhauser Boston Inc., Boston, MA, 2011, pp. 367–392, DOI 10.1007/978-0-8176-4741-4 11. MR2743769 (2012a:22033)

[4] Elizabeth Dan-Cohen, Borel subalgebras of root-reductive Lie algebras, J. Lie Theory 18(2008), no. 1, 215–241. MR2413961 (2009b:17054)

[5] Elizabeth Dan-Cohen and Ivan Penkov, Parabolic and Levi subalgebras of finitary Lie

algebras, Int. Math. Res. Not. IMRN 6 (2010), 1062–1101, DOI 10.1093/imrn/rnp169.MR2601065 (2011e:17037)

[6] Elizabeth Dan-Cohen and Ivan Penkov, Levi components of parabolic subalgebras of finitaryLie algebras, Representation theory and mathematical physics, Contemp. Math., vol. 557,Amer. Math. Soc., Providence, RI, 2011, pp. 129–149, DOI 10.1090/conm/557/11029.MR2848923

[7] Elizabeth Dan-Cohen, Ivan Penkov, and Joseph A. Wolf, Parabolic subgroups of real directlimit Lie groups, Groups, rings and group rings, Contemp. Math., vol. 499, Amer. Math. Soc.,Providence, RI, 2009, pp. 47–59, DOI 10.1090/conm/499/09790. MR2581925 (2011f:17037)

[8] Ivan Dimitrov and Ivan Penkov, Weight modules of direct limit Lie algebras, Internat. Math.Res. Notices 5 (1999), 223–249, DOI 10.1155/S1073792899000124. MR1675979 (2000a:17005)

[9] Ivan Dimitrov and Ivan Penkov, Borel subalgebras of gl(∞), Resenhas 6 (2004), no. 2-3,153–163. MR2215977 (2007a:17035)

[10] Ivan Dimitrov and Ivan Penkov, Locally semisimple and maximal subalgebras of the finitaryLie algebras gl(∞), sl(∞), so(∞), and sp(∞), J. Algebra 322 (2009), no. 6, 2069–2081, DOI10.1016/j.jalgebra.2009.06.011. MR2542831 (2010h:17010)

[11] A. A. Kirillov, Representations of the infinite-dimensional unitary group, Dokl. Akad. Nauk.SSSR 212 (1973), 288–290 (Russian). MR0340487 (49 #5239)

[12] George W. Mackey, On infinite-dimensional linear spaces, Trans. Amer. Math. Soc. 57(1945), 155–207. MR0012204 (6,274d)

[13] Loki Natarajan, Enriqueta Rodrıguez-Carrington, and Joseph A. Wolf, Differentiablestructure for direct limit groups, Lett. Math. Phys. 23 (1991), no. 2, 99–109, DOI10.1007/BF00703721. MR1148500 (92k:22035)

[14] Loki Natarajan, Enriqueta Rodrıguez-Carrington, and Joseph A. Wolf, Locally convex Liegroups, Nova J. Algebra Geom. 2 (1993), no. 1, 59–87. MR1254153 (94j:22022)

[15] G. Olafsson & J. A. Wolf, Separating vector bundle sections by invariant means. {arXiv1210.5494 (math.RT; math.DG, math.GR)}

[16] G. I. Ol′shanskiı, Unitary representations of infinite-dimensional pairs (G,K) and the for-malism of R. Howe, Representation of Lie groups and related topics, Adv. Stud. Contemp.Math., vol. 7, Gordon and Breach, New York, 1990, pp. 269–463. MR1104279 (92c:22043)

[17] I. E. Segal, The structure of a class of representations of the unitary group on a Hilbertspace, Proc. Amer. Math. Soc. 8 (1957), 197–203. MR0084122 (18,812f)

[18] Serban Stratila and Dan Voiculescu, Representations of AF-algebras and of the groupU(∞), Lecture Notes in Mathematics, Vol. 486, Springer-Verlag, Berlin, 1975. MR0458188(56 #16391)

[19] Serban Stratila and Dan Voiculescu, A survey on representations of the unitary group U(∞),Spectral theory (Warsaw, 1977), Banach Center Publ., vol. 8, PWN, Warsaw, 1982, pp. 415–434. MR738307 (85i:22036)

[20] Serban Stratila and Dan Voiculescu, On a class of KMS states for the unitary group U(∞),Math. Ann. 235 (1978), no. 1, 87–110. MR0482248 (58 #2327)

[21] Nina Stumme, Automorphisms and conjugacy of compact real forms of the classical in-finite dimensional matrix Lie algebras, Forum Math. 13 (2001), no. 6, 817–851, DOI10.1515/form.2001.036. MR1861251 (2003a:17029)

[22] D. Voiculescu, Sur let representations factorielles finies du U(∞) et autres groupes semblables,C. R. Acad. Sci. Paris 279 (1972), 321–323.

[23] Hermann Weyl, The classical groups, Princeton Landmarks in Mathematics, Princeton Uni-versity Press, Princeton, NJ, 1997. Their invariants and representations; Fifteenth printing;Princeton Paperbacks. MR1488158 (98k:01049)

[24] Joseph A. Wolf, Principal series representations of direct limit groups, Compos. Math. 141(2005), no. 6, 1504–1530, DOI 10.1112/S0010437X05001430. MR2188447 (2007d:22010)

280 JOSEPH A. WOLF

[25] J. A. Wolf, Principal series representations of direct limit Lie groups, in Math. Forschungsin-stitut Oberwolfach Report 51/210, “Infinite Dimensional Lie Theory” (2010), 2999–3003.

[26] Joseph A. Wolf, Infinite dimensional multiplicity free spaces III: matrix coefficients andregular functions, Math. Ann. 349 (2011), no. 2, 263–299, DOI 10.1007/s00208-010-0525-3.MR2753823 (2012c:22028)

[27] J. A. Wolf, Principal series representations of infinite dimensional Lie groups, I: Minimalparabolic subgroups, Progress in Mathematics, to appear. {arXiv: 1204.1357 (math.RT)}

[28] J. A. Wolf, Principal series representations of infinite dimensional Lie groups, III: Functiontheory on symmetric spaces. In preparation.

Department of Mathematics, University of California, Berkeley, California 94720–

3840


Published Titles in This Series

598 Eric Todd Quinto, Fulton Gonzalez, and Jens Gerlach Christensen, Editors,Geometric Analysis and Integral Geometry, 2013

592 Arkady Berenstein and Vladimir Retakh, Editors, Noncommutative BirationalGeometry, Representations and Combinatorics, 2013

590 Ursula Hamenstadt, Alan W. Reid, Rubı Rodrıguez, Steffen Rohde,and Michael Wolf, Editors, In the Tradition of Ahlfors-Bers, VI, 2013

588 David A. Bader, Henning Meyerhenke, Peter Sanders, and Dorothea Wagner,Editors, Graph Partitioning and Graph Clustering, 2013

587 Wai Kiu Chan, Lenny Fukshansky, Rainer Schulze-Pillot, and Jeffrey D.Vaaler, Editors, Diophantine Methods, Lattices, and Arithmetic Theory of QuadraticForms, 2013

586 Jichun Li, Hongtao Yang, and Eric Machorro, Editors, Recent Advances inScientific Computing and Applications, 2013

585 Nicolas Andruskiewitsch, Juan Cuadra, and Blas Torrecillas, Editors, Hopf

Algebras and Tensor Categories, 2013

584 Clara L. Aldana, Maxim Braverman, Bruno Iochum, and Carolina NeiraJimenez, Editors, Analysis, Geometry and Quantum Field Theory, 2012

583 Sam Evens, Michael Gekhtman, Brian C. Hall, Xiaobo Liu, and Claudia Polini,Editors, Mathematical Aspects of Quantization, 2012

582 Benjamin Fine, Delaram Kahrobaei, and Gerhard Rosenberger, Editors,Computational and Combinatorial Group Theory and Cryptography, 2012

581 Andrea R. Nahmod, Christopher D. Sogge, Xiaoyi Zhang, and Shijun Zheng,Editors, Recent Advances in Harmonic Analysis and Partial Differential Equations, 2012

580 Chris Athorne, Diane Maclagan, and Ian Strachan, Editors, Tropical Geometryand Integrable Systems, 2012

579 Michel Lavrauw, Gary L. Mullen, Svetla Nikova, Daniel Panario, and LeoStorme, Editors, Theory and Applications of Finite Fields, 2012

578 G. Lopez Lagomasino, Recent Advances in Orthogonal Polynomials, Special Functions,and Their Applications, 2012

577 Habib Ammari, Yves Capdeboscq, and Hyeonbae Kang, Editors, Multi-Scaleand High-Contrast PDE, 2012

576 Lutz Strungmann, Manfred Droste, Laszlo Fuchs, and Katrin Tent, Editors,Groups and Model Theory, 2012

575 Yunping Jiang and Sudeb Mitra, Editors, Quasiconformal Mappings, RiemannSurfaces, and Teichmuller Spaces, 2012

574 Yves Aubry, Christophe Ritzenthaler, and Alexey Zykin, Editors, Arithmetic,Geometry, Cryptography and Coding Theory, 2012

573 Francis Bonahon, Robert L. Devaney, Frederick P. Gardiner, and DragomirSaric, Editors, Conformal Dynamics and Hyperbolic Geometry, 2012

572 Mika Seppala and Emil Volcheck, Editors, Computational Algebraic and AnalyticGeometry, 2012

571 Jose Ignacio Burgos Gil, Rob de Jeu, James D. Lewis, Juan Carlos Naranjo,Wayne Raskind, and Xavier Xarles, Editors, Regulators, 2012

570 Joaquın Perez and Jose A. Galvez, Editors, Geometric Analysis, 2012

569 Victor Goryunov, Kevin Houston, and Roberta Wik-Atique, Editors, Real andComplex Singularities, 2012

568 Simeon Reich and Alexander J. Zaslavski, Editors, Optimization Theory and

Related Topics, 2012

567 Lewis Bowen, Rostislav Grigorchuk, and Yaroslav Vorobets, Editors, DynamicalSystems and Group Actions, 2012

566 Antonio Campillo, Gabriel Cardona, Alejandro Melle-Hernandez, Wim Veys,and Wilson A. Zuniga-Galindo, Editors, Zeta Functions in Algebra and Geometry,2012

This volume contains the proceedings of the AMS Special Session on Radon Transformsand Geometric Analysis, in honor of Sigurdur Helgason’s 85th Birthday, held from January4–7, 2012, in Boston, MA, and the Tufts University Workshop on Geometric Analysis onEuclidean and Homogeneous Spaces, held from January 8–9, 2012, in Medford, MA.

This volume provides an historical overview of several decades in integral geometry andgeometric analysis as well as recent advances in these fields and closely related areas. Itcontains several articles focusing on the mathematical work of Sigurdur Helgason, includ-ing an overview of his research by Gestur Olafsson and Robert Stanton. The first article inthe volume contains Helgason’s own reminiscences about the development of the group-theoretical aspects of the Radon transform and its relation to geometric analysis. Othercontributions cover Radon transforms, harmonic analysis, Penrose transforms, represen-tation theory, wavelets, partial differential operators on groups, and inverse problems intomography and cloaking that are related to integral geometry.

Many articles contain both an overview of their respective fields as well as new researchresults. The volume will therefore appeal to experienced researchers as well as a youngergeneration of mathematicians. With a good blend of pure and applied topics the volumewill be a valuable source for interdisciplinary research.

ISBN978-0-8218-8738-7

9 780821 887387

CONM/598

CO

NM

598G

eo

me

tricA

nalysis

and

Integ

ralG

eo

me

try•

Quinto

,Go

nzale

z,and

Christe

nsen,Ed

itors

AM

S

geometric analysis and integral geometry: ams special session on radon transforms and geometric...

Documents