1

PLUS Factorization of Matrices and Its Applications

Pengwei Hao

Center for Information Science, Peking University

Queen Mary, University of London

Contents

1. Background

2. Main Achievements

3. Applications

4. Unsolved Problems

1. Background

Where the problem from

Storage

transmission

Image C

oding

Spatial Transform

B/W images

Color images

Multiple Component images

Color Space Transform

MCT

Image D

ecoding

Inverse Spatial Trans.

Color images

Multiple Component images

Inverse Color Transform

Inverse MCT

B/W images

Lossless?

Lossless? Lossless?

Modeling of the General Problem

Visual Information

Remotely Sensed Data

Other types of

Information

Processing

Storage

Transmission

En

codin

g

Sign

al Tran

sform

Sign

al Acq

uisition

digital signal

Applications

integers

floating-point computation

transform domain

integers

Mathematics : integer reversible?

What’s integer reversible transform?

The transform maps integer x to integer y, and its inverse recovers integer xexactly the same from integer y.

If the inverse transform is also required to be “integer reversible”, the transform has to be one-to-one integer mapping.

2

A Simple Example

A simple 2D transform:

What happens if mapping integers directly?

It’s not one-to-one integer mapping, so not reversible!

What if more bits kept?

1 1

2 2

4

1/ 4

y x

y x

= ⋅

4 1 4

1/ 4 0 0 4Roundoff

04 1 4

1/ 4 1 0.25

⋅ ⇒ ⇒ ⇒ ⋅ ⇒

What if take more bits?In 2000, some NASA researchers were trying to find out how many bits behind the point are needed for lossless transform.

Their experiments show that it doesn’t help if more bits kept. And the cost is about 1 bit for 1 bit!

By direct rounding after transformation:

The rounding error:

If the error is independent of integers:Then, the transform matrix must satisfy:

i.e. we must have and thus, If the inverse transform is also integer reversible: However, Therefore, we have

This condition is too strict.It’s why NASA failed.

11 ==∞

−∞

AA

Analysis of Direct Integer Transformation

][][ˆ bAxbAxbAxyyye +−+=+−=−=

ˆ [ ]= + ⇒ = +y Ax b y Ax b

1 1 1 1ˆ[ ( )] [ ( )] [ ] [ ]x− − − −= − = + − − = − = + −x A y b A Ax b e b x A e A e

[ ] [ ]n c n c+ = +

0=− − ][ 1eA 11 ≤∞

−A

1∞≤A

111 =≥∞

−

∞

−∞

AAAA

Research in Integer Transforms

• Integer version of simple wavelet transforms

S transform (Blume & Fand, 1989)

TS transform (Zandi et al, 1995)

S+P transform (Said & Pearlman, 1996)

• Ladder structure (Bruekers & van den Enden, 1992)

• The lifting scheme (2D, Sweldens, 1996)

• Rough color space transform (3D, Gormish et al, 1997)

• Factorization of wavelet transforms (2D, QY Shi, 1998, Daubechies et al, 1998)

Previous WorkS-Transform – reversible wavelets

H. Blume and A. Fand, "Reversible and Irreversible Image Data Compression Using the S Transform and Lempel-Ziv coding", Proc. SPIE Medical Imaging III: Image Capture and Display, vol. 1091, pp. 2-18, 1989.

Forward transform:

Its Inverse:

Carefully combined!!!

( )( ) (2 ) (2 1) 2

( ) (2 ) (2 1)

s n x n x n

d n x n x n

= + + +

( )(2 ) ( ) ( ) 1 2

(2 1) ( ) ( ) 2

x n s n d n

x n s n d n

= + +

+ =

Previous WorkTS-Transform – reversible wavelets

A. Zandi, J. D. Allen, E. L. Schwartz, and M. Boliek. "CREW: Compression with reversible wavelets ", In Proc. of IEEE Data Compression Conference, pp. 212-221, Snowbird, UT, 1995.

Forward :

Its Inverse:

Carefully combined!

( )( )

( ) (2 ) (2 1) 2

( ) (2 2) (2 3) ( 2) ( ) 2 4

s n x n x n

d n x n x n s n s n

= + +

= + + + + +

( )( )

( ) ( 1) ( 1) ( 1) 2 4

(2 ) ( ) ( ) 1 2

(2 1) ( ) ( ) 2

p n d n s n s n

x n s n p n

x n s n p n

= − − + − − + = + + + = −

3

Previous WorkS+P Transform – reversible wavelets

A. Said, W. A. Pearlman, "An image multiresolution representation for lossless and lossy compression" , IEEE Transactions on Image Processing, vol. 5, pp. 1303-1310, Sept. 1996.

S-Transform + Prediction

Carefully combined!

Previous WorkColor Space Transform

M. J. Gormish, E. L. Schwartz, A. F. Keith, M. P. Boliek, and A.Zandi, "Lossless and nearly lossless compression for high quality images", Proceedings of SPIE, vol. 3025, pp. 62-70, 1997.

A Rough approximation:

Its Inverse:

Carefully combined!!!

( )2 2Yr R G B

Ur R G

Vr B G

= + + = − = −

( ) 4G Yr Ur Vr

R Ur G

B Vr G

= − + = + = +

Previous WorkLadder Structure

F. A. M. L. Bruekers, A. W. M. van den Enden, "New networks for perfect inversion and perfect reconstruction", IEEE J. on Selected Areas in Communications, vol. 10, pp. 130-137, 1992.

Proposed a new network for perfect inversion

Previous WorkThe Lifting Scheme

W. Sweldens, "The Lifting Scheme: A new philosophy inbiorthogonal wavelet constructions", in A. F. Laine and M. Unser, editors, Wavelet Applications in Signal and Image Processing III, Proc. SPIE, vol. 2569, pp. 68-79, 1995.

W. Sweldens, "The lifting scheme: a custom-design construction of biorthogonal wavelets", J. of Applied and Computational Harmonic Analysis, vol. 3, No. 2, pp. 186-200, 1996.

W. Sweldens, "The lifting scheme: A construction of second generation wavelets", SIAM J. Math. Anal, vol. 29, No. 2, pp. 511-546, 1997.

In order to increase the moments of wavelets

Previous WorkInteger mapping with lifting scheme

A. R. Calderbank, I. Daubechies, W. Sweldens, B-L. Yeo, "Wavelet transforms that map integers to integers", J. of Applied and Computational Harmonic Analysis, 5(3): 332-369 , 1998.

First time relate lifting scheme to integer mapping

Previous WorkFactoring wavelet transform into lifting steps

I. Daubechies, W. Sweldens, "Factoring wavelet transforms into lifting steps", J. of Fourier Analysis and Application, vol. 4, No. 3, pp. 247-269, 1998.

Qingyun Shi, "Biorthogonal wavelet theory and techniques for image coding", Proceedings of SPIE, vol. 3545(ISMIP'98, Oct. 1998): 24-32, 1998.

Split the polyphase representation into even and odd coefficients:

A general theory of 2 channel polyphase matrix factorizations

1

( ) ( ) 1 01 ( ) 0( )

( ) ( ) ( ) 10 1 0 1

me o i

ie o i

h z h z s z Kz

g z g z t z K=

= = ⋅

∏P

4

Then…… for N-D?

Can anyone combine an integer reversible implementation for higher dimensional transforms?

Even for experts?

Too difficult!!!

We must find some new way out.

• Shears can be implemented fast by memory-moving with hardware.

x yshear shear

• A viewing transform can be represented by a matrix if homogeneous coordinates are employed.

• Problem: Can we use some shears to implement the transform so as to speed-up it?

Problem in Computer Graphics

• 2-shear-scaling factorization for 2D rotation (Catmull& Smith, SIGGRAPH, 1980)

• 3-shear factorization for 2D rotation (Paeth ,1986)• 3-shear-scaling factorization for 3D affine transforms

(Hanrahan, SIGGRAPH, 1990)• 3-shear factorization for 3D rotation (Wittenbrink &

Somani, SIGGRAPH, 1993)• Shear-warp factorization for 3D projection (Lacroute

& Levoy, SIGGRAPH, 1994)• 4-shear factorization for 3D rotation (Chen & Kaufman,

2000)

Research in Shear Factorizations

• A shear matrix is a unit triangular matrix

• Shear factorizations are to find those factors of unit triangular matrices

• Shear factorizations and the factorization for reversible integer transformation are equivalent in mathematics: Customizable Factorization

Relations between Shear Factorizations and Integer Reversible Transforms

Generic Linear Transforms

If we can compute in some special way…

We find some

Elementary Reversible Matrices

can do it!

Our Solution: PLUS FactorizationPengwei Hao and Qingyun Shi, "Invertible linear transforms implemented by integer mapping", Science in China, Series E, (in Chinese), vol. 30(2), pp. 132-141, 2000.

Pengwei Hao and Qingyun Shi, "Proposal of reversible integer implementation for multiple component transforms", ISO/IEC JTC1/SC29/WG1N 1720, Arles, France, July 3-7, 2000.

Pengwei Hao and Qingyun Shi, "Matrix factorizations for reversible integer mapping", IEEE Transactions on Signal Processing, vol. 49, No. 10, pp. 2314-2324, Oct. 2001.

Yiyuan She and Pengwei Hao, "Block TERM Factorization of Uniform Block Matrices", Science in China (Series F), Vol. 47, No. 4, pp. 421-436, 2004.

Pengwei Hao, "Customizable Triangular Factorizations of Matrices", Linear Algebra and Its Applications, Vol. 382, pp. 135-154, May 2004.

A general theory of finite dimensional matrix factorizations

5

2. Main Achievements

Reversible Condition for just One Number

If two integers x and y satisfy both equation:

y=ax+b

x=(y b)/a

Then, a, b, 1/a and b/a must be all integers

If integer b is replaced by rounding a fractional number: [b]

a and 1/a must be some special integers: integer factors (integer units)

Integer Factors

Integer units

Integer factor: j

• Real numbers: 1, -1

• Complex numbers: 1, -1, i , -i

Integer reversibility: y=jx+[b]

Property 1: integer factors don’t change the magnitude of an integer

Property 2: the inverse of an integer factor is still an integer factor

b

[ ]

x j y+

+

b

x1/j

b

[ ]

-

+

Elementary structure of integer number mappingj is an integer factor, 1, -1, i, -i

For Integer Numbers,How Reversible Mapping Works?

y=jx+[b] x=(1/j).(y+[b])

If j=1, it is the same as the ladder structure byBruekers & van den Enden and the lifting scheme by Sweldens.

Rounding arithmetic: chopping, rounding, or bits before decimal point

Implementation of Linear Transforms

Elementary computing units are the same as the transformation of numbers: y=jx+[b]

Transform matrices: The diagonal elements are integer factors

Forward transform: b must be independent of x

Inverse transform: b must be independent of y

Elementary Reversible Matrix (ERM)

Elementary Reversible Matrices

• Diagonal elements are all integer factors

• Triangular ERM (TERM)– Upper TERM

– Lower TERM

• Single-row ERM (SERM)–

• Other matrices that can be converted into ERMs by row permutations only

Tm m m= +S J e s

6

Transform with TERMsUpper TERMs

Forward Reverse

Lower TERMs

Forward Reverse

[ ]

[ ]( )1

m m m mn nn m

m m

m mm

y j x s x

j x b

x y bj

≠

= + = +

= −

∑

Transform with SERMs

Forward transform calculation :

Its reverse :

Properties of TERMs

• The product of two upper TERMs is also an upper TERM, and the product of two lower TERMs makes a lower TERM.

• The determinant of a TERM is an integer factor.

• The inverse of a TERM is a TERM.

• A 2D TERM is also a SERM.For any SERM : is an integer factor.

Properties of SERMs

Unit SERMs :

where

Tmmm seIS +=

( )

1

,1 ,2 ,1 ,2

, det 1Tm m m m

T

m m m m m

− = − =

⋅ = + +

S I e s S

S S I e s s

TN 00 seIS +=

the m-th standard basis vector formed as the m-th column of the identity matrix

a vector that the m-th element smm is 0

me

{ }mnm s=s

det mS

Problems We Must Solve

• Can a matrix be factorized into some ERMs?

• If not always, what’s the sufficient/necessary conditions?

• Under the conditions it can be factorized, how?

• Is the factorization unique?

• If not unique, what’s the optimal factorization?

• What’s the error between the original theoretical transform and its integer reversible implementation?

Factorization of Two Special Matrices

−

−

−

=

−1sin)cos1(

01

10

sin1

1sin)cos1(

01

cossin

sincos

θθθ

θθθθθθ

−

−=

10

sin)1(cos1

1sin

01

10

sin)1(cos1 θθθ

θθ

0 1 0 1 1 1 0 1 1

0 1 1 1 1 0 1 1 1 0 1

1 0 1 1 1 0 1 1 1

1 1 0 1 1 1 0 1

α αα α α

α αα

− = − − − −

= −

2D rotation:

Scaling:

7

Factorization of Nonsingular Matrices

Any nonsingular matrix : A=PLDU

Problem: permutation P, unit triangular Land U are all ERMs, how to factorize D?

With 7 TERMs:

1 2( , , , )N O E R E O Rdiag d d d= ⋅⋅ ⋅ = ⋅ ⋅ = ⋅ ⋅D D D D D D D

⋅⋅⋅⋅⋅⋅

=−−

−−

odd is if )1,1,,,1,,1,(

even is if )1,,,1,,1,(

223311

113311

Ndiag

Ndiag

NN

NNO λλλλλλ

λλλλλλD

⋅⋅⋅⋅⋅⋅

=−−

−−

odd is if )1,,,1,,1,,1(

even is if )1,1,,,1,,1,,1(

114422

224422

Ndiag

Ndiag

NN

NNE λλλλλλ

λλλλλλD

1 2m md d dλ = ⋅⋅⋅

1 2 3 4 1 2 3 4

1 2 3 4 5 6 7

O O O O E E E E R

R

==

A PLV V V V V V V V D U

PVV V V V V V D

),1,,1( NR diag λ=D …

We prove: the necessary and sufficient condition to factorize a matrix into up to 3 TERMs

is

where

The Least Number of Factor Matrices?

0USPLDA R=

0detdet ≠= RT DAP

]0,,,,[ 12100 −⋅+=+= NNT

N ssseIseIS( ))det(,1,,1,1 APD T

R Diag= ……

…

Theorem of SERM Factorizations

Matrix A has a unit SERM factorization of

A=PSNSN-1 … S1S0

if and only if is an integer factor,

where are SERMs.

j=Adet

),,2,1,0( Nmm ⋅⋅⋅=S

SERM Factorization

11 12 11 12

1 121 22 21 11 22 21 11 12

− −

= −

A A I 0 A A

A A A A A A A A 0 I

11 12 122 21 11 12 11

21 22

det det( ) det( )− = − ⋅

A AA A A A A

A A

Owing to the equation:

SERM factorization can be derived from TERM factorization: A=PLUS0

A=PSNSN-1 … S1S0

( )( )

( )

11 1

1 11 2 2 1

1 1 11 2 1 1 2 1

2 1

N N

N

−

− −

− − −− −

=

=

=

=

=

LU LUS S

LUS S S S

LUS S S S S S

S S S

…

… …

…

Necessary and Sufficient Condition for ERM factorizations

The absolute value of the determinant of the matrix is 1.

|det(A)|=1 is much looser than the reversible condition for direct rounding

11

==−

∞ AA

If the Determinant is Not 1

For nonsingular matrices, we can apply some scaling to make the determinant 1:

To multiply a diagonal matrix or a nonzero scalar number.

Scaling can also:• Reduce the number of factor matrices• Control the transform dynamic range• Increase the transformation efficiency• ……

8

Factorization Algorithm1. Factorize into 3 TERMs :

A=PLUS0 similar to LU factorization2. Further factorize LU into N SERMs :

LU=SNSN-1 … S1 extract row-by-rowExample:

2 3 0 1 1 0 1 4 1 0

3 4 1 0 1/ 2 1 0 1 1/ 2 1

= ⋅ ⋅ ⋅

2 0 0 1 1 0 1 1/ 2 1 0

2 1/ 2 1 0 2 1 0 1 2 1

1 1 1 0 1 1/ 2 1 0

0 1 1 1 0 1 2 1

= ⋅ ⋅ ⋅ −

− = ⋅ ⋅ ⋅

.

.

.

P

.

.

.

S0

.

.

.

.

.

.

x yLU

.

.

.

TERM Factorization for Integer TransformComputational flow chart

A=PLUS0

.

.

.

P

.

.

.

S0

.

.

.

.

.

.

x ySNS1

.

.

.

S2

.

.

.

S3

.

.

.

SN-1

.

.

.

. . .

. . .

. . .

. . .

. . .

. . .

SERM Factorization for Integer TransformComputational flow chart

A=PSNSN-1 … S1S0

Implementation Addition Multiplication Rounding Permutation

Original Transform NN −2 2N N No

TERM factorization 12 −N 12 −N 2 1N − Needed

SERM factorization 12 −N 12 −N 1+N Needed

A TERM or a SERM-factorized transform at most has a computational complexity very close to the naive transformation.

Computational Complexity

Error Estimation

( )( ) ( )321321 LUuLuuPUuuLuPu ++=+⋅+⋅=

( ) ( )321321 LUuLuuPLUuLuuPu ++≤++=

( )∞∞∞

++⋅≤ Nu LUeLu 1

0PLUSA =

Properties of ERMs

• S0 is not unique, so neither are ERM factorizations.

• A real matrix has a real ERM factorization.

• Error bounds depend on factorization itself, are independent of the data to be transformed.

9

Advantages of ERM Factorization

Best approximation of given linear transform

Integer reversible ( lossless )Flexible rounding arithmetic ( [b] )

In-place calculation ( add to: += )

Simple inverse ( subtract from: -= )

Acceptable computation complexity

Generalization - Customizable

A is just nonsingular for almost every transforms

P is unit triangular pseudo-permutation

U has customizable diagonal freely designated by users

S has many possible forms freely chosen by users

PLUS factorization: A=PLUS

(customizable triangular factorization)

Forms of Special Matrix S

Single-row

Single-column

Bidiagonal

Other…?

Factorization of Block Matrices

Factorization of block matrices is a generalization for element matrices

For parallel and efficient computation

Problem: some blocks may be singular

Questions: How to block? What permutation works? How to make it more efficient?

Solution: define a determinant matrix

BLUS factorization (Yiyuan She)

linear invertible |detA|=1 11 ==∞

−∞

AA

Solved Problems

3. Applications

10

Applications in Image Coding

• Signals acquired and stored are almost integers• Lossless data/signal compression• Unified lossy/lossless compression system• Region-of-interest (ROI) decoding• Lossy compression with lossless reproduction of ROI• Progressive lossy-lossless transmission• Integer-reversible implementation of transforms

makes lossy coding methods lossless and reusable.

App1. Color Image Coding

Color Images

Color Trans.

Image Coding

R

G

B

C1

C2

C3

−−−−=

BGR

CbCrY

500.0331.0169.0081.0419.0500.0114.0587.0299.0

The integer reversibility of the color transform is the key problem of lossless

color image coding.

App1. Color Transform (JPEG 2000)

−−−−=

B

G

R

Cb

Cr

Y

500.0331.0169.0

081.0419.0500.0

114.0587.0299.0

( )[ ]

−=−=

++=

GBCb

GRCr

BGRY 42 ( )[ ]

+=+=

+−=

GCbB

GCrR

CbCrYG 4

JPEG2000 – irreversible transform

JPEG2000 – reversible transform

• Roughly approximated• Different from the transform for lossy coding

App1. Integer Reversible Color Transform

=

kdd

cc

bb

aaRL

21

21

21

21

1

1

1

1

1

1

1

1

1

1

1

APP

1det ±== RLk APP

Our matrix factorization method:

It can be applied to all 3D linear transforms, thus all possible transforms can be compared for lossless coding.

App2. Multiple Component ImagesXB- Hyperspectral image- 80 bands (highly correlated)- 346x512- Road, water, rice paddy

App2. Multiple Component Transform

多成分图像数据

Decorrelating transforms of multiple component

JPEG 2000 compression ontransformed components

- No transform (NoT)- Discrete wavelet transform (DWT)- Direct prediction transform (DPT)- Tasselled cap transform for TM (TCT)- Karhunen-Loeve transform (KLT)- DCT, DHT, …...

Input Data Cube

11

App2. MCT Image Coding

Previous solutions:

• Different methods for lossy and lossless coding

• Lossy transform + Lossless residue coding

MCT matrix may be of large size, 100x100, 512x512…

Our solution (JPEG 2000 proposal No. 1720): reversible integer implementation by using matrix factorization for MCT (including KLT)

App2. Lossless Compression(XB, Bit Rate, BPPPB)

RKLT gives the best performance

App2. Lossy Compression(XB, 0.5BPPPB, PSNR, dB)

RKLT gives the performance close to the best TM5-3-2,Beijing(Huairou)

Preview(1/256) & ROI 2%, 33.8dB

46%, Lossless

Matrix F

actorization for

Reversib

le Integer M

app

ing

Image Coding Based on ROI

Applications in Information Security

Transform with cipher still keeps some good properties, such as orthogonal

Covert communication: to hide some secret data in a carrier signal

Digital watermarking: for copyright protection, source detection, …

App4. Data Hidingtransform without encryption

TransformTransformTransformTransformOriginalOriginalOriginalOriginalImageImageImageImage

SecretSecretSecretSecretDataDataDataData

EncryptionEncryptionEncryptionEncryption

InverseInverseInverseInverseTransformTransformTransformTransform

ImageImageImageImagewithwithwithwithDataDataDataData

TransformTransformTransformTransformHiddenHiddenHiddenHiddenDataDataDataData

DecryptDecryptDecryptDecryptDataDataDataData

ExtractionExtractionExtractionExtraction

ImageImageImageImagewithwithwithwithDataDataDataData

Data hiding

Data extraction

Image transfer

12

TransformTransformTransformTransformOriginalOriginalOriginalOriginalImageImageImageImage

SecretSecretSecretSecretDataDataDataData

EncryptionEncryptionEncryptionEncryption

InverseInverseInverseInverseTransformTransformTransformTransform

ImageImageImageImagewithwithwithwithDataDataDataData

CipherCipherCipherCipher

TransformTransformTransformTransformHiddenHiddenHiddenHiddenDataDataDataData

DecryptionDecryptionDecryptionDecryptionDataDataDataData

ExtractionExtractionExtractionExtraction

ImageImageImageImagewithwithwithwithDataDataDataData

CipherCipherCipherCipher

Data hiding

Data extraction

Image transfer

App4. Data Hidingtransform with encryption

PSNR = 36.37dBOriginal image (512x512)

App5. Data HidingExperiment with huge data: 70K bytes

App5. Data HidingThe hidden data (7K bytes)

北京大学视觉与听觉信息处理室验室视觉与听觉信息处理实验室于1986年3月通过专家委员会论证,1986年6

月经国家计委批准,1988年12月通过国家验收,是国家在北京大学建立的第一个重点实验室实验室学术委员会主任为中科院院士唐孝威教授,实验室学术委员会顾

问为中科院院士程民德教授 实验室主任为唐世谓教授实验室的主要研究方向是紧密结合智能计算机,智能机器人,智能武器及

指挥系统,管理自动化等社会主义四个现代化的重大课题和发展需要,开展机器视觉与听觉信息处理方面的基础研究与应用基础研究 特别是关于机器视觉 机器听觉 语音 文字的识别处理与自然语言理解,图像识别和图像数据库,智能系统与知识工程,视觉与听觉的神经计算模型和人工神经网络等方面的基本理论与基本方法研究,以及具有显著经济和社会效益的应用研究中的基础性工作 以期为实现实用的机器视觉系统与听觉系统提供可用的方法与技术,为实现自然的人机通信提供基础技术手段并在某些应用领域提供具有国际先进水平的设计思想与技术,指导实用产品的开发 为此实验室装备了先进的计算机及其他设备根据学术委员会确定的研究方向和目标,实验室积极开展了相关课题的

研究 包括国家科技攻关 863高技术 攀登计划 科学基金 博士点基金以及部委合作项目等,近60项 在进行这些课题的研究中实验室发挥北京大学重视基础研究和应用基础研究的传统和学科相互交叉渗透的良好学风,互相配合,共同奋斗在所承担的一些项目上取得了创新成果 “七.五”国家重点攻关项目“模式识别图像数据库”被鉴定委员会认为是取得一系列具有国际水平 国际先进水平和国际领先水平的成果 特别对实验室完成的“指纹自动识别系统”给予了极高的评价,认为“系统采用领先的指纹软件技术成为第一个在工作站上实现的技术最先进的指纹自动识别系统”

“是一项居于国际领先地位的科技成果” 这一成果获得了国家教委科技进步一等奖和国家科技进步 ?等奖并在国际竞争中战胜了实力强大的对手进入国际市场 “八五”以来该项目在上海,珠海,杭州,广州等十个公安部门进一步推广应用,取得了良好的社会效益 此外在计算机视觉,说话人识别,图象数据库,智能系统与知识工程以及视觉与听觉的生理与心理模型方面也取得了一些显著成果几年来实验室在国内外学术会议和刊物上发表了论文近300篇,开展了广

泛的学术交流活动.一些年青的硕士,博士毕业生也很快的成长起来,在研究上成为独挡一面的骨干 目前在实验室课题指南涉及的各个方面有年青研究人员从事独立研究,且正在成长为学术接班人 实验室自1988年底通过国家验收以来,还实施了八轮开放课题研究计划,支持了147项课题.邮政编码 100871通讯地址 北京大学信息科学中心电 话 (010) 62751935传 真 (010) 62755654E-mail chi@cis.pku.edu.cnURL http://www.cis.pku.edu.cn/

THE NATIONAL LABORATORY ON MACHINE PERCEPTIONThe proposal of setting up a National Laboratory on Machine Perception was discussed and accepted

by the Expert Committee in March 1986. The laboratory construction begun in June 1986 after being approved by the State Planning Committee, and finished in December 1988, which is the first National Key Laboratory settled in Peking University.

Our leaders: Chairman of the laboratory's Academic committee: Acad. Prof. TANG XiaoWei Director of the laboratory: Prof. TANG ShiWei Consultant of the Academic Committee: Acad. Prof. CHENG Minde The laboratory's main research objects are fundamental and applications oriented basic studies in the

field of machine visual perception and auditory information processing. These researches are closely related to artificial intelligence, robotics, intelligent control systems, management automation and other important subjects demanded by socialist modernization and development.

Special concern is given to basic theory and basic methodology for machine visual and auditory perception, voice and text recognition and natural language understanding, image recognition and image database, intelligent system and knowledge engineering, neural computational modeling and artificial neural networks, and other fundamental works with evident social and economical impact. The purpose of these researches is to provide methods and techniques for realization of the practically applicable machine visual and auditory perception system, to provide basic technical solution for natural human-machine communication systems, to provide, in certain application fields, principles for designing techniques at advanced international standards, to lead the development of practical products. According to those purposes, the laboratory is equipped with advanced computers, workstations and other special facilities.

Based on the decision of the Academic Committee for the research orientation and purpose, the laboratory has been actively carrying out researches under related projects. These include the National Key projects, including fundamental and high-tech projects the Foundation for, the Foundation forPh.D. projects, the National Natural Science Foundation projects, and other projects with collaboration to other Ministries and Committees, totals almost 60 projects.

Researchers in the laboratory enhanced Peking University's tradition to promote the fundamental and application oriented basic researches, and in the good style of interdisciplinary study, collaborated closely with each other, obtained original results in certain projects.

On the7th National Five-year Plan's "Pattern Recognition and Image Database" was considered by the Evaluation Committee as having a series of results with internationally advanced achievement. Especially the Automated Fingerprint Identification System, which was considered as the most advanced system, realized for the first time on workstations adopting special software technique received a high praise. It has received the first rank award from the State Education Committee for progress in science and technology and the second rank National Award for progress in science and technology. Since the 8th National Five-Year Plan, this system was applied in Shanghai, Zhuhai,Hangzhou, Guangzhou and other Police Departments, and obtained a good social effect. It also entered the international market.

Good results were also obtained in Computer Vision, Speaker Identification, Image Database, Intelligent Systems and Knowledge Engineering and Physiological and Psychological Modeling for Vision or Auditory Systems.

In last years, the laboratory published near 300 papers, collaboration and exchange activities been rapidly developed.

Now, more and more young research fellows and Ph.D. candidates are actively participating in the laboratory's projects. Since the end of 1988, the laboratory already supported 147 projects for guest scientists and visiting scholars. Center for Information SciencesPeking University, Beijing, 100871, P. R. of ChinaTel. : 86-010-62751935Fax : 86-010-62755654E-mail : chi@cis.pku.edu.cnURL : http://www.cis.pku.edu.cn/

App6. Digital Watermarking

Image with watermark(PSNR=33.92dB)

Watermark北京大学信息科学中心北京大学信息科学中心北京大学信息科学中心北京大学信息科学中心电话电话电话电话:(010)62752461:(010)62752461:(010)62752461:(010)62752461传真传真传真传真:(010)62755654:(010)62755654:(010)62755654:(010)62755654URL: http://www.URL: http://www.URL: http://www.URL: http://www.ciscisciscis....pkupkupkupku....eduedueduedu....cncncncn////(R) All rights reserved, 1999(R) All rights reserved, 1999(R) All rights reserved, 1999(R) All rights reserved, 1999年年年年3333月月月月

版权所有版权所有版权所有版权所有

Digital watermarks should survive commonly

used image processing, compression,

deformation and cutting

Watermark北京大学信息科学中心北京大学信息科学中心北京大学信息科学中心北京大学信息科学中心电话电话电话电话:(010)62752461:(010)62752461:(010)62752461:(010)62752461传真传真传真传真:(010)62755654:(010)62755654:(010)62755654:(010)62755654URL: http://www.URL: http://www.URL: http://www.URL: http://www.ciscisciscis....pkupkupkupku....eduedueduedu....cncncncn////(R) All rights reserved, 1999(R) All rights reserved, 1999(R) All rights reserved, 1999(R) All rights reserved, 1999年年年年3333月月月月

版权所有版权所有版权所有版权所有

Applications in Computer Graphics

To speedup the geometric transforms (affine or linear) by factorizing them into a series of shear transforms (3-4 shears in 2D, 4-5 shears in 3D) and a possible resize.

• Image or volume data display/rendering

• Image or volume data deformation

• Image or volume data registration

• Image or volume data merging

App7. Transforms of Volumetric Data

Original

Image

Scaled

and

Rotated

Naive transformation:

All the pixels after geometric transform have to be re-located

Some pixels have to be obtained by interpolation

Auxiliary memory is often required

Transformed data cannot be recovered losslessly

Transform

13

App7. Average Time for 3D Transforms (ms)

volume size

transforms 643 1283 2563 5123

x 0.26 5.73 46.97 361.55

y 0.10 3.94 36.52 293.40

z 0.09 4.13 50.88 445.38 translations

3D 0.22 5.61 58.19 462.84

xy/z 0.10 4.04 35.17 287.66

xz/y 0.19 7.20 51.82 366.14 slice

shears yz/x 1.98 183.28 1866.25 19261.0

x/yz 0.48 5.75 37.10 297.76

y/xz 1.37 14.46 115.89 2117.59 beam

shears z/xy 4.83 203.84 5422.13 34434.7

x 10.85 87.73 683.58 5435.71

y 0.39 6.78 47.67 380.84

z 0.12 6.39 44.96 363.26 resizes

3D 11.37 91.33 709.62 5608.07

xy 0.31 5.95 41.71 490.67 transposes

yz 0.38 5.66 48.21 1097.42

rotation 50.27 403.18 3570.23 29338.0 naive

transforms linear 56.08 471.58 4252.21 34347.3

App7. Speedup Geometric Transforms

Shear-resize factorization: T=DxLUSx=DxSxSySx

xshear

x shear

xresize

Equivalent

Scaled &

Rotated

1 1 0

0 1 1

a

b

= =

x yS S

(Ying Chen)

Originaly

shear

App8. Accelerate Volumetric Data Registration

Take one as the reference and the other as the floating image, and iteratively to find the optimal geometric transform such that the two images are registered.

Reference

imageFloating

image

App8. Multi-Modal Volumetric Data Registration - slices

Side

View

Top

View

Front

View

Reference Floating Registered

App8. Multi-Modal Volumetric Data Registration - rendered

Side

View

Top

View

Front

View

Reference Floating Registered

App8. Multi-Modal Volumetric Data Registration -rendered and compared

14

App8. Time for 3D Registration (ms/iteration)

Image size

Transform 643 1283 2563

Translation 0.30 6.39 54.07

Resize 1.79 28.15 208.80

3 shears 1.52 27.12 160.20

Memory

Reallocation 1.46 19.13 135.42

Transform

using

Shear-Resize

Factorization

Total 5.06 80.79 558.50

Naive Transformation 63.33 508.31 4163.69

The transformation with our shear-resize factorization is about 10 times faster than the naive transformation.

Advantages of Shear-Resize Factorization

Transform computation is just once for each layer

Interpolation can be accelerated

No auxiliary memory is needed

The data can be recovered losslessly if the nearest neighbor interpolation is employed and the area or volume is not reduced.

4. Unsolved Problems

(I) Forms of Generic S for PLUS Factorization

We have found the necessary and sufficient conditions for several forms of special matrix S, and two necessary conditions and one sufficient condition for generic S

What necessary and sufficient condition should S satisfy so that the PLUS factorization exists?

(II) PLUS Factorization with the Least Rounding Error

Now that PLUS factorization is not unique and the rounding error bounds only depend on the factorization

Then, what factorization is with the least rounding error? How to find it? What pivoting rules should be followed to find it?

( ) ( )321321 LUuLuuPLUuLuuPu ++≤++=

(III) Stability of PLUS factorization

When the matrix size is large, the PLUS factorization may be unstable, the elements in the factor matrices may be extremely large.

Why does PLUS factorization turn unstable? What’s the boundary condition? How to make factorization stable?

15

(IV) Perturbation Analysis for PLUS Factorization

A=PLUS

A+dA =P(L+dL)(U+dU)(S+dS)

What’s the influence of perturbation?What matrices suffer more from perturbation?

How to reduce the influence before factorization?How to reduce the influence during factorization?

(V) Other Applications of PLUS Factorization

With development and dissemination of PLUS factorization, we believe that more applications will be found and PLUS factorization will make more applications possible and attractive.

Thank You!

phao@cis.pku.edu.cn

phao@dcs.qmul.ac.uk

http://www.dcs.qmul.ac.uk/~phao