Multispectral Image Registration and Accuracy Analysis of ZY-3 Satellite

Xiaoyong Zhu1，Bin Liu2，Guo Zhang3, Xinming Tang1 1Satellite Surveying and Mapping Application Center,NASG,Beijing ,China

2Institute of Remote Sensing Applications, Chinese Academy of Sciences,Beijing,100101,China 3State Key Laboratory of Information Engineering in Surveying Mapping and Remote Sensing,Wuhan, China

1Satellite Surveying and Mapping Application Center,NASG,Beijing,100830,China e-mail:zhuxy@sasmac.cn, liub@irsa.ac.cn,guozhang@whu.edu.cn, txm@sasmac.cn

Abstract—While using multispectral remote sensing images for quantitative application, registration accuracy between spectrums will directly affect the accuracy of each quantitative production, thus, precise multispectral image registration is a base step for further processing and application. ZY-3 is China's first civilian mapping satellite, which attracts extensive attention for its excellent positioning and mapping ability as well as high precision of radiation characteristic, and greatly enhances the applications of domestic satellite. However, for the spectral camera optical system carried on ZY-3 satellite, there exists the problem of geometry deformation, which seems to be inconsistent for each spectrum in the “Multi-Spectrum Integral” device. Therefore, traditional image registration method does not show high accuracy, and “False color” phenomenon exists in the edge of image field of view. In order to solve this problem, a method of the virtual CCD line array re-imaging algorithm on spectrum registration will be introduced in this paper by analyzing the effects of displacement in image caused by virtual CCD re-imaging; the validation of this method will also be verified in principle. Based on ZY-3 spectrum registration experiment in AnPing, HeBei, the visual evaluation shows that the “False color” phenomenon in edge of multispectral image could be effectively solved. Precise evaluation has been performed to the registration of multispectral image products using artificial targets. it can be seen from the result, the maximum error of relative coordinates difference for each spectrum is 0.25 pixels, standard deviation in CCD line direction is better than 0.15 pixels. While it is 0.1 pixels on orbit direction, which is comparative accuracy with that of the manual cases. The experiment verifies that it’s feasible to produce multispectral images by using virtual CCD line array re-imaging algorithm, and high accuracy is achieved in ZY-3 multispectral image registration.

Keywords-ZY-3 Satellite; Multispectral Remote Sensing; Virtual CCD Line Array; Band to Band Registration

I. INTRODUCTION ZY-3 is China's first civilian high resolution stereo

mapping satellite, the main purpose of this satellite is to achieve stereo mapping with scale1:50000, and to improve the existed 1:25000 maps using 3 line array images. ZY-3 is also China's first civilian high resolution optical stereo mapping satellite, which carries a nadir TDICCD camera with resolution of 2.1 m, 1 forward and 1 backward TDICCD camera with resolution better than 3.6 m, and 1 multispectral camera with

resolution better than 6 m. For the multispectral camera, spectrums are integrated in one CCD, and 3 CCDs are staggered installed in the transmission and the reflection area in focal plane, forming a continuous CCD array in an approximate straight line(Fig. 1,Fig. 2)[1].

Figure 1. ZY-3 multispectral CCD bands relationship.

Figure 2. ZY-3 multispectral 3-CCD arrangement

The multi-spectrums of Beijing-1 satellite (3 bands) are composed of 6 independent cameras; registration error and deformation between bands are inconsistent. Pan Jun obtained a large number of registration control points through image matching between spectrums, then, performed spectrum registration by using tiny facet differential rectification with large amount of calculation[2][3].In CBERS-02B satellite, “multi spectrum Integral” devices used. WANG Honghai performed spectrum registration through overall translation among images

[4]. In CBERS-02B, it carries “multi spectrum Integral” device and a plurality of CCD splicing, and the relation between spectrums is similar to ZY-3, but it is impossible to achieve high registration accuracy through simple translation due to the

effects of satellite orbit, attitude, integral time, CCD splicing, optical deformation, etc. For RapidEye satellite, Keith Beckett et al has achieved precise image registration between multi-spectrums by removing the deformation in optical system

[5].The multi-spectrums and the whole field of view of RapidEye is single one CCD, thus, relation between spectrums is simple, and registration accuracy could be around 0.1 after removing deformation from optical system. While for ZY-3 satellite, relation between spectrums is relatively more complicated due to the plurality of CCD splicing, therefore this method is not compliant with ZY-3, but still possess certain reference significance.

In this paper, based on the characteristics of the multi-spectral camera in ZY-3, an accurate calibration of the direction for each spectral CCD element in camera coordinate system is introduced. While using the virtual CCD re-imaging algorithm, high registration accuracy could be achieved by the image registration products generation for each spectrum using visual and artificial target evaluation,

II. REGISTRATION PROCESSING METHOD

A. Interior orientation acquisition for each CCD through camera calibration Normally, precise camera interior orientations are

composed by focal length coordinates of principal point, arrangement and size of CCD element, together with calibration parameters. In which, calibration parameters of the camera is a fitting of the deformation in optical system, lens deformation, change of pixel size, CCD points transition and a plurality of CCD splicing, this process is pretty complicated. In this paper, the field angular model of interior orientations is introduced, in which the principal point, focal length and deformation will be represented together to reduce the complexity of model. Fig. 3 shows the sketch of interior orientation parameters, O-XYZ stands for camera coordinate system in satellite, and base point O is the imaging center. The purpose of interior calibration is to measure two angles of direction ,x yψ ψ in this system w.r.t 1u .

Xo

Y

Z

1u0>xψ

0<yψP

Figure 3. Schematic diagram of calibration parameters.

Based on the presentation of interior orientations above, geometric calibration method for interior orientations in linear array camera is introduced. High precision and resolution images, DTM and ideal interior orientation elements model for aero camera are adopted to simulate image, and the real position of each CCD element in linear array could be computed from the analysis of match errors. By matching the simulated image and real image,

After the launch of ZY-3, each element of 4 spectrum bands in 3 CCD was precisely calibrated by using ground data like DEM and DOM in scale 1:2000 from various regions. Results of calibration could be seen as following.

Figure 4. ZY-3 pointing angle (rad) of 4 spectrums for all 3 CCD.

Figure 5. ZY-3 Pointing angle (rad) of B2.

Figure 6. ZY-3 B2 spliting of CCD 1 and CCD 2.

Figure 7. diagrammatic sketch of ZY-3 Band 2 in along-track direction w.r.t CCD direction in 1000X.

B. Virtual CCD arraybased high-precision Registration For spectral camera of ZY-3 satellite, the imaging of each

spectrum is not in the same position in focal plane, since three pieces of CCD are spliced, in order to achieve image matching between spectrums in the way named “whole scene” and “Whole stripe. A virtual CCD method is used in this paper, the

installed position of multiple CCD along track direction is taken as a reference, the virtual CCD is " installed" in the center line of multiple CCD along track direction. As shown in Fig. 8, the solid line represents the actual multiple CCD linear array, a total of 4bands, and each band is composed of 3 spliced CCD. The dotted line represents the virtual CCD, D1 and D2 represent the maximum distance in two along track directions of virtual CCD and real CCD, respectively [1][9].

Figure 8. Relative position for virtual CCD and practical CCD in the focal plane.

Virtual CCD are aligned ideal in the focal plane without distortion, and during imaging process the integral time is unified, thus, each spectrum image in virtual imaging could be regarded as an ideal linear push broom image with center projection.

C. Virtual CCDarray based Re-imaging algorithm The pointing angle of each CCD detecting element in the

camera coordinate system is ( ,x yψ ψ ), which stands for the geometric relation of photography beam of each detecting element in the camera coordinate system [1]. Similarly, according to the "Install" position of virtual CCD, the pointing angle of each detecting element of virtual CCD could be figured out in the camera coordinate system based on ideal imaging principle with center projection. Based on this, virtual CCD based image split joint is made according to the following steps:

1) For an arbitrary point in virtual CCD image, precise imaging geometric model of virtual CCD image could be set up, using related pointing angle, real orbit parameters and attitude parameter. With this model, point will be projected to SRTM-DEM or the mean elevation plane of this area, so as to obtain corresponding geodetic coordinates.

2) Based on precise imaging geometric model of real CCD image, geodetic coordinated derived from step 1) could be projected to real CCD image.

3) Perform gray resampling to real CCD image; corresponding pixel is assigned to virtual detecting element after re-imaging.

4) Repeat 1) 2) and 3), until re-imaging is completed in the whole image.

Flow chart of Virtual CCD based Re-imaging algorithm could be seen in Fig. 9:

Figure 9. Virtual CCD based Re-imaging algorithm.

D. Accurate analysis of Virtual CCDarraybasedregistration method

In virtual CCD re-imaging algorithm, elevation values provided by SRTM are used for image conversion; the real images are projected into the virtual CCD image. Because there is certain deviation in STRM height, the elevation deviation will affect the virtual CCD imaging accuracy if displacement exists in the along-track direction for the virtual CCD "Install" position and the real CCD location, [1].Fig. 10 shows the section which is perpendicular to the CCD line during push-broom imaging process. An angle deviation of δ α exists in the pointing direction of imaging light for cross point (Black point) between the cross section plane with the virtual CCD and real CCD, the angle between the angle bisector of δ α and its perpendicular line is namedφ .

δαφ

h

d

Figure 10. Factors for the CCD image splicing error.

h (as shown in Fig. 10) is the height difference between the elevation of projection point (topographic height in figure) of virtual CCD detecting element light and its true elevation,

then, the image deviation d (distance in space) caused by topography undulation is:

tan( / 2) tan( / 2)d h hφ δα φ δα= + − − (1) According to trigonometric function, equation (1) can be

expanded to equation (2) 2 2 22 tan( / 2)[(1 tan ) / (1 tan *tan / 2)]d h δα φ φ δα= + − (2)

Side-View angle of the spectrum camera in ZY-3 is 0.1043 (rad), the maximum observation angle deviation between the real CCD and virtual CCD in along orbit direction is 0.0023 (rad), while the elevation error of SRTM is 50 m[10].

d<2*100*tan(0.00115)*[(1+tan2(0.1043))/(1-tan2(0.1043)*tan2(0.00115))]=0.1163meter (3)

For the spectrum resolution of 5.8 m, because of the maximum registration error is approximate 0.04 pixels due to elevation error, it could be neglected here.

III. EXPERIMENT AND ACCURACY ANALYSIS

A. Experiment Data To verify the algorithm in this paper, a scene of ZY-3

multispectral image was selected, and corresponding Beijing Time is 18th Feb, 2012, 11:18 am, with area in An Ping, HeBei. During imaging period, artificial target was laid out on ground in size 40m*40m. Images of target in each spectral band could be found in Fig. 11, in which band 1, band2, band3 and band4 are listed from left to right.

Figure 11. Image of artificial target in each spectral band.

B. Virtual CCD based registration of ZY-3 multi-spectral image Using the above virtual CCD construction method, virtual

CCD scanning lines of multispectral camera are constructed (see red line in Fig. 12), with horizontal axis for CCD direction and vertical axis for along-track direction, in this figure, B1, B2, B3 and B4 are presented with blue lines from top to bottom respectively. Registration images of four spectral bands could be generated by re-imaging algorithms mentioned in chapter 2.3 (See Fig. 13).

Figure 12. Diagrammatic Sketch of Virtual CCD Position.

Figure 13. Registration Image of Spectral band.

C. Visual Effect Analysis As to non-precise registration between spectrums for

homonymous points, blurred color could be found in boundary lines with obvious characteristics. According to the relative positions of each spectral CCD in focal plane, take image B2as reference, if an overall translation perform to image B1 in along-track direction by 152pixels, to image B3 by 128 pixels and B4 by 256 pixels, Fig. 14( a ),14( C ) could be derived. In this paper registration images of four spectrums are shown in Fig. 14(b), 14(d), in which 14( a ) and14( b ) are true color images combing bands 3,2,1, while 14( c ) and14( d ) are pseudo color images for the bands 4,3,2.

（a） （b）

(c) (d) Figure 14. Analysis and comparison of registration visual effects.

D. Accuracy Verification of Artificial TargetUsing the transition edge from high refle

reflection zone in artificial target, coordinatestarget centers to be precisely determined, whWhile registration accuracy between speevaluated by using 7 targets that are not orientation calibrations. And from the produced registration images, coordinates ofeach spectral band could be derived, and thewith the band B2 is used to evaluate the regFig. 15(a) and (b) show the difference coordinates between bands B1、B3、B4 targets, in CCD direction and alongrespectively.

(a) In CCD direction

(b) along-track direction

Figure 15. Relative errors between each spectrums w

From the results of artificial target expemaximum errors of relative coordinate dispectrums is 0.243 pixels, with standard 0.1296 pixels in CCD direction, and 0.069track direction, which is relatively good regFrom accuracy. From the visual analysis inedge, the method in this paper is effectiproblem as multiple multispectral CCD splicregistration in the field edge.

IV. CONCLUSION AND SUGGE

In this paper, aiming the imaging charamultiple multispectral CCD, high precprocessing method between spectrums was using the virtual CCD re-imaging algoritregistration images were generated for subseand this algorithm has been successfully

/通用格式

/通用格式

/通用格式

1 3 5 7 9 11 13 1

/通用格式

/通用格式

/通用格式

1 3 5 7 9 11 13 1

ts ection zone to low s of image point in hich is 0.15 pixels. ectrums could be

used for interior above mentioned f image points for

e difference of that gistration accuracy.

of image points and B2 in all 7

g-track direction

with B2 as reference.

eriments, it shows: ifference between errors better than 8 pixels in along-

gistration accuracy. n the characteristic ive to solve such cing and spectrum

ESTION acteristics of ZY-3 cision registration

introduced, while thm, multispectral equent processing, applied to -ZY-3

application system for standardthe geometric relation of eaimaging, this method has sobetween spectrums, which is wmultispectral images presentedprecision in calculation. While depends on the calibration aelements, since complicated mais not involved, therefore, cphysical meaning.

ACKNOWL

This paper is carried out witof science and technolog2011BAB01B02) and the natioproject" ZY-3 satellite data proctest of key technology research”

REFER

[1] Tang Xinming,ZhangGuo,Zhu XGeometry Model of ZiYuan-3 SActaGeodaetica et Cartographica

[2] ChenZhengchao,LuoWenfei,ZhanBeijing-1 Microsatellite MultisSensing, vol. 10(5), pp. 690-696,

[3] Pan Jun,ZhuYing,WangMiREGISTRATION FOR HJ-1A/1International Symposium on Imapp. 191-198, Yunnan,China,2011

[4] Wang Honghai,Xu Jianyan,LongMethod of Sub-pixel Band-to-baMultispectral Image,” SPACSENSING, vol. 31(3), pp. 55-63,

[5] Keith Beckett,ChrisRampersad,assessment.Sensors,Systems,and by Roland Meynart,Steven P. Nvol. 7474,74741L, 2009.

[6] R.Gachet,“SPOT5 in-fight command HRS instruments, in Proc,”Istanbul, Turkey, vol. 35, 2004.

[7] ZhuXiaoyong, ZhangGuo, TangXCBRS02B Image Geometric ExtInformation Science, vol. 25(3), p

[8] Zhang Guo,“Rectification for HUnder Lack of Ground Control2005.

[9] Zhang Guo, Liu Bin ,Jiang WaInner FOV stitching of SpaceboCCD line,” Journal of Image and

[10] Brown, C.G Jr,Sarabandi, K,PierTopography Mission height datand Remote Sensing, 2005.

15 17 19 21

15 17 19 21

d products’ generation. Based on ach detecting elements during lved the registration problems

well applied in the processing of d by pointing angles-with high the registration precision mainly

accuracy of interior orientation athematics and image processing calculation speed is fast with

LEDGMENT th full support from the Ministry

gy project (2011BAB01B01, nal defense science and industry cessing, application and on-orbit ”.

RENCES

Xiaoyong,“Triple Linear-array Imaging Surveying Satellite and its Validation,” a Sinica, vol. 41(2), pp. 191-198, 2012. ngHao,“The Geometric Evaluation of spectral Images,” Journal of Remote 2012. ,“PARALLEL BAND-TO-BAND B CCD IMAGES USING OPENMP, ” age and Data Fusion(ISIDF), vol. 248,

1. g Xiaoxiang,“An Automated Evaluation and Registration Accuracy for Satellite CECRAFT RECOVERY&REMOTE , 2010. ,RonyPutih,RapidEye product quality

Next-Generation Satellites XIII,edited Neeck,HaruhisaShimoda,Proc. of SPIE

missioning: Inner orientation of HRG ” XXth ISPRS Congr., Commission I,

Xin-ming,“Research and Application of erior Calibration,” Geography and Geo-pp. 16-18, 2009.

High Resolution Remote Sensing Image l Points,” Wuhan：Wuhan University,

angshou,“Research on an algorithm of orne optical sensor based on the virtual d Graphics., in press. rce,“L.E.Validation of the Shuttle Radar ta,” IEEE Transactions on Geoscience