data integration with xds -...

Data Integration with XDS

Wolfgang Kabsch, MPI HeidelbergReference: Tables of Crystallography F, pp. 218–225

presented by Tim Grüne

Göttingen, December 12th, 2003

Data Integration with XDS Overview

• characteristics of XDS

– Spot location and definition

– Flow-scheme for integration process

• Setting up XDS

– image directory and working directory

– The parameter file XDS.INP

• Running XDS: Refinement of input parameters; control

Data Integration with XDS Principles of XDS

Characteristics of XDS

• intended as fully automated integration program

• it is easy to set up and to run

• template files available for many different detector types

• very well documented

• no command line option — all parameters read from one single file: XDS.INP

• no graphical interface

Data Integration with XDS Principles of XDS

Co-ordinate Systems

All co-ordinate systems are right-handed. They are usually properly set-up in template file.

Detector System based on an idealised plane detector. Origin at closest point to crystal, not where beamhits the detector.

X-direction: direction of “fast” pixels of the detector

Y-direction: direction of “slow” pixels of the detector

Laboratory System origin at intersection of beam and crystal rotation axis; directions as with ideal de-tector at θ = 0◦.

Ewald Sphere System a “per reflection” co-ordinate system with its origin on the Ewald Sphere, whereintegration takes place. Based on incoming and reflected beam direction.

Data Integration with XDS Locating Spots

Locating Spots

1. find strong pixels on a sequence offrames

2. all strong pixels that can be directly con-nected make one spot

3. map onto Ewald sphere. This createsmore uniform spot shapes because it re-moves distortions due to θ angle andnon-perpendicular traversion through theEwald sphere.

4. assume Gaussian distributions in all threedimensions

5. find co-ordinates• X and Y co-ordinates as centroids• full reflection: Z as centre of frame• partial reflection: Z as weighted mean

of all frames. Weights are calculatedfrom presumed Gaussian distribution.


Locating Spots






of all frames


Locating Spots






of all frames

� ��

� ��

� ��

� ��

� ��

� ��

� ��

� ��

� � ��

5

7

4

3

2Inte

nsity

wei

ghts

Frame / degrees0° 1° 2° 3° 4°

Z =2 ∗ 0.5◦ + 5 ∗ 1.5◦ + 7 ∗ 2.5◦ + 4 ∗ 3.5◦ + 3 ∗ 4.5◦

2 + 5 + 7 + 4 + 3= 2.55◦

Data Integration with XDS XDS flow scheme

Flow scheme of XDS

Input parameterswavelengthgeometry

Integration

reduced cellorientation

improved parameters

Indexingoptional:

parameters

Correction

all Data

final set of parametersintegrated intensities

optional:re−integratewith improvedparameters

determinespace group

Datastrong spots

standard profiles3−D profile fitting

re−index to improve

XDS uses the measured reflection co-ordinates (X, Y, Z) and their indices (h, k, l) to refine the inputparameters for cell parameters, beam direction, rotation axis, detector distance.

Data Integration with XDS Setting up XDS

Setting up XDS

1. XDS has fixed names for input and output files⇒ every project should be run in separate directory⇒ rename log-files before test runs

2. path name where to find images limited to 50 characters

• work in directory “near” images or

• set link from working directory:

ln -s long_path_name_image_directory images

3. select and edit appropriate input file


Choosing the right templateThere is only one input file for XDS, XDS.INP . It contains all parameters required to run XDS.


Choosing the right template


Selection of important input parameters

XDS does not read any header information, everything must be given in the XDS.INP input parameter file

DIRECTION OF DETECTORY AXIS for detectors with adjustable θ angleNX, NY, QX, QY number of pixels, pixel widthORGX, ORGY detector origin, start with NX/2, NY/2OVERLOAD Maximum valid contents of an image pixel.

Set to 90% of actual overload value to remain in the linear region ofthe detector.

NAMETEMPLATEOF DATAFRAMES e.g. ./images/frame ???.mar2300 DIRECT MAR345

SPACEGROUPNUMBER Set to 0: Integration can be carried out in P1UNIT CELL CONSTANTS only required if space group specified

REFINE(IDXREF) Parameters that are to be refined duringREFINE(INTEGRATE) indexing, integration, final correctionREFINE(CORRECT) combination of: BEAM AXIS ORIENTATION CELL DISTANCE

The JOBcard tells XDS what to do:

JOB = XYCORR INIT COLSPOT IDXREF DEFPIX XPLAN INTEGRATE CORRECT

or

JOB = ALL

Data Integration with XDS Running XDS

Running XDS

In the directory containing XDS.INP , type xds or (on multiprocessor machines) xds par for parallelisedversion.

Every step of the JOBcard prints out a log file (ending with .LP ).

Some steps print control images files (ending with .pck , compressed CCP4 image format), that can (andshould!) be viewed with the program VIEW:

ABS.pck and BKGPIX.pck (DEFPIX) To set and check the VALUERANGEFORTRUSTEDDETECTORPIXELS .

Data Integration with XDS The JOBswitches

JOB = XYCORR INIT COLSPOT IDXREF DEFPIX INTEGRATE CORRECT

XYCORRSets up a look-up table with X-,Y- distortions per pixel in three possible ways:

1. calibration image with brass-plate and point source ( e.g. Fe55). Recommended for Siemens,Bruker, CCDD2AM detectors.

2. For MAR, MAC detectors (spiral read-out): ROFFand TOFFfor radial and tangential off-set.

3. otherwise set to 0 if no other information available, assuming that corrections are already in frames

INIT determines

1. background, either by given dark current image, by a fixed offset, or from the four corner points ofa few frame

2. gain, the variance of the background noise of each pixel. Determined from a box (default: 7 × 7

pixels) around the pixel

3. initial background, determined from a few frames (card BACKGROUNDRANGE).


JOB = XYCORR INIT COLSPOT IDXREF DEFPIXINTEGRATE CORRECT

COLSPOTFinds strong spots and their X-, Y-, Z- coordinates used for indexing. It maps spots to the Ewaldsphere at ϕ = ϕ0.

IDXREF Indexes reflections using the “local indexing method”. It avoids scaling up errors in the initialreciprocal basis.

1. find reciprocal basis that best explains differences of reflection vectors.

2. index differences of reflection vectors using the reciprocal basis.

It reports the 10 largest subtrees that could be indexed with the chosen basis, e.g.

SUBTREE POPULATION1 29562 23 14 15 16 17 18 19 1

10 1

DEFPIX Excludes shaded detector regions and region outside resolution range.



INTEGRATE Integrate reflections in two steps. Spots are mapped onto Ewald sphere and split into boxes9× 9× 9 pixels. Parameters are refined every certain number of frames (REFINE(INTEGRATE) ).

learn profiles from strong spots. Individual profiles are determined in 9 equal parts of the detectorand every 5◦. Only strong reflections that are close to their predicted location are taken intoaccount. Determines their profiles by fitting Gaussian distributions in all three directions.

7

4

1

8

5

2

9

6

3

1 2 3 4 5



INTEGRATE Integrate reflections in two steps. Spots are mapped onto Ewald sphere and split into boxes9× 9× 9 pixels. Parameters are refined every certain number of frames (REFINE(INTEGRATE) ).

learn profiles from strong spots. Individual profiles are determined in 9 equal parts of the detectorand every 5◦. Only strong reflections that are close to their predicted location are taken intoaccount.

measure intensities 1. get intensities around reflection locations (h, k, l) as predicted from cell pa-rameters.

2. remove spots from overlaps — pixels are assigned to nearest reflection

3. map all spot pixels into Ewald system and divide into 9× 9× 9 boxes.

4. determine intensity of each reflection by profile fitting: minimise

Ψ(I) =∑i∈D

(ci − I ∗ pi − bi)2/vi

where: ci =measured contentsI = intensity to be determinedpi =expected fraction in pixel i (from profile fitting)bi =background of pixelvi =variance of pixel, iteratively determined starting at vi = bi + Ipi


JOB = XYCORR INIT COLSPOT IDXREF DEFPIX INTEGRATECORRECT

The CORRECTstep applies correction factors to the measured intensities and their std. deviations topartially compensate for radiation damage and absorption effects. Reflections that do not follow the Wilsondistribution are excluded (the more are excluded the lower the WFACcard).

The CORRECTallows the determination of the space group:IDXREF.LP contains a table with the “Quality of Fit” of the given cell with the different Bravais-lattices,together with the required reindexing card (REIDX).

→ choose highest compatible bravais type and re-run CORRECTstep (i.e., change JOB=... to JOB

= CORRECTin XDS.INP . Choose correct space group based on statistics reported in CORRECT.LPbyseveral runs of this last step.

Or use the program xdsconv to convert XDS list of reflections XDSASCII.HKL to SHELX format anduse xprep . . .

Data Integration with XDS Other useful programs

Other programs that come with XDS

xscale Scaling program that applies Kabsch’s local scaling algorithm

xdsconv Conversion program from XDS HKL-format to:

1. shelx

2. cns (usefull for conversion to mtz-format)

3. mir

4. fall

xdsconv can also flag reflections as working and test set.

VIEW Program to view image files.

Data Integration with XDS Acknowledgments

Acknowledgements

I would like to thank Cornelius Zeth, Serge Cohen, and Raimond Ravelli for introducing me to XDS.

data integration with xds -...

Documents