Uriel Zajaczkovski

Data Visualization with Paraview

Scripps Institution of OceanographyFeb-11 2013

VTK and Paraview

VTK (Vizualization ToolKit)

The graphical output is hardware-accelerated by using the OpenGL library

Written in C++ but there also bindings for scripting languages like Python and Tcl

Graphical user interface based on VTK

VTK + Paraview

User-defined pipeline which passes the data from the source (input file) to the sink (image on the screen or output file) through several filters.

Paraview

Paraview can handle large datasets via multi-core processing (MPI library / cluster)

Object-oriented high-level library for data processing and visualization

Example: Southern Ocean SST, SSH and Sea Ice concentration (SOSE)

Example: Southern Ocean velocity field

(I) Prepare the data

(II) Work in Paraview

(III) Create the animation

Typical workflow

- Use MATLAB and/or NCO (NetCDF operators) to write the data in netCDF format- Use a scripting language to automate the task (Perl, Python, etc)

(a) Wrap the images in a video container format - Sequimago (free), Quicktime (not free), etc

(b) Compress the video: - ffmeg (http://www.ffmpeg.org/) - handbrake (http://handbrake.fr/)

- Apply filters (Calculator, Cell data to Point Data, Contour, etc)- Use the Animation Interface to create the desired motion- Save the output as images (PNG)

fhdr = '~/data/SOSE/';disp('loading data...')ini = 1; fin = 219;U (1:2160,1:320,1:fin-ini+1) = 0;V (1:2160,1:320,1:fin-ini+1) = 0;for i=ini:fin Utemp = rdda([fhdr 'iter60/UVEL.0000000060.data'],[2160 320 42],i,'real*4'); U(:,:,i-ini+1) = Utemp(:,:,1); Vtemp = rdda([fhdr 'iter60/VVEL.0000000060.data'],[2160 320 42],i,'real*4'); V(:,:,i-ini+1) = Vtemp(:,:,1);endU(U==0) = -99; V(V==0) = -99;load([fhdr 'grid.mat']);

ncid = netcdf.create('SOSE_VEL_5day_219rec.nc','CLOBBER'); lon_map = squeeze(XC(:,1));lat_map = squeeze(YC(1,:));timevec = [ini:fin]; nx = length(XC(:,1));ny = length(XC(1,:));nts = length(timevec);

dimidcol = netcdf.defDim(ncid,'lon',nx);dimidrow = netcdf.defDim(ncid,'lat',ny);dimidt = netcdf.defDim(ncid,'time',nts); LonID = netcdf.defVar(ncid,'lon','double',dimidcol);LatID = netcdf.defVar(ncid,'lat','double',dimidrow);TimID = netcdf.defVar(ncid,'time','double',dimidt);uID = netcdf.defVar(ncid,'U','NC_DOUBLE',[dimidcol dimidrow dimidt]);vID = netcdf.defVar(ncid,'V','NC_DOUBLE',[dimidcol dimidrow dimidt]);

netcdf.putAtt(ncid,LonID,'long_name','Longitudes')netcdf.putAtt(ncid,LonID,'units','degrees_east')netcdf.putAtt(ncid,LatID,'long_name','Latitudes')netcdf.putAtt(ncid,LatID,'units','degrees_north')netcdf.putAtt(ncid,uID,'long_name','UVEL');netcdf.putAtt(ncid,vID,'long_name','VVEL');netcdf.putAtt(ncid,uID,'_FillValue',-99)netcdf.putAtt(ncid,vID,'_FillValue',-99)

netcdf.endDef(ncid); netcdf.putVar(ncid,LonID,lon_map)netcdf.putVar(ncid,LatID,lat_map)netcdf.putVar(ncid,TimID,timevec)netcdf.putVar(ncid,uID,U);netcdf.putVar(ncid,vID,V); netcdf.close(ncid);

MATLAB NetCDF package

Create new NetCDF file

Create NetCDF dimensions

Create NetCDF variables

Write NetCDF attributes

End NetCDF file define mode

Write data to NetCDF variables

Close NetCDF file

NCO (NetCDF Operators)

Set of command line (scriptable) utilities that operate on netCDF files

Source: http://nco.sourceforge.net/Binaries also available for OS X / WindowsAlternatively install package via Fink or MacPorts

• ncap2 netCDF Arithmetic Processor

• ncatted netCDF ATTribute EDitor

• ncbo netCDF Binary Operator (includes ncadd, ncsubtract, ncmultiply, ncdivide)

• ncea netCDF Ensemble Averager

• ncecat netCDF Ensemble conCATenator

• ncflint netCDF FiLe INTerpolator

• ncks netCDF Kitchen Sink

• ncpdq netCDF Permute Dimensions Quickly, Pack Data Quietly

• ncra netCDF Record Averager

• ncrcat netCDF Record conCATenator

• ncrename netCDF RENAMEer

• ncwa netCDF Weighted Averager

Dealing with Time

ncecat -O -h in.nc out.nc

ncpdq -O -h -a time,record out.nc out.nc

ncpdq: Permute dimensions-a reorder

ncwa -O -h -a record out.nc out.nc

Weighted averager“-a record”: average variables over record dimension

source: NCO user’s guide (http://nco.sourceforge.net/nco.html#dmn_rcd_mk)

(I) Adding a record dimension to a file

Dealing with Time

In the COARDS convention time is identified by the units attribute on the time variable. The ParaView reader looks for a string of the form "<time length> since <date>" or "<time length> until <date>".

We can add this string with ncap2:

ncap2 -O -s \"time\@units=\\\"days since YYYY-MM-DD HH:MM:SS\\\"\" input.nc output.nc

(II) Making Paraview aware of the time variable

#!/usr/bin/perl -w @files = </Users/uriel/data/Ssalto.Duacs.h/msla/h/*>;

foreach $file (@files) { $n = length($file);

$c1 = substr($file,0,$n-3) . "_c1.nc"; $c2 = substr($file,0,$n-3) . "_c2.nc"; $c3 = substr($file,0,$n-3) . "_c3.nc"; $c4 = substr($file,0,$n-3) . "_c4.nc"; $c5 = substr($file,0,$n-3) . "_c5.nc"; # define time dimension (see param.nco) $cmd="ncap2 -A -v -S param.nco $file"; print "$cmd \n"; system($cmd);

# define ssha variable $cmd="ncap2 -O -s 'ssha[\$time,\$NbLongitudes,\$NbLatitudes]=0.0' $file $c1"; print "$cmd \n"; system($cmd);

# copy values from Grid_0001 $cmd="ncap2 -O -F -s 'ssha(1,:,:)=Grid_0001(:,:)' $c1 $c2"; print "$cmd \n"; system($cmd);

# the next 3 commands set the time as the record dimension $cmd="ncecat -O -h $c2 $c3"; print "$cmd \n"; system($cmd);

$cmd="ncpdq -O -h -a time,record $c3 $c4"; print "$cmd \n"; system($cmd);

$cmd="ncwa -O -h -a record $c4 $c5"; print "$cmd \n"; system($cmd); # delete intermediate files $cmd="rm *_c1.nc *_c2.nc *_c3.nc *_c4.nc"; print "$cmd \n"; system($cmd); }

Using scripting languages to automate the taskexample: Aviso MSLA

NetCDF and Paraview

When NetCDF files are open in Paraview they are interpreted as Structured (curvilinear) grid and they are mapped to a sphere.

- Cell data to Point Data

The workflow in Paraview consists of a user-defined pipeline which passes the data from the source (input file) to the sink (image on the screen or output file) through several filters:

Some of the most common filters are:

data source filter(s) display/rendered

DATA

- Calculator

- Threshold, Extract subset, Contour...

The Calculator filter: Coordinate transformation

We can use the calculator filter to transform our data from spherical coordinates to whatever projection we want.

For example to visualize out data in a polar stereographic projection we can apply the following transformation:

The syntax for the Calculator filter:

iHat*cos(coordsY)*sin(coordsX)-jHat*sin(coordsY)

(x, y) = cos(y) ⇤ sin(x)x� sin(y)y

The Calculator filter: Exaggerated topography

(1 + (depth/6370000) * factor) * ( iHat * cos(asin(coordsZ)) * cos(atan(coordsY/coordsX)) * coordsX/abs(coordsX) + jHat * cos(asin(coordsZ)) * sin(atan(coordsY/coordsX)) * coordsX/abs(coordsX)+ kHat * coordsZ )

In the syntax of the calculator filter:

cos(sin�1Z) ⇤ cos(tan�1

Y ⇤X�1) ⇤X ⇤ |X|�1 ⇤ x(x, y, z) = (1 + Z ⇤R�1 ⇤ f)⇤+y ⇤ cos(sin�1

Z) ⇤ sin(tan�1Y X

�1) ⇤X ⇤ |X|�1 +z ⇤ Z)

Bugayevskiy & Snyder “Map Projections: A Reference Manual”

Frederick Pearson: Map Projections: Theory and Applications 

?