jörg uhlenbrock universität hannover institut für meteorologie und klimatologie palm-seminar...

Jörg Uhlenbrock Universität Hannover

Institut für Meteorologie und Klimatologie PALM-Seminar 8.-14.7.2003

How to work with the „user-interface“-routine in PALM

Contents:

• Purpose of the user-interface

• Embedding in PALM

• Realisation

• Hints for new users



Purpose of the user-interface

• a standardarized model cannot count for all specific interests of the users, but provides the basic simulation and most common outputs

=> the user should be able to add special, non-standard features in an interface of the model WITHOUT changing the standard code

• standard version of PALM allows (excerpts):– forcing through time-invariant homogeneous or randomly distributed

surface heat flux or temperature

– output of time-series, 2D-slices, 3D binary data (AVS), time-averaged profiles and spectra of several meteorological variables

– prescription of time-invariant stratification (θ-profile)

– …



Purpose of the user-interface

• examples of extended features:– time-averaged 2D- and 3D-data– adaption of experimental approaches (e.g. adaption of coarse

tomography-grid, prescription of time-variant θ- and wind-profiles)– inhomogeneous surface forcing– …

Inhomogeneous surface forcing



Embedding in PALM

• the name of the user-interface is …/SOURCE/user_interface.f90 and provides the framework for the entry of user code

My recommendations:1. create an additional directory, e.g. „…/SOURCE/USER_CODE“2. create an additional directory named equal to the new run you want

to simulate, eg. „…/SOURCE/USER_CODE/example“ -> in .mrun.config there is a search path integrated, so that the model will find your user-files: %add_source_path $HOME/palm/current_version/SOURCE/USER_CODE/$fname

3. copy the original „…/SOURCE/user_interface.f90“ into this directory4. add your own user-code and use mrun with the option

-s „user_interface.f90“ (ensure that this file has write permits!)

(hint: „mrun –s TEST“ compiles all .f90-files with write-permits)



• to grant access to important parts of the model, there are several subroutines in the user-interface, which are called by the main program:

SUBROUTINE user_last_actions

Initialization:

Run-time:

Finishing:

SUBROUTINES user_parinuser_header

user_init

SUBROUTINESuser_actions (location)user_actions (location)

user_statistics

SUBROUTINEuser_init_particlesuser_particles_attrib

utesuser_dvrp_coltab



InitializationPALM

parin

init_pegrid

header

check_parameters

init_grid

read_var_list

user_parin

package_parin

user_header

init_3d_model

init_***

user_init

leap_frog

user_init_particles



Run-time leap_frogleap_frog

user_actions (before_timestep)

prognostic_equations_fastprognostic_equations

exchange_horiz asselin_filter

swap_timelevel boundary_conds

pres

prandtl_fluxes

diffusivities check_for_restart

flow_statistics run_control plot_***

poisfftsor poismg

timestep

poisfft_hybrid

1D topol.

standard advection

advec_particles user_particles_attribute

user_dvrp_coltab

user_statistics

user_actions(*-tendency)

user_actions (after_timestep)



Realisation

The framework: source code of user_interface.f90

Notepad



Realisation: location „user_actions“

Excerpt from leap_frog.f90:

• !• !-- Start of the time loop• DO WHILE ( simulated_time < end_time .AND. .NOT. stop_dt .AND. &• .NOT. terminate_run )

• CALL cpu_log( log_point_s(10), 'timesteps', 'start' )

• !• !-- Execute the user-defined actions• #if defined( __decalpha )• PRINT*, '+++ WARNING: leap_frog: no call of user_actions due to'• PRINT*, ' internal compiler error'• #else• CALL user_actions( 'before_timestep' )• #endif

• !• !-- Solve the prognostic equations. A fast cache optimized version with• !-- only one single loop is used in case of Piascek-Williams advection• !-- scheme. NEC vector machines use a different version, because• !-- in the other versions a good vectorization is prohibited due to• !-- inlining problems.• IF ( host(1:3) == 'nec' ) THEN• CALL prognostic_equations_vec• ELSE• IF ( impulse_advec == 'ups-scheme' .OR. &• scalar_advec == 'ups-scheme' .OR. &• scalar_advec == 'bc-scheme' ) &• THEN• CALL prognostic_equations• ELSE• CALL prognostic_equations_fast• ENDIF• ENDIF…..

…..!!-- Execute user-defined actions#if defined( __decalpha ) PRINT*, '+++ WARNING: leap_frog: no call of user_actions due to' PRINT*, ' internal compiler error'#else

CALL user_actions( 'after_timestep' )#endif

!!-- Determine size of next time step CALL timestep



Realisation: location „user_actions“

Excerpts from „prognostic_equations.f90“…!!-- Tendency terms for u-velocity component tend(:,j,i) = 0.0 IF ( bt == 2.0 ) THEN CALL advec_u_pw( i, j ) CALL diffusion_u( i, j, ddzu, ddzw, km_m, tend, u_m, usws_m, v_m, & w_m ) ELSE CALL advec_u_up( i, j ) CALL diffusion_u( i, j, ddzu, ddzw, km, tend, u, usws, v, w ) ENDIF CALL coriolis( i, j, 1 ) IF ( sloping_surface ) CALL buoyancy( i, j, pt, 1, 4 )

CALL user_actions( i, j, 'u-tendency' )

!!-- Prognostic equation for u-velocity component DO k = nzb+1, nzt u_p(k,j,i) = ( 1-at ) * u_m(k,j,i) + at * u(k,j,i) + & bt * dt_3d * tend(k,j,i) - & dt_3d * ( p(k,j,i) - p(k,j,i-1) ) * ddx - & bt * rdf(k) * ( u(k,j,i) - ug ) ENDDO

!!-- Tendency terms for v-velocity component tend(:,j,i) = 0.0 IF ( bt == 2.0 ) THEN CALL advec_v_pw( i, j ) CALL diffusion_v( i, j, ddzu, ddzw, km_m, tend, u_m, v_m, vsws_m, & w_m ) ELSE CALL advec_v_up( i, j ) CALL diffusion_v( i, j, ddzu, ddzw, km, tend, u, v, vsws, w ) ENDIF CALL coriolis( i, j, 2 )

CALL user_actions( i, j, 'v-tendency' )……

For the call of tendency-terms on the ibm-regatta, a FORTRAN95-functionality called „function-overloading“ is used to allow different argument lists for the same SUBROUTINE, e.g.:

CALL user_actions( ‘before_timestep‘ )

in leap_frog.f90, but

CALL user_actions(i, j, ‘u-tendency‘)

in prognostic_equations.f90



Realisation: prerequisites for „function overloading“

• the subroutines using this technique have to be embedded in a modular programming structure:

MODULE user_actions_mod

!-------------------------------------------------------------------------------!!! Description:! ------------! Execution of user-defined actions before or after single timesteps!-------------------------------------------------------------------------------!

PRIVATE PUBLIC user_actions

INTERFACE user_actions MODULE PROCEDURE user_actions MODULE PROCEDURE user_actions_ij END INTERFACE user_actions

CONTAINS……

……!------------------------------------------------------------------------------!! Call for all grid points!------------------------------------------------------------------------------! SUBROUTINE user_actions( location ) …. END SUBROUTINE user_actions

!------------------------------------------------------------------------------!! Call for grid point i,j!------------------------------------------------------------------------------! SUBROUTINE user_actions_ij( i, j, location ) … END SUBROUTINE user_actions_ij

END MODULE user_actions_mod



Realisation: change of DO-loop

• A fast method for computing on the IBM is realized through the change of DO-Loop structure. The call of SUBROUTINES is faster than the direct computation of the arrays within a loop.

SUBROUTINE user_actions DO k = …

DO j = … DO i = … array1(k,j,i) = array2(k,j,i) + SIN(i) ENDDO ENDDO

ENDDOEND SUBROUTINE user_actions

is changed to:SUBROUTINE prognostic_equations

DO j = … DO i = …

CALL SUBROUTINE user_actions(i, j, location) ENDDO

ENDDOEND SUBROUTINE prognostic_equations

SUBROUTINE user_actions(i, j, location)DO k = … array1(k,j,i) = array2(k,j,i) + SIN(i)ENDDO

END SUBROUTINE user_actions



Final stepsPALM

leap_frog

write_3d_binary

header

user_last_actions

cpu_statistics

write_var_list



Hints for new users

• Most important:

IF NOT SPECIFIED, ALL OF YOUR ADDED COMMANDS ARE EXECTUED BY ALL PEs!!!!

• Not less important:

YOUR ENTRIES IN THE TENDENCY-LOCATIONS OF SUBROUTINE „USER_ACTIONS“ ACTUALLY ARE IN A TWOFOLD LOOP OF INDICES „i“ and „j“ AND THEREFORE WILL BE COMPUTED i*j TIMES BY EACH PE!!!!

In most cases, this is only intended in a computation with at least 2D-arrays in order to compute each element of the array.

• Use the MODULE „user“ for the declaration of variables and arrays, etc. This leads to a more transparent code.

• Use SUBROUTINES „user_init“ and „user_last_actions“ to restore and save your data for restart_runs.



Hints

in „user_last_actions“:

!– write fields for restart run

IF ( write_binary(1:4) == 'true' ) THEN

WRITE (14) 'example_array '; WRITE (14) example_array

WRITE (14) 'example_integer '; WRITE (14) example_integer

!-- Set an end mark

WRITE ( 14 ) '** end_user_entry ** '

ENDIF<= UNIT (14): BINOUT,

all necessary model parameters are stored in here



Hints

Example:

in „user_init“: IF …

ELSEIF ( TRIM( initializing_actions ) == 'read_fields_from_prior_run' ) THEN!!-- Read data frtom prior run. READ ( 13 ) field_chr DO WHILE ( TRIM( field_chr ) /= '** end_user_entry **' )

SELECT CASE ( TRIM( field_chr ) ) CASE ('example_array') READ (13) example_array CASE ('example_integer') READ (13) example_integer CASE DEFAULT PRINT*, '+++ user_init: unkown field "', & TRIM( field_chr ), '" find in' PRINT*, ' the data of prior run on PE ', myid STOP END SELECT!!-- Read next character line READ ( 13 ) field_chr

ENDDO ENDIF !initializing_actions

<= UNIT (13): BININ,

all necessary model parameters are returned from here



The End

Any questions?



Embedding in PALM: subroutine calls in the main program

SUBROUTINE user_last_actions

SUBROUTINES user_parinuser_header

user_init

SUBROUTINESuser_init_particles

user_particles_attributeuser_dvrp_coltab

SUBROUTINESuser_actionsuser_actions

user_statistics

call in main program: parin.f90header.f90

init_3d_model.f90

call in main program: init_particles.f90

advec_particles.f90plot_dvrp.f90

call in main program: leap_frog.f90leap_frog.f90

(for locations „before/after timestep“)prognostic_equations.f90 prognostic_equations.f90

(for locations „u,v,w,pt,e,q-tendency“)flow_statistics.f90

call in main program: palm.f90

jörg uhlenbrock universität hannover institut für meteorologie und klimatologie palm-seminar...

Documents