soil acidity adaptive control problem

ORIGINAL PAPER

Soil acidity adaptive control problem

Victor P. Yakushev • Vladimir V. Karelin •

Vladimir M. Bure • Elena M. Parilina

� Springer-Verlag Berlin Heidelberg 2014

Abstract Problem of soil acidity regularization is mod-

eled as stochastic adaptive control problem with a linear

difference equation of the dynamics of a field pH level.

Stochastic component in the equation represents an indi-

vidual time variability of soil acidity of an elementary

section. We use Bayesian approach to determine a poste-

riori probability density function of the unknown parame-

ters of the stochastic transition process. The Kullback–

Leibler information divergence is used as a measure of

difference between true distribution and its estimation.

Algorithm for the construction of an adaptive stabilizing

control in such a linear control system is proposed in the

paper. Numerical realization of the algorithm is repre-

sented for a problem of a field soil acidity control.

Keywords Soil acidity and liming � Stochastic control �Adaptive control

1 Introduction

The necessity of control of a complex object arises with the

expansion of the computer technology applications. In

most cases, the complex objects do not have appropriate

mathematical models, and principles describing objects

behavior are not completely known. The new approach of

solving control problems is based on the idea which is in

adapting control system to the properties of a particular

object. The knowledge about the object is limited by a

priori class of objects which it belongs to. We consider this

type of a control problem.

Following Nebolsin and Nebolsina (2010), Shilnikov

et al. (2003), soil liming is a very effective method of

fertility increase. The main goal of liming is the elimina-

tion of extra soil acidity. Soil acidity is determined by pH

level, and pH from interval 6.3–6.5 is usually considered as

standard Nebolsin and Nebolsina (2010), Shilnikov et al.

(2003). In Shilnikov et al. (2003) it is noticed that in soils

with standard pH level radionuclide and heavy metal

infiltration into the plants is reduced in 3-8 times, and its

succeeding infiltration into groundwater is also reduced in

several times. In soils with standard pH level (pH 6.3-6.5)

the plants productivity significantly increases. Without

supporting liming, soil acidity increases over time (pH

level decreases), and soil degradation takes place (see

Nebolsin and Nebolsina (2010), Shilnikov et al. (2003)).

The most typical soil pH level in the north-western region

of Russian Federation is from 4.0 to 6.5. Following the

studies (http://pss.uvm.edu/ppp/pubs/oh34.htm) we repre-

sent the appropriate soil pH level for gathering different

agricultural plants which are traditionally grown in the

north-western region of Russian Federation (see Table 1).

In the studies of soils in different European countries

(e.g. England, Germany) the phenomenon of a drastic

V. P. Yakushev � V. M. Bure

Agrophysical Research Institute, Grazhdansky pr. 14,

Saint Petersburg 195220, Russia

e-mail: [email protected]

V. M. Bure


V. V. Karelin � V. M. Bure � E. M. Parilina (&)

Faculty of Applied Mathematics and Control Processes, Saint

Petersburg State University, Universitetskii pr. 35, Petergof,

Saint Petersburg 198504, Russia


V. V. Karelin


123

Stoch Environ Res Risk Assess

DOI 10.1007/s00477-014-0965-5

http://pss.uvm.edu/ppp/pubs/oh34.htm

change in the soil pH level has been repeatedly noted, and

this situation leads to the necessity of using the expensive

methods to normalize this very important agrochemical

indicator. In Shilnikov et al. (2003) it is shown that the rate

of soil acidity increase (pH level decrease) depends on a

particular pH level, i.e. the higher pH level, the relatively

quicker this level decreases. The average annual reduction

of the pH level or average annual increase of soil acidity

are usually studied in the literature. In accordance with this,

we suggest to describe the dynamics of the pH level

approximately by the stochastic linear difference equation

of the first order, and the presence of stochastic variable in

the equation is due to the soil heterogeneity. Given

‘‘inevitable variability’’ of the soil, plants, micro-organisms

Nebolsin and Nebolsina (2010), we may talk about the pH

level distribution for the entire field, while we assume that

the normal value of acidity is known Shilnikov et al. (2003)

and is denoted by K. Following the approach proposed in

Yakushev et al. (2012), Yakushev et al. (2013), assume that

the field is divided into some elementary sections, wherein

the pH level is the same inside of an elementary section

(Fig. 1), but there is a difference among elementary areas,

and this difference leads to the necessity of considering

stochastic component. Therefore, the existing variability in

elementary section acidities over time is supposed to be

modeled by considering stochastic variable. Therefore, we

describe the difference between the normal value of acidity

K and the measured level of soil acidity (pH level) at time

(year) t by stochastic controlled process xt.

There is a range of literature devoted to the management

problems of environmental indicators with stochastic

component. In most cases they consider a statistical model,

for example, regression model describing the relation

between some agrophysical index and several factors, e.g.

in Kros et al. (2002) the regression model of soil acidifi-

cation is constructed, where surface is divided into non-

overlapping areas, each area has its own regression model

coefficients.

In the paper Bole et al. (2012) authors propose a

dynamic stochastic model of a defect growth and future

environmental stresses which is represented by a general-

ized Markov process. The dynamic model of soil acidity

management where the dynamics of soil acidity depends on

a set of some soil physical and biochemical indicators is

considered in Schmieman et al. (2002). In the paper Boland

et al. (2010) authors represent a stochastic model of deci-

sion-making with different criteria (discounted expected

utility and expected average utility) and discuss the prob-

lem of optimal control using these criteria, and also provide

a model of using solar panels for water desalination.

Game-theoretic modelling can be used as an alternative

approach to find the ways of the management of environ-

mental indicators in a case there is participants’ conflict in

the considered problem (e.g. see Jorgensen et al. (2010)).

This approach is mostly often used for modelling regula-

tion of environmental policies at the level of large com-

panies or countries.

Close to the current paper work Guo et al. (2012) is

devoted the adaptive optimal control problem for discrete-

time, infinite horizon, stochastic control ‘‘non-parametric’’

systems where stochastic variables are i.i.d. random ele-

ments with unknown distribution. In the paper Hernandez

Lerma and Marcus (1987) authors consider the first passage

problem (maximization of the expected total cost before the

first period when ‘‘trajectory’’ enters a given target set) for

discrete-time, non-stationary, stochastic control systems.

Table 1 Appropriate pH level

of a soil for selected plants

grown in the north-western

region of Russian Federation

Plant Soil pH level

Apple 5.5–6.5

Plum 6.5–8.5

Strawberry 5.5–6.5

Broccoli 6.0–7.0

Cabbage 6.0–7.5

Carrots 5.5–7.0

Cucumbers 5.5–7.0

Onions 6.0–7.0

Potatoes 4.8–6.5

Tomatoes 5.5–7.5

Fig. 1 A field divided into elementary sections with the different soil

pH levels


123

The paper is organized as follows. In Sect. 2 we state the

agricultural problem and consider the properties of con-

trolled stochastic process. Then, in Sect. 3 we suggest a

model of adaptive optimal control and algorithm of con-

structing adaptive stabilizing control. Main theoretical

results are also represented in Sect. 3. Section 4 contains a

numerical example illustrating the work of the algorithm.

2 Model

Let y be a field soil pH level. Let K be the field standard pH

level taking into account the agricultural crops that are

planned to be planted in the field. The standard pH level

should be set by an agriculture specialist of a particular

region. Value x ¼ K � y represents the deviation of the

true pH level from the standard pH level. Assume that the

deviation increases over time, i. e. the average pH level

decreases, time changes discretely and one time period

lasts six months. We describe the dynamics of the field pH

level by stochastic difference equation with control

variable:

xtþ1 ¼ ð1þ aÞxt þ but þ ftþ1; ð1Þ

where a; b are unknown parameters (specific characteristics

of a field), a [ 0, fftg are i.i.d. random variables repre-

senting Gaussian white noise with zero mean and standard

deviation r, ut is a control variable in time period t rep-

resenting a dose of some contributed fertilizer.

We use function

J ¼ limT!1

1

T

XT

t¼1

Eðx2t Þ: ð2Þ

as a reward criterion. Minimization of criterion (2) has the

following practical meaning. A control is chosen to stabi-

lize the field acidity around the standard acidity level.

Parameters a and b are specific for the field and unknown,

i. e. their evaluation must be carried out in parallel with the

control choice, and this fact complicates the problem

solution.

The described argicultural problem can be modeled

using the stochastic optimal control methods. Controlled

stochastic process xt is completely defined if the transition

distribution of the process is known Aoki (1989), Dinkin

and Ushkevich (1975):

Pðxtþ1jxt; utÞ; ð3Þ

where xt ¼ fx1; x2; . . .; xtg is a history of the first t time

periods, and also when initial distribution Pðx1Þ and some

admissible strategy ut ¼ ðu1; . . .; utÞ are known. If transi-

tion function (3) depends on the last known values xt and

ut, then it is called Markov transition function. If we use

Markov strategies in Markov transition function of con-

trolled process, then the process is a Markov process. In

applied problems, transition function may depend on the

unknown parameter h 2 H, i. e. transition distribution is as

follows:

Pðxtþ1jxt; ut; hÞ:

At the same time, the knowledge of parameter h may be not

necessarily needed for control selection, but it is advisable

if the distribution estimation is close to the true distribu-

tion. Moreover, the measure of closeness of the true

probability measure P2 and its estimation P1 can be

introduced in the sense of the following metric:

qðP1;P2Þ ¼ supfAig

X

i

jP1ðAiÞ � P2ðAiÞj: ð4Þ

We propose to consider the measure of information dif-

ference between the true distribution P2ðAiÞ and its esti-

mation P1ðAiÞ in terms of Kullback-Leibler information

divergence Kullback and Leibler (1951):

IðP1; P2Þ ¼ Iðp1; p2Þ ¼Z

Y

lnp1ðyÞp2ðyÞ

� �p1ðyÞlðdyÞ; ð5Þ

where P1ðyÞ and P2ðyÞ are probability distributions defined

on measurable space Y with positive measure lðdyÞ, and

p1ðyÞ, p2ðyÞ are two corresponding probability density

functions.

We may notice that value IðP1; P2Þ from (5) is not a

metric, but it has some properties allowing to use it as a

measure of difference between P2ðAiÞ and P1ðAiÞ. If the

integral in (5) exists, then IðP1; P2Þ � 0, and IðP1; P2Þ ¼0 if and only if P1 ¼ P2 almost everywhere relatively to

lðdyÞ. And the following inequality holds Karelin (1991):

IðP1; P2Þ �1

8

Z

Y

jp1ðyÞ � p2ðyÞjlðdyÞ

2

4

3

52

: ð6Þ

Therefore, an upper bound of metrics (4) is 2ffiffiffi2p

I, and

convergence I ! 0 is sufficient for convergence q! 0.

The following useful property is proved for the family of

normal distributions. Let P0ðxÞ is a probability distribution

in Rn with mean a and covariance matrix V , and p0ðxÞ is a

corresponding probability density function. Consider the

family of normal distributions Q in Rn, and denote normal

probability density function with mean a and covariance

matrix V by q0ðxÞ.

Theorem 1 (See Karelin 1991) The inequality Iðp0; q0Þ� Iðp0; qÞ holds for any qðxÞ 2 Q. And Iðp0; q0Þ ¼Iðp0; qÞ iff q � q0.

Denote the conditional distribution of process xt which

depends on the unknown parameter h 2 H as Pðxtþ1jxt; hÞ.


123

We consider Bayesian approach, in which some distribu-

tion mðdhÞ on ðh;IÞ of the unknown parameter h is sug-

gested as a priori distribution of unknown h. Denote

transition distribution function Pðxtþ1jxt; hÞ by Phðxtþ1jxtÞwith transition probability density function phðxtþ1jxtÞ. Let

mðdhÞ belong to a family of @ which is dominated by a

measure nðdhÞ. Denote density mðdhÞ relatively to nðdhÞ as

mðhÞ. Suppose that uðhÞ is a measurable function of

parameter h 2 H, then Bayesian estimation of this function

is

�uðxtÞ ¼Z

H

uðhÞmtðhjxtÞnðdhÞ;

where mtðhjxtÞ is a posteriori probability density function of

parameter h of the following form:

mtþ1ðhjxtþ1Þ ¼ mtðhjxtÞ phðxtþ1ÞRH

phðxtþ1ÞmtðhjxtÞnðdhÞ : ð7Þ

If we consider partly observed Markov process xt in the

phase space H� X, it can be determined by phðxtþ1jxtÞ.Suppose that xt can be observed but the probability density

function ktðhÞ is unknown. The transition distribution

function of the process fxt; ktg by expressions:

Pðxtþ1jxt; ktÞ ¼Z

H

phðxtþ1jxtÞktðhÞnðdhÞ; ð8Þ

ktþ1ðhÞ ¼ ktðhÞphðxtþ1jxtÞ

Pðxtþ1jxt; ktÞ: ð9Þ

To find an estimation of function ktðhÞ we suggest the

following scheme. We consider space fX;@g where @ is a

space of distributions on H as phase space which is a Borel

space. Define the transition distribution function on space

fX;@g by the following way:

Pðxtþ1jxt; mtÞ ¼Z

H

phðxtþ1jxtÞmtðhjxtÞnðdhÞ; ð10Þ

where the probability density function mtðhjxtÞ is defined by

(7) and takes the form:

mtþ1ðhjxtþ1Þ ¼ mtðhjxtÞphðxtþ1jxtÞ

Pðxtþ1jxt; mtðhjxtÞÞ: ð11Þ

The initial distribution P1ðx1Þ and a priori probability

density function m1ðhÞ ¼ dm1=dn are given to set the pro-

cess. As an estimation of function ktðhÞ select distribution

mtðhÞ defined by recurrent expression (11) with any initial

value m1ðhÞ.Consider the asymptotic properties of the sequence mt,

t ¼ 1; 2; . . .. In the paper Karelin (1991) the fact that

transition function Pðxtþ1jxt; mtÞ becomes close to

Pðxtþ1jxt; ktÞ when t goes to infinity is verified. Therefore,

we may state the convergence of the transition function to

unstable transition function Phðxtþ1jxtÞ. If the set H is

finite, the neighborhood of the point hi coincides with the

point hi (any discrete topology may be chosen). In this

case, the integrals take the form of finite or infinite sums,

and functions mtðhÞ are converted into probabilistic

sequences fmtðhiÞg, i ¼ 1; 2; . . .

3 Adaptive optimal control problem

Consider the linear difference equation of the nth order:

xtþ1 ¼Xn�1

k¼0

akxt�k þXm

k¼0

bkut�k þ ftþ1; ð12Þ

where ak, bk are constants, b0 6¼ 0, fftg is a sequence of

independent normally distributed random variables with

zero mean and standard deviation r, and ut is a control or

strategy.

Assume that parameters bk are observed and xt, ut are

known, but parameters ak are unknown. Assume that the

system defined by (12) is minimum-phase, i. e. the roots of

polynomial

bðkÞ ¼Xm

k¼0

bkkk

are inside the unit circle. The quality of the control can be

estimated by criterion (2). Admissible controls are any

measurable functions of the history, i.e. ut ¼ wtðxtÞ. Show

that strategy w defined by equation:

w ¼ arg minut

Efhðxtþ1; mtþ1jxt; mt; utÞg þ ~qðxt; mt; utÞ½ �;

ð13Þ

for some general scheme provides mean square stabilization

of the trajectory given by equation (12). Let mtðhÞ be the

probability density function of a random normally distributed

vector with vector mean ht and covariance matrix St. Denote

the phase vector of equation (12) by zTt ¼ ðxt; xt�1; . . .; xt�nÞ

and rewrite equation (12) in the following way:

ztþ1 ¼ Azt þ b hT zt þXm

k¼0

bkut�k þ ftþ1

!; ð14Þ

where

A ¼

0 0 . . . 0 0

1 0 . . . 0 0

. . .0 0 . . . 0 1

0BB@

1CCA; b ¼

1

0

..

.

0

0BB@

1CCA;

Assume that quadratic form VðztÞ of vector zt takes the

form:


123

VðztÞ ¼ nx2t þ ðn� 1Þx2

t�1 þ ::: þ x2t�nþ1: ð15Þ

Compute conditional mathematical expectation EfVðztþ1Þjzt; mt; utg: The above discussion implies that to find the

conditional distribution of vector ztþ1 with Bayesian

approach, it is necessary to consider h in formula (14) as a

stochastic vector with distribution mtðhÞ. Therefore,

EfVðztþ1Þjztg ¼ VðbÞEf 2tþ1 þ

Z

H

VðAzt þ bðhT zt

þXm

k¼0

bkut�kÞÞmtðhÞdh ¼ VðbÞEf 2tþ1 þ VðAzt þ bðhT zt

þXm

k¼0

bkut�kÞÞ þ VðbÞZ

H

jðh� htÞT ztj2mtðhÞdh

where

ht ¼Z

H

hmtðhÞdh: ð16Þ

It is clear thatZ

H

jðh� htÞT ztj2mtðhÞdh ¼ zTt Stzt;

St ¼Z

H

ðh� htÞðh� htÞTmtðhÞdh:

Making notations: VðbÞ ¼ q, Ef 2tþ1 ¼ d2, we finally get:

EfVðztþ1Þjxtg ¼ qd2 þ qzTt Stzt

þ V Azt þ bðhTt zt þ

Xm

k¼0

bkut�kÞ !

:

ð17Þ

Given form (15) of VðzÞ we can prove that

VðAzt þ bvÞ þ z2t ¼ VðzÞ þ nv2

VðbÞ ¼ n

Rewrite (17) in the following way:

EðVðztþ1ÞjxtÞ þ ðz2t � qzT

t StztÞ

¼ n d2 þ jhTt zt þ

Xm

k¼0

bkut�kj2 !

þ VðztÞ ð18Þ

and, finally, it proves the theorem.

Theorem 2 Control futg satisfying equation

Xm

k¼0

bkut�k ¼ �hTt zt

minimizes the reward criterion

JT ¼ Em

XT�1

t¼0

zTt ðI � ST nÞzt þ VðzTÞ

( )

for each T , and infu

JT ¼ nTd2 þ Vðz0Þ, where ht ¼RH

hmtðhÞdh; d2 ¼ Ef 2tþ1 and St ¼

RHðh� htÞðh� htÞ0mtðhÞdh.

Corollary 1 Let the inequality I � qST [ �0I be true for

some t. Then, there exists q, 0 \ q\ 1, that the following

inequality holds:

E Vðztþ1Þjzt; mtf g � qVðztÞ þ nd2:

Corollary 2 Let kðhÞ be a priori probability density

function satisfying condition: kðhÞ � CmðhÞ. Then, the

following inequality holds:

Ek

XT

t

z2t

!� CEm

XT

t

z2t

!:

Theorem 3 LetPm

k¼0 bkut�k ¼ �hTt zt and mtðhÞ be a

sequence of a posteriori probabilities that corresponding to

the normal a priori distribution mtðhÞ, in addition, nS1\I.

Let kðhÞ is a probability density function concentrated in a

limited region, satisfying condition: kðhÞ\CmðhÞ. Then

the inequalities hold:

Efz2t g � const; Efu2

t g � const:

Proof Since the sequence of matrices St monotonically

decreases, the inequality is satisfied:

I � nSt � I � nS0 � e0I

for very small e0. Therefore,

EVðztþ1Þ\qEVðztÞ þ nd2;

where q\1. Hence, Ez2t \const. The boundedness of

Efu2t g can be proved in a similar way. h

Now we consider the special case of equation (12):

xtþ1 ¼ axt þ but þ ftþ1; ð19Þ

where a, b are unknown parameters, fftg is i.i.d. Gaussian

variables with zero mean and standard deviation r, ut is a

control variable. And the relations take the form of finite

sums, and functions mt are the stochastic sequences mðiÞt ,

i ¼ 1; 2; . . . We can prove the following theorem about

optimal control for system (19).


123

Theorem 4 Consider the stochastic system which

dynamics determined by (19). Reward criterion has form

(2). The optimal control for the problem is as follows:

ut ¼ �Pn

j¼1 ajbjmjtPn

j¼1 b2j m

jt

xt; ð20Þ

where mjtþ1 satisfies the recurrent formula:

m jtþ1 ¼ m j

t

uðxtþ1 � ajxt � bjutÞPnk¼1 uðxtþ1 � akxt � bkutÞmk

t

: ð21Þ

4 Numerical simulations

In this section we present a numerical simulation of the

method suggested in Sect. 3, specifically, in Theorem 4.

Consider stochastic difference equation describing the

dynamics of a field pH level:

xtþ1 ¼ ð1þ aÞxt þ but þ ftþ1; ð22Þ

where a; b are unknown parameters, a [ 0, fftg are i.i.d.

random variables representing Gaussian white noise with

zero mean and standard deviation r, ut is a control variable

in time period t representing a dose of some contributed

fertilizer. State variable xt represents the deviation of a soil

pH level from the standard pH level at time period t. The

reward criterion has form (2).

Vector of unknown parameters of system defined by

(22) is ða; bÞ. Applying Bayesian approach we assume that

a priori distribution of the unknown parameters is intro-

duced. We make a discretization of the range of unknown

parameters determining the finite grid in which nodes are

considered as possible values of unknown parameters.

Then we discuss the results of the algorithm work in two

ways. First, we suppose that the unknown true values of

parameters ða; bÞ coincide with some grid node (Option 1).

Second, the true values of parameters ða; bÞ do not coincide

with any grid nodes (Option 2).

The results of the algorithm work are summarized in two

tables (Tables 2 and 3). We set the initial data: x1 ¼ 2. Sto-

chastic components fftg have normal distribution with zero

mean and a standard deviation r. Consider the discretization

of the set of unknown parameters by a finite grid: (1, –2),

(1.25, –1.75), (1.5, –1.5), (1.75, –1.25), (2, –1), (2.25, –0.75),

(2.5, –0.75), (2.75, –0.25), (3, 0). To analyze the results we

consider the true values of parameters as follows: a ¼ 1:5

and b ¼ �1:5. For Option 2 we consequently choose points:

(1, –2,25), (1, –1.4), (1.7, –1.4) as values of unknown

parameters, these points are not included in the grid. In

Option 1 calculations are made for the following values of

the standard deviation r: 0:25, 0:35, 0:45. For each value of rwe made at least 30 simulations, and each simulation consists

of generation of normally distributed sample with zero mean

and corresponding value of r. In Option 2 only one value

r ¼ 0:35 is considered, and 30 simulations are made. In all

simulations, the absolute values of xt in the third and fourth

time periods are calculated by formula (22). Moreover, the

probability of true parameter values in the third and fourth

time periods (the third and fourth iterations) in Option 1, and

maximal probabilities in the third and fourth time periods in

Option 2 are calculated. Notice, that in applied problems the

method of discretization can be realized based on the infor-

mation obtained as a result of the earlier empirical studies.

Following Theorem 4, the optimal control for reward

criterion (2) with feedback control takes form:

ut ¼ �P9

j¼1ð1þ ajÞbjmjt

P9j¼1 b2

j mjt

xt;

where

m jtþ1 ¼ m j

t

uðxtþ1 � ð1þ ajÞxt � bjutÞP9k¼1 uðxtþ1 � ð1þ akÞxt � bkutÞmk

t

;

where the uniform distribution on the grid points is taken as

the initial distribution, and function uðzÞ is the probability

density function of the normal distribution with zero mean

and standard deviation r. The choice of considering three

and four time periods is associated to the fact that simu-

lations show that the duration of an ‘‘adaptation’’ period in

all simulations does not exceed three time periods. And in

most simulations, in Option 1, algorithm in the second time

period determines the true values of the unknown param-

eters with the high probability (more than 0.8). All calcu-

lations including simulations of normally distributed

variables are made in Excel.

Table 2 contains the average for all the simulations

values of the following characteristics: absolute value of xt,

i.e. deviation of the observed xt from the standard one, for

t ¼ 3 and t ¼ 4, probability of the true values parameters

(Option 1) and probabilities of the nearest grid point sub-

ject to the true values of the unknown parameters (Option

2).

Table 3 represents the values of statistics characterizing

the degree of dispersion of the absolute deviations xt

among all simulations.

From Table 2 and 3 we may notice that in average

among all simulations with r ¼ 0:25 (in Option 1) the

Table 2 Average absolute values of xt , t ¼ 3; 4, and probabilities of

true parameter values

Average value jx3j jx4j Pð3Þ Pð4Þ

r = 0.25 (Option 1) 0.1502323 0.219876 0.99995 0.999956

r = 0.35 (Option 1) 0.288575 0.367264 0.941073 0.953571

r = 0.45 (Option 1) 0.294356 0.47225 0.899275 0.922981

r = 0.35 (Option 2) 0.258486 0.296266 0.990658 0.991047


123

deviation of the absolute value is equal to 0.1502323, and

the absolute value of the deviation increases as the variance

of the random component increases. Qualitatively, the

similar results are also obtained in the next iteration. And

with the presence of random components the influence of

the unrecorded latent factors leads to the fact that the

absolute deviations in the fourth period are a bit larger than

at the third one. The proposed algorithm stabilize only the

deviations xt, but in stochastic environment, it is impossi-

ble to achieve zero deviations because of the presence of

random component with constant variance characterizing

the variability over time. Reducing variance, the influence

of random components can be also reduced. Selection of

the ‘‘appropriate’’ variance is possible only after making a

lot of practical experiments but even in this case it may

also vary depending on the specific conditions. We should

notice that Table 3 demonstrates the following: the dis-

persion in the absolute values of xt increases as the vari-

ance of the random component increases. Table 2 shows

that, in average, the proposed algorithm allows to achieve a

substantial reduction of deviation xt. As you can see at the

initial time period deviation is 2, but in time period 3 and 4,

it becomes less than 0.5. In all computational experiments

in time period 3, obtained values of unknown parameters

are the closest to the true values (Option 1), and the closest

to the true values of the unknown parameters grid point

(Option 2) with a high probability.

Acknowledgments We thank two anonymous referees for helpful

comments and remarks. The work of the fourth author was partly

supported by research project 9.38.245.2014 of Saint Petersburg State

University.

References

Aoki M (1989) Optimization of stochastic systems. Academic Press,

California

Boland JW, Filar JA, Howlett PG (2010) Environmental problems,

uncertainty, and mathematical modeling. Notices AMS

57(10):1286–1294

Bole B, Goebel K, Field M (2012) Using Markov model of fault

growth physics and environmental stresses to optimize control

actions, American Institute of Aeronautics and Astronautics 1–7

Dinkin EB, Ushkevich AA (1975) Controlled Markov processes and

their applications, 338 p. Nauka, Moscow

Guo X, Hernandez-del-Valle A, Hernandez-Lerma O (2012) First

passage problems for nonstationary discrete-time stochastic

control systems. Eur J Control 6:528–538

Hernandez-Lerma O, Marcus SI (1987) Adaptive policies for

discrete-time stochastic control systems with unknown distur-

bance distribution. Syst Control Lett 9:307–315

Jorgensen S, Martin-Herran G, Zaccour G (2010) Dynamic games in

the economics and management of pollution. Environ Model

Assess 15:433–467

Karelin VV (1991) Adaptive optimal strategies in controlled Markov

processes, Advances in Optimization Proceedings of 6 the

French-German Colloquium of Optimization, pp 518–525

Kros J, Mol-Dijkstra JP, Pebesma EJ (2002) Assessment of the

prediction error in a large-scale application of a dynamic soil

acidification model. Stoch Environ Res Risk Assess 16:279–306

Kullback S, Leibler RA (1951) On information and sufficiency.

Annals Math Stat 22(1):79–86

Nebolsin AN, Nebolsina ZP (2010) Soil liming. RASHN, GNU

LNIISCH, Saint Petersburg

Schmieman E, de Vries W, Hordijk L, Kroeze C, Posch M, Reinds

GJ, van Ierland E (2002) Dynamic cost-effective reduction

strategies for acidification in Europe: an application to Ireland

and the United Kingdom. Environ Model Assess 7:163–178

Shilnikov IA, Akanova NI, Barinov VN (2003) The method of soil

acidity predicting and calculation of calcium balance in agricul-

ture of Nechernozemie in Russian Federation. In: Pryanishnikov

DA, VNII of agricultural chemistry, Moscow, p 24

Yakushev VP, Bure VM, Yakushev VV, Bure AV (2012) Stochastic

modeling and optimal solutions on soil liming. Agrophysics

2:24–29

Yakushev VP, Karelin VV, Bure VM (2013) Bayesian approach for

soil acidity control, vol 10, 3. Vestnik St. Petersburg University,

Saint Petersburg, pp 168–179

Table 3 Sample statistics for simulations

Sample statistics St. deviation jxð3Þj max jx3j St. deviation jxð4Þj max jx4j

r ¼ 0:25 (Option 1) 0.11517851 0.349956 0.11855403 0.408598

r ¼ 0:35 (Option 1) 0.16405918 0.58084 0.16617282 0.5969

r ¼ 0:45 (Option 1) 0.32631627 1.1521 0.28192888 1.096765

r ¼ 0:35 (Option 2) 0.259902143 0.890608 0.20515263 0.552861


123

soil acidity adaptive control problem

Documents