formal computational modelling of bone physiology and ... · modelling of bone physiology and...

Formal Computational Modelling ofBone Physiology and Disease Processes

Candidate: Nicola PaolettiSupervisor: Emanuela Merelli

Examiners: Luca Bortolussi, Guido Sanguinetti

PhD course in Information Science and Complex Systems - XVI cycle - Universityof Camerino

Camerino, 24 March 2014

Formal ComputationalModelling of Bone

Physiology andDisease Processes

N. Paoletti

Outline

ME

TH

OD

S

Sect1.1

Sect1.2

Sect2

D − CGF(extension

of CGF PA)

Shape Calculus(Spatial PA)

ControlledSwitchedSystems

StochasticHybrid

Automata

Hybrid Ap-proximation

StochasticABM

CTMC

ODE

Simulation

ProbabilisticModel Checking

Stabili-zation

LANGUAGES MODELSANALYSIS

TECHNIQUESBIOLOGICALPROPERTIES

Simulation

Sensitivity Analysis

Model Checking

ParameterSynthesis

Optimal Control

Bone microstructure

Osteopo-rosis

Osteomy-elitis

STI(Therapy

Scheduling)



N. Paoletti

The bone remodelling process



N. Paoletti

The bone remodelling process

Bone remodelling. . . why?

• Multiscale process: multiscale effects, from the molecularsignalling level to the tissue level

• A paradigm for other physiological systems (likeepithelium renewal and haematopoiesis)

• Clinical and social impact: for diseases like osteoporosis,Paget’s disease, osteomyelitis.

• Collaboration with IOR (Istituto Ortopedico Rizzoli)

Shape Calculus(Spatial PA)

StochasticABM

Simulation

Bone microstructure

Osteopo-rosis

1.1- ShapeCalculus &agent-basedsimulation



N. Paoletti

Shape Calculus

• A bio-inspired spatial process algebra for describing 3Dprocesses, entities characterized by a behaviour B (a laTimed CCS) and a shape S .

• We extend the calculus with additional terms: Iteration,Thanatos and Duration

S ∈ S - 3D shapeσ = 〈V ,m, p, v〉 Basic shapeS〈X 〉S Complex shape

B ∈ B - Behaviornil Null behavior〈α,X 〉.B Bindω(α,X ).B Weak splitρ(L).B Strong splitε(t).B DelayB + B Choice(B)i IterationΘ Thanatosδ(t,B) DurationK Process name

P ∈ 3DP - 3D processS[B] S ∈ S,B ∈ BP〈a,X 〉P Composed 3D process

N ∈ N - 3D networknil Empty networkP P ∈ 3DPN‖N Parallel 3D processes



N. Paoletti

Shape Calculus

S0[B0]

S1[B1]

S0[B ′0]〈a,Y 〉S1[B ′1]

〈a,X1〉〈a,X0〉〈b,X ′1〉

〈c ,X ′0〉

S0[B ′′0 ]

S1[B ′′1 ]

〈a,Y 〉

〈P0,P1,Y 〉t ρ∆

Evolution of 3D processes in the Shape Calculus. Processes S0[B0] and

S1[B1] are involved in a bind on the channel 〈a,Y 〉 and in a subsequent strong

split.



N. Paoletti

Shape Calculus model for BR

• Cells communicateeach other via directcontact

• RANK/RANKL/OPGare surface-boundproteins

• Cells as 3D Processes,communicating by binding

• Biochemical signals aschannels exposed at thesurface of the 3D process



N. Paoletti

Shape Calculus model for BR

• Cells communicate eachother via direct contact

• RANK/RANKL/OPG aresurface-bound proteins

• Cells as 3D Processes,communicating by binding

• Biochemical signals as channelsexposed at the surface of the 3Dprocess

RANKL/OPG signaling. (pre-osteoblast, mature osteoblast) → (mature osteoblast,active osteoblast)

ρ(rankl,X1) 〈rankl,X2〉

〈rankl,Y 〉

ρ(rankl,X2)BOb〈rankl,X2〉

〈rankl,X1〉

SOb

SOb

SObSOb

SObSOb

SOb [〈rankl,X1〉.ρ(rankl,X1).BOPG] ‖ SOb [〈rankl,X2〉.ρ(rankl,X2).BOb]↓

SOb [ρ(rankl,X1).BOPG]〈rankl,Y〉SOb [ρ(rankl,X2).BOb]↓

SOb [BOPG] ‖ SOb [BOb]



N. Paoletti

BR specification

Tissue, BMU

Tissuedef= (‖ABMUi )

ai=1 ‖ (‖QBMUj )

qj=1

ABMUdef= (‖Oyi )

nOy

i=1 ‖ (‖Ocj )nOcj=1 ‖ (‖Obk )

nObk=1

QBMUdef= (‖Oyi )

nOy

i=1Osteocyte

Oydef= SOy [(〈can,X〉+ 〈can,X〉)kOy + 〈consume,X〉.Θ]

Osteoclast

Ocdef= SOc [BOc]

BOcdef= δ(tOc, (〈consume,X〉.resorb)∞).Θ.〈mineral,X〉kOc

Osteoblast

Obdef= SOb[〈rank,X〉.ρ(rank,X).BOPG + ε(tPb).BOPG]

BOPGdef= 〈rank,X〉.ρ(rank,X).BOb + BOb

BObdef= δ(tOb, (〈mineral,X〉.form)∞).Θ



N. Paoletti

Encoding of 3D processes as agents

• Stochastic actions Binding affected by the rates of theactions involved (rated output/passive input)

• Perception distance Agents can communicate within agiven radius

• Prototype in Repast Simphony: support for ShapeCalculus models (simplified shapes), stochastic simulationimplemented through discrete-event scheduler

A B

a,w2a,w1a, λ



N. Paoletti





A (a,λ)d−−−→B a,w1−−→

D a,w3−−→

C a,w2−−→

d



N. Paoletti







N. Paoletti

BR model

Motion

• Output channels release “molecule concentrations”(RANKL and Oc death factors) diffusing according to aCA-like rule

• Cells move according to a biased random walk, influencedby compatible molecular bias

Parametrization

• cells number, lifetime, size, resorption and formation ratetaken from experimental works

• diffusion coefficients and perception radii tuned for havingrealistic remodelling times



N. Paoletti

Results

Experimental evidence: RANKL concentration inversely relatedto bone turnover and density. Aging affects bone structure.

Two parameter configurations

• healthy, with regular RANKL production and cellular activity;

• pathological, with an overproduction of RANKL and a reduced cellularactivity (i.e. aging).

HealthyOsteoporoticStdev HStdev O

time [days]

BM

D [

mg

/cm

2]

40 simulations of agent-based model, 1 remodelling cycle



N. Paoletti

Results

Experimental evidence: RANKL concentration inversely relatedto bone turnover and density. Aging affects bone structure.

Two parameter configurations

• healthy, with regular RANKL production and cellular activity;

• pathological, with an overproduction of RANKL and a reduced cellularactivity (i.e. aging).

Normal

Osteopenia

Osteoporosis

After 7 remodelling cycles (∼ 7 years), pathological patient goes osteoporosis.



N. Paoletti

Results

H

O

H

O

H

O

1st cycle 2nd cycle 3rd cycle

Positioning of signalling osteocytes affect bone microstructure



N. Paoletti

Conclusion

• Extension of Shape Calculus

• New agent-based, stochastic and spatial model of BR, basedon formal language specification, and able to reproducenormal and defective dynamics

• Too many parameters if using the full expressive power of thecalculus. Practically, simplifications are needed.

[1] N. Paoletti, P. Lio, E. Merelli and M. Viceconti.Multi-level Computational Modeling and Quantitative Analysis of BoneRemodeling.In IEEE/ACM TCBB, 9(5), pp. 1366-1378, 2012.

[2] N. Paoletti, P. Lio, E. Merelli and M. Viceconti.Osteoporosis: a multiscale modeling viewpoint.In CMSB 2011, 9(5), ACM, pp. 183-193, 2011.

[3] P. Lio, E. Merelli, N. Paoletti and M. Viceconti.A combined process algebraic and stochastic approach to bone remodeling.In CS2Bio 2011, ENTCS 277, pp. 41-52, 2011.

1.2- Formal Analysis ofBone Pathologies

Hybrid Ap-proximation

CTMC

ODE

ProbabilisticModel Checking

Stabili-zation

Simulation

Sensitivity Analysis

Model Checking

Parameter Synthesis

Osteopo-rosis

Osteomy-elitis



N. Paoletti

ODE model

From [Komarova et al. Bone 33.2 (2003): 206-215]

(Osteoclasts) x1 =α1x1g11x2

g21 − β1x1

(Osteoblasts) x2 =α2x1g12x2

g22 − β2x2

(Bone density) z =− k1x1 + k2x2

0 5 10 15 20

0

5

10

Time (d)

Osteoclasts

0 200 400

0

50

100

Time (d)

Osteoblasts

0 200 400

−40

−20

0

Time (d)

”Bone”



N. Paoletti

Calibration

Aim: Tune ODE parameters (growth and death rates) so that maxnum. of bone cells agree with more recent evidence (max 10 Ocand 100 Ob, against 20 Oc and 2000 Ob of the original model).

Workflow

• Local sensitivity of bone cells

• Global sensitivity of max Oc and Ob

• Fitting and estimation against (made up) observations

0 10 20−6

−4

−2

0

2

Time (d)

Oc sensitivity

α1

β1

α2

β2

0 10 20

−100

0

100

Time (d)

Ob sensitivity

α1

β1

α2

β2



N. Paoletti

Calibration


Workflow




●●●

●

● ●●

● ●

●

●

●

●

● ●

●

●

●●

●

●

●

●

●

●●●

●

●

●

●●

●●

●

●

●

●●

●

●

●● ●

●

●●

●

●●●

●

●

●●

●

●

●

● ●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

● ●

●●●

●●

●●

● ●

●

●

●

●

●

● ●

●

●

●

●

●

●

●●

●●●

●●

●●

●●

●

●

●

●

● ● ●

●

●

●

●

●●

●

●

●

●●●

●

●

●

●

●

●

●

●●

●

●

● ●

●

●●

●

●●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●

●●

●● ●

●

●

●

●

●

●

● ●●

0.0 0.2 0.4 0.6 0.8 1.0

050

100

150

200

Osteoblasts

alpha1

Max

imum

cel

l num

ber

●●●

●

● ●●

●●

●

●

●

●

● ●

●

●

●●

●

●

●

●

●

●●●

●

●

●

●●

●●●

●

●

●●

●

●

●● ●

●

●●

●

●●●

●

●

●●

●

●

●

● ●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

● ●

●●●

●●

● ●

● ●

●

●

●

●

●

● ●

●

●

●

●

●

●

●●

●●●

●●

●●

●●

●

●

●

●

● ●●

●

●

●

●

●●

●

●

●

●●●

●

●

●

●

●

●

●

●●

●

●

● ●

●

●●

●

●●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●

●●

●● ●

●

●

●

●

●

●

● ●●

0.5 0.6 0.7 0.8 0.9 1.0

050

100

150

200

Osteoblasts

beta1

Max

imum

cel

l num

ber



N. Paoletti

Calibration


Workflow






N. Paoletti

CTMC model

Derived from ODE model and implemented in the model checkerPRISM

Osteoclasts:[ ] 0 < x1 < xmax

1 ∧ x2 > 0 → α1xg111 x

g212 : x1 = x1 + 1

[ ] x1 > 0 → β1x1 : x1 = x1 − 1[resorb] x1 > 0→ k1x1 : true

Osteoblasts:[ ] 0 < x2 < xmax

2 ∧ x1 > 0 → α2xg121 x

g222 : x2 = x2 + 1

[ ] x2 > 0 → β2x2 : x2 = x2 − 1[form] x2 > 0→ k2x2 : true

Bone reward:[resorb] true : 1[form] true : 1

0 100 200 300 400

020

4060

8010

0

0 5 10 15 20

02

46

810

Min−MaxMean+−sd

Min−MaxMean+−sd

Bone

Rel

ativ

e D

ensi

ty

Time [days]0 100 200 300

20

0

-20

-40

-60

-80



N. Paoletti

Hybrid approximation

• Optimal approximation of the non-linear ODEs into apiecewise multiaffine (PMA) system

• Overapproximation into a transition system where states arehyper-rectangles (reachable sets) and transitions are possibletrajectories between reachable sets. Analyzed with the toolRoVerGeNe by Batt et al.

x1 = α1

ns1+1∑i=1

r(x1, θ(1)i , θ

(1)i+1, y

(1)i , y

(1)i+1))

ns2+1∑i=1

r(x2, θ(2)i , θ

(2)i+1, y

(2)i , y

(2)i+1))− β1x1

- r is the ramp function

r(x , θ1, θ2, y1, y2) =

y2 (x ≥ θ2)

y1 + (y2 − y1) (x−θ1)(θ2−θ1)

(θ1 ≤ x < θ2)

y1 (x < θ1)

- θ(k)1 = xmin

k < θ(k)2 < · · · < θ

(k)ns1+1 = xmax

k are the segmentsidentified by the approximation algorithm for xk

- y(k)i is the i th sampled point for xk



N. Paoletti

Hybrid approximation

• Optimal approximation of the non-linear ODEs into apiecewise multiaffine (PMA) system

• Overapproximation into a transition system where states arehyper-rectangles (reachable sets) and transitions are possibletrajectories between reachable sets. Analyzed with the toolRoVerGeNe by Batt et al.

By the convexity property of multi-affine funs, we just look at the vertices



N. Paoletti

Stabilization properties

• Stabilisation is usually the existence of a unique fixpoint(state) or attractor (region) that is always eventually reached:in LTL, FG (prop).

• In our case, we follow the pattern φ =⇒ G (prop), focusingon particular regions (precondition φ) of the state space

• Note: with conservative extension of PMA, we can be sureonly of positive answers (simulation weakly preserves LTLSAT relation)

Down-regulation of Oc by Ob

With high numbers of Obs, Ocs never grow

• PMA model√

(x2 > θ(2)3 )→ G (∧11

i=2(x1 < θ(1)i ∧x2 > θ

(2)3 ) =⇒ X (x1 < θ

(1)i ))

• CTMC model√,∀θ(1) = 1, . . . , xmax

1

P=1 [G ((x1 < θ(1) ∧ x2 > θ(2)3 ) =⇒ X (x1 < θ(1)))]



N. Paoletti

Stabilization properties

• Stabilisation is usually the existence of a unique fixpoint(state) or attractor (region) that is always eventually reached:in LTL, FG (prop).

• In our case, we follow the pattern φ =⇒ G (prop), focusingon particular regions (precondition φ) of the state space

• Note: with conservative extension of PMA, we can be sureonly of positive answers (simulation weakly preserves LTLSAT relation)

Converse

With low numbers of Obs, Ocs never decay

• PMA model√

(x2 < θ(2)2 ) =⇒ G (∧11

i=4(x1 > θ(1)i ∧x2 < θ

(2)2 ) =⇒ X (x1 > θ

(1)i ))

• CTMC model√,∀θ(1) = 1, . . . , xmax

1

P=1 [G ((x1 > θ(1) ∧ x2 < θ(2)1 ) =⇒ X (x1 > θ(1)))]



N. Paoletti

Parameter synthesis

• We find the regions of the parameter space s.t. Ocs and Obshave a fixed upper bound

((x1 < θ(1)10 ) ∧ (x2 < θ

(2)20 ))→ G((x1 < θ

(1)10 ) ∧ (x2 < θ

(2)20 ))

• Note: synthesis 6= estimation/identification (where asingle param value is provided)

• Synthesis method (Batt et al.), based on hierarchicaldecomposition of param space guided by affine inequalitiesover the parameters (termination ensured by the existence ofparam equivalence classes).

3.7

3

4.5

44.6

5 8

·10−

21.6

2.04

2.4

2.8

β2

β1



N. Paoletti

Robust stabilisation wrt init conditions

Stabilization (bone homeostasis) reached regardless init num ofosteoclasts

ODE model, global sensitivity analysis

0 5 10 15 20

05

1015

20

Osteoclasts

Time [days]

Num

ber

of c

ells

Min−MaxMean+−sd

0 100 200 300 400

050

100

150

Osteoblasts

Time [days]

Num

ber

of c

ells

Min−MaxMean+−sd

0 100 200 300 400

−80

−60

−40

−20

0

Bone

Time [days]

Rel

ativ

e D

ensi

ty

Min−MaxMean+−sd

CTMC model, probabilistic (quantitative) model checking,parametric analysis

0 5 10 15 20

05

1015

20

Osteoclasts

Time [days]

Exp

ecte

d nu

mbe

r of

cel

ls

x10 = 0

x10 = 5

x10 = 10

x10 = 15

x10 = 20

0 100 200 300 400

050

100

150

Osteoblasts

Time [days]

Exp

ecte

d nu

mbe

r of

cel

ls

x10 = 0

x10 = 5

x10 = 10

x10 = 15

x10 = 20

0 100 200 300 400

−60

−40

−20

0

Bone

Time [days]

Exp

ecte

d R

elat

ive

Den

sity

x10 = 0

x10 = 5

x10 = 10

x10 = 15

x10 = 20



N. Paoletti

Defective Dynamics

Osteoporosis: characterized by low bone mass and structuralfragility, results from ageing and defects in theRANK/RANKL/OPG pathway.

Osteomyelitis: a bacterial infection of bone (S. aureus) thataffects the signalling level, and rapidly leads to bone loss andnecrosis.



N. Paoletti

Defective Dynamics - Extended ODEs

(Osteoclasts) x1 =α1xg11(1+f11

Bs )

1 xg21(1−f21

Bs )+gpor

2 − kageingβ1x1

(Osteoblasts) x2 =α2xg12/(1+f12

Bs )

1 xg22−f22

Bs

2 − kageingβ2x2

(S. aureus) B =(γB − V )B · log(s

B)

• B follows a logistic growth

• fij : effects of the infection on auto- and para- regulationfactors gij

• V : medical treatment (V > γB , bacteriocide; V = γB ,bacteriostatic)

• gpor : effectiveness of RANKL expression

• kageing: ageing factor increasing cells death rate



N. Paoletti

Defective Dynamics - Extended ODEs

(Osteoclasts) x1 =α1xg11(1+f11

Bs )

1 xg21(1−f21

Bs )+gpor

2 − kageingβ1x1

(Osteoblasts) x2 =α2xg12/(1+f12

Bs )

1 xg22−f22

Bs

2 − kageingβ2x2

(S. aureus) B =(γB − V )B · log(s

B)

0 500 1000 1500 2000 2500

020

4060

80

STAPHYLOCOCCUS AUREUS

Time [days]

Num

of b

acte

ria

BA

CT

ER

IOS

TA

TIC

TR

EA

TM

EN

TB

AC

TE

RIO

CID

ET

RE

AT

ME

NT

0 500 1000 1500 2000 2500

020

4060

80

STAPHYLOCOCCUS AUREUS

Time [days]

Num

of b

acte

ria

0 500 1000 1500 2000 2500

020

4060

8010

012

0

BONE DENSITY

Time [days]

Den

sity

%

0 500 1000 1500 2000 2500

020

4060

8010

012

0

BONE DENSITY

Time [days]

Den

sity

%

Simulation of bacteriostatic (V = γB ) and antibiotic (V > γB )treatments.



N. Paoletti

Probabilistic model checking results

Stochastic vs ODE model

0 200 400 600 800 1000 1200

−0.

50.

00.

5

0 200 400 600 800 1000 1200

−0.

50.

00.

5

0 200 400 600 800 1000 1200

−0.

50.

00.

5

a) b) c)

Model checking-based clinical estimator: density change rate.



N. Paoletti

Conclusion

• Formal analysis gives not only biological insights(stabilization), but also estimators of clinical importance(bone diseases)

• Stochastic model closer to original ODEs → Probabilisticmodel checking results are “to be trusted”

• Unlike hybrid approximation, no effective methods forparameter synthesis of CTMCs

[1] E. Bartocci, P. Lio, E. Merelli and N. Paoletti.Multiple verification in complex biological systems: the bone remodelling casestudy.In TCSB XIV, LNCS 7625, pp. 53-76, 2012.

[2] P. Lio, N. Paoletti, M.A. Moni, K. Atwell, E. Merelli and M. Viceconti.Modelling osteomyelitis.In BMC Bioinformatics, 13(Suppl 14), 2012.

[3] P. Lio, E. Merelli and N. Paoletti.Multiple verification in computational modeling of bone pathologies.In CompMod2011, EPTCS 67, pp. 82-96, 2011.

2- Hybrid Approach tothe Scheduling of

Multiple Therapies

D − CGF (extensionof CGF PA)

ControlledSwitchedSystems

StochasticHybrid

Automata

Optimal ControlSTI (Ther-

apy Schedul-ing)



N. Paoletti

Optimal Scheduling of Multiple Therapies

Aim

Structured Therapy Interruption (STI) is the programmedinterruption of a medication for a period of time.STI Problem. Find the “best” combination of multiple on-offtherapies to recover a dynamic disease model

CGF −→ ODEsystems

• Based on CGF (Chemical Ground Form) (Cardelli, TCS2008, pp. 261–281), a subset of stochastic CCS

• (Bortolussi et al., Mathematics in Computer Science 2(3), pp.465–491, 2009): hybrid automata semantics for CGF

• D-CGF (Disease CGF) extends CGF to describe complexdisease processes, with a hybrid dynamical systemssemantics.



N. Paoletti


Aim


CGF −→ Hybridautomata






N. Paoletti


Aim


D-CGF −→ Hybrid dynamicalsystems






N. Paoletti

Syntax

Two kind of processes:

• Individuals: standard CGF processes that represent the populationin the disease model (e.g. susceptible, infected, . . . ). Interpretedas continuous variables in the hybrid semantics.

• Therapies: processes for modelling discrete interventions on thedisease scenario (drug dosage). Interpreted as discrete switchesin the semantics.

E ::= 0 | X = M,E ReagentsM ::= 0 | π.P + M MoleculeP ::= 0 |(X‖P) Solutionπ ::= τ r | ?x r | !x r Actions (r ∈ R+)

D ::= 0 | Y = C ,D Set of therapy definitionsC ::= 0 | π.T + C TherapyT ::= 0 |(Y ‖T ) Combination of therapies

D − CGF ::= (E ,P,D,T ) A D-CGF model



N. Paoletti

Epidemics with therapies

We consider a SIR epidemic model + therapies:

• D1, a vaccination (makes immune the susceptiblepopulation)

• D2, an antiviral (increases the recovery rate)



N. Paoletti

SIR + therapies D-CGF model

We build on a open population SIR (births and deaths):

D-CGF model M = (E ,P ,D,T )

E :

S = τbS1.(S‖S) + τµS2

.0 + ?jρ.R + ?iβ .I

I = τbI1.(I‖S) + τµI2

.0 + ?hk .R + !iβ .I + τνI3 .R

R = τbR1.(R‖S) + τµR2

.0

D:

D1off = τ r1on1on .D1on

D1on = τr1off1off .D1off + !jρ.D1on

D2off = τ r2on2on .D2on

D2on = τr2off2off .D2off + !hk .D2on

C : D1off ‖D2off

List of reactionsS + D1on →ρ R + D1on Susceptible becomes recovered with therapy 1

I + D2on →k R + D2on Infected becomes recovered with therapy 2D1off →r1on D1on D1 switched onD1on →r1off

D1off D1 switched offD2off →r2on D2on D2 switched onD2on →r2off

D2off D2 switched off



N. Paoletti

Derivation of hybrid dynamical systems

We focus on Controlled Switched Systems (CSS), a class ofhybrid dynamical systems where the external control input is givenby the discrete operation mode.

System

x = f (x,q)y = g(x,q)

Controller

Drug scheduler

yObservables (e.g. pa-

tient medical data)

qTherapy (combi-

nation of drugs)

Control loop (x continuous state, y observable output, q discrete mode)



N. Paoletti

Derivation of hybrid dynamical system

For a correct derivation of the semantics, the D-CGF model mustbe well-formed, i.e. informally, its drug terms can be interpretedas discrete variables.

1 Build the SD-graph, a digraph whose nodes are therapyterms in D and arcs connect couples (Yi ,Yj ) involved in aswitch, i.e. an action π that consumes Yi , and produces Yj :∆(π,Yj ) = −1 and ∆(π,Yj ) = 1

2 Consider its connected components



N. Paoletti


3 Derivation of the discrete modes from the SD-graph

4 Modified rate vector:For each mode qi , φqi is obtained according to the drug terms active inqi :

. . . τ1on τ1off τ2on τ2off j hφ = . . . T1off r1on T1onr1off T2off r2on T2onr2off T1onρS T2onkI

φq =

q1 . . . r1on 0 r2on 0 0 0q2 . . . 0 r1off r2on 0 ρS 0q3 . . . r1on 0 0 r2off 0 kIq4 . . . 0 r1off 0 r2off ρS kI



N. Paoletti


5 Controlled switched system given, at each mode qi , by theproduct of the stochiometric matrix (restricted to i) and therate vector at qi

x(qi ) = M|E · φqi

x(q1) =

S = b(S + I + R)− βSI − µS

I = βSI − µI − νI

R = νI − µR

x(q2) =

S = b(S + I + R)− βSI − µS − ρS

I = βSI − µI − νI

R = νI − µR + ρS

x(q3) =

S = b(S + I + R)− βSI − µS

I = βSI − µI − (ν + k)I

R = (ν + k)I − µR

x(q4) =

S = b(S + I + R)− βSI − µS − ρS

I = βSI − µI − (ν + k)I

R = (ν + k)I − µR + ρS



N. Paoletti

Non-linear optimal control definition

We define the following Model Predictive Control (MPC) problemto compute the optimal administration of therapies.

minq

t+Tp∫t{‖Rq(k)‖1 + ‖Qx(k)‖1} dk

subj . to x(t) ∈ [0, 1]3

q(t) ∈ {0, 1}4∑i qi (t) = 1

x(t) = f (x(t), q(t))x(0) = x0

x(Tf ) ∈ Xf

∀t ∈ [0,Tf ]

• Tp is the prediction horizon

• ∆t = 1/365 is the discrete timestep (1 day)

• q: controlled modes; x:continuous state

• Xf = {[S I R]T ∈[0, 1]× [0, I0]× [0, 1]} is set ofterminal states (the final num. ofinfected will not be higher thanthe init one)

• R and Q are theweights/penalties on thecontrolled inputs and on thesystem state



N. Paoletti

Controlling non-linear hybrid systems

No exact algorithms for the optimal control of generic non-linearhybrid systems (except for specific classes, up to PMA)

Embedding approach

• Original system (SOCP) is embedded into a larger class ofsystems without discontinuities (EOCP)

• Relaxes mode vector q ∈ {0, 1}n into q ∈ [0, 1]n

• The optimal continuous modes of the EOCP are projectedback as discrete modes

• Facts:• SOCP has solution =⇒ EOCP has solution• EOCP solution can be approximated with arbitrary precision

to SOCP solution• Mode projection is theoretically justified and practically

effective to approximate EOCP solution



N. Paoletti

Embedded problem

minq

t+Tp∫t

{qi (k)(‖R·,i‖1 + ‖Qx(k)‖1)} dk

subj . to x(t) ∈ [0, 1]3

q(t) ∈ [0, 1]4∑i qi = 1

˙x(t) =∑

i qi · fqi (x(t))x(0) = x0

x(Tf ) ∈ Xf

∀t ∈ [0,Tf ]



N. Paoletti

Application to the H1N1 Case Study

We start from a SIR model of H1N1 influenza occurred in US in2009 (Feng et al. AAPS, 13(3):427–437, 2011)

• Large R −→ drug dosage tends to be avoided (severe sideeffects or early stage of the disease)

• Small R −→ more prominent drug dosage (little sideeffects or mature stage of disease)

Treatment scenarios

1 Low penalties to D1 and D2

2 High penalty to D1 (antiviral favoured)

3 High penalty to D2 (vaccination favoured)

4 High penalty to the combination of D1 and D2



N. Paoletti

Results

Evolution of epidemics and administration of vaccination andantiviral treatments

01

Dru

gs

Scenario 1D1 D2

0

0.5

1

Po

pu

lati

on

01

Dru

gs

Scenario 2

0 50 100 150 200 250 300 3500

0.5

1

Time (d)

Po

pu

lati

on



N. Paoletti

Results

0 100 200 300

0

20

40

60

80

4.85

86.5

4.864.87

Time (d)

Cu

mu

lati

veIn

fect

ion

s Scenario 1

Scenario 2

Scenario 3

Scenario 4

Cumulated cases of infection under the four treatment scenarios

0 100 200 300

0

100

200

300

400

52.66

367.46

55.0154.85

Time (d)

Cu

mu

late

dco

st

Scenario 1

Scenario 2

Scenario 3

Scenario 4

Cumulated costs of optimal strategies



N. Paoletti

Conclusions

• From (stochastic) process algebra to hybrid non-linearoptimal control

• Approximated methods effective for the control of non-linearhybrid systems

• Directions: game-theoretic extension, stochasticity incontrolled system, . . .

[1] P. Lio, E. Merelli and N. Paoletti.Disease processes as hybrid dynamical systems.In of HSB 2012, EPTCS 92, pp. 152-166, 2012.

formal computational modelling of bone physiology and ... · modelling of bone physiology and...

Documents