cabeq 2012-03 verzija 3silverstripe.fkit.hr/cabeq/assets/uploads/cabeq-2012-03-3.pdf · process...

Optimization of Recipe Based Batch Control Systems Using Neural Networks

A. Šoštarec,a,* D. Gosak,c and N. Hlupiæb

aPliva Croatia Ltd., Prilaz baruna Filipoviæa 25, ZagrebbUniversity of Zagreb, Faculty of Electrical Engineering and Computing,Department of Applied Computing, Unska 3, Zagrebcpresent address: Hospira Zagreb Ltd., Prilaz baruna Filipoviæa 27, Zagreb

In the modern pharmaceutical industry many flexible batch plants operate under anintegrated business and production system, using ISA S95 and ISA S88 standards formodels and terminology, and implementing flexible recipe-based production.

In the environment of constantly changing market conditions, adjustment to sur-roundings is a business necessity. To support necessary production improvement, re-gulatory authorities have introduced the risk based approach for the control of processdevelopment, production based on the quality by design (QbD) principle, and processanalytical technology (PAT).

In this work, the method for practical implementation of an adaptable control rec-ipe, that allows process improvement inside the previously established design space, isproposed, based on the neural network process model.

Based on the neural network model, the three methods for recipe-controlled processimprovement and optimization were introduced – neural-based software sensor, genericneural model control, and process optimization using iterative dynamic programming.

Suitability of the proposed method was tested in a mini reaction plant ChemreactorBüchi, running the wastewater treatment batch, controlled by the production recipe basedon S88 standard.

Key words:ISA S88, neural network model, recipe-controlled process, design space

Introduction

Recipe-based automated flexiblepharmaceutical plants

Modern pharmaceutical production is a branchof industry faced with many different challenges. Itis characterized by the necessity to maintain highproductivity and flexibility, while at the same timecomplying with strict regulations imposed by dif-ferent government agencies (FDA, EMEA, etc.).Therefore, to maintain their competitive edge, manypharmaceutical companies (API, as well as finalforms producers) use various organizational andprocess control software tools. Many of the avail-able support and process control tools are based onISA S95 and S88 standards.1,2

Flexible pharmaceutical plants, organized inaccordance with integrated ISA S95 – S88 strategy,have the general structure depicted in Fig. 1. Whilethe various S 95 levels are used for a structured ap-proach to business planning and logistics, as well asmanufacturing operations and control (so-calledMES level – manufacturing execution systemlevel), the S88 standard is used on the lowest level

to actually control the production process within aparticular plant.

ISA S88.01 models and terminology (1995 and2011) are created to fulfill the task of unifying termi-nology and models, so that the experts from differentfields (automation, equipment and technology) caneasily cooperate. The standard, as a basis, defines therecipe which uniquely specifies the ingredients andsteps necessary to produce a certain product. Apply-ing S88 requires separation of physical equipment

A. ŠOŠTAREC et al., Optimization of Recipe Based Batch Control Systems …, Chem. Biochem. Eng. Q. 26 (3) 175–190 (2012) 175

Original scientific paperReceived: May 25, 2012

Accepted: August, 22, 2012

*Corresponding author: [email protected]

F i g . 1 – ISA S95 and S88 integration through enterprise,MES and control levels as defined in correspond-ing standards

from production procedures. This is achieved by ap-plying three models: the process model, physicalmodel and procedural control model. The modelsand their mapping are depicted in Fig. 2. The pur-pose of mapping is to tie the procedural controlmodel to the physical model, in order to provide pro-cessing described in the process model.

Practical implementation of S88 models, afterperforming of mapping on existing process equip-ment, is usually done as depicted in Fig. 3.

Quality by design and PAT methodology

Application of the mentioned methods pro-vides clear benefits, such as improved processreproducibility, lower production costs and in-creased plant flexibility for introduction of newproducts. In spite of all this, until recently, processoptimization of a particular product was an expen-sive and time-consuming task due to regulatory re-quirements that effectively hinder process changesnecessary for improvement.

In order to encourage process improvementand, at the same time, ensure high regulatory (GMP– good manufacturing practice) levels in the flexi-ble pharmaceutical production, the FDA has intro-duced two initiatives – quality by design (QbD)3

and process analytical technology.4

The key concept in QbD is the design space –the established range of process parameters insideof which product quality does not diminish; there-fore, movement within these boundaries for optimi-zation purposes is a regulatory allowed change. Ascan be seen from Fig. 4, the control space – theworking process space that spans around processset points – can generally be moved within the de-

176 A. ŠOŠTAREC et al., Optimization of Recipe Based Batch Control Systems …, Chem. Biochem. Eng. Q. 26 (3) 175–190 (2012)

F i g . 2 – Mapping of models according to ISA S88.01 stan-dard

F i g . 3 – Common way of practical realization of ISA S88 models on control equipment

sign space using some optimization technique (nu-merical or experimental design method) in searchof the optimum point.

Another important, recently-introduced para-digm is the process analytical technology (PAT) ini-tiative. The essence of PAT initiative is to allow re-placement of classical sampling by off-line analysiswith timely (on-line) process parameters measure-ment, via analyzers as well as applying softwaresensors when appropriate.

This paper proposes a method based on theneural network model that extends the ISA S88methodology-based control recipe mapped on aparticular physical equipment unit, in a way that al-lows gradual movement of control space toward theoptimal point, as well as application of softwaresensors.

Flexible batch plants as hybrid systems

As can be seen from available literature(Sanchez et al.,5 Moor and Raisch,6 Potoènik etal.7), flexible batch plants represent so-called hy-brid systems. Hybrid systems are broad classes ofsystems whose behavior is characterized by exhibit-ing continuous and discrete process dynamics.While there are many mathematically rigorous de-scriptions of such systems (Goebel et al.,8 Lennart-son et al.9), the state space approach of Barton,Banga and Galan10 is here presented. According totheir definition, a general hybrid system can be de-scribed by state space S S kk

nk�

�1� where everymode S k is characterized by:

– Set of variables � �� , , , ,( ) ( ) ( ) ( )x x y u

k k k k t

– Set of equationsf x x y u

( ) ( ) ( ) ( ) ( )( � , , , , )k k k k k t � 0

– Set of transitions possibly containing transi-tion conditions L x x y uj

k k k k k t( ) ( ) ( ) ( ) ( )( � , , , , ) � 0 for-

med by logical propositions that trigger switching

when they become true, and transition functionsT x x y u x x y uj

k k k k k j j j j t( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )( � , , , , � , , , , ) � 0

associated with particular conditions

– Set of constraints that could be path con-straints h x x y u

( ) ( ) ( ) ( ) ( )( � , , , , )k k k k k t �0, and point con-

straints c x x y urk k k k k

rt( ) ( ) ( ) ( ) ( )( � , , , , )�0, r nr

k� �{ , , }( )0that must be satisfied only in certain moments.

Based on hybrid system methodology, Lennart-son et al.9 propose the two approaches to practicalrealization of the hybrid process control. The firstapproach is to replace hybrid plant model with fullydiscrete model. This task is performed by partition-ing the continuous state space into regions, and thencreating the supervisor for apparent discrete pro-cess. The main advantage of this approach is easyimplementation, while the main disadvantage ispossible poor approximation of continuous processparts. The second approach is the direct synthesis ofhybrid supervisor controller. The main advantage ofthis approach is the better control that can beachieved, but at the expense of applying much morecomplex model.

Another method for realization of hybrid pro-cess control is proposed by Sanchez et al.5 Theirmethod is applicable to hybrid systems with pre-dominant discrete parts. The basis of this method isthe division of an overall plant model into the pro-cess model and discrete event controller model. Theprocess model is further divided into equipmentmodels and interface models. Each equipmentmodel describes the hybrid dynamic of one processunit. The second part of the process model links theunit operation model with the discrete event con-troller models. It is also made of two separate mod-els: device drivers and state observers. Their role isto connect equipment with the discrete controllerand to model discrete events instruments. Supervis-ing the discrete event controller includes an inter-face for connection with lower levels, and actualimplemented control system. Controller is designedto be in compliance with the ISA S88 standard, andtherefore consists of phases, operations, and unitprocedures.

Recently, Fabre et al.11 used PrODHyS hybridsimulation environment coupled with schedulingmodule ProSched in order to model batch processes(ISA S88 compliant), and optimize interaction be-tween scheduling and simulation models. Recipesare modeled with extended resource task networks,while scheduling is performed by mixed integer lin-ear programming.

Process modeling using neural networks

Developing the physical, thermodynami-cally-based mathematical model of a complex pro-


F i g . 4 – Positioning of parameter space according to qual-ity by design initiative

cess is a difficult and time-consuming activity. Incase of the flexible batch plant that producesmany different products, delays between switchingproducts must be minimized for economic reasons,while at the same time optimization is needed.Therefore, faster ways of developing process mod-els are desired. If only experimental data are avail-able, using the neural network model is a practicaloption.

Chemical engineering unit operation processes,such as reaction, distillation, crystallization etc. canbe generally described with a nonlinear dynamicalrelation that relates state coordinates x with processinputs u and outputs y (eqs. 1 and 2)

d

d

xf x u

t� ( , ) (1)

y h x u� ( , ) (2)

Static input – output relation (eq. 2) can be di-rectly approximated by neural network if data forlearning are available. In the literature,12 the neuralnetwork which is most often used, is a multilayerfeedforward perceptron because of its universallearning capability. Multilayer feedforward percep-tron neural network is depicted in Fig. 5, and eqs.3–5 provide its mathematical representation.

x f netp p p� ( ) (3)

net x w wp p p p p� � 1 1 0 1 (4)

f nete

p p net p( ) �

�

1

1(5)

Learning the net means adjusting the weightcoefficients in a way that approximation error (eqs.6 and 7) is minimized.

E y O et ri rii

I

ii

I

� ��

1

2

1

22

0

2

0

( ) (6)

E ET tr

R

��

1

(7)

Minimization of the weight coefficients mustbe done by some minimization method. The mostbasic is the method of steepest descent (eq. 8)

w i w iE

wpp ppT

pp

� � 1 1 11

1( ) ( ) ��

�(8)

Many more sophisticated methods can be ap-plied; deterministic like Levenberg Marquard orconjugate gradient (Masters13) as well as stochastic,for example genetic algorithm. The necessary par-tial derivatives of errors with regard to weight coef-ficients can be calculated analytically by using,so-called backpropagation algorithm. eqs. 9–12

�

�

�

�

�

�

E

we

x

net

net

wt

iji

i

i

i

ij

�( )

( )(9)

�

�

�

�

�

�

�

�

�

�

E

w

net

w

x

nete

x

net

nett

jk

j

kj

j

ji

i

i

i�( )

( ) ( )

( )

x ji

I

�

0

(10)

�

�

( )net

xw

p

ppp

�

11 (11)

�

�

x

wx

p

ppp

�

11 (12)

Many modifications of the presented basic al-gorithm are developed in order to improve conver-gence and speed up the computation. While trainingthe net, one must bear in mind that if the network ischosen to have more neurons that are actually nec-essary to learn a particular relation, it can oftenoverfit the data. The easiest method to prevent thisfrom happening is to divide the entire data set into atraining set and a verification set. During the netlearning minimization results of training set arechecked against the validation set and minimizationroutine is stopped at the optimal point.

Regarding the learning of the dynamical rela-tion (eq. 1) two main approaches are developed.The first one is the so-called tapped delay linemethod (Hrycej,14 Paengjuntuek et al.15). The prin-ciple of this method is to use a static network, butwith additional inputs of past values for network in-puts as well as outputs (eq. 13)

x f x x u ut t t d t t d� � � �1NN ( , , , , , ) (13)


F i g . 5 – Schematic representation of multilayer feedforwardnetwork

In eq. 13 d is the measurement index, a numberwhich determines how many past network inputsand outputs must be used in order to approxi-mate eq. 1 with sufficient accuracy. The clearadvantage of this method is the ability to applythe static network, but quality of approximation isstrongly dependent on the time step and choice of dvalue.

Another general way for achieving the dy-namic process learning by neural network is to userecurrent networks. The recurrent networks may beinternally or externally recurrent. Internally recur-rent nets are nets whose neurons are fully con-nected with each other, as well as with themselves.Activation functions of these kinds of neurons aredifferential equations (eq. 14)

p

p

p p p

x

tx f net

d

d� � ( ) (14)

To train this kind of network, two principalgroups of methods can be applied (De Jesus andHagan16). The first group is called BackpropagationTrough Time (BPTT) and its main principle is de-veloped by Werbos17 who introduced the idea todiscretize eq. 14 around current point ti, using amethod for calculation of finite differences. Afteradjusting backpropagation equations to reflect thenew expression for neuronal activation function,calculation is performed backwards in time fora number of past inputs appearing in discretizedeq. 14. The second group of methods is calledReal Time Recurrent Learning (RTRL) originallyproposed by Williams and Zipser.18 The maincharacteristic of this method is learning in realtime, i.e. while network is running by integrationof neuron activation function (eq. 14). The net-work weights are updated after every integrationstep.

The internally recurrent neural network learn-ing process suffers from a problem called vanishinggradient (Haykin12). This problem makes learningof the behavior that depends on the data belongingto a distant past, with respect to the current time,very difficult due to the local nature of both learn-ing methods. Therefore, another kind of recurrentnetwork, namely externally recurrent neural net-works, are introduced. The simplest form of exter-nally recurrent neural network is introduced byNerrand et al.19 (eq. 15) and called the canonicform of recurrent network.

x x f x i u i( ) ( ) [ ( ), ( )]i i NN� � �1 (15)

While the network given by eq. 15 is taught bythe standard algorithm, and thus does not sufferfrom the vanishing gradient problem, the practicalperformance of the net can be less than satisfactory

due to the fact that applied simple discretizationmay not be sufficient in the case of approximationof highly nonlinear systems, especially if they be-long to the class of “stiff” dynamical systems. Forthis reason, Gosak and Vampola20 used continuouscanonic form (eq. 16).

d

d

xf x u

tNN� ( , ) (16)

In order to perform the task of learning the re-current network described by eq. 16, derivatives ofmeasured state vector data must be calculated ateach point, and the quality of approximationheavily depends on the chosen numerical method.Additionally, resulting continuous differential equa-tion model must be solved by integration. Despiteall this, the resulting network may be the networkof choice because of its ability to cope with difficultnonlinear system, by enabling any method of inte-gration – multistep, semi implicit or implicit to beapplied.

Methodology

Outline of neural network-based extensionto recipe-based control

A recipe-based flexible plant is normally de-signed to produce many different products withonly two common characteristics – allowableranges of process parameters (temperature, pressureetc.) and equipment construction materials. Usualdesign procedure includes several steps, such as de-fining the process units equipment modules, defin-ing the basic control modules, and set of equipmentphases designed to perform predefined module tran-sitions. Whenever a new product is introduced tothe production plant, only product-specific controlrecipe needs to be developed.

While this design procedure allows quick prod-uct switching and high batch reproducibility, thefact that the resulting control system is not in anyway built in for any particular product introducespotential drawbacks. The main drawbacks are thefollowing: lack of specific sensors needed foron-line measurement (e.g., concentration, density,conductivity), control loops may suffer from stronginteractions, and particular operations may operatefar from optimum points.

It is well known that all the mentioned draw-backs could be eliminated by the use of an appro-priate process model for the model based optimiza-tion and process control improvement. A suitableform of a flexible batch plant model is a hybridmodel generally described in the text above, i.e. amodel that takes into account the fact that the un-


derlying dynamical process consists of mixed dis-crete and continuous parts. The main problem,when formal models such as neural networks areused, that depends solely on experimental data ishow to perform the successful training. Particularly,it would be very difficult for neural model to learnthe exact switching times of a hybrid system pro-cess. Therefore, the control recipe should be con-structed in a way that a resulting process learningby neural network can be accomplished. One wayto do this is to ensure that the batch system behavesas a hybrid automaton. The concept of hybrid au-tomata was introduced by Alur21 and are defined asthe particular form of hybrid systems modeled as aset of continuous dynamic nodes that become activein a predefined order based on some set of discreteevents.

In order to unambiguously define the struc-ture of a hybrid automaton model that corres-ponds to a particular process, it is appropriate toimpose some additional constraints on the controlrecipe.

The first constraint is that the equipment phasebehaves in a way that it performs some transitionon the equipment module. This transition can gen-erally be described in the form of piecewise linearfunction for a set point of variables controlled inparticular phases (eqs. 17–19)

l t tk� (17)

� kksp

ksp

k k

y y

t t�

�

�

1

1

(18)

y ty l k K t t

t tsp k

spk

Phase

Phase( ), , ,

��

� �

� 1 max

Ksp

maxy tOperationmax

��

(19)

By applying eqs. 17–19, any transitionfunction with arbitrary precision can be defined,although fixed set points or ramps are mostlyused, for example temperature ramp on temperaturecontrol module or pressure ramp on pressurecontrol module. After the final value of set pointvalue has been achieved (e.g., final temperatureor pressure), the phase remains in this state (withfinal set point value) until the new operationstarts.

The second constraint is that recipe operationrepresents the set of phases that all start at the sametime and operation ends when the last phase in theoperation completes its transition, thus ensuringproper ending of continuous nodes. Fig. 6 showsthis graphically, with dashed part of a polyline rep-resenting a set point of a finished phase waiting forthe end of operation.

While these additional constraints make devel-opment of the hybrid automaton model easier, theydo not impose significant limits on recipe capabilitybecause, if necessary, any operation can be splitinto several shorter ones, thereby avoiding creationof time lags.

Having in mind all the operations in a recipe –actually interesting for modeling are only those spe-cific to a particular process (reaction, distillation,crystallization), while others are common to allproducts (inertization, sterilization, cleaning, fill-ing, emptying). Fig. 7 shows an example schematicof a control recipe in usual SFC (sequential func-tion chart) representation, and corresponding hybridautomaton model in a state diagram form (dashedboxes represent common operations not necessaryfor the model).

After training the neural network for everycontinuous mode, the network can be used as asoftware sensor to enhance process control perfor-mance, as well as for process optimization. Sincethe introduction of neural models should be op-tional, and should only support the existing basiccontrol instead of replacement it, the most suitableway to extend a control system with a neural net-work is to form a neural process abstraction layer asdepicted in Fig. 8.

Detailed structure of a network layer is givenin Fig. 9. As can be seen from the figure, the net-work switches on when modeled operation startsusing the appropriate parameter set, and it switchesoff after the operation with modeled continuousmode finishes. Depending on the task required forthe operation neural model, software sensing, con-troller improvement or optimization of set pointcurve defined by eq. 19 is performed.


F i g . 6 – Set point transitions for particular phases in oper-ations. Shaded area shows regions in which phase 1 waits foroperation completion by holding the last active set point(dashed line).


F i g . 7 – Representation of recipe operations and equivalent modes of hybrid automation (with MODE 2 chosen not to be modeled)

F i g . 8 – Modification of recipe control system by insertation of neural layer

The development of neural networkprocess model

The neural network needs to model processeson all continuous modes that correspond to opera-tions whose behavior is characteristic for a particu-lar production process. Continuous modes can begenerally modeled with equations describing statevariable dynamic (eq. 16), output variables relation(eq. 20), and an equation defining controller actions(eq. 21) of involved equipment module basic con-trol.

y h x u� NN ( , ) (20)

u r x y� ( , ) (21)

Neural networks are needed to approximateeqs. 16 and 20. Both relations are generally nonlin-ear, and eq. 16 represents the system of differentialequations. In order to train the networks, threequestions need to be answered: what will be thenetwork topology, how will training data sets becollected, and which training method will be used.

In this paper, for mapping the static relations,the multilayer feedforward neural network (Fig. 5)is chosen. Dynamic process data was mapped bycontinuous externally-recurrent modification of themultilayered feedforward network.

The next issue is gathering and preparation of atraining data set. Since in this particular problemthe design space represents the allowable span ofprocess variables, there are two options for datacollection – the first is moving the control space in-side the design space by changing set points of de-sired process outputs, and the second way is to ex-tend the control space to the entire design space. Inthe second case, process inputs may be varied with-out controller feedback in order to cover the entiredesign space. Running several trial batches experi-mentally (without the need to produce a product

within the specification) may significantly speed upthe data collection process. Of course, data for statevariables that are not directly measurable must becollected off-line by taking and analyzing samplesduring batch runs.

After data collection is performed, the staticoutput relation (eq. 20) can be learned by the neuralnetwork. Unfortunately, state space dynamic rela-tions (eq. 16) cannot be learned directly from thecollected data, so some form of data preparationmust be done. In physics, a well-known methodcalled vector field reconstruction22 is used for pa-rameter estimation in ill-defined systems. The mainprinciple of the method is to sample data as a timeseries and then use a numerical method to approxi-mate time derivatives for particular trajectories. Thesame can be done with collected state space vari-ables data. For this purpose, a smoothing cubicspline polynomial23 is used (eq. 22) that interpolatesthe cubic polynomial between each pair of experi-mental points with additional smoothing criteriagiven by eq. 23.

S t a t t b t t c t t di i i i i i i i( ) ( ) ( ) ( )� � � �3 2

i n� �1, ,(22)

SSQ p W y S t pS

tti i i

i

� � �

��

�

�� [ ( )] ( )2

2

2

2

1d

dd (23)

After analytical derivation of the cubic polyno-mials, derivatives on the left-hand side of eq. 16 areobtained for every training point, so static neuralnetwork (eqs. 3 – 5) can be used.

The third important decision is the choice ofa training method. As already mentioned, themultilayer perceptron network is trained by the useof analytical determination of error gradients –backpropagation algorithm along with parameter tofind a numerical method such as the steepest de-scent, conjugate gradient or Levenberg-Marquardmethods. Due to the fact that classical backpropaga-tion algorithm is slowly convergent, in this paper,modification of the basic algorithm, the so-calledOWO-HWO method by Manry et al.,24 was ap-plied. This modification allows much faster conver-gence, but the network must have additional con-straints because liner output neurons must be usedwhile the net may have only one hidden layer ofneurons.

After the networks have been trained, i.e. theiroptimal coefficients have been determined, soft sen-sor for immeasurable coordinates of state vector isobtained for every particular mode (operation) byintegrating the dynamical system (eqs. 16 and 20).In practice, mismatch problems may arise due tothe mismatch between the actual process and the


F i g . 9 – Detailed internal view of neural layer construc-tion

network model and/or existence of measurementnoise. Therefore, state observer25 could be used (eq.24)

d

d

�( � , ) [ ( � , )]

xf x u K y h x u

t� � (24)

The matrix K represents observer gain. Usuallyin the case of model mismatch only, the gain is cal-culated using the extended Luenberger observer al-gorithm, whereas if process noise is also present,the extended Kalman estimator method for gain cal-culation is used. In both cases, model gradientsneed to be calculated and this could be accom-plished analytically using a backpropagation algo-rithm, so this is an additional indication in favor ofusing a multilayer feedforward network.

Generic neural model control

During the flexible plant design phase, the ba-sic control for particular equipment modules is usu-ally defined as classical PID control, either in a sin-gle loop or in cascade. The main problem that couldoccur in process implementation is control perfor-mance deterioration, caused by strong loop interac-tion (e.g. dosing and temperature). This effect canbe minimized by using a neural network model as ageneric process model and implementing genericneural model control (GNMC). In the literature, thesame method has already been applied by Abonyiet al.,26 and Gosak and Vampola.20 The method usesa neural model as the particular model in Lee andSullivan,27 the well-known generic model controlmethod. The basis of the method is the idea that theoutput set point dynamic can be maintained at a de-sired level as shown by expression 25.

d

dd

yK y y K y y

tt

SPSP SP

t��

�� 1 2

0

( ) ( ) (25)

By the transformation shown in eq. 26, the out-put can be related to model the following equation

d

d

d

d

d

d

y h x u h

x

x h

xx u

t t tf� � �

( , )( , )

�

�

�

�(26)

Finally, the model is introduced in the control-ler expression as shown in eq. 27

�

�

h x uf x u

( , )( , )

t�

� � �K y y K y y1 20

( ) ( )SP SPt

td(27)

To apply generic model control, eq. 27 shouldbe solved by control input vector u. Depending onmodel complexity, this can be done analytically or

numerically. In the case of a neural network model,the first multiplicand expression on the left-handside of eq. 27 can be calculated analytically bybackpropagation algorithm, and then the resultingequation must be solved numerically. However,there is a special case of GNMC when state vari-ables are directly controlled, either if they are mea-surable or calculated by software sensor. In thiscase, eq. 28 is used instead of eq. 25:

d

dd

xK x x K x x

tt

SPSP SP

t��

�� 1 2

0

( ) ( ) (28)

Then, eq. 27 equivalent expression is given byeq. 29

f x u K x x K x x( , ) ( ) ( )� � �1 20

SP SPt

td (29)

If eq. 28 is the equation of the controller, thenin the case of a neural model, instead of the numeri-cal solution of eq. 28, the partial inversion of theneural network can be done by generating a set ofdata using eq. 16 and then using this data set fortraining the network (eq. 30) by exchanging placesof vectors u and dx/dt (assuming they have thesame dimension).

u f xx

��

��NNinv

t,d

d(30)

The generic neural model control techniquehas, in the context of used neural network abstrac-tion layer, a distinct advantage that in eqs. 27 and29 neural model plays the part of PI controller biasterm, therefore no change in equipment modulesbasic control is necessary; neural model processcontrol enhancement can be switched on and offwithout changes to the initially designed equipmentmodule or recipe.

Process optimization

The third possibility to enhance the initial rec-ipe with neural model is to use it as a basis for addi-tional optimization. The process inputs that are notengaged in the feedback loop can be used to opti-mize process performance if a suitable optimizationcriterion can be set. According to the optimal con-trol theory, a process can be optimized by minimiz-ing a suitable criterion (eq. 31), taking the processmodel into account, as well as the control inputboundaries.

J t t tt

t j

[ ( )] [ ( )] ( , )x x x u0 0

0

� �� d

� �i i iu� �(31)


In literature, there are many methods for solv-ing the optimal control problem, but in this work,the iterative dynamic programming method byLuus28,29 is chosen, because it uses preset bound-aries in the search for a solution, and from the com-putational point of view, it only requires integrationof a process model. These characteristics make thismethod suitable for the present purpose.

The basis of the iterative dynamic program-ming (IDP) method is to search for P piecewiseconstant or linear control policy over P time stagesof equal time length L. The performance index isthus approximated by eq. 32;

J t P t t t tj Kt

t

K

P

K

K

[ ( ), ] [ ( )] [ ( ), ( )]x x x u0 11 1

� � �

�� d (32)

The algorithm works as follows (Bojkov andLoos):

1. Divide the total time interval into P stages oflength L.

2. Choose the number of x grid points N andthe number of M allowable values for each controlvariable.

3. Choose the initial size of a search region ri

for each control variable.

4. Integrate the process model from t0 to tf withN evenly distributed control values in the allowableregion, thus generating N values of x in each stage.

5. Starting with stage P, for each x grid pointintegrate the model from tf – L to tf with each al-lowable control value. For each grid point choosecontrol that minimizes the value of J.

6. Move to stage P-1 and repeat the integrationfrom this segment to final time choosing (from step5) control that gives the nearest x value.

7. Repeat step 6 for all stages.

8. Reduce the size of the search region by �(eq. 33)

r r( ) ( )j j� �1 � (33)

9. Increment iteration index j by 1 and go tostep 4. Stop the procedure when there is no im-provement in minimization of J.

As previously mentioned, the method allowseasy checking of search boundaries ensuring thatconstraints given by design space are satisfied.Also, it uses neural model only for integration (andnot for calculation of higher order derivatives andother transformations), which enables the neuralnetwork to be used only for the purpose it wastrained for. All this increases modeling accuracy.

Experimental

Automated mini plant Chemreactor Büchi

The experimental study was performed in anautomated reaction and distillation mini plantChemreactor C60 produced by Büchi Glass Uster.The control system used for actual method imple-mentation was Siemens PCS7. System PCS7 con-sists of one AS station (Simatic S7 400 processor)and one OS station, based on Pentium PC computerrunning MS Windows OS.

Basic control layer was built using SiemensCFC tool. Adjacent phase logic level is pro-grammed using PCS7 specific SCL language. Userinterface and trend visualization were developed onSiemens WinCC SCADA system. Recipes were im-plemented30 using Batch Flexible tool which oper-ates as an add-on to WinCC software. Fig. 10shows the schematic description of the process unit.

During software design the following equip-ment modules were implemented:

– Heating/cooling module

– Mixing module

– Pressure module

– Distillation module

– pH module

– Dosing module.

For each module a set of equipment phaseswere created. As an example, Fig. 11 shows phasesof Heating/cooling module and the parameters thatneed to be set for one phase of the module.

By using the method of recipe creation as de-scribed in Methodology, new recipes can be createdefficiently in a very short time by defining opera-tions, choosing phases for parallel execution inthem, and parameterization of phases using SFC ed-itor of Batch Flexible software.

Wastewater treatment by distillationand oxidation

As an example of a batch process that can beconstructed using the described equipment,wastewater treatment is performed. This processconsists of several operations, i.e., charging ofwastewater, purging the equipment with nitrogen,removal of solvent by batch distillation, adjustmentof pH value, addition of hydrogen peroxide for de-toxification, degradation of the peroxide surplusamount and cooling down the reaction mixture. Fig.12 shows and example of a complete batch run per-formed in the test plant with marked operationswitches. The purpose of this run was validation ofrecipe without the use of neural optimization.


Testing the process improvement achievedby neural network

While completion of all phases in the recipe isnecessary for successful wastewater treatment batchrun, improvements can be expected in only two ofthem – removal of solvent and detoxification by hy-drogen peroxide. Other operations are not specificto this process, and are normally optimized duringplant startup period.

Normally, the developed method should usedata gathered from either on-line or off-line mea-surement obtained during the batch run. In order todo better testing of the developed neural modelingand optimization method for those two operationsor modes of hybrid automation, thermodynami-cally-based mathematical models were developedusing process and literature data, and performingthe reaction kinetic determination experiments inthe laboratory. Data from the simulation using de-veloped models were then used for training net-works.

The solvent removal phase is actually batchdistillation of methanol – water mixture intendedfor the removal of methanol from the wastewater. Ifthe principle of theoretical equilibrium stage is


F i g . 1 0 – Schematic representation of Chemreactor Buchi Uster with defined process modules

F i g . 1 1 – Phases of Heating/cooling module implementedon Chemreactor unit and parameters of reactortemperature phase

F i g . 1 2 – Wastewater treatment process data log pre-sentation for typical batch with labeled opera-tions

used, the resulting model is given by eqs. 34–37,while vapor liquid equilibrium is given by eq. 38with the assumption of near constant relativevolatilities.

d

d

H

tD tB � ( ) (34)

d

d

XX Y X X

B

BN B B Nt H

V D� � 1

[ ( ) ( )] (35)

d

d

XX X

n

nn nt H

V� �

11[ ( )

� � D Vn n n n( ) ( )]X X Y Y1 1

(36)

d

d

XY X

D

DDt H

V� 1

1[ ( )] (37)

YX

Xi

i i

j jj

n�

� �

�

�1 11

1

( )

i n� � 1 1, , (38)

The corresponding neural model is given byeqs. 39 – 41

d

d

X

tf X X H DB NN

B D B� ( , , , ) (39)

d

d

X

tf X X H DD NN

B D B� ( , , , ) (40)

d

d

H

tD tB � ( ) (41)

Model training has been done using a data setobtained by several simulated runs on different con-stant values of distillate flow. The data collected fortop and bottom composition was then interpolatedby cubic spline and derivatives were calculated forall points generated by spline interpolation. Fig. 13

shows interpolated data, derivation data, and corre-sponding network approximation for one batch run.

In order to cover the complete process range ofinterest, sets of batch runs are performed, and train-ing and validation sets are formed. Because datagathered in this way belongs to different time se-ries, poor fit or overfit inside particular runs can beprevented by adjusting the number of spline inter-polated points. This process can be easily con-trolled graphically by plotting data against time. Agreater problem is ensuring the quality of networkinterpolation of runs that belong to the time seriesof data not used in training. This was done by form-ing the validation sets from batch runs not includedin the training sets (i.e. obtained from batches withdistillate flows not used for training). As long as theresults were unsatisfactory, the training set was in-creased by performing new batch runs by simula-tion using the physical model. Figs. 14 and 15 showthe final results of training. Fig. 14 shows networktraining results on the training set, and Fig. 15 onthe validation set.

The developed neural model is used to performthe task of optimizing the distillate concentration dur-ing the batch run. If methanol concentration in thedistillate is constant and sufficiently high, the regener-


F i g . 1 3 – Learning of spline approximated derivative oftop and bottom methanol concentration duringdistillation operation

F i g . 1 4 – Neural network approximation of distillationtraining set

F i g . 1 5 – Neural network approximation of distillationvalidation set

ated solvent can be used directly in the production.For this purpose, the optimizing criteria is given byeq. 42, i.e. the optimal process is carried out if, duringthe run, the concentration of methanol as a product iskept constant, and in the end the desired quantity ofmethanol-free wastewater remains in the reactor.

J H t H X t X tf f f i i

t f

12 2

0

� � �[ ( ) ] [ ( ) ]* * d (42)

Optimization was performed using the neuralmodel with ten distillate flow segments. Fig. 16shows the calculated optimal distillate flow thatkeeps methanol concentration at 99.5 % and 95.0 %respectively during the batch distillation operation.

Generic neural model control was tested on ox-idation operation. Considering the fact that hydro-gen peroxide oxidizes a large number of unknowncomponents present in the wastewater, the formalkinetic relation based on peroxide concentrationonly is developed (eq. 43)

r k CA AaT b� � � (43)

Based on the kinetic model, the material andenergy balance of the process is given by eqs. 44 – 46:

d

d

d

d

C

t VF C V k C C

V

tA

rin A

inr A

aT bA

r� � � � !

"#$

�1 ( ) (44)

d

d

T

t V CF C T T

r pAin in pA in ref�

� � �

1

%%[ ( )

� � � � � � k V C H U A T TT

V

V

tr A jr& ( )]

d

d

(45)

d

d

T

tF

mT T

U A

C mT T

j

cmcm

cmcm j

pcm cmj� �

�

%( ) ( ) (46)

The neural network equivalents for the modeldescribed by eqs. 44 – 46 is given by eqs. 47 – 49

d

d

C

tf F C T VA NN

in A r� ( , , , ) (47)

d

d

T

tf F C T T VNN

in A j r� ( , , , , ) (48)

d

d

T

tf T T T

j NNj cm� ( , , ) (49)

Networks 47 – 49 were trained in open loopprocess setup using data generated by process model(eqs. 44 – 46) by alternating sinusoidal control func-tions and random step functions (with inlet peroxideflow and thermostat temperature as inputs) in orderto capture complete possible dynamic ranges of vari-ables. The concentration network was the most diffi-cult to train (eq. 47) due to instant changes in con-centration in the reactor directly following inputflow change. As for distillation, the total data set wasdivided into training and validation sets. Fig. 17shows the results of concentration network simu-lation compared with complete training set, whileFig. 18 shows the same for the validation set.


F i g . 1 7 – Concentration network approximations in com-parison with training set

F i g . 1 8 – Concentration network approximation in com-parison with validation set

F i g . 1 6 – Optimal distillate flow calculated with IDPmethod with goal of keeping the top concentration of methanolin distillate at two fixed values (XD = 99.5 and 99.0)

After the training, the network models are usedfor generic neural model control testing. For thispurpose, networks 47 and 49 were partially invertedas in eq. 29 in order to obtain explicit relations forcontrol inputs Fin and Tcm. The comparison betweenclassical PI cascade control of temperature and re-sulting GNMC control calculated by simulation isshown in Fig. 19.

Discussion

Besides the method here proposed, the task ofoptimization and batch process improvement can beachieved by direct recipe manipulation with thepurpose of optimization (Verwater-Lukszo, Šel etal.).31,32 Verwater-Lukszo proposed a method for di-rect recipe optimization called FRIS (Flexible rec-ipe improvement system) based on recipe adapta-tion. This adaptation is practically accomplished bythe development and application of recipe adapta-tion set. The recipe adaptation set is prepared byutilizing combination of design of experiments,modeling and experimental optimization.

The main practical difference between recipemodification and the method introducing neuralprocess abstraction layer here proposed is that thelatter is tailored specifically to be in accordancewith the quality by design approach. Since QbD ap-proach allows free changes in the process as long asthey do not move the process state space outside theboundaries of design space, no change in batch doc-umentation is necessary from run to run. The neuralnetwork model works on process control inputs(manipulated variables) only, which means thatbatch data log is sufficient to document the differ-ences between batch runs.

The necessary requirement for application ofthe proposed method is to build the recipe in theway described in this work. While this modification

does not affect recipe flexibility, a problem mayarise when attempting to apply this method to somebatch plant with a different method of recipe build-ing (ISA S88.01 allows and expects high flexibilityin recipe formulation). The proposed method canstill be applied, but change in recipe formulationmay change the way the hybrid process model isbuilt, so instead of hybrid automata, some other hy-brid system model may be more appropriate. Con-sequently, changes in neural model structure andtraining may be necessary.

Training of neural network and its usage arethe most important issues in application of the pro-posed method. Most of the difficulties arise fromthe necessity to approximate dynamical nonlinearmodel with neural network. In literature, there aremany different methods developed involving recur-rent (and for this reason either discrete or continu-ously dynamical) networks training. Unfortunately,most dynamical networks suffer from one or bothcommon problems – complex and hardly conver-gent methods for training, as well as so-called “van-ishing gradient problem” (gradually forgetting ofolder data dynamics behavior while learning fromthe more recent data set). For this reason, continu-ous state space representation of the neural networkis used. Use of the spline smoothing and derivationof resulting polynomials enables overcoming thetypical abovementioned problems during learning.One problem should nevertheless be taken into ac-count – the problem of network regularization. It iswell known that the main problem in using anymultivariable nonparametric regression is the prob-lem of data overfitting. Overfitting usually occurswhen an applied network is able to learn more com-plex information than actually needed for a givendata set (too many neurons or too many hidden lay-ers in the network). When approximation of themultivariable function is performed, it is very diffi-cult to predict possible overfitting. The simplestway to deal with this problem is to divide the train-ing set into learning and verification sets. If verifi-cation fails, it is possible to resort to prematurestopping of numerical minimization (learning algo-rithm) or repeating the learning with a less complexneural network. Things are somewhat different inthe case of trajectory learning. The task of the neu-ral network in this case is to approximate the trajec-tories presented as the training set, as well as to in-terpolate correctly between the trajectories, i.e. ap-proximate correctly dynamical behavior of the sys-tem. Learning the trajectories in the training setmay be accomplished by increasing the numberof learning points after smoothing spline interpola-tion has been performed. Approximation quality(smoothness of trajectories) can be easily tested byplotting. On the other hand, interpolating between


F i g . 1 9 – Comparison of reactor temperature control dur-ing oxidation reaction with classical PID controland using generic neural model control

trajectories is much more difficult to achieve. Inthis work, it was attained by dividing the smoothedtrajectory data into two sets. One set was used forlearning and the other for verification. If verifica-tion had failed, the network would be retrained withcombined sets and another verification set would beproduced from plant data. This procedure was re-peated until verification passed. While this methodis operational, its drawback is that sufficiently largedata sets must be available. In the case of insufficientdata, the alternative would be the application of theleave-one-out method as described by Specht33–35

(treating trajectory as data point in static functionapproximation), but so far there is not enough expe-rience to predict applicability of this method.

While in this work only neural models areused, due to the desire to achieve maximum possi-ble generality, it is always advantageous to incorpo-rate additional existing process knowledge into adeveloped model. If the use of a full thermodynam-ically-based model is impractical, there are twomain approaches for incorporation of knowledgenot accessible by the neural model alone.

The first method involves development of hy-brid models which include some parts of the physi-cal model. For example, Hosen et al.36 used hybridneural model to control polystyrene batch polymer-ization reaction where the first principle kineticmodel was used as a part of the overall neural model.

The second method involves the combining ofAI methods in order to incorporate different kindsof process knowledge offered by different algo-rithms. Especially interesting is the combination ofneural and fuzzy systems due to favorable incorpo-ration of symbolic knowledge into numeric neuralmodel. Examples of such neuro-fuzzy systems aregiven by Causa et al.37 and Simon and Hunger-buhler.38

In practical implementation of a developedmethod it is also important to address the problemof software realization of neural network applied. Itis obvious that software modules for data gatheringand network training should be preferably placedon SCADA system or some separate computer onindustrial LAN network. On the other hand, the net-work execution modules along with weight setsneed to be in PLC or DCU system, because theyhave the logical role of a layer between IO modulesand basic control of equipment modules. In the caseof the PCS7 system, they can be implemented withthe same tool used for basic control programming –continuous function chart (CFC) tool. If PLC doesnot have sufficient processing power or a lowerprogramming language is used for software devel-opment, a possible solution is to use a separate pro-cessor for network execution.

Conclusion

The presented method of installing the neuralnetwork layer between the process and basic con-trol level in a recipe-controlled batch plant, isshown to have potential in dealing with the task ofconstant gradual improvement by movement ofcontrol space (parameter operating range space) in-side the design space.

In future work, attention should be paid to fur-ther development of methods of hybrid neural mod-eling that would allow implementation of neuralprocess layer to a broader class of recipe structures.In addition, dynamic neural network training meth-ods with reduced data set should also be a topic ofinterest.

L i s t o f s y m b o l s

H liquid holdup, molD distillate flow, mol s–1

X molar fraction in liquid phaseY molar fraction in vapor phaseV vapor flow, mol s–1

t time, s� relative volatilityCA molar concentration, mol L–1

F flow, L h–1

Vr reactor volume, LT temperature, K (oC)% density, kg m–3

&H reaction enthalpy, J mol–1

A area, m2

U heat transfer coefficient, W m–2 K–1

Cp heat capacity, J mol–1

m mass, kg

I n d i c e s

B bottomD distillaten theoretical stage numberf finalr reactorj jacketin inletcm cooling mediumNN neural network

L i t e r a t u r e

1. ANSI/ISA-95.00.01–2000, Enterprise-Control System In-tegration, Part 1: Models and Terminology.

2. ANSI/ISA S88.01–1995(2011), Batch Control Part 1:Models and Terminology.


3. Lepore, J., Spavins, J., Journal of Pharmaceutical Innova-tion,. 3 (2008) 79.

4. Davies, B., Ellis, S., Pharmaceutical Technology Europe,17 (2005) 17, 22.

5. Sanchez, A., Para, L., Baird, R., Macchietto, S., ISATransactions 42 (2003) 401.

6. Moor, T., Raisch, J., Supervisory Control of Hybrid Sys-tems In: Behavioural Framework. Systems and ControlLetters, Elsevier, New York, 1999, 157–166.

7. Potoènik, B., Bemporad, A., Torrisi, F., Mušiè, G., Zu-panèiè, B., Control Engineering Practice 12 (2004) 1127.

8. Goebel, R., Sanfelice, R., Teel, A., Hybrid Dynamical Sys-tems, Princeton University Press New Yersey, 2012.

9. Lennartson, B., Tittus, M., Egardt., B., Pettersson, S.,IEEE Control Systems 16 (1996) 45.

10. Barton, P., Banga, J., Galan, S., Computers and ChemicalEngineering 24 (2000) 2171.

11. Fabre, F., Hetreux, G., Le Lann, J., Zarate, P., Computersand Chemical Engineering 35 (2011) 2098.

12. Haykin, S., Neural networks and learning machine,Prentice Hall, New York, 2009.

13. Masters, T., Advanced algorithms for neural networks,John Wiley, New York, 1995.

14. Hrycej, T., Neurocontrol, John Wiley, New York, 1997.15. Paengjuntuek, W., Thanasinthana, L., Arpornwichanop,

A., Neural Neurocomputing 83 (2012) 158.16. De Jesus, O., Hagan, M., IEEE Transaction on Neural

Network 18 (2007) 15.17. Werbos, P., Proceedings of IEEE 78 (1990) 1550.18. Williams, R., Zipser, D., Neural Computation 1 (1989) 270.19. Nerrand, O., Roussel-Ragot, P., Urbani, D., Personnaz, L.,

Dreyfus, G., IEEE Trans. on Neural Networks 5 (1994)178.

20. Gosak, D., Vampola, M., Generic Neural Model Control ofBatch Distillation Process In: Advances in Process Control6 , Love, J. (Ed.). Rugby: ICHEME UK, 2001. 137 – 144.

21. Alur, R., Henzinger, T. A., Lafferriere, G., Pappas, G., Pro-ceedings of the IEEE 88 (2000) 971.

22. Le Sceller, L., Letellier, C., Gouestbed, G., Physics LettersA 211 (1996) 211.

23. De Boor, C., Practical guide to splines, Springer, NewYork, 2001.

24. Chen, H. H., Manry, M. T., Chandrasekaran, H., Neuro-computing 25 (1–3) (1999) 55.

25. Grewal, M., Andrews, A., Kalman filtering theory andpractice using MATLAB, John Wiley and Sons, NewYork, 2001.

26. Abonyi, J., Madar, J., Szeifert, F., Combining first princi-ples models and neural networks for generic model controlIn: Soft Computing in Industrial Applications – RecentAdvances, Roy, R., Koppen, M., Ovaska, S., Furuhashi, T.,Homann, F. (Eds.), Springer Engineering Series, 2001,111–122.

27. Lee, P., Sullivan, G., Computers and Chemical Engineer-ing 12 (6) (1988) 573.

28. Bojkov, B., Luus, R., Industrial & Engineering ChemistryResearch 31 (1992) 1308.

29. Schroder, A., Mendes, M., Computers and Chemical Engi-neering Supplement (1999) S491.

30. Vampola, M., Gosak, D., Šoštarec, A, Pavlièiæ, D., Vraneš,N., Hraniloviæ, M., Development of the Batch ReactionControl System Using the ISA S88 Batch Control Stan-dard, In: Computers in Technical Systems and IntelligentSystems, Budin, L. (Ed.). Rijeka MIPRO, 2002.

31. Verwater-Lukszo, Z., Computers in Industry 36 (1998) 279.32. Šel, D., Hvala, N., Strmènik, S., Milaniè, S., Šuk-Lubej, B.,

Control Engineering Practice 7 (1999) 1191.33. Larsen, J., Svarer, C., Nonboe Andersen, L., Hansen, L. K.,

Adaptive Regularization in Neural Network Modeling In:Orr, G. B., Müller, K. (Eds.) Neural Networks: Tricks ofthe Trade, Lecture Notes in Computer Science 1524, Ger-many: Springer-Verlag, 1998, 113–132.

34. Specht, D., IEEE Transactions on Neural 2 (6) (1991).35. Boser, B., Guyon, I., Vapnik, V., A training algorithm for

optimal margin classifiers In: 5th Annual Workshop onComputational Learning Theory, Pittsburgh, ACM, 1992,144–152.

36. Hosen, M., Husain, M., Mjalli, F., Control EngineeringPractice 19 (2011) 454.

37. Causa, J., Karer, G., Nunez, A., Saez, D., Škrjanc, I., Zu-panèiè, B., Computers and Chemical Engineering 12(2008) 3254.

38. Simon, L., Hungerbuhler, K., Chemical Engineering Jour-nal 157 (2010) 568.


cabeq 2012-03 verzija 3silverstripe.fkit.hr/cabeq/assets/uploads/cabeq-2012-03-3.pdf · process...

Documents