data-driven modeling and dynamic programming applied to ...€¦ · batch cooling crystallization....

Data-Driven Modeling and Dynamic Programming Applied to BatchCooling CrystallizationDaniel J. Griffin, Martha A. Grover,* Yoshiaki Kawajiri, and Ronald W. Rousseau

School of Chemical & Biomolecular Engineering, Georgia Institute of Technology, Atlanta, Georgia 30332-0100, United States

*S Supporting Information

ABSTRACT: In this article, we demonstrate a model-based approach for controlling the average size of crystals produced bybatch cooling crystallization. The method is distinguished most notably in the modeling strategy. Rather than developinga crystallization model within the population-balance framework, as is usually done, we apply a machine-learning technique toidentify an empirical model from measurement data. The model is low-dimensional and can therefore be discretized and usedwith dynamic programming to obtain optimal control policies for producing crystals of targeted average sizes in prespecifiedbatch run times. Experimental results are reported that demonstrate the use of the identified policies to produce crystals of thedesired average sizes in the specified run times.

1. INTRODUCTION

Batch crystallization is a cornerstone operation in the productionof high-purity pharmaceuticals and fine chemicals; yet, establish-ing control over these operations often represents a majorbottleneck in the overall manufacturing process. The controlchallenge stems from complicated nonlinear and stochasticcrystallization dynamics. If no in-process measurements areavailable, such dynamics make it difficult to establish a relation-ship between the process inputs (e.g., temperature) and the finalcrystal properties.Fortunately, the pharmaceutical and fine-chemical industries

have recently championed the development of process analyticaltechnology (PAT).1,2 PAT tools, which can be used to monitorkey solution-state and aggregate crystal-state properties in realtime, facilitate the characterization of crystallization dynamicsand make it easier to connect the operating inputs with the keyproperties of the final crystals. Moreover, these tools enable theapplication of feedback control.Indeed, the development of PAT has markedly improved

control over crystallizationwith many researchers leveragingPAT tools to achieve some degree of control over crystal size,shape, purity, and polymorphic form.1−12 Despite advancements,however, control of crystallization in industry remains anoutstanding issue. In our view, progress might be impeded bythe restrictive dichotomy of control approaches, which almostalways fall into one of two categories: population-balance-model-based control or model-free control. These two approaches arebriefly reviewed below.The population balance (PB) provides a rigorous framework

for modeling the time evolution of dispersed-phase systemproperties, such as the size distribution of crystals.13−17 Toestablish a population-balance model, rate equations for theunderlying phenomena must be specified. For example, to trackthe crystal size distribution, a PB model must include a rateexpression for crystal growthpossibly along with expressionsfor nucleation, agglomeration, and breakage. In addition tothese rate expressions, energy and mass balance equations areneeded to connect the time evolution of the dispersed-phase

system properties to the process variables that can bemanipulated.3 The result is a mathematical model composedof integro partial differential algebraic equations.Once established, population-balance models can be used to

develop optimal control schemes, but PB models are complexand must be solved numerically, making their use for controldifficult. Even more troubling is the fact that, for all of theircomplexity, PB models often fall short of capturing importantattributes of the observed dynamics. If not properly taken intoaccount, this model mismatch can degrade the performance ofthe model-based control scheme.11,12,18,19

Model-free control schemes, on the other hand, aredeveloped from expert intuition and heuristics rather thanrigorous mathematical models. Although many model-freecontrol schemes have been developed, two have becomeparticularly popular: supersaturation control (SSC)20−27 anddirect nucleation control (DNC).28−33 Both of these schemescan be used to indirectly control final crystal properties bymanipulating relevant variables measured in real time (super-saturation in SSC and some measure of the number of crystalsin DNC).Model-free schemes, such as SSC and DNC, are generally

more conducive to application than those that rely onpopulation-balance models, but there are also disadvantagesto a model-free approach. The link established between theoperating variables and the crystal properties is heuristic innature and usually indirect. This makes it hard to control thecrystal properties to reach prespecified target values and evenharder to evaluate the optimality of the operation.In comparison, population-balance-model-based control

approaches are rigorous but cumbersome and difficult to execute,whereas model-free approaches are conducive to application butalso heuristic and indirect.

Received: September 28, 2015Revised: December 23, 2015Accepted: January 18, 2016Published: January 19, 2016

Article

pubs.acs.org/IECR

© 2016 American Chemical Society 1361 DOI: 10.1021/acs.iecr.5b03635Ind. Eng. Chem. Res. 2016, 55, 1361−1372

pubs.acs.org/IECR

http://dx.doi.org/10.1021/acs.iecr.5b03635

Some effort has been made to bridge this gap by developingmore direct model-free schemes. For example, Nagy andco-workers34,35 proposed a methodology for selecting a super-saturation set point to achieve a prespecified target crystal sizedistribution. (The set point can then be implemented in themodel-free SSC scheme or implemented by “direct design”.8)However, to arrive at the appropriate supersaturation setpoint, a population-balance model was developed and usedunder the assumption that growth is the only crystallizationmechanism.Recently, we have also explored more direct model-free

control schemes, including a rule-based scheme for producingcrystals of target average sizes.36 In that case, a feedback schemewas used to control the mass per counta metric related toaverage crystal size that can be measured online. By such atechnique, specific average crystal sizes could be achieved, butwithout the use of a mathematical model, the required batchtime could not be calculated and no definitive conclusionscould be made about the optimality of the operation.The present work considers a control approach for unseeded

batch cooling crystallization that does not fit into the above-described dichotomy of established control strategies. Thecontrol scheme is model-basedand therefore can be used toformulate an optimal policybut the model is not developedwithin the population-balance framework. Instead, the modelis empirical and learned from experimental data gathered usingcommon process analytical technology.The mass-count framework developed previously36 is a

precursor for the data-driven modeling strategy developedhere. In particular, it was demonstrated that crystal size andyield could be controlled by manipulating two properties thatare measured online: the total crystal mass measured onlineby attenuated total reflectance Fourier transform infrared(ATR-FTIR) spectroscopy and the total chord count measuredby the focused-beam reflectance measurement (FBRM)technique. The empirical model that we develop herein describesthe dynamics of these two properties as a function of theircurrent values and the prevailing supersaturation. Such a modelprovides a link between the supersaturation profile (input) andthe average size and yield of the product crystals (output).Although this model is not capable of capturing the same

level of detail as a population-balance-equation model, thereis a notable advantage in the model’s simplicity. The low-dimensional state space and bounded input can be easilydiscretized without significantly compromising accuracy. Undersuch a formulation, one can then apply dynamic programming37,38

to identify optimal state-feedback control policies for producingsalt crystals of prespecified average sizes in set batch times.

The control strategy is tested in a series of experimentsimplemented on a laboratory-scale crystallizer. Specifically, wedesign three test runs with different target crystal sizes and batchrun times. For these runs, optimal control policies are developedby the described empirical modeling−dynamic programmingstrategy and then implemented as part of a cascade feedbackloop. At the end of each run, the crystals are filtered and sizedby sieve analysis. The results demonstrate that the strategy canbe implemented, in practice, to produce crystals of the desiredaverage sizes in the prespecified batch times.

2. CONTROL METHODOLOGY: DATA-DRIVENMODELING AND DYNAMIC PROGRAMMING

Crystallization dynamics are complex, stochastic, and highlynonlinear. This poses challenges for control. To establishreliable control over this type of system, new innovative controlstrategies are needed. One strategy that shows potential is thecombination data-driven modeling and dynamic programming.39,40

We aim to develop such a strategy here.Figure 1 provides a schematic outline of the methodology

that we propose for establishing a control policy. In the firststep, a learning method is applied to identify an empirical modelof the crystallization dynamics using measurement data recordedin previous crystallization runs. In the second step, dynamicprogramming is applied to identify the optimal state-feedbackcontrol policy. The overall effect is the extraction of a controlpolicy from past run data.

2.1. Data-Driven Modeling: Empirical Model ofAggregate Crystallization Dynamics. Crystallization dy-namics are commonly modeled from “first principles” usingpopulation-balance-equation models. We propose an alternativemodeling strategy in which an empirical model of crystallizationdynamics, expressed in terms of aggregate crystal-state pro-perties, is learned from measurement data. This section presentsthe model framework and describes the learning method.

2.1.1. Model Framework. The mass-count frameworkdeveloped in a previous work36 serves as a precursor to theempirical modeling strategy developed here. In this pastwork, the average crystal size was shown to be a function oftwo aggregate crystal-state properties that could be measuredonline, namely, the total chord count and the total crystal mass,denoted as c and m, respectively. We use these two propertiesto define the crystal state, x

≡ ⎡⎣⎢

⎤⎦⎥

cmx

Two comments should be made about this definition. First,whereas the crystal mass is an intrinsic property of the crystal

Figure 1. Schematic outline of the proposed methodology for establishing a control policy.

Industrial & Engineering Chemistry Research Article

DOI: 10.1021/acs.iecr.5b03635Ind. Eng. Chem. Res. 2016, 55, 1361−1372

1362


population irrespective of the measurement system, the totalchord count is a value obtained from a particular instrument.In treating the total chord count as a property of the crystals,we are considering an abstraction of the measure. That is, weconsider each crystal population to be associated with a truechord count, of which focused-beam reflectance measurementsprovide an unbiased measure. Second, x, which contains justtwo aggregate properties, does not fully capture the crystal statein the sense that it contains all of the information neededto uniquely describe the crystal population. We are workingoff the hypothesis that this reduced state representation cancapture the key dynamics and that feedback control can be usedto mitigate unmodeled effects.Treating the total chord count and crystal mass as the two

relevant crystal-state properties, the objective is to identify amathematical expression that predicts the evolution of theseproperties under the influence of any adjustable input variables.Taking supersaturation to be the single input variable, wewrite the following discrete-time expression for crystallizationdynamics

− = Δτ τ τ τ+ F u tx x x( , )1 (1)

where τ denotes the time index, xτ+1 is the crystal state at timetτ+1, xτ is the state at time tτ, uτ is the supersaturation (input) atthis same time, and Δt is the time interval from tτ to tτ+1.This expression assumes that the dynamics are Markovian

(i.e., the dynamics depend on only the current state and input).Because our state dimension is only 2, an easy-to-understandphase-plane picture is associated with such a model. This pictureis shown in Figure 2.

To complete the model, the function F(xτ,uτ) in eq 1 mustbe specified. A priori we have some qualitative understandingof the crystallization dynamics that can be used to inform thefunction choice. We assume that supersaturation is the primaryvariable, driving crystallization when positive (when the solutionis supersaturated) and driving dissolution when negative (whenthe solution is undersaturated). More specifically, in terms of theaggregate crystal-state properties, we expect the rates of changeof the total chord count and crystal mass to

(1) be zero when the solution is saturated,(2) be positive when the solution is supersaturated and

negative when the solution is undersaturated, and(3) increase monotonically with the level of supersaturation.

This a priori knowledge can be expressed mathematically asthe following constraints on the function F(xτ,uτ):

(1) F(x,u) = 0 if u = 0 for any x,

(2) uF(x,u) ≥ 0 for any x, and

(3) ∂F/∂u ≥ 0 for a fixed x.

These constraints narrow the choice of functions somewhat,but a wide class to choose from still remains. To identify theappropriate function to express the dynamics, we take a data-driven approach. That is, we allow the model to be learnedfrom past run data.Although a number of strategies, such as artificial neural

networks and Gaussian process modeling, exist for learninggeneral nonlinear functions from data,41 our a priori knowledgemotivates the use of a tailored approach. The learning methodthat we propose is locally weighted, constrained least-squaresregression, with local weighting to capture the unknownnonlinear effect of the crystal state on the dynamics andconstrained regression to incorporate the a priori knowledge.This learning method is described in greater detail in thefollowing subsection.

2.1.2. Learning Method. The learning method is defined bythe hypothesis set (i.e., the pool of candidate functions) and thelearning algorithm (i.e., the mechanism for choosing among thecandidate functions).41

Hypothesis Set. We restrict the hypothesis set to polynomialfunctions with respect to supersaturation u. For our application,a sixth-order polynomial with zero constant provides enoughflexibility and easy computation. Adding the expressedconstraints yields the following hypothesis set

β

β

= | =

∈ ≥ ∂ ∂ ≥×

F F u u u u

uF u F u

x x

x R x

{ ( , ) [ ... ] ( ),

( ) , ( , ) 0, and / 0}

2 6

2 6 (H)

This hypothesis set enforces the desired constraints with respect tothe effect of supersaturation on the dynamics but leaves the effectof the crystal state on the dynamics unspecified with one caveat:The influence of the crystal state on the dynamics is secondaryto the influence of supersaturation. This is accomplished byincorporating the effect of the crystal state through thepolynomial coefficients (β).

Learning Algorithm. The learning algorithm that we use toselect the function from the hypothesis set is locally weightedleast-squares regression. This algorithm can be expressed asfollows

∑

|| Δ − Δ ||

∈

=

w k F u t

F u

x

x x x x

x

for a given position

minimize [ ( , ; ) ( , ) ]

subject to ( , )

F j

N

j j j1

22

train

where Ntrain is the number of samples in the training data set,w(x, xj;k) = exp(−||x − xj||2

2/k) is a weight function with oneadjustable parameter denoted by k, uj represents the jthmeasured supersaturation in the training set, and Δxj representsthe jth measured subsequent change in state over the “next”time interval Δt.Taking into account the specific hypothesis set H, the overall

learning method can be written explicitly as follows

Figure 2. Phase-plane picture associated with the mathematical modelof crystallization dynamics, F(x,u). At each point x, the function Fpredicts the forward change in position for a given level ofsupersaturation, u.



1363


∑β β

β

β β

β β

β β

β β

β β

= || Δ − Δ ||

≥

≥

β

∈ =×

⎡

⎣

⎢⎢⎢⎢⎢⎢⎢⎢

⎤

⎦

⎥⎥⎥⎥⎥⎥⎥⎥

w k u u u t

u u u u u

u u u u u u

x

x x x

0

0

for a given position

arg min( ( , ; ) [ , , ..., ] )

subject to [ ... ] , for all

[2 3 4 5 6 ] , for all

j

N

j j j j jx

R

[ ]

1

2 622

2 6

2 3 4 5

2,1 2,2

3,1 3,2

4,1 4,2

5,1 5,2

6,1 6,2

2 6

train

(A)

where the solution, denoted β[x], parametrizes the local model

β= F u u u[ , , ..., ]j j jx2 6 [ ] around x. The method used to solve

eq A, and the selection of the weight function is described inSupporting Information sections S.1.2 and S.1.3.2.1.3. Final Form of the Empirical Model. The empirical

model obtained from the learning method has coefficientsβ that change with each queried crystal state. As a result, onecannot write an explicit equation for the global model [F(x,u)for all x]. The local model is, however, expressed by a simpleset of equations. In particular, the local model at x is given bytwo polynomial functions of u, with the parameters found bysolving eq A

β β β

β β β

ΔΔ

= = + + +

ΔΔ

= = + + +

xt

F u u u u

xt

F u u u u

x

x

( , ) ...

( , ) ...

x

x x x

x

x x x

11 1,1

[ ]1,2[ ] 2

1,6[ ] 6

22 2,1

[ ]2,2[ ] 2

2,6[ ] 6

Alternatively, the global model F(x,u) can be visualized withvector fields in the mass-count phase plane that show themodel-predicted movement from different states under differ-ent inputs. This idea is introduced in Figure 2, and such avisualization is used to show the final model learned from datain Figure 5 in section 4.2.2. Dynamic Programming: Optimal State-Feedback

Policies for Targeted Crystal Sizes. This section formalizesthe control objectives and then outlines the application ofdynamic programming to obtain an optimal state-feedback controlpolicy.2.2.1. Conceptual Control Objectives and the Correspond-

ing Optimal Control Formulation. Controlling the averagecrystal size and yield corresponds to controlling the final chordcount and crystal mass (cf. section 3.4). Consequently, ourprimary control objective is to reach a target final mass-countposition, x⊕, in a fixed batch time. Additionally, we would liketo run the operation with minimal energy input. The collectivecontrol objective is formalized as follows

∑ρ ε

τ

+

= Δ + = −

=

ττ

τ τ τ

=

−⊕

+

τ−

u d

F u t N

x x

x x x

x x

minimize[ ( ) ( , )]

subject to ( , ) , 0, ..., 1

u u

N

N

r

,..., 0

1

1

0init

N0 1

In this expression, N denotes the total number of time intervals(i.e., tN is the batch time), and the cost function contains two

terms, a running cost that we weight by the adjustable parameterρ ∈ R+, and a terminal cost.The running cost is given by the summation of the input-

ef fort function

ε ≡u u( ) 2

The terminal cost is given by the distance-to-target functionevaluated for the final position, d(xN,x

⊕), defined as

λ≡ − − =⊕ ⊕ ⊕ ⎡

⎣⎢⎤⎦⎥d x x x x Q x x Q( , ) ( ) ( ), where

1 0

0N N N

T2

The input-effort function reflects the following facts: Crystalliza-tion or dissolution will spontaneously bring the system tosaturation; maintaining undersaturation or supersaturationduring the operation requires a continual temperature changeand, consequently, the input of energy.The distance-to-target function corresponds to the normalized,

squared Euclidean distance between the mass-count position andthe target position. In this measure, we have scaled the chordcount by λ = 1/25 (gram/no.) to provide approximately equalweights to the chord count (no.) and mass measurements (ingrams).Given the input-effort and distance-to-target functions, the

optimization problem reads as follows: “Find the time series ofminimal inputs (supersaturation set points) that brings thesystem to the target position in the given batch time.” This alignswith the conceptual objectives already specified, but it misses apractical consideration, which is that we want the system to besettled at the end of the run; that is, we do not want the crystalmass and chord count to be transient at the end of the batch.To preference trajectories that move toward the target as directly

as possible and then settle, we add the term (tτ/tN)γd(xτ, x

⊕) to therunning cost, resulting in the following optimal control formulation

∑ ρε

τ

+ +

= Δ + = −

=

ττ

γτ τ

τ τ τ

=

−⊕ ⊕

+

−

t t d u d

F u t N

x x x x

x x x

x x

minimize{ [( / ) ( , ) ( ) ( , )]}

subject to ( , ) , 0, ..., 1

u u

N

N N

r

,..., 0

1

1

0init

N0 1

(OPT)

where γ ∈ R+ represents a second adjustable parameter. Thechoice of optimization parameters is discussed in more detail inSupporting Information section S.2.

2.2.2. Dynamic Programming to Identify the OptimalState-Feedback Policy. The control formulation in eq OPTyields the optimal open-loop control input profile. As thedeterministic model of dynamics F(xτ,uτ) is only an approxima-tion of the true dynamics, we expect improved control to comefrom a state-feedback or closed-loop control strategy.The current standard approach for establishing such

control in industry is online model predictive control (MPC).This strategy, as applied classically, is as follows. At each timeinterval, a finite-horizon open-loop control problem is posed

∑ ρε

τ

+

+

= Δ + = −

=

ττ

γτ τ

τγ

τ

=

+ −⊕

+⊕

+

+ −

t t d u

t t d

F u t t N

x x

x x

x x x

x x

minimize{ [( / ) ( , ) ( )]

( / ) ( , )}

subject to ( , ) , , ..., 1

u u t

t

N

N

r

t t

,...,

1

1

t0 1



1364

http://pubs.acs.org/doi/suppl/10.1021/acs.iecr.5b03635/suppl_file/ie5b03635_si_001.pdf



where xt is the current measured state and is the selectedtime horizon.Let ut*, ..., ut+τ−1* , represent the solution to this optimization

problem. Once solved, the first input in the series, ut*, isapplied, and the process is repeated: With each new position,the computation is repeated for the shifted time horizon, andthe first input of the new sequence of optimal control moves isapplied. This, in effect, is a complicated state-feedback controlstrategy, ut = ϕ mpc(xt). However, to apply such a strategy, onemust have a fast and robust method for solving the givennonlinear optimization problem online.With a low-dimensional state and input, another approach is

available: One can treat the state and input as discrete variablesand then use dynamic programming to obtain the optimal state-feedback control policy offline in the form of a lookup table(such a strategy is sometimes referred to as explicit MPC42).This approach is briefly outlined here.Let χ denote a set of center points for cells of the discretized

state space, and let u denote a discrete set of inputs. Underdiscretization, we change the representation of the dynamics.Where we previously described the point-to-point dynamics,we now describe the dynamics with a cell-to-cell mapping43,44

= Δ ≡ || − Δ + ||τ τ τχ

τ τ τ τ+∈

+τ+

u t F u tx x x x x( , , ) arg min [ ( , ) ]x

1 1 22

1

where maps the current position xτ ∈ χ under the inputuτ ∈ to a new position xτ+1 ∈ χ over the time step Δt.With the dynamics represented as a cell-to-cell mapping, one

can apply the Bellman or dynamic programming principleto obtain the optimal state-feedback control policy.45,46 Thisalgorithm is outlined in Table 1.It is useful to express the feedback policy obtained offline

from dynamic programming in a slightly different way, so that itcan be compared with online MPC. The optimal control policyfrom dynamic programming can be expressed as follows

∑ ρε

τ

ϕ

* | * | = +

+

= Δ −

= * |

ττ

τγ

τ τ

τ τ τ

τ τ

−∈

−⊕

⊕

+

τ ττ

τ

u u t t d u

d

u t N

u

x x

x x

x x

x

, ..., arg min{ [( / ) ( , ) ( )]

( , )}

subject to ( , , ), , ..., 1

( )

Nu

N

N

N

t

x x

x

dp, dp, 1

1

1

dp, dp,

Here, one can see that, as long as the discretization does notintroduce significant error, dynamic programming yields a policy

that is the same as would be obtained by applying online MPCif the time horizon is selected to span from the current timeinterval to the final batch time. The advantage of dynamicprogramming in this case is clear: One can obtain the optimalstate-feedback control policy offline in the form of a lookuptable and avoid having to perform online optimization.

2.2.3. Form of the Optimal State-Feedback Control PolicyObtained from Dynamic Programming. The state-feedbackcontrol policy obtained from dynamic programming is expressedas a lookup table. This table indicates the input (supersaturationset point) to apply at each time step at each different discretestate, x ∈ χ. Such a policy can be shown as a time-varying colormap of mass-count space, where the color at each position(state) indicates the input suggested by the lookup table. Thistype of visualization is used to show policies for reachingdifferent targets in Figures 6−8 in section 4.

2.3. Additional Details and Open-Source Code.Additional details on the empirical modeling strategy and optimalcontrol formulation can be found in Supporting Informationsections S.1 and S.2. The MATLAB code developed toimplement the learning algorithm and apply dynamic program-ming is publically available and can be found at http://dgriffin36.github.io/Crystallization-Control-Feedback-Policy-from-Data/.

3. EXPERIMENTAL SYSTEM: MEASUREMENTS ANDMETHODS

3.1. Crystallization System. Crystallization ofNa3SO4NO3·H2O from an aqueous solution containingNa+, SO4

−, and NO3− was studied. Each crystallization run

started from a clear solution containing 110 g of NaNO3/100 gof H2O and 7.25 g of Na2SO4/100 g of H2O at 80 °C underatmospheric pressure. Batch cooling crystallizations wereimplemented using an OptiMax workstation from Mettler-Toledo that was equipped with instruments for focused-beamreflectance (FBR) measurements, attenuated total reflectanceFourier transform infrared (ATR-FTIR) absorbance measure-ments, and temperature measurements.We used this same crystallization system, which represents

a very simple simulant of electrolytic nuclear waste solutions,in previously reported studies.36,47−49 The online monitoringmethodologies developed for these earlier studies are alsoused here. In addition, the data set used to learn the empiricalmodel that we present was constructed from data recorded forcrystallization runs reported in these past studies.

Table 1. Algorithm: Dynamic Programming to Obtain the Optimal State-Feedback Control Policy (Discretized State and Input)



1365


http://dgriffin36.github.io/Crystallization-Control-Feedback-Policy-from-Data/

http://dgriffin36.github.io/Crystallization-Control-Feedback-Policy-from-Data/


3.2. Online Measurements. Solution-state and aggregatecrystal-state properties were tracked in real time using FBR,ATR-FTIR, and temperature measurements.3.2.1. Concentration from ATR-FTIR Measurements. Infra-

red absorbance measurements were used to infer the con-centrations of SO4

− and NO3− (in grams of solute per gram of

solvent) during batch cooling operations. Solution infraredabsorbance measurements were made with a ReactIR systemfrom Mettler-Toledo. The calibration strategy used to quantifythe relationship between infrared absorbance measurementsand concentration is given in previous work.47 The resultingcalibration model provided an overall unbiased prediction ofthe anion concentrations with an average absolute relative errorof less than 5%.To correct for day-to-day measurement fluctuations, we also

implemented a slope-bias correction procedure at the start of

each run. In this procedure, 10 infrared spectra were recordedfor pure water, and 10 infrared spectra recorded for the initialmixture with the solute fully dissolved. The mismatch betweenthe calibration-model predictions and the known concen-trations for these samples was then used to establish a slope-bias correction.

3.2.2. Solubility from the Solubility Trace Methodology.Prior to each crystallization run, the solubility of Na3SO4NO3·H2O in the multicomponent solution was identified with thesolubility trace method.49 The procedure is as follows: Thesolution is cooled to drive crystallization, allowed to equilibrateat a low temperature, and then slowly heated. During the heatingstage, the solution composition, which should be approximatelyin equilibrium, is tracked by ATR-FTIR spectroscopy. Thecomposition−temperature curve obtained during the heatingstage provides the solubility−temperature relationship.

3.2.3. Supersaturation from Concentration and Temper-ature Measurements. For single-component, nondissociatingsolutes, the relative supersaturation provides a measure of thedriving force for crystallization. This measure is defined as

σ = −C C TC T

( )( )

S

S

where C is the concentration of the solute in solution andCS(T) is the solubility concentration at the given temperature T.Molar supersaturation is the analogue of this measure for

dissociating salts.49 For Na3SO4NO3·H2O, the molar super-saturation is given by

σ = −

−

+ − −

+ − −

+ − −

X X X X

X T X T X T X T

X T X T X T X T

(( )

{[ ( )] ( ) ( ) ( )})

/{[ ( )] ( ) ( ) ( )}

x[ ]Na

3SO NO H O

NaS 3

SOS

NOS

H OS

NaS 3

SOS

NOS

H OS

42

3 2

42

3 2

42

3 2

where Xj denotes the mole fraction of the jth component, andXjS(T) denotes the solubility mole fraction of that component

at the current temperature T. In this work, the term super-saturation specifically refers to the molar supersaturation ofNa3SO4NO3·H2O.

Figure 3. Experimental correlation observed for the mass per countmeasured online and the average crystal size measured by sieveanalysis offline. The region between the dashed lines provides the 95%confidence interval based on the given correlation.

Figure 4. Schematic of the cascade feedback loop used to apply optimal state-feedback control policies obtained from dynamic programming.



1366


3.2.4. Total Crystal Mass from Concentration Measurements.For the closed system (unseeded batch cooling crystallization),a mass balance was applied to infer the total crystal mass atany given time from the initial solution composition and thecurrent measured solution composition (by IR spectroscopy).In this way, the total crystal mass was obtained from infraredabsorbance measurements made online.3.2.5. Total Chord Count from FBRM Measurements. The

total chord count from FBRM is an approximate, relativemeasure of the number of crystals suspended in solution. In thiswork, measurements were made over 30-s intervals. For ourparticular instrument, the minimum size particle that could bedetected was approximately 1 μm.3.3. Offline Crystal Size Measurement by Sieve

Analysis. Sieve analysis was applied offline to obtain anindependent measure of the average crystal size, defined as

∑ ≡=

s m s M( )/i

n

i im1

bins

where mi is the mass of crystals in the ith sieve bin, si is themidpoint sieve diameter of the bin, nbins denotes the number ofbins in the stack, and M is the total mass of the sieved crystals.Sieve analysis was always applied at the end of the crystalli-

zation using the entire crystal sample following a two-stepprocedure: (1) The crystals were filtered with a Buchner funneland washed sparingly with cold deionized water. (2) The still-wetcrystals were distributed evenly across the top sieve tray, and thestack was placed in a RO-TAP sieve shaker for 100 min.3.4. Mass per Count as an Online Measure of the

Average Crystal Size. We use the mass-per-count metric as

an online measure of the average crystal size.48 To validate themeasurement and quantify the relationship, we compared themass per count measured online against the average crystal sizemeasured by sieve analysis offline. The experimental data andcorrelation used in this work are shown in Figure 3.

3.5. Cascade Feedback Loop Used to Apply OptimalControl Policies. Optimal control policies (expressed aslookup tables) can be applied using a cascade feedback loopwith three tiers: The supersaturation set point is determinedaccording to the policy in the outermost loop; the correspond-ing temperature set point is then calculated in the middle loop;and finally, proportional−integral (PI) temperature control isapplied in the innermost loop. The cascade structure is shownin Figure 4.

4. RESULTS AND DISCUSSIONIn gathering results, our objective was to experimentally test theoverall control methodology. The procedure used was simple:We first selected three size-yield targets and specified batchtimes. Next, we established the optimal control policies for

Figure 5. Visualization of the empirical model, F(x,u). The arrows give the forward change in crystal state over a 30-s interval predicted by F for eachdifferent crystal state (base of the arrow) under the supersaturation indicated in the title of each subplot.

Table 2. Run Targets and Batch Times for the ThreeDesigned Test Runs

runtarget final position(count, mass (g))

target crystal size(μm)

batch time(min)

1 (300, 7) 420 (261−579)a 302 (200, 10) 540 (381−701) 603 (75, 11) 775 (612−937) 120

a95% confidence interval for the measured average crystal size if themass-count target is hit exactly.



1367


reaching each of these targets (by the method proposed insection 2). Finally, we applied the control policies to theexperimental system, measuring the average size of the crystalsproduced in each case. The results demonstrate that themethodology canfor the studied system, at leastbe appliedto control batch cooling crystallizations to produce crystals ofthe desired average size in the specified batch times.The results are organized into four subsections: Section 4.1

describes the empirical model of aggregate crystallizationdynamics learned from data; section 4.2 outlines the designedtest runs; section 4.3 presents the control policies developed

for each of these runs; and finally, section 4.4 reports theexperimental results.

4.1. Empirical Model of Aggregate CrystallizationDynamics. As described in section 2.1, experimental datawere used to develop a mathematical model of the dynamicsin terms of the measurable aggregate crystal-state proper-ties [F(x,u) in eq 1]. This model is nonparametric withrespect to the crystal state, and therefore, the global modeldoes not have a fixed analytical expression. Nevertheless, themodel can be represented graphically with a series of phase-plane diagrams. Figure 5 presents a graphic representation of

Figure 6. Visualization of policy 1, the optimal state-feedback control policy for run 1.




1368


the model learned from data for the crystallization ofNa3SO4NO3·H2O.4.2. Test Run Design. We designed three runs to test the

control methodology. For each run, we set the batch timeand selected a mass-count target, which corresponds to a targetaverage crystal size and yield; these are given in Table 2. Therun targets were selected to span a significant portion ofthe reachable mass-count region (see Supporting Information

section S.3.1). The batch times were chosen to be slightly longerthan the minimum required batch time calculated according theempirical model (Supporting Information section S.3.2).

4.3. State-Feedback Control Policies for Reaching theRun Targets. For each designed test run, we applied dynamicprogramming to identify the optimal state-feedback policyaccording to the empirical model show in Figure 5. These policiesspecify the optimal supersaturation set point as a function of the


Figure 9. Deterministic simulation results for crystallizations under each control policy: (a) crystallization trajectories, (b) distance-to-target profiles,(c) temperature profiles, (d) optimal supersaturation profiles.



1369




current mass-count position and time. That is, each policy is givenby a lookup table that matches the current mass-count positionwith a supersaturation set point at each time instant. Thesepolicies can be visualized as a time-varying color map in mass-count space; the optimal state-feedback control policies for eachof the three designed runs are shown in Figures 6−8.From these figures, it is clear that the optimal policy varies

depending on the run target and set batch time. Even so,we can make a general observation from these color maps thatmight be anticipated: When both the crystal mass and chordcount are below the target, the policies almost always suggestpositive supersaturation to drive crystallization; conversely,when both the crystal mass and chord count are above the target,the policies almost always dictate undersaturation (negativesupersaturation) to drive dissolution.Color maps of the control policies alone, however, do not

provide much insight into the expected performance of thecontrol scheme. To further examine the expected performanceof the control policies, simulations are useful. The followingsubsection presents deterministic simulation results for batchcooling crystallizations under the three different policies. In

addition, stochastic simulation results are provided in SupportingInformation section 4.2.

4.3.1. Deterministic Simulations: Best-Case Performanceof the Control Policies. We ran deterministic simulations toprobe the control performance in the ideal case where theprocess-model mismatch is zero, the measurements are exact,and the input (supersaturation) is applied precisely withoutdelay.The simulated results for the three test runs under the

identified optimal control policies are displayed in Figure 9.Figure 9a shows the simulated crystallization trajectories undercontrol. Panels b−d of Figure 9 show the corresponding timeprofiles for the distance-to-target, temperature, and super-saturation, respectively. For each run, the policies drive thesimulated crystallization trajectory to end at mass-countpositions near the target positions in the desired batch time.The expected final crystal sizes (according to the simulated finalmass-count positions) are listed in Table 3.

4.4. Experimental Results. We experimentally tested thecontrol policies identified for the three designed test runs.Figure 10 shows the measured crystallization trajectories forthese three runs. As can be seen, in each run, the appliedcontrol policy was successful in guiding the MC trajectory toend at a position near the target position.In addition to monitoring the crystallization trajectories,

the crystals produced by each run were filtered from solutionand sieved. From sieve analysis, we obtained a measure of theaverage crystal size. These results, presented in Table 4, showthat the measured average crystal sieve diameter aligns closelywith target average crystal size for each of the three runs.The key end results are highlighted in Figure 11, which

compares the target mass per count and crystal size with the

Table 3. End Results Predicted by Deterministic Simulation

run

target finalposition (count,

mass (g))target average

crystal size (μm)

predicted finalposition (count,

mass (g))predicted averagecrystal size (μm)

1 (300, 7) 420 (261−579)a (299, 7.2) 424 (265−584)a

2 (200, 10) 540 (381−701) (215, 10.4) 535 (376−696)3 (75, 11) 775 (612−937) (95, 11.2) 724 (559−882)

a95% confidence interval for the measured average crystal size for thecorresponding mass-count positions.

Figure 10. Measured profiles for experimental crystallizations under each control policy: (a) measured crystallization trajectories, (b) measureddistance-to-target profiles, (c) measured temperature profiles, (d) supersaturation set point profiles.



1370




measured final mass per count and average crystal size. Theseresults support the claim that the proposed methodology canbe used to control the final mass-count position and, thereby,control the average crystal size.

5. CONCLUSIONSIn this article, a data-driven model of batch cooling crystalliza-tion dynamics was developed outside the population-balanceframework. The simplicity of the model, which is low-dimensional, facilitated the use of dynamic programming toidentify optimal feedback control policies for producing crystals ofspecified average sizes in set run times. These policies were testedusing a lab-scale crystallizer. The results demonstrated that theobtained feedback policies could indeed be applied to producecrystals of the targeted average sizes in the specified batch times.

■ ASSOCIATED CONTENT*S Supporting InformationThe Supporting Information is available free of charge on theACS Publications website at DOI: 10.1021/acs.iecr.5b03635.

(S.1) Empirical modeling methodology, (S.2) optimalcontrol formulation, (S.3) use of the empirical model toanswer design questions, (S.4) model-based simulationsof the control performance, and (S5) application toindustrially relevant system (PDF)

■ AUTHOR INFORMATIONCorresponding Author*Tel.: +1 404 894 2878. Fax: + 1 404 894 2866. E-mail:[email protected].

NotesThe authors declare no competing financial interest.

■ ACKNOWLEDGMENTSFinancial support was provided by the Consortium for RiskEvaluation with Stakeholder Participation (CRESP), the NuclearEnergy University Program (NEUP), the Georgia ResearchAlliance, and the Cecil J. “Pete” Silas Endowment. We alsoacknowledge Professor Evangelos A. Theodorou in the AerospaceEngineering department at Georgia Tech for early discussionsand input on the application of dynamic programming.

■ REFERENCES(1) Yu, L. X.; Lionberger, R. A.; Raw, A. S.; D’Costa, R.; Wu, H. Q.;Hussain, A. S. Applications of Process Analytical Technology toCrystallization Processes. Adv. Drug Delivery Rev. 2004, 56, 349−369.(2) Simon, L. L.; Pataki, H.; Marosi, G.; Meemken, F.; Hungerbuhler,K.; Baiker, A.; Tummala, S.; Glennon, B.; Kuentz, M.; Steele, G.;Kramer, H. J. M.; Rydzak, J. W.; Chen, Z.; Morris, J.; Kjell, F.; Singh,R.; Gani, R.; Gernaey, K. V.; Louhi-Kultanen, M.; O’Reilly, J.; Sandler,N.; Antikainen, O.; Yliruusi, J.; Frohberg, P.; Ulrich, J.; Braatz, R. D.;Leyssens, T.; Von Stosch, M.; Oliveira, R.; Tan, R. B. H.; Wu, H.;Khan, M.; O’Grady, D.; Pandey, A.; Westra, R.; Delle-Case, E.; Pape,D.; Angelosante, D.; Maret, Y.; Steiger, O.; Lenner, M.; Abbou-Oucherif, K.; Nagy, Z. K.; Litster, J. D.; Kamaraju, V. K.; Chiu, M.-S.Assessment of Recent Process Analytical Technology (PAT) Trends:A Multiauthor Review. Org. Process Res. Dev. 2015, 19, 3−62.(3) Rawlings, J. B.; Miller, S. M.; Witkowski, W. R. ModelIdentification and Control of Solution Crystallization Processes: AReview. Ind. Eng. Chem. Res. 1993, 32, 1275−1296.(4) Braatz, R. D. Advanced Control of Crystallization Processes.Annu. Rev. Control 2002, 26, 87−99.(5) Braatz, R. D.; Fujiwara, M.; Ma, D. L.; Togkalidou, T.; Tafti, D.K. Simulation and New Sensor Technologies for IndustrialCrystallization: A Review. Int. J. Mod. Phys. B 2002, 16, 346−353.(6) Ulrich, J.; Jones, M. J. Industrial Crystallization: Developments inResearch and Technology. Chem. Eng. Res. Des. 2004, 82, 1567−1570.(7) Barrett, P.; Smith, B.; Worlitschek, J.; Bracken, V.; O’Sullivan, B.;O’Grady, D. A Review of the Use of Process Analytical Technology forthe Understanding and Optimization of Production Batch Crystal-lization Processes. Org. Process Res. Dev. 2005, 9, 348−355.(8) Fujiwara, M.; Nagy, Z. K.; Chew, J. W.; Braatz, R. D. First-Principles and Direct Design Approaches for the Control ofPharmaceutical Crystallization. J. Process Control 2005, 15, 493−504.(9) Lovette, M. A.; Browning, A. R.; Griffin, D. W.; Sizemore, J. P.;Snyder, R. C.; Doherty, M. F. Crystal Shape Engineering. Ind. Eng.Chem. Res. 2008, 47, 9812−9833.(10) Heinrich, J.; Ulrich, J. Application of Laser-BackscatteringInstruments for In Situ Monitoring of Crystallization ProcessesAReview. Chem. Eng. Technol. 2012, 35, 967−979.(11) Nagy, Z. K.; Braatz, R. D. Advances and New Directions inCrystallization Control. Annu. Rev. Chem. Biomol. Eng. 2012, 3, 55−75.(12) Nagy, Z. K.; Fevotte, G.; Kramer, H.; Simon, L. L. RecentAdvances in the Monitoring, Modelling and Control of CrystallizationSystems. Chem. Eng. Res. Des. 2013, 91, 1903−1922.(13) Hulburt, H. M.; Katz, S. Some Problems in Particle Technology:A Statistical Mechanical Formulation. Chem. Eng. Sci. 1964, 19, 555−574.(14) Randolph, A. D. Population Balance For Countable Entities.Can. J. Chem. Eng. 1964, 42, 280−281.(15) Randolph, A. D.; Larson, M. A. Theory of Particulate Processes:Analysis and Techniques of Continuous Crystallization, 2nd ed.;Academic Press: San Diego, 1988.(16) Ramkrishna, D. Population Balances: Theory and Applications toParticulate Systems in Engineering; Academic Press: San Diego, 2000.(17) Ramkrishna, D.; Singh, M. R. Population Balance Modeling:Current Status and Future Prospects. Annu. Rev. Chem. Biomol. Eng.2014, 5, 123−146.

Table 4. End Results for the Three ExperimentalCrystallizations Runs Compared against the Targets forEach Run

run

target finalposition (count,

mass (g))

target averagecrystal size

(μm)

measured finalposition (count,

mass (g))

measuredaverage crystalsize (μm)

1 (300, 7) 420 (301, 5.3) 3862 (200, 10) 540 (198, 10.1) 5173 (75, 11) 775 (91, 10.8) 731

Figure 11. Comparison of the target mass per count and averagecrystal sizes with the measured final mass per count and average crystalsizes.



1371

http://pubs.acs.org

http://pubs.acs.org/doi/abs/10.1021/acs.iecr.5b03635


mailto:[email protected]


(18) Nagy, Z. K. Model Based Robust Control Approach for BatchCrystallization Product Design. Comput. Chem. Eng. 2009, 33, 1685−1691.(19) Zhang, K.; Nadri, M.; Xu, C. Z. Reachability-Based FeedbackControl of Crystal Size Distribution in Batch Crystallization Processes.J. Process Control 2012, 22, 1856−1864.(20) Fujiwara, M.; Chow, P. S.; Ma, D. L.; Braatz, R. D. ParacetamolCrystallization Using Laser Backscattering and ATR-FTIR Spectros-copy: Metastability, Agglomeration, and Control. Cryst. Growth Des.2002, 2, 363−370.(21) Feng, L. L.; Berglund, K. A. ATR-FTIR for DeterminingOptimal Cooling Curves for Batch Crystallization of Succinic Acid.Cryst. Growth Des. 2002, 2, 449−452.(22) Gron, H.; Borissova, A.; Roberts, K. J. In-Process ATR-FTIRSpectroscopy for Closed-Loop Supersaturation Control of a BatchCrystallizer Producing Monosodium Glutamate Crystals of DefinedSize. Ind. Eng. Chem. Res. 2003, 42, 198−206.(23) Liotta, V.; Sabesan, V. Monitoring and Feedback Control ofSupersaturation using ATR-FTIR to Produce an Active PharmaceuticalIngredient of a Desired Crystal Size. Org. Process Res. Dev. 2004, 8,488−494.(24) Zhou, G. X.; Fujiwara, M.; Woo, X. Y.; Rusli, E.; Tung, H. H.;Starbuck, C.; Davidson, O.; Ge, Z. H.; Braatz, R. D. Direct Design ofPharmaceutical Antisolvent Crystallization Through ConcentrationControl. Cryst. Growth Des. 2006, 6, 892−898.(25) Yu, Z. Q.; Chow, P. S.; Tan, R. B. H. Seeding and Constant-Supersaturation Control by ATR-FTIR in Anti-Solvent Crystallization.Org. Process Res. Dev. 2006, 10, 717−722.(26) Nagy, Z. K.; Chew, J. W.; Fujiwara, M.; Braatz, R. D.Comparative Performance of Concentration and TemperatureControlled Batch Crystallizations. J. Process Control 2008, 18, 399−407.(27) Hermanto, M. W.; Phua, A.; Chow, P. S.; Tan, R. B. H.Improved C-Control of Crystallization with Reduced CalibrationEffort via Conductometry. Chem. Eng. Sci. 2013, 97, 126−138.(28) Doki, N.; Seki, H.; Takano, K.; Asatani, H.; Yokota, M.; Kubota,N. Process Control of Seeded Batch Cooling Crystallization of theMetastable α-Form Glycine Using an In-Situ ATR-FTIR Spectrometerand an In-Situ FBRM Particle Counter. Cryst. Growth Des. 2004, 4,949−953.(29) Chew, J. W.; Chow, P. S.; Tan, R. B. H. Automated In-lineTechnique Using FBRM to Achieve Consistent Product Quality inCooling Crystallization. Cryst. Growth Des. 2007, 7, 1416−1422.(30) Woo, X. Y.; Nagy, Z. K.; Tan, R. B. H.; Braatz, R. D. AdaptiveConcentration Control of Cooling and Antisolvent Crystallizationwith Laser Backscattering Measurement. Cryst. Growth Des. 2009, 9,182−191.(31) Abu Bakar, M. R.; Nagy, Z. K.; Saleemi, A. N.; Rielly, C. D. TheImpact of Direct Nucleation Control on Crystal Size Distribution inPharmaceutical Crystallization Processes. Cryst. Growth Des. 2009, 9,1378−1384.(32) Abu Bakar, M. R.; Nagy, Z. K.; Rielly, C. D. Seeded BatchCooling Crystallization with Temperature Cycling for the Control ofSize Uniformity and Polymorphic Purity of Sulfathiazole Crystals. Org.Process Res. Dev. 2009, 13, 1343−1356.(33) Saleemi, A.; Rielly, C.; Nagy, Z. K. Automated DirectNucleation Control for In Situ Dynamic Fines Removal in BatchCooling Crystallization. CrystEngComm 2012, 14, 2196−2203.(34) Nagy, Z. K.; Aamir, E. Systematic Design of SupersaturationControlled Crystallization Processes for Shaping the Crystal SizeDistribution using an Analytical Estimator. Chem. Eng. Sci. 2012, 84,656−670.(35) Aamir, E.; Rielly, C. D.; Nagy, Z. K. Experimental Evaluation ofthe Targeted Direct Design of Temperature Trajectories for Growth-Dominated Crystallization Processes Using an Analytical Crystal SizeDistribution Estimator. Ind. Eng. Chem. Res. 2012, 51, 16677−16687.(36) Griffin, D. J.; Grover, M. A.; Kawajiri, Y.; Rousseau, R. W. Mass-Count Plots for Crystal Size Control. Chem. Eng. Sci. 2015, 137, 338−351.

(37) Bertsekas, D. P. Dynamic Programming and Optimal Control, 3rded.; Athena Scientific: Belmont, MA, 2005; Vol. 1.(38) Puterman, M. L. Markov Decision Processes: Discrete StochasticDynamic Programming; John Wiley & Sons, Inc.: Hoboken, NJ, 2005.(39) Oguz, C.; Gallivan, M. A. Optimization of a Thin FilmDeposition Process Using a Dynamic Model Extracted from MolecularSimulations. Automatica. 2008, 44, 1958−1969.(40) Lee, J. H.; Wong, W. Approximate Dynamic ProgrammingApproach for Process Control. J. Process Control 2010, 20, 1038−1048.(41) Abu-Mostafa, Y. S.; Magdon-Ismail, M.; Lin, H.-T.; LearningFrom Data: A Short Course; AMLBook, 2012.(42) Alessio, A.; Bemporad, A. A Survey on Explicit Model PredictiveControl. Nonlinear Model Predictive Control: Towards New ChallengingApplications 2009, 384, 345−369.(43) Bursal, F. H.; Hsu, C. S. Application of a Cell-Mapping Methodto Optimal-Control Problems. Int. J. Control 1989, 49, 1505−1522.(44) Sun, J.-Q. Control of Nonlinear Dynamic Systems with the CellMapping Method. Advances in Intelligent Systems and Computing. 2013,175, 3−18.(45) Bellman, R. On the Theory of Dynamic Programming. Proc.Natl. Acad. Sci. U. S. A. 1952, 38, 716−719.(46) Bellman, R. E. Dynamic Programming; Princeton UniversityPress: Princeton, NJ, 1957.(47) Griffin, D. J.; Grover, M. A.; Kawajiri, Y.; Rousseau, R. W.Robust Multicomponent IR-to-Concentration Model Regression.Chem. Eng. Sci. 2014, 116, 77−90.(48) Griffin, D. J.; Grover, M. A.; Kawajiri, Y.; Rousseau, R. W.Combining ATR-FTIR and FBRM for Feedback on Crystal Size. InProceedings of the American Control Conference; IEEE Press: Piscataway,NJ, 2015; pp 4308−4313.(49) Griffin, D. J.; Kawajiri, Y.; Grover, M. A.; Rousseau, R. W.Feedback Control of Multicomponent Salt Crystallization. Cryst.Growth Des. 2015, 15, 305−317.



1372


data-driven modeling and dynamic programming applied to ...€¦ · batch cooling crystallization....

Documents