1. INTRODUCTION

1.1 A microorganism: a microscopic biochemical factory

A single cell is considered as a microscopic biochemical factory. Materials such as carbon, nitrogen, oxygen and others are brought into the cell and converted within the cell via the hundreds of reactions to the various constituents of the cell as well as to biochemical products which may be retained or transported back into the environment outside the cell . 1 metabolic activities inside the cell are regulated at various levels both inside and outside the cell. Moreover, biological activity of the cell is extremely sensitive to the environment it is exposed to. Because of this multi - level complex regulation, by an engineering point of view, it is of utmost importance to understand the nutritional and environmental factors affecting cell metabolism.

1.2 Bioreactor as a controlled environment for the growth of microorganisms

An engineer is always interested in consistently producing large quantities of product of interest over long periods of time. The best way to achieve this goal wants to be to grow the cells in a bioreactor where the cellular activity can be controlled efficiently. The three basic modes of bioreactor operation are batch, fed-batch and continuous. The control issues related to each of these will be discussed in later sections.

1.3 Importance of process control in bioreactors

Development of methods for monitoring and control of commercial bioprocessing including reliable and robust real-time sensors has been listed on outreaching priority for federal investment in a report published in July 95 from the Biotechnology Research Subcommittee (BRS) of the Committee on Fundamental Science of the The National Science and Technology Council. The best performance of the upstream processing can not be achieved without the knowledge of the state of the system and on control algorithms that can optimize the process. Controls of Bioprocesses is challenging, particularly in batch and fed-batch bioreactors, due to high degree of nonlinearity (meaning that nonlinear differential equations are required for mathematical modeling), and their potential for instability when they involve high-yield mutant or recombinant microorganism. These problems are further complicated by the scarcity of on-line real-time sensors and realistic models that capture the intricate complexities of biological systems.

2. Parameter characterizing a bioreactor

The presence of the living microorganisms inside the bioreactor makes it more complicated than the conventional chemical reactor. As already indicated, it is extremely important to gather the knowledge about the state of the bioreactor prior to design and implementation of any control system for the reactor. The complete state of the biochemical reactor can be assessed by knowing the following parameters: physical parameters, chemical (extracellular) parameters, biochemical (intracellular), and biological parameters. 3 The following subsections will list the various parameters in each category. The common methods for the determination of these parameters are discussed in section 9.2.

2.1 Physical parameters

The following are the important physical parameters for the operation of a bioreactor: agitation power, agitation speed, broth volume, color, expanded broth volume (density), foaming, gas flow rate, gas humidity, heat generation rate, heat transfer rate, liquid feed rate, liquid level, mass, osmotic pressure, pressure, shear rate, tip speed, temperature, turnover time, and viscosity. 3 Many of these parameters have important implications in the control of bioreactors.

2.2 Chemical Parameters

The following list gives the different parameters that define the chemical environment inside the reactor: amino acids, carbon dioxide (gas), cation level, conductivity, inhibitor, intermediate (s), ionic strength, Malliard reaction products, nitrogen (free and total) , nutrient composition, oxygen, pH, phosphorous, precursor, product, redox and substrate. 3

2.3 Biochemical (intracellular) parameters

Biochemical parameters are the intracellular parameters that indicate the metabolic state of the cell at any given time during the cell growth. These include amino acids, ATP / ADP / AMP, carbohydrates, cell mass composition, enzymes, intermediates, NAD / NADH, nucleic acids , total protein, and vitamins.

2.4 Biological parameters

Biological parameters characterize the bioreactor in terms of what is happening inside the reactor at the cellular level. The list includes age / age distribution, aggregation, contamination, degeneration, doubling time, genetic instability, morphology, mutation, size / size distribution, total cell count and viable cell count.

3. Control Strategies for Bioprocesses

All the bioreactors used oxygen at the present time use control strategies for three basic environmental factors: pH, temperature and dissolved. Invariably, these control implementations are achieved through regulation of flow rate of acid / base, flow rate of fluid through the cooling coil, and agitation respectively. Needless to say, these three parameters are extremely important for optimal cellular activity. But they alone do not guarantee the maximum productivity, which is the objective for most of the industrial fermentations. This paper will explore the control strategies which are used to accomplish this goal.

Before attempting to understand the details of the control strategies used for bioprocess, one should be familiar with the common features in the field of controls. One of the feature integrated into any control system is control algorithm. The control algorithm is that part of the control system that takes the available measurements and level of process understanding and decides on the best way to influence the process with the available manipulated variable to achieve The desired objective. 4   A control system can not be implemented unless the process under consideration is understood. An efficient way of understanding the process is a mathematical model of the process. A good process model is an invaluable tool to deveolp a control algorithm. It is not implied that controllers can not control poorly understood processes, indeed, that is often their function. However, an expensive, and time - consuming trial and error adjustment of the control algorithm is required in that case. 4

A common approach to obtain a simple, empirical model for controller design is to make small step changes in the inputs and observe the dynamic behavior of the outputs. One can then obtain a linear time - invariant process model in a straightforward fashion. A more fundamental approach is to formulate mass and energy balances for different components, resulting in a set of nonlinear ordinary differential equations. The latter approach has an advantage that the nonlinear model may better represent the process over a significant range of state values, whereas the linear empirical model resulted from the former approach may not be reliable for process states away from the state at which model is identified. This is particularly important in case of batch and fed - batch fermentations in which the process state changes significantly during operation. The disadvantage of the latter approach is that the available controllers - design tools are less developed for the nonlinear models. 4

There are three modes of bioreactor operation: batch, fed-batch and continuous. As discussed in subsequent sections of this paper, each of these modes presents different challenges to the control algorithm.

4. Bioreactor dynamics

4.1 Bioreactor as a multivariable system with nonlinear dynamics   Prior to discussing specific control applications, some general features of bioreactors relevant to control applications should be mentioned. The two main characteristics that are important to know before designing a control system for a bioreactor are:

Multivariable system: As one would anticipate, control of a bioreactor will involve a system involving many variables.

Nonlinear dynamics: The control of a bioreactor is complicated by the fact that nonsteady-state behavior is nonlinear. This has several consequences. Hysteresis is often observed. For example, a step increase in reactor feed rate in case of CSTBR (continuously stirred tank bioreactor) will result in a transient that will be different than when the corresponding equivalent step decrease in feed rate back to initial conditions is made. Moreover, multiple steady states are often observed for identical feed conditions, and in certain cases, exotic dynamics like limit cycles, oscillatory transients, long time lags may be exhibited. 1

The reasons for the above mentioned behaviors are ultimately related to the complexities of living cells. Finally, many of the important variables which are desirable for monitoring and control are only measurable with Large time lags or not measurable at all This gives scope for accurate mathematical models and / or state estimation techniques. Fortunately, simple models and single input - single output feedback loops are available and work well in many cases.

4.2 Cell growth modeling in a batch reactor

The simplest way to model cell growth will be to consider in unstructured, unsegregated model for cell growth. For this kind of model,   r x = dX / dt = mX (1)

where, r x = rate of cell generation (g / l-hr)

X = cell concentration (g / l) m = specific growth rate (hr -1)

The most commonly used expression that relates the specific growth rate of the cell to the substrate concentration is Monod's equation, which is given as   m = m max S / (K s + S) (2) where, m = specific growth rate (hr -1) m max = maximum specific growth (hr -1) S = substrate concentration (g / l) K S = saturation constant for substrate (g / l)   Figure 1 depicts the dependence of? on S according to Monod's equation. One should note that Monod's equation is empirical and does not have any mechanistic basis. 5 The equation is only valid for an exponentially growing culture under condition of balanced growth. The equation does not fair well in transient conditions. Despite its simplicity and no fundamental basis, it works

Figure 1 Monod's growth curve

surprisingly well in a large number of steady state and dynamic situations. This characteristic has important implications in control of bioreactors.

4.3 Continuous bioreactor dynamics

For a continuously fed bioreactor, the cells are continuously supplied substrate at growth limiting level, and hence they remain in the exponential phase. Since the cells remain in the exponential phase, Monod's equation can be applied. A cell balance on the reactor can be written as

FX - FX f + V (dX / dt) = r x (3) where F = volumetric flow rate (l / hr) X = cell concentration inside the reactor and in the outlet stream (g / l) F X = cell concentration in the feed (g / l) V = reactor volume (l) r x = rate of cell generation (g / l-hr)

For a sterile feed (X f = 0)) and noting that the reaction rate can be written in terms of the specific growth rate (r x = mX, equation (3) can be reduced to

dX / dt = (m - D) X (4) where D = dilution rate = F / V (hr -1)

A balance on the substrate yields the following equation

FS - FS f + V (dS / dt) = r s V (5) where F = volumetric flow rate (l / hr) S = cell concentration inside the bioreactor and in the outlet stream (g / l) S f = substrate concentration in the feed (g / l) V = reactor volume (l) r s = rate of substrate consumption (g / l-hr)

A yield parameters (Y x / s) is defined that relates the amount of cell mass produced per amount of substrate consumed, and is mathematically represented as

Y x / s = mass of cells produced / mass of substrate consumed x = r / r s (6)

Combining equations (1), (5), and (6) yields

dS / dt = D (S f - S) - mX / Y x / s (7)

The CSTBR (continuous stirred tank bioreactor) is now completely described by equations (4) and (7) with m given by equation (2). At steady state (with fixed Sf and D), the following are the values for m (specific growth rate), S (substrate concentration) and cell concentration (X)

m = D (8)

DK S = S / (m max - D) (9) X = Y x / s (S f - S) (10)

There are a few characteristics of an open-loop CSTBR that are conceptually different from that of a chemical reactor which are important to know before any control system for a bioreactor can be designed. Figure 2 shows that D must be less than m max for a realistic value of S to be achieved. The same conclusion can

Figure 2 Relationship between dilution rate and specific growth rate for a steady state CSTBR

be derived by looking at the steady state solution of equation (4). The two solutions are equation (8) and

X = 0 (11)

The corresponding substrate concentration is

S = S f (12)

Equation (11) and (12) define a situation called washout. This situation is encountered whenever the value of dilution rate equals or exceeds m max. A rigorous discussion of washout would point to the fact that whenever m (S f), ie, m f evaluated at S is less than? Max, then the critical dilution rate for washout will occur at D = m (S f), and not at D = m max. 1 The control algorithm should be completely aware of this unproductive state.

For the given set of equations, numerical solution is required since the system is described by two coupled nonlinear differential equations, ie, equations (4) and (7). Linear control theory can be applied only in a limited sense, ie, only near the steady state when the system model is linearized. 1

Start up is an important consideration as well. The general procedure in the start up avoiding washout would be to initiate cell growth in a batch mode until the exponential phase is reached. At this point, the sterile feed would be started with a dilution rate such that D <m (Sf). A non washout steady state would be reached after a transient phase. 1

4.4 Multiplicity and stability of steady states in a continuous bioreactor

Though the control loop of a CSTBR is simple, the system is complicated by the presence of multiple steady states and the stability considerations of these steady states. The following discussion will highlight these problems.

As already implied, the control design of a biological reactor described by equation (4) and (7) should take into account the nonlinear nature of these differential equations. Multiple critical points are common with nonlinear systems. This has been shown earlier in the discussion of washout. A systematic approach to an efficient control design will involve 1. calculation of the number of steady states 2. characterization of the nature of the steady states with respect to their stability 3. design of appropriate control loops based on the results from step 1 and step 2

4.4.1 Calculation of multiple steady states

Once the governing equations describing the system are in place, the steady states are found by replacing all time derivatives by zero. This can be done by inspection and algebric solution. For high order or complex models, a nonlinear root finding technique should be employed. 1

4.4.2 Stability of a steady-state

A steady state is stable if, for initial conditions near the steady state, all transients converge to it. If the transients diverge, steady state is called unstable. The diverging transients always end at some other stable state. Stability analysis of a steady state would involve Whether the steady state under consideration is stable or not and the information about state - to - state transitions in case of unstable steady states. 1

The information about the local stability and dynamics of the steady states is accomplished through a linear stability analysis. It should be borne in mind that the results of the linear stability analysis are good only near the steady state. For general (nonlocal) behavior and information about state - to - state transitions, generation of the phase plane is suitable. 1   4.5 Proportoinal control of a CSTBR with Monod's kinetics

Before designing the closed - loop continuous bioreactor, one should understand the open - loop CSTBR fully since the scope of closed - loop CSTBR will be given only by the knowledge about the open - loop CSTBR. Linear stability analysis and phase plane analysis for open - loop CSTBR and closed - loop CSTBR are detailed below.

4.5.1 Stability analysis of open - loop CSTBR

4.5.1.1 Linear stability analysis for open - loop CSTBR

As already discussed, for an open - loop CSTBR with Monod's kinetics, there exist two steady states, ie, a nontrivial steady state (defined by equation (8), (9), and (10)), washout and steady-state (defined by equation (11) and (12)). The Jacobian J for the system defined by equation (4), and (7) with m given by equation (2) is

where X '= dX / dt S '= dS / dt

Substituting the values for X 'and S' from equations (4) and (7) yields the value of Jacobian for the system as

For the nontrivial steady state, stability is guaranteed if the following equations are satisfied

Trace J <0 (15) Det J> 0 (16)

Which yields

-D - m'X / Y x / s <0 (! 7) Xm'm / Y x / s> 0 (18)

In case of Monod's equation, m '> 0 for all p. Hence, nontrivial state is always stable. For the washout state, the conditions for stability are derived from a similar procedure

m (S f) - D <0 (19) (m (S f) - D) (D)> 0 (20)

Equations (19) and (20) indicate that D must be greater than m (S f) for the washout steady state to be stable. Thus, any dilution rate which gives any realistic solution (X> 0, and S> 0) will result in washout being unstable. The same conclusion can be derived by the analysis phase plane which is also discussed in the next subsection.

4.5.1.2 Phase plane analysis for open - loop CSTBR

Construction of the phase plane for open - loop CSTBR be achieved through integrating equations (5) and (8), selecting several time points, plotting the values of S and X at each point, then repeating for new initial conditions or sketched directly from the results of the linear analysis. As shown in Figure 3, all-state initial conditions result in the achievement of the desired steady. 1

The motive for controlling this reactor would be to maintain a closed - loop system such that washout could be avoided regardless of flow fluctuations. An easy-state approach to achieve this would be to measure the cell concentration and manipulate the flow rate to force the reactor to nontrivial steady. This can

Figure 3 Open loop phase plane for bioreactor with Monod's kinetics (S R is the feed substrate concentration and alpha and beta are the steady-state cell and substrate concentrations)

easily be accomplished with a simple proportional controller Whose stability analysis is discussed in next subsection.

4.5.2 Stability analysis of closed - loop bioreactor

The governing equation for the proportional controller which manipulates the flow rate as a response to changing cell concentration inside the reactor is given by

D = D ss + K c (X - X sp) (21) Where, D = dilution rate that is manipulated by the controller (hr -1) D ss = dilution rate corresponding to the nontrivial steady state for X = X sp in

open loop CSTBR (hr -1) K c = controller gain (l / g-hr) X = cell concentration in the reactor (g / l) X sp = controller set point and the desired cell concentration in the reactor (g / l)

Substituting the value of D from equation (21) into equations (4) and (7) shows that X = 0, S = S f is no longer a steady-state solution. A rigorous analysis for this system will show that the worst case for this system as X approaches 0 corresponds to D = 0 This is equivalent to saying that the CSTBR wants to approach the behavior of a batch reactor.

The conditions for the stability of the system under consideration according to linear stability analysis are

K c - D ss - m '/ Yx / s <0 (22) K c + m '/ Y x / s> 0 (23)

Any positive value of K c is sufficient to satisfy equations (22) and (23), and hence guarantee stability. This state is not surprising keeping in mind the stability of nontrivial steady in the open - loop CSTBR. It seems fair to expect the closed - loop phase plane similar to open - loop phase plane for reasonable values of K c.

The whole discussion can be summarized as follows. Since the nontrivial state is always stable for realistic D values, there is little incentive for closed - loop operation other than to prevent washout from large flow disturbances. The incentive for closed - loop operation increases significantly if the growth kinetics are more complex, eg, substrate inhibited growth kinetics. This is discussed in the next section.

4.6 Control of a continuous bioreactor with substrate inhibition kinetics

Though Monod's kinetics makes a nice model for substrate - limited cases, it does not approximate the real cases very well since all the biological systems are inhibited by high substrate concentration. Hence, understanding these kind of reactors are important. The dynamics of the CSTBR with substrate inhibition kinetics leads to an interesting control problem. Interested reader may find detailed analysis by Dibiasio for an open - loop as well as closed - loop CSTBR with substrate. 1   A number of reports have been published regarding control issues related to

continuous bioreactors.   4.7 Fed batch reactor dynamics

Though CSTBR is an excellent tool to study bacterial metabolism, it is not extensively used in biotechnology - industry. The most widely used mode for fermentation industrial production of biochemicals is fed - batch fermentation. The state-fed - batch system is an interesting system to study since it does not have a true steady. In this case, evaluation of the state variables will locate the position of the system on a trajectory through the operational cycle. Since these state variables can not be measured online, the estimation of state becomes an important element of optimization and control of the reactor. There are numerous reports about the theoretical and experimental issues related to the fed - batch fermentation.

4.8 Bifurcation analysis

A convenient method of classifying the various types of possible dynamic behaviors that can be exhibited by a fermentation model is provided by bifurcation analysis. This theory has a large literature, and the interested reader is directed to Razon and Schmitz.

Feedback control is an action by which PID controllers as well as the controllers based on advanced control strategies implement their control action. Feedback control is an action by which PID controllers as well as the controllers based on advanced control strategies implement their control action. In feedback control system, the controlled variable is measured and compared to the setpoint. The feedback control system, the controlled variable is measured and compared to the setpoint. Subsequently, an error signal is generated by subtracting the setpoint from the value of the controlled variable. Subsequently, variable an error signal is generated by subtracting the value from the setpoint of the controlled. Then the controller calculates the the appropriate corrective action, to be implemented by the manipulated variable, by using the value of the error signal. 16 Then the controller calculates the the appropriate corrective action, to be implemented by the manipulated variable, by using the value of the error signal. 16