Control Engineering Practice 20 (2012) 674–683

Contents lists available at SciVerse ScienceDirect

Control Engineering Practice

0967-06

doi:10.1

$Thisn Corr

E-m

Robert_

journal homepage: www.elsevier.com/locate/conengprac

Laser pulse shaping via extremum seeking$

Beibei Ren a,n, Paul Frihauf a, Robert J. Rafac b, Miroslav Krstic a

a Department of Mechanical and Aerospace Engineering, University of California, San Diego 9500 Gilman Drive, La Jolla, CA 92093-0411, USAb Cymer Inc., 17075 Thornmint Court, San Diego, CA 92127, USA

a r t i c l e i n f o

Article history:

Received 13 July 2011

Accepted 11 March 2012Available online 29 March 2012

Keywords:

Extremum seeking

Optimization

Laser

Pulse shaping

61/$ - see front matter & 2012 Elsevier Ltd. A

016/j.conengprac.2012.03.006

paper was not presented at any IFAC meeti

esponding author. Tel.: þ1 858 822 1374; fax

ail addresses: bren@ucsd.edu (B. Ren), pfrihau

Rafac@cymer.com (R.J. Rafac), krstic@ucsd.ed

a b s t r a c t

Extremum seeking, a non-model based optimization scheme, is employed to design laser pulse shapes

that maximize the amount of stored energy extracted from the amplifier gain medium for a fixed input

energy and inversion density. For this pulse shaping problem, a double-pass laser amplifier whose

dynamics are fully coupled and composed of two nonlinear, first-order hyperbolic partial differential

equations, with time delays in the boundary conditions, and a nonlinear ordinary differential equation,

is considered. These complex dynamics make the optimization problem difficult, if not impossible, to

solve analytically and make the application of non-model based optimization techniques necessary.

Hence, the laser pulse shaping problem is formulated as a finite-time optimal control problem, which is

solved by first, parameterizing the input pulse and pumping rate over the system’s finite time interval

and then, utilizing extremum seeking to maximize the associated cost function. The advantage of the

approach is that the model information is not required for optimization. The extremum seeking

methodology reveals that a rather non-obvious laser pumping rate waveform increases the laser gain

by inducing a resonant response in the laser’s nonlinear dynamics. Numerical simulations illustrate the

effectiveness of the approach proposed in the paper.

& 2012 Elsevier Ltd. All rights reserved.

1. Introduction

Background and motivation: Maximizing the energy extractedby a laser pulse is critical in many laser applications. In thepolysilicon process for manufacturing flat panel displays, one ofthe outstanding problems is obtaining enough instantaneous laserpower to melt as large an area as desired (e.g., one continuousmelt of several meters along the substrate for a flat panel TVdisplay). One engineering solution is to use several laser ampli-fiers and combine their outputs, but using a single amplifier moreefficiently is preferable. The energy efficiency is also a growingconcern in photolithography, where a drive to increase scannerwafer-per-hour throughout means that the optical exposures(which have a fixed energy dose required to print an image, setby the chemistry of the photoresist) must be accomplished withfewer pulses of higher energy. The goal in this paper is tooptimize laser pulse shapes to maximize the amount of storedenergy extracted from the amplifier gain medium for a fixed inputenergy and inversion density.

In Frantz and Nodvik (1963), the growth of a radiation pulse ina laser amplifier was described by nonlinear, time-dependent

ll rights reserved.

ng.

: þ1 858 822 3107.

f@ucsd.edu (P. Frihauf),

u (M. Krstic).

photon transport equations, which account for the effect of theradiation on the medium as well as vice versa. In Akashi, Sakai,and Tagashira (1995), a one-dimensional model including Pois-son’s equation to consider the space-charge for a discharge-excited ArF excimer laser has been developed. In the recent work(Ren, Frihauf, Krstic, & Rafac, 2011), a single-pass laser model isdescribed by a coupled nonlinear first-order hyperbolic partialdifferential equation (PDE) and a nonlinear ordinary differentialequation (ODE). This model is extended from the classical model(Frantz & Nodvik, 1963). In this paper, a double-pass laser modelis considered with partial overlap and optical feedback in theamplifier, which adds another PDE and time delays in the boundaryconditions. This setup allows for more efficient energy extractionand a longer output pulse length.

Design tool—extremum seeking: For two coupled first-orderPDEs, with nonlinear coupling of Lotka–Volterra type, boundarycontrol was developed in Pavel and Chang (2009) to drive thestate at the end of the spatial domain to the desired constantreference values. Instead of stabilization, the laser pulse shapeoptimization problem is addressed in this paper. However, thecomplex dynamics in the laser amplifier model makes theoptimization problem difficult, if not impossible, to solve analy-tically and make the application of non-model based optimizationtechniques necessary. The extremum seeking method (Ariyur &Krstic, 2003) is employed to design the finite-time optimal inputsignal and pumping rate to maximize the amount of stored

Table 1Parameters of the laser system.

Parameter Description Value (units)

L Length of gain medium 0.5 (m)

T Evolution duration time of laser dynamics 300 (ns)

c Speed of light 3�108 (m/s)

s Stimulated emission cross section 2.8�10�16 (cm2)

a Distributed loss 0.006 (cm�1)

B. Ren et al. / Control Engineering Practice 20 (2012) 674–683 675

energy extracted from the amplifier gain medium for a fixed inputenergy and inversion density.

Extremum seeking is a real-time, non-model based optimiza-tion approach for dynamic problems where the system isassumed to have a potentially nonlinear equilibrium map withlocal minima or maxima, but is otherwise unknown. This methodperforms optimization by estimating the gradient of a costfunction and driving this gradient to zero. The gradient estimationis performed using perturbation signals that are typically deter-ministic periodic signals, e.g., sinusoids (Ariyur & Krstic, 2003),but can be replaced by stochastic excitation signals (Liu & Krstic,2010; Manzie & Krstic, 2009), or existing disturbances affectingthe system (Carnevale et al., 2009). A popular tool in controlapplications in the 1940–1950s, extremum seeking has seen aresurgence after its stability analysis was established in Krstic andWang (2000) for continuous-time systems and in Choi, Krstic,Ariyur, and Lee (2002) for discrete-time systems. Many successfulapplications of extremum seeking have been reported in theliterature, including particle beam matching (Schuster et al., 2007),flow control (Becker, King, Petz, & Nitsche, 2007), tokamak fusiondevices (Ou et al., 2008), source seeking with nonholonomicunicycles (Cochran, Kanso, Kelly, Xiong, & Krstic, 2009; Zhang,Arnold, Ghods, Siranosian, & Krstic, 2007), control of combustioninstability (Banaszuk, Ariyur, Krstic, & Jacobson, 2004), maximizingthe pressure rise in an axial flow compressor (Wang, Yeung, & Krstic,2000), limit cycle minimization (Wang & Krstic, 2000), optimalpositioning of mobile sensors under stochastic noise (Stankovic &Stipanovic, 2010), control of thermoacoustic instability (Moase,Manzie, & Brear, 2010), and optical fibre amplifiers (Dower, Farrell,& Nesic, 2008). However, extremum seeking has never before beenused in high-performance photolithography light source systems.

Results: Motivated by Frihauf, Krstic, and Bas-ar (2011), whereextremum seeking was introduced to solve noncooperative gameswith infinitely-many players, the laser pulse shaping problem isformulated as a finite-time optimal control problem, by parameter-izing the evolution of the input pulse and pumping rate in the timeinterval ½0,T�, and employing extremum seeking to maximize theassociated cost function. Both a Gaussian input pulse shape para-meterization and a general input pulse shape parameterization thatconsists of a summation of weighted finite characteristic functionsare considered. The advantage of the approach is that modelinginformation of the laser amplifier is not required in the pulseshaping optimization via extremum seeking, and the effectivenessof this approach is shown through numerous simulations. Thoughthe optimal pulse shape may not converge to the same shape fordifferent initial conditions, the amplifier gain does improve for eachcase. In general, it is difficult to achieve global optimization usingextremum seeking when multiple extrema exist (Tan, Nesic,Mareels, & Astolfi, 2009), which is expected for the high-dimen-sional extremum seeking problem considered in this paper. A rathernon-obvious laser pumping rate waveform, which increases thelaser gain by inducing a resonant response in the laser’s nonlineardynamics, is obtained using the extremum seeking approach.

Organization: The optimization problem is formulated in Section 2and solved using extremum seeking for a Gaussian input peak timing

Fig. 1. Double-pass laser amplifier with partial overlap and optical feedback.

optimization in Section 3 and a general input pulse shape optimiza-tion in Section 4. In Section 5, both the input pulse and the pumpingrate are optimized simultaneously. Throughout this work, theeffectiveness of the proposed method is demonstrated by extensivenumerical studies. Finally, some concluding remarks are given inSection 6.

2. Double-pass laser dynamics with partial overlap andoptical feedback

The intensity dynamics of the laser beam in Fig. 1 are described asfollows for zAð0,L�,tZ0:

@Ilrðz,tÞ

@t¼ �c

@Ilrðz,tÞ

@zþscnðz,tÞIlrðx,tÞ

�aIlrðz,tÞ,

@Irlðz,tÞ

@t¼ c

@Irlðz,tÞ

@zþscnðz,tÞIrlðz,tÞ

�aIrlðz,tÞ,

@nðz,tÞ

@t¼ �

1

Fsat½Ilrðz,tÞþ Irlðz,tÞ�nðz,tÞ

�1

tnðz,tÞþrðtÞ,

8>>>>>>>>>>>>>>>><>>>>>>>>>>>>>>>>:

ð1Þ

with boundary conditions

Ilrð0,tÞ ¼ k2Irlð0,t�2d2Þþð1�k2ÞIinðt�d2Þ,

boundary condition=input,

IrlðL,tÞ ¼ k1IlrðL,t�2d1Þ,

boundary condition,

IoutðtÞ ¼ ð1�k2ÞIrlð0,t�d2Þþk2IinðtÞ,

boundary value=output,

8>>>>>>>>><>>>>>>>>>:

ð2Þ

where Ilr and Irl are the rightward and leftward irradiance, respec-tively, n is the population difference between the upper and lowerlaser levels, and r is the pumping rate, k1 and k2 are partial overlapand optical feedback gains, lext,1 and lext,2 are the right and leftextended lengths, and d1 ¼ lext,1=c and d2 ¼ lext,2=c are time delayscaused by the extended lengths. The physical meanings and values ofthe parameters are listed in Table 1.

The objective is to seek the optimal input pulse shape IinðtÞ

over the time interval tA ½0,T� such that the amplifier gain

G¼ Eout

Ein, ð3Þ

t Lifetime of laser state 1.75 (ns)

Fsat Saturation fluence 3.67 (mJ/cm2)

k1 Partial overlap gain 0.85

k2 Optical feedback gain 0.95

lext,1 Extended length (right) 0.7 (m)

lext,2 Extended length (left) 0.8 (m)

d1 Time delay (right) 2.3 (ns)

d2 Time delay (left) 2.7 (ns)

Ein Input energy Fixed

Eout Output energy To be optimized

Vr Inversion density Fixed

G¼ Eout

Ein

Amplifier gain To be optimized

B. Ren et al. / Control Engineering Practice 20 (2012) 674–683676

i.e., the ratio of the output energy Eout ¼R T

0 IoutðtÞ dt to the inputenergy Ein ¼

R T0 IinðtÞ dt, is maximized while holding the input

energy and the inversion density Vr ¼R T

0 rðtÞ dt fixed. Hence,IinðtÞ needs to be found to solve

maxIinðtÞZ0

G subject to Ein and Vr fixed: ð4Þ

To ensure that the input energy and the inversion density arefixed, the input signal and the pumping rate are modulated overthe period ½0,T� as

Imodin ðtÞ ¼

IinðtÞR T0 IinðtÞ dt

E0in, ð5Þ

rmodðtÞ ¼rðtÞR T

0 rðtÞ dtV0r, ð6Þ

where E0in and V0

r are the user-defined input energy level andinversion density, respectively.

In the following subsections, the response of the double-passlaser models (1)–(2) is investigated with different input pulseshapes and pumping rates, highlighting their effect on the outputpulse Iout(t). Since the analytical solution of the models (1)–(2) isdifficult, if not impossible, to find, the implicit finite differenceapproximation method (Tveito & Winther, 1998) is adopted toobtain its approximate solution. The space ½0,L� is discretized intoNz intervals and the time interval ½0,T� into Nt intervals withthe spatial discretization dz¼ L=Nz and temporal discretizationdt¼ T=Nt . For the simulations, the parameters are chosen asNz¼100, Nt¼1000, and thus dz¼0.5 cm, dt¼0.3 ns.

0 50 100 150 200 250 3000

0.5

1

1.5

2

2.5

3

3.5x 107

Time (ns)

I out

(m−

3)

0 50 1000

2

4

6

8

10

12

14x 108

Tim

I out

(m−

3)

Fig. 2. Output pulse with different Gaussian pumping rate magnitudes Ar , while

1021 m�3 s�1, and (c) Ar ¼ 10� 1021 m�3 s�1.

0 50 100 150 200 250 3000

1

2

3

4

5

6

7x 109

Time (ns)

I out

(m−

3)

0 50 100 10

0.5

1

1.5

2

2.5

3x 109

Tim

I out

(m−

3 )

Fig. 3. Output pulse with different Gaussian pumping rate pulse widths sr , and while A

and (c) sr ¼ 20 ns.

2.1. Response with Gaussian input pulse and pumping rate

Consider the case when both the input pulse and pumping rateare approximated by the Gaussian functions of time t

IinðtÞ ¼1ffiffiffiffiffiffi

2pp

sin

e�ððt�minÞ2=2s2

inÞ, ð7Þ

rðtÞ ¼ Ar1ffiffiffiffiffiffi

2pp

sre�ððt�mrÞ

2=2s2rÞ, ð8Þ

where min and mr are the mean values (peak times), sin and sr arethe standard deviations (pulse ‘‘width’’), and Ar is the magnitudeof pumping rate.

Fig. 2 shows the output pulse IoutðtÞ with different Gaussianpumping rate magnitudes Ar, when min and mr, sin and sr are fixed.It can be seen that IoutðtÞ increases in magnitude and becomes moreoscillatory when Ar increases. Fig. 3 shows the output pulse IoutðtÞ

with different pulse widths sr of the Gaussian pumping rate. Withthe increase of sr, the oscillations of IoutðtÞ are reduced, but themagnitude of IoutðtÞ decreases too. The amplifier gain G for Fig. 3(a),(b), and (c) is 139.32, 68.95 and 46.10, respectively, for the fixedinversion density V0

r ¼ 5� 1021 m�3.

2.2. Response with constant input pulse and sinusoidal

pumping rate

The modulation of the pumping rate are introduced in thefollowing form:

rðtÞ ¼ Ar½1þar sinð2pvrtÞ�, ð9Þ

150 200 250 300

e (ns)0 50 100 150 200 250 300

0

1

2

3

4

5

6x 10

9

Time (ns)

I out

(m−

3)

min ¼ mr ¼ 60 ns, and sin ¼ sr ¼ 12:8 ns. (a) Ar ¼ 1� 1021 m�3 s�1, (b) Ar ¼ 5�

50 200 250 300

e (ns)

0 50 100 150 200 250 3000

1

2

3

4

5

6

7x 108

Time (ns)

I out

(m−

3)

r ¼ 5� 1021 m�3 s�1, min ¼ mr ¼ 60 ns and sin ¼ 12:8 ns. (a) sr ¼ 3 ns, (b) sr ¼ 8 ns,

B. Ren et al. / Control Engineering Practice 20 (2012) 674–683 677

where vr is the modulation frequency, ar is the modulationdepth, and Ar is modulation mean. The effect of vr, ar and Ar onthe response of laser dynamics is investigated and illustratedin Figs. 4–6. In Fig. 4, the fixed inversion density is V0

r ¼ 5�1022 m�3 over the period ½0,T�, while the amplifier gain is 51.76,51.41, and 51.26 for (a), (b), and (c), respectively. In Fig. 5, theincrease of Ar results in the increase of the inversion density, andthus, the increase of the amplifier gain. In particular, for Fig. 5 (a),(b), and (c), the inversion density is 7�1020 m�3, 7�1021 m�3,7�1022 m�3, and the corresponding amplifier gain is 1.18, 2.28and 54.16, respectively. In Fig. 6, the fixed inversion density is

300 350 400 450 5004

6

8

10x 10

9

Time (ns)

I out

(m−

3)

3 3.5 4 4.5 5

x 10

0

1

2x 10

29

Time (ns)

�(m

−3·

s−1)

300 350 404

6

8x 10

9

Time

I out

(m−

3)

3 3.5 40

1

2x 10

29

Time

�(m

−3·

s−1)

Fig. 4. Response with different sinusoidal pumping rate modulation frequencies vr(b) vr ¼ 2� 107, and (c) vr ¼ 5� 107.

400 450 500 550 6001.19

1.2

1.21

1.22x 10

8

Time (ns)

I out

(m−

3)

4 4.5 5 5.5 6

x 10−7

0

1

2x 10

27

Time (ns)

�(m

−3

·s−

1)

400 4502

2.5

3

3.5x 10

8

Tim

I out

(m−

3)

4 4.50

1

2x 10

28

Tim

�(m

−3· s

−1)

Fig. 5. Response with different sinusoidal pumping rate modulation mean magnitudes

(b) Ar ¼ 1� 1028 m�3 s�1, and (c) Ar ¼ 1� 1029 m�3 s�1.

600 620 640 660 680 7006.1

6.2

6.3

6.4x 10

9

Time (ns)

I out

(m−

3)

6 6.2 6.4 6.6 6.8 7

x 10−7

0.95

1

1.05x 10

29

Time (ns)

�(m

−3·s

−1)

I out

(m−

3)

�(m

−3·s

−1)

600 620 6405

6

7

8x 10

9

Tim

6 6.2 6.40.5

1

1.5

2x 10

29

Tim

Fig. 6. Response with different sinusoidal pumping rate modulation depths ar , w

(b) ar ¼ 0:5, and (c) ar ¼ 1.

V0r ¼ 5� 1022 m�3 over the period ½0,T�, while the amplifier gain is

51.45, 51.48, and 51.04 for (a), (b), and (c), respectively.From Figs. 2–6, it is noted that the performance of the double-

pass laser model (1)–(2) is affected by the shapes of inputpulse IinðtÞ and pumping rate rðtÞ. In the following sections,the pulse shaping optimization problem (4) is investigated forGaussian peak timing optimization, where the input signal isGaussian, and for a general input pulse shape, parameterized bythe summation of weighted finite characteristic functions. Finally,both the input pulse shape and the pumping rate are optimizedsimultaneously.

0 450 500

(ns)

4.5 5

x 10(ns)

300 350 400 450 5004

6

8x 10

9

Time (ns)

I out

(m−

3)

3 3.5 4 4.5 5

x 10

0

1

2x 10

29

Time (ns)

�(m

−3·

s−1)

, while Ar ¼ 1� 1029 m�3 s�1, ar ¼ 1, and IinðtÞ ¼ 1� 108 m�3. (a) vr ¼ 1� 107,

500 550 600

e (ns)

5 5.5 6

x 10−7e (ns)

400 450 500 550 6004

6

8x 10

9

Time (ns)

I out

(m−

3)

4 4.5 5 5.5 6

x 10−7

0

1

2x 10

29

Time (ns)

�(m

−3 ·

s−1 )

Ar , while vr ¼ 2� 107, ar ¼ 1, and IinðtÞ ¼ 1� 108 m�3. (a) Ar ¼ 1� 1027 m�3 s�1,

I out

(m−

3)

�(m

−3·s

−1)

660 680 700

e (ns)

6.6 6.8 7

x 10−7e (ns)

600 620 640 660 680 7004

6

8x 10

9

Time (ns)

6 6.2 6.4 6.6 6.8 7

x 10−7

0

1

2x 10

29

Time (ns)

hile Ar ¼ 1� 1029 m�3 s�1, vr ¼ 2� 107, and IinðtÞ ¼ 1� 108 m�3. (a) ar ¼ 0:05,

B. Ren et al. / Control Engineering Practice 20 (2012) 674–683678

3. Gaussian input peak timing optimization using extremumseeking scheme

Let both the input pulse and the pumping rate be modeled asGaussian functions shown in (7) and (8). The objective is to findan optimal peak timing min for the input pulse, so that theamplifier gain is maximized for a fixed input energy, with thepulse width sin and pumping rate rðtÞ given. The optimizationproblem (4) is reformulated as

maxmin 40

G

subject to sin,rðtÞ given and

Z T

0IinðtÞ dt¼ E0

in: ð10Þ

To solve the optimization problem (10), discrete-time extre-mum seeking is used to tune the parameter minðkÞ, where k is thediscrete-time index, to determine the Gaussian-shaped inputpulse Iin(t) that optimizes the amplifier gain. Specifically, givena peak time minðkÞ, which determines the input pulse Ik

inðtÞ, theoutput pulse Ik

inðtÞ is found and amplifier gain Gk is computed. Theextremum seeking loop, driven by Gk, updates the peak timeparameter estimate to obtain minðkþ1Þ and the new input pulseIkþ1in ðtÞ. The process is then repeated.

A block diagram of the system is shown in Fig. 7, where min isthe estimate of min, gm is the adaptation gain, ol and oh are lowand high pass filter cutoff frequencies, a sin ok is the perturbationsignal, and oAð0,pÞ is the perturbation frequency. In equationform, the extremum seeking algorithm is given by

Zðkþ1Þ ¼ ð1�ohÞZðkÞþohGk, ð11Þ

xðkþ1Þ ¼ ð1�olÞxðkÞþolðGk�ZðkÞÞa sin ok, ð12Þ

Las

ω l

z − (1 −� �

z − 1+

a sin �k

�in

Modulation

�in ξ

I in (t)

Fig. 7. Discrete-time extremum seeking scheme f

0 500 1000 150010

20

30

40

50

60

70

80

90

100

ES steps

G

0 50040

50

60

70

80

90

ES s

�(n

s)

Fig. 8. The evolution of the amplifier gain G (a) and Gaussian input peak time min (b)

input Iin, and output signal Iout.

minðkþ1Þ ¼ minðkÞþgmxðkÞ, ð13Þ

minðkÞ ¼ minðkÞþa sin ok: ð14Þ

The high pass filter removes the DC component of the costfunction, and the low pass filter attenuates the oscillations causedby the sinusoidal perturbation and smooths the parameterestimate min. The stability of the extremum seeking feedbackscheme can be established by employing the tools of averagingand singular perturbation analysis (Ariyur & Krstic, 2003; Choiet al., 2002; Krstic & Wang, 2000).

Simulation results: In this study, the initial IinðtÞ and rðtÞ aretaken as the form of (7) and (8) with Ain ¼ 1:0 m�3, min ¼ 60 ns,sin ¼ 12:8 ns, while Ar ¼ 4� 1021 m�3 s�1, mr ¼ 60 ns, sr ¼ 3 ns.The initial amplifier gain G0

¼ 86:76. Other extremum seekingparameters in Fig. 7 are chosen as: gm ¼ 1, a¼ 0:6, o¼ 1,ol ¼ 0:024, oh ¼ 0:020. Fig. 8 shows the evolution of the amplifiergain G and Gaussian input peak timing min when min is tuned usingextremum seeking. It can be seen that the optimal amplifier gainGn ¼ 93:76 and optimal peak time mn

in ¼ 44:97 ns are achievedafter 1000 extremum seeking steps. Thus, when the amplifier gainG is maximized, the Gaussian peak time of the input signal is15.03 ns ahead of the pumping rate peak.

4. General input pulse shaping optimization

In Section 3, using extremum seeking, the peak time for the scaledGaussian single-hump function has been optimized to maximizethe amplifier gain. However, the amplifier gain may potentially beincreased further by more complex input pulse shapes that cannot bemodeled by Gaussian single-humps. Therefore, in this section thegeneral input pulse shaping optimization problem is considered.

er System Cost FunctionG

z − 1z − (1− ωh)

×

a sin ωk

ω l )G − η

or Gaussian input peak timing optimization.

1000 1500

teps0 50 100 150 200 250 300

0

1

2

3

4

5x 10

9

Laser time (ns)

I in

(m−

3),

I out

(m−

3),

�(m

−3

·s) I in × 10

Iout�/ max (�) × 5 × 109

using extremum seeking when sin ¼ 12:8 ns; and (c) the pumping rate r, optimal

B. Ren et al. / Control Engineering Practice 20 (2012) 674–683 679

Motivated by Frihauf et al. (2011), where extremum seekingwas introduced to solve noncooperative games with infinitely-many players, the problem in this paper is formulated as aninfinite-dimensional optimal control problem by the parametri-zation of the evolution of the input pulse IinðtÞ in the time interval½0,T�. The input pulse IinðtÞ, tA ½0,T� is approximated in the spaceL2½0,T� as follows:

IinðtÞ ¼XN

j ¼ 1

yj1½tj ,tjþ 1 �ðtÞ, tA ½tj,tjþ1�, ð15Þ

where ½0,T� is divided into a finite number (N) of intervals ½tj,tjþ1�,

j¼ 1;2, . . . ,N, with t1 ¼ 0, tNþ1 ¼ T , yj is the weight for the jth

characteristic function, and 1½tj ,tjþ 1ÞðtÞ denotes the characteristic

function in the interval ½tj,tjþ1�, i.e.

1½tj ,tjþ 1 �ðtÞ ¼1, tA ½tj,tjþ1Þ,

0 otherwise:

�ð16Þ

Extremum seeking is applied to determine optimal values for the

weights yj.

In practice, some constraints on the input function IinðtÞ haveto be considered due to the dynamics and magnitude constraintsin the electronics that generate the input signal. For example,IinðtÞ cannot be too short or too long in duration, and IinðtÞ cannotchange too rapidly. The constraints on the input pulse shape canbe adjusted by sin when a Gaussian-like shape is assumed as isdone in Section 3. For a general input pulse, however, there is nodirect control over the optimized shape; therefore, a penalty onthe input gradient is included in the cost function to penalizeoverly narrow input signals, resulting in the following optimiza-tion problem:

maxy40

JðyÞ with JðyÞ ¼ G�bZ T

0ð_I inÞ

2ðtÞ dt

subject to rðtÞ given and

Z T

0IinðtÞ dt¼ E0

in, ð17Þ

where b40 is the penalty weight, y¼ ½y1,y2, . . . ,yN�T is defined in

(15), and E0in is a user-defined input energy level. In simulation (or

in practice, when sampling the continuous-time laser amplifierdynamics (1)–(2)), the idealized, integral penalty term in (17)must be approximated. In this work, a forward finite-differenceapproximation is adopted for the time derivative of the inputpulse, namely

_I inðtjÞ �Iinðtjþ1Þ�IinðtjÞ

dt, j¼ 1;2, . . . ,Nt ,

¼yjþ1�yj

dt, ð18Þ

Las

�l

z − (1 −� �

j

z − 1+

aj sin �j k

θj

Modulation

�j �j

I in (t)

Fig. 9. Discrete-time extremum seeking scheme for gen

where dt¼T/N. The integral term is then approximated by aRiemann sum, i.e.Z T

0ð_I inÞ

2ðtÞ dt�

XN

j ¼ 1

yjþ1�yj

dt

� �2

dt,

¼1

dt

XN

j ¼ 1

ðyjþ1�yjÞ2: ð19Þ

The multi-parameter extremum seeking scheme used to max-imize (17) is

Zðkþ1Þ ¼ ð1�ohÞZðkÞþohJðyðkÞÞ, ð20Þ

xjðkþ1Þ ¼ ð1�olÞxjðkÞþolðJðyðkÞÞ�ZðkÞÞaj sin ojk, ð21Þ

yjðkþ1Þ ¼ yjðkÞþgyj xðkÞ, ð22Þ

yjðkÞ ¼ yjðkÞþaj sin ojk, ð23Þ

where, for j¼ 1;2, . . . ,N, yj is the estimate of the optimal weight

vector yn, gyj is the adaptation gain, ol and oh are low and high

pass filter cutoff frequencies, aj sin ojk is the perturbation signal,

and ojAð0,pÞ is the perturbation frequency. For multi-parameter

extremum seeking designs, the perturbation frequencies must

also satisfy oiaoj, ia j (Durr, Stankovic, & Johansson, 2011). The

extremum seeking scheme used to update yj is depicted in Fig. 9.

Simulation results: The simulation results are shown in Figs. 10–12,where the time interval ½0,T� for the laser amplifier dynamics hasbeen discretized into N¼500 intervals. In Fig. 10, the initial and finalsignal pulse shapes for the input IinðtÞ and the output IoutðtÞ areshown in (a) and (b), and the evolution of the cost JðyÞ and amplifiergain G are depicted in (c). The initial input pulse is chosen as asingle-hump shape with energy E0

in ¼ 1:0 m�3 s, and penalty weightb¼ 2:25� 10�25 m6 s (where the penalty weight b units aregiven so that the cost JðyÞ is a nondimensional value). The pumpingrate rðtÞ is described by a Gaussian function as (8) with Ar ¼

4� 1021 m�3 s�1, mr ¼ 60 ns, sr ¼ 3 ns.The extremum seeking parameters are selected as a¼

½a1,a2, . . . ,aN�T , o¼ ½o1,o2, . . . ,oN�

T and gy by a random draw froma uniform distribution. Specifically, a is sampled from the distributionUað2� 106,4� 106

Þ, o from Uoð0:3,0:6Þ, and gy ¼ ½gy1,gy2, . . . ,gyN�

T

from Ugð0:3,0:4Þ, where Uða,bÞ denotes the uniform distribution with

probability density 1=ðb�aÞ. Note, the probability that oi ¼oj is zero.One could also choose o as the value of a monotonically increasing ordecreasing function so that oiaoj. o is selected from a uniformdistribution so that comparatively fast and slow frequencies arespread throughout the parameter space. The low pass and high passfilter parameters are ol ¼ 0:024 and oh ¼ 0:020.

It is observed that:

(i)

er Sy

�l )

eral

The optimal cost Jn ¼ 130:82 and optimal amplifier gainGn ¼ 134:54 are achieved after 1.0�105 extremum seeking

stem Cost Function

z − 1z − (1 − �h)

×

aj sin �j k

J (�) − �

input pulse shaping with the yj loop depicted.

Fig.(c) t

E0in ¼

Fig.(c) t

E0in ¼

Fig.Ar ¼

B. Ren et al. / Control Engineering Practice 20 (2012) 674–683680

steps, while the initial cost J0¼ 93:70 and initial amplifier

gain G0¼ 93:71.

(ii)

The input pulse shape converges to an optimal multi-hump pulse.

While Fig. 10 shows a result with a single-hump initial pulse,Fig. 11 shows a result with a double-hump initial input signalpulse. Similar results are obtained though the initial input pulseis different. In Fig. 12 the effect of the penalty weight b on the

0 50 100 150 200 250 3000

2

4

6

8

10

12

14

16

18x 10

7

Laser time (ns)

I in

(m−

3)

initialfinal

0 50 100 150

1

2

3

4

5

6

7x 10

9

Laser tim

I out

(m−

3)

10. Initial and final (a) input IinðtÞ and (b) output IoutðtÞ signals when using the ex

he evolution of the cost function JðyÞ and amplifier gain G. Convergence is ach

1:0 m�3 s and the penalty weight b¼ 2:25� 10�25 m6 s.

0 50 100 150 200 250 3000

2

4

6

8

10

12

14

16x 10

7

Laser time (ns)

I in

(m−

3)

initialfinal

0 50 100 150

1

2

3

4

5

6

7x 10

9

Laser ti

I out

(m−

3)

11. Initial and final (a) input IinðtÞ and (b) output IoutðtÞ signals when using the ext

he evolution of the cost function JðyÞ and amplifier gain G. Convergence is ach

1:0 m�3 s and the penalty weight b¼ 2:25� 10�25 m6 s.

0 50 100 150 200 250 3000

0.5

1

1.5

2

2.5

3

3.5

4

4.5x 10

7

Laser time (ns)

I in

(m−

3)

initialfinal

0 50 100 150

1

2

3

4

5

6x 10

9

Laser ti

I out

(m−

3)

12. Results with the same conditions as in Fig. 11 except for the penalty w

4� 1021 m�3 s�1, mr ¼ 60 ns, sr ¼ 3 ns, and E0in ¼ 1:0 m�3 s and the penalty weigh

performance is investigated, where b¼ 6:75� 10�24 m6 s and theinitial input pulse is double-hump. Based on Figs. 11 and 12, it isobserved that:

(i)

0 2

e (n

tremu

ieved

0

me (

remu

ieved

0 2

me (n

eight

t b¼

The peak value of the optimal input pulse decreases and theduration of the pulse increases, i.e., its rate of changedecreases as b is increased.

(ii)

The achieved amplifier gain and extremum seeking conver-gence rate both decrease as b is increased.

00 250 300

s)

initialfinal

0 2 4 6 8 10

x 104

80

90

100

110

120

130

140

ES steps

J,G

J

G

m seeking algorithm with N¼500 for a single-hump initial input pulse, and

in 4.0�104 steps, when Ar ¼ 4� 1021 m�3 s�1, mr ¼ 60 ns, sr ¼ 3 ns, and

200 250 300

ns)

initialfinal

0 2 4 6 8 10

x 104

80

90

100

110

120

130

140

ES steps

J,G

J

G

m seeking algorithm with N¼500 for a double-hump initial input pulse, and

in 4.0�104 steps, when Ar ¼ 4� 1021 m�3 s�1, mr ¼ 60 ns, sr ¼ 3 ns, and

00 250 300

s)

initialfinal

0 0.5 1 1.5 2 2.5

x 105

40

50

60

70

80

90

100

110

120

ES steps

J ,G

J

G

b¼ 6:75� 10�24 m6 s. Convergence is achieved in 2.0�105 steps, when

2:25� 10�24 m6 s.

B. Ren et al. / Control Engineering Practice 20 (2012) 674–683 681

(iii)

Fig. 1initia

and t

It is not surprising that the extremum seeking scheme couldconverge to one of the local extrema if they exist, in particular,for the high-dimensional extremum seeking problem consideredin this paper. For different initial conditions and cost functionparameters, the optimal pulse shape may not converge to thesame shape, however, the amplifier gain does improve indeedfor each case.

La

�

z − (1

� Ij

z − 1+

aIj sin �I

j k

� Ij

Modulation

�Ij

�

z − (1

� �j

z − 1+

a�j sin ��

j k

I in (t)

� (t)

��j

Fig. 13. Discrete-time extremum seeking scheme for both input pulse shaping an

0 50 100 150 200 250 3000

2

4

6

8

10

12

14

16x 10

7

Laser time (ns)

I in

(m−

3)

initialfinal

0 50 100 150 200 250 3000

1

2

3

4

5

6x 10

29

Laser time (ns)

�(m

−3

·s)

initialfinal

4. Initial and final (a) input IinðtÞ, (b) output IoutðtÞ and (c) pumping rate rðtÞ sign

l input pulse, and (d) the evolution of the cost function JðyÞ and amplifier gain G. Co

he penalty weights b¼ 2:25� 10�25 m6 s, g¼ 9� 10�67 m6 s3.

5. Optimization of both input pulse and pumping rate

In Sections 3 and 4, the input pulse IinðtÞ has been opti-mized with the pumping rate rðtÞ given. Now extremumseeking is used to optimize both IinðtÞ and rðtÞ to maximizethe extraction of the stored energy from the amplifier gainmedium.

ser System Cost Function

z − 1z − (1 − �h )

×

aIj sin �I

j k

l

− �l )

z − 1z − (1 − �h )

×

a�j sin ��

j k

l

− �l )

d pumping rate pulse shaping optimization with loops for yIj and yrj shown.

0 50 100 150 200 250 3000

1

2

3

4

5x 10

9

Laser time (ns)

I out

(m−

3)

initialfinal

0 1 2 3 4 5

x 105

0

50

100

150

200

ES steps

J,G

J

G

als when using the extremum seeking algorithm with N¼500 for a double-hump

nvergence is achieved in 5.0�104 steps, when E0in ¼ 1:0 m�3 s, V0

r ¼ 4:0� 1021 m�3

B. Ren et al. / Control Engineering Practice 20 (2012) 674–683682

It is assumed that the input pulse IinðtÞ and pumping rate rðtÞ,tA ½0,T�, are approximated in the space L2

½0,T� as follows:

IinðtÞ ¼XN

j ¼ 1

yIj1½tj ,tjþ 1 �ðtÞ, tA ½tj,tjþ1� ð24Þ

rðtÞ ¼XN

j ¼ 1

yrj 1½tj ,tjþ 1 �ðtÞ, tA ½tj,tjþ1� ð25Þ

where ½0,T� is separated into N intervals ½tj,tjþ1�, j¼ 1;2, . . . ,N,t1 ¼ 0 and tNþ1 ¼ T , and where 1½tj ,tjþ 1 �

ðtÞ denotes the character-istic function given by (16). The weights for the jth characteristicfunction of IinðtÞ and rðtÞ are denoted by yI

j and yrj , respectively.Due to practical constraints on the input function IinðtÞ and the

pumping rate rðtÞ, both a penalty on the input gradient and apenalty on the pumping rate gradient are included in the costfunction to prevent IinðtÞ and rðtÞ from having a rapid rate ofchange. Hence, the optimization problem becomes

maxyI 40,yr40

JðyI ,yrÞ,

with JðyI ,yrÞ ¼ G�bZ T

0ð_I inÞ

2ðtÞ dt�g

Z T

0ð _rÞ2ðtÞ dt

subject to

Z T

0IinðtÞ dt ¼ E0

in,

Z T

0rðtÞ dt¼ V0

r, ð26Þ

where b40 and g40 are the penalty weights for the input

gradient and the pumping rate gradient, yI¼ ½yI

1,yI2, . . . ,yI

N�T and

yr ¼ ½yr1 ,yr2 , . . . ,yrN�T , which are defined in (24) and (25), and

V0r ¼

R T0 r0ðtÞ dt. In implementation, the integral penalty terms

are numerically approximated as is done in Section 4 (cf.(17)–(19)).

0 50 100 150 200 250 3000

0.5

1

1.5

2

2.5

3

3.5

4

4.5x 10

7

Laser time (ns)

I in

(m−

3)

initialfinal

0 50 100 150 200 250 3000

1

2

3

4

5

6x 10

29

Laser time (ns)

�(m

−3· s

)

initialfinal

Fig. 15. Initial and final (a) input IinðtÞ, (b) output IoutðtÞ and (c) pumping rate rðtÞ sign

initial input pulse, and (d) the evolution of the cost function JðyÞ and amplifier gain G. Co

and the penalty weights b¼ 6:75� 10�24 m6 s, g¼ 9� 10�67 m6 s3.

The multi-parameter extremum seeking scheme used to max-imize (26) are

Zðkþ1Þ ¼ ð1�ohÞZðkÞþohJðyIðkÞ,yrðkÞÞ, ð27Þ

xIjðkþ1Þ ¼ ð1�olÞx

IjðkÞþolðJðy

IðkÞ,yrðkÞÞ�ZðkÞÞaI

j sin oIjk, ð28Þ

yI

jðkþ1Þ ¼ yI

jðkÞþgIjx

IðkÞ, ð29Þ

yIjðkÞ ¼ y

I

jðkÞþaIj sin oI

jk, ð30Þ

xrj ðkþ1Þ ¼ ð1�olÞxrj ðkÞþolðJðy

IðkÞ,yrðkÞÞ�ZðkÞÞarj sin or

j k, ð31Þ

yrj ðkþ1Þ ¼ y

rj ðkÞþg

rj x

rðkÞ, ð32Þ

yrj ðkÞ ¼ yrj ðkÞþarj sin or

j k, ð33Þ

where j¼ 1;2, . . . ,N, yIðkÞ and yrðkÞ are the weight vectors of IinðtÞ

and rðtÞ to be optimized; yIðkÞ and y

rðkÞ are the estimates of yI

ðkÞ

and yrðkÞ, respectively; and aI sin oIk, and ar sin ork are perturba-tion signals. The extremum seeking scheme is depicted in Fig. 13.

Simulation results: Fig. 14(a)–(c) show the initial and final signalpulse shapes for the input IinðtÞ, output IoutðtÞ, and pumping rate rðtÞsignals, respectively, when the time interval ½0,T� is discretized intoN¼500 intervals. The evolution of the cost JðyI ,yrÞ and amplifier gainG is also shown in Fig. 14(d). The extremum seeking parameters ol,oh, and those used to tune yI are the same as those selectedin simulations presented in Section 4. And ar ¼ ½ar1 ,ar2 , . . . ,arN� issampled from the distribution Ua

rð2� 1027,4� 1028Þ, or from Uo

rð0:3,0:6Þ, and gr ¼ ½gr1 ,gr2 , . . . ,grN�

T from Ugrð0:1,0:2Þ, where Uða,bÞ

denotes the uniform distribution with probability density 1=ðb�aÞ.

0 50 100 150 200 250 3000

1

2

3

4

5x 10

9

Laser time (ns)

I out

(m−

3)

initialfinal

0 2 4 6 8

x 105

−50

0

50

100

150

200

ES steps

J,G

J

G

als when using the extremum seeking algorithm with N¼500 for a double-hump

nvergence is achieved in 5.0�104 steps, when E0in ¼ 1:0 m�3 s, V0

r ¼ 4:0� 1021 m�3

B. Ren et al. / Control Engineering Practice 20 (2012) 674–683 683

For the fixed input energy Ein ¼ 1:0 m�3 s, inversion densityVr ¼ 4:0� 1021 m�3 and penalty weights b¼ 2:25� 10�25 m6 s,g¼ 9:0� 10�67 m6 s3:

(i)

the optimal cost Jn ¼ 114:59 and optimal amplifier gainGn ¼ 179:83 are achieved after 5�105 extremum seeking steps,while the initial cost J0

¼ 15:22 and initial amplifier gainG0¼ 90:50,

(ii)

the input pulse converges from a double-hump pulse to anoptimal three-hump pulse,

(iii)

the pumping rate pulse also converges to an optimal multi-hump pulse, which seems to oscillate at a frequency thatpromotes a resonance in the laser dynamics, and

(iv)

compared with the extremum seeking results in Fig. 11 whenthe pumping rate rðtÞ is a given value, the energy amplifiergain is increased from 126.09 to 179.83.

Fig. 15 shows the effect of the penalty weight b on theperformance where b¼ 6:75� 10�24 m6 s. Compared with theresults in Fig. 14, it can be seen that:

(i)

The peak value of the optimal input pulse decreases and theduration of the pulse increases, i.e., its rate of changedecreases when b is increased.

(ii)

Both the achieved amplifier gain and extremum seekingconvergence rate decrease when b is increased.

(iii)

For different cost function parameters, the optimal pulseshape may not converge to the same shape. However, theamplifier gain does improve for each case.

A comparison of Figs. 12 and 15 (or Figs. 11 and 14) providesan interesting physical insight. Extremum seeking finds a non-obvious pumping rate rðtÞ that improves the gain G by about 40%by inducing a resonant response in the laser dynamics.

6. Conclusion

In this paper the input pulse and pumping rate shape optimi-zation have been investigated for a laser amplifier through thehigh-dimensional extremum seeking method. It has been shownthat the optimized input/pumping rate signal can extract max-imum stored energy from the amplifier gain medium for differentinitial conditions and different cost function parameters. Nomodel information is required in the pulse shaping optimization.A non-obvious pumping rate rðtÞ has been found to improve thegain G by about 40% by inducing a resonant response in the laserdynamics.

References

Akashi, H., Sakai, Y., & Tagashira, H. (1995). Modelling of a self-sustaineddischarge-excited ArF excimer laser: the influence of photo-ionization and

photodetachment by laser light on the discharge development. Journal of

Physics D: Applied Physics, 28, 445–451.Ariyur, K., & Krstic, M. (2003). Real-time optimization by extremum seeking control.

Hoboken, NJ: Wiley-Interscience.Banaszuk, A., Ariyur, K. B., Krstic, M., & Jacobson, C. A. (2004). An adaptive

algorithm for control of combustion instability. Automatica, 40(11),1965–1972.

Becker, R., King, R., Petz, R., & Nitsche, W. (2007). Adaptive closed-loop separationcontrol on a high-lift configuration using extremum seeking. AIAA Journal,45(6), 1382–1392.

Carnevale, D., Astolfi, A., Centioli, C., Podda, S., Vitale, V., & Zaccarian, L. (2009). Anew extremum seeking technique and its application to maximize rf heating

on ftu. Fusion Engineering and Design, 84(2–6), 554–558.Choi, J.-Y., Krstic, M., Ariyur, K. B., & Lee, J. S. (2002). Extremum seeking control for

discrete-time systems. IEEE Transactions on Automatic Control, 47(2), 318–323.Cochran, J., Kanso, E., Kelly, S. D., Xiong, H., & Krstic, M. (2009). Source seeking for

two nonholonomic models of fish locomotion. IEEE Transactions on Robotics

and Automation, 25(5), 1166–1176.Dower, P. M., Farrell, P. M., & Nesic, D. (2008). Extremum seeking control of

cascaded raman optical amplifiers. IEEE Transactions on Control Systems

Technology, 16(3), 396–407.Durr, H.-B., Stankovic, M. S., & Johansson, K. H. (2011). Distributed positioning of

autonomous mobile sensors with application to coverage control. In 2011

American control conference.Frantz, L. M., & Nodvik, J. S. (1963). Theory of pulse propagation in a laser

amplifier. Journal of Applied Physics, 34(8), 2346–2349.Frihauf, P., Krstic, M., & Bas-ar, T. (2011). Nash equilibrium seeking with infinitely

many players. In 2011 American control conference.Krstic, M., & Wang, H. H. (2000). Stability of extremum seeking feedback for

general nonlinear dynamic systems. Automatica, 36(4), 595–601.Liu, S.-J., & Krstic, M. (2010). Stochastic averaging in continuous time and its

applications to extremum seeking. IEEE Transactions on Automatic Control,55(10), 2235–2250.

Manzie, C., & Krstic, M. (2009). Extremum seeking with stochastic perturbations.IEEE Transactions on Automatic Control, 54(3), 580–585.

Moase, W. H., Manzie, C., & Brear, M. J. (2010). Newton-like extremum-seeking for

the control of thermoacoustic instability. IEEE Transactions on Automatic

Control, 55(9), 2094–2105.Ou, Y., Xu, C., Schuster, E., Luce, T. C., Ferron, J., Walker, M. L., & Humphreys, D. A.

(2008). Design and simulation of extremum-seeking open-loop optimal con-

trol of current profile in the DIII-D Tokamak. Plasma Physics and Controlled

Fusion, 50(3), 115001.Pavel, L., & Chang, L. (2009). Lyapunov-based boundary control for 2�2 hyper-

bolic Lotka-Volterra systems. In Joint 48th IEEE conference on decision and

control and 28th Chinese control conference.Ren, B., Frihauf, P., Krstic, M., & Rafac, R. J. (2011). Laser pulse shaping via iterative

learning control and infinite-dimensional extremum seeking. In 2011 ASME

dynamic systems and control conference.Schuster, E., Xu, C., Torres, N., Morinaga, E., Allen, C., & Krstic, M. (2007). Beam

matching adaptive control via extremum seeking. Nuclear Instruments and

Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and

Associated Equipment, 581(3), 799–815.Stankovic, M. S., & Stipanovic, D. M. (2010). Extremum seeking under stochastic

noise and applications to mobile sensors. Automatica, 46(8), 1243–1251.Tan, Y., Nesic, D., Mareels, I. M. Y., & Astolfi, A. (2009). On global extremum seeking

in the presence of local extrema. Automatica, 45(1), 245–251.Tveito, A., & Winther, R. (1998). Introduction to partial differential a computational

approach. New York: Springer-Verlag.Wang, H.-H., & Krstic, M. (2000). Extremum seeking for limit cycle minimization.

IEEE Transactions on Robotics and Automation, 45(12), 2432–2437.Wang, H.-H., Yeung, S., & Krstic, M. (2000). Experimental application of extremum

seeking on an axial-flow compressor. IEEE Transactions on Control Systems

Technology, 8(2), 300–309.Zhang, C., Arnold, D., Ghods, N., Siranosian, A., & Krstic, M. (2007). Source seeking

with nonholonomic unicycle without position measurement and with tuning

of forward velocity. Systems & Control Letters, 56(3), 245–252.