Slide 1

Sensitivity derivativesCan obtain sensitivity derivatives of structural response at several levelsFinite difference sensitivity (section 7.1)Analytical sensitivity of continuum equations (Chapter 8)Analytical sensitivities of discretized equations (Chapter 7)Analytical sensitivities of computer program (survey paper by van Keulen, Haftka and Kim).

We normally calculate structural response with a structural analysis program, usually finite-element based. To calculate derivatives we can simply use finite differences. This is what Matlab would do for us if we call an optimization routine like fmincon and do not provide it with a function that calculates gradients.

It turns out that this is sometimes tricky, and so it is the main subject of todays lecture, and it is covered in Section 7.1 of Haftka and Gurdals Elements of Structural Optimization.

Other approaches will be covered in other lectures and are available in textbooks or papers. It is possible to differentiate the continuum equations that are solved by finite elements, an approach tht is covered in Chapter 8 of Haftka and Gurdal. It is possible to differentiate the finite element equations, which is the main topic of Chapter 7.

Finally, it is possible to differentiate the finite element computer program, something that is covered in the survey paper

Van Keulen, F., Haftka, R.T, and Kim, N.H., Review of options for structural design sensitivity analysis. Part 1: Linear systems, Computer Methods in Applied Mechanics and Engineering, Volume 194, Issues 30-33 , 12 August 2005, Pages 3213-32431Finite difference derivativesForward divided differences

Central divided differences

Central differences usually more accurate. Why dont we usually use them?Higher order formulae also available

We will do most of the derivations for the derivatives of structural response for the displacement vector u. The most commonly used finite difference approximations of the derivatives are the forward difference and central difference approximations.

At first glance both appear to require two function evaluations, and central differences is usually more accurate. So it would appear we should always use central differences.However, when there are n design variables, forward differences require n+1 function evaluations compared to 2n for central differences.2Truncation errorTaylor series expansion

Leading to the following truncation error in forward difference approximation

Similarly for the central difference approx.

When do we expect forward to be more accurate?

The errors associated with the forward and central finite difference derivatives are obtained from the Taylor series expansion. The errors are due to the fact that in order to obtain these two estimates we take only f finite number of terms in the series, so they are called truncation errors. For example, the forward difference approximation is obtained if we take only two terms in the Taylor series expansion.

To estimate the truncation error, we add one more term to the series. It turns out that you are allowed to have a Taylor series with finite number of terms, if the final term has the derivative evaluated somewhere in the middle of the interval instead of the beginning.

Doing the analysis on the basis of the Taylor series with one more term we see that for the forward difference approach the error is proportional to the interval and the second derivative. For the central differences method, the error is proportional to the square of the interval length times the third derivative. That is when we halve the interval, the error in forward differences is halved, while the error in the central difference approximation is reduced by a factor of 4.

Of course, since different derivatives are involved, we can get cases where forward will be more accurate than central. For example, if the second derivative is very small and the third derivative is large.3Example 7.1.1u found as solution to:Derivative at x=100

We take an example of a somewhat ill-conditioned system of equations. At x=100, the two eigenvalues of the matrix of the equations are 0.5 and 200.5, which means that the solution will be sensitive to small errors.

We solve these equations using only single precision arithmetic, and calculate the derivative of u with different step sizes as shown in the figure. The exact value of the derivative du/dx is -0.1. For a range of step sizes from 0.01 to 0.0001, both methods provide accurate derivatives. For lower step sizes the values jump around because of round-off errors in the solution. For higher step sizes the truncation error dominates, and it is clear that central differences stays accurate a bit longer.

Note that the step size in the figure is the relative rather than the absolute step size, so 0.01 corresponds to changing x from 100 to 101.4Now with poorer conditioningNew system for solving for uDerivative at x=10,000

We now change the system to be more ill conditioned. Now the two eigenvalues are 0.5 and 20,000. the region where the derivatives are of acceptable accuracy narrows. For forward differences it is less than a decade, for central differences it is a bit more.5Condition errorFor small step sizes we are limited by the scatter in the numerical calculation of uThis scatter can be caused by: Round-off error due to the use of finite-digit calculations.Convergence criterion for iterative solution techniquesAutomatic remeshingGeneral name for this error is condition error

6Optimal step size for forward difference approximationFormulaTruncation error

Bound for total errorOptimal step size

Example: Calculate optimum step size for

With large errors when the step size is too small or too large, there will be an optimal step size. We recall the equations for the forward-difference derivative and for its truncation errors, and assume that we have a bound sb for the magnitude of the second derivative.

The error in the derivative due to the condition error is maximal when the error is positive at x and negative for x+delta-x or vice versa. Together with the bound on the truncation error we can get a bound for the total error and differentiate it for the step size to get the optimal step size.7Problems step sizeDerive the truncation error for the central difference derivative.Provide an estimate for the optimal step size when using the central difference formula for derivativesHow will you reduce the truncation error with a given step size when you cannot use the central difference formula because you can use only a positive step size (function is not defined on the left)Generate the figure on Slide 5 for the precision used in Matlab

8Effect of derivative magnitudeLarge derivatives are easier to estimate than small ones.What does it mean to say that derivative is large?One measure is logarithmic derivative

What does it mean to have a logarithmic derivative equal to one?

Large derivatives are easier to estimate than small derivatives because they are less affected by noise. It is also common to reduce the number of variables for optimization problems by eliminating variables that have small derivatives. The question do we mean by large or small derivative.

Without some normalization the value of the derivative may be misleading. Derivatives measure the influence of the variable on the function, and so a natural way of normalizing the derivative is asking what percent change in the function will be caused by one percent change in the variable.

This is the idea behind the logarithmic derivative. So if the logarithmic derivative is equal to one, it means that one percent change in x will cause one percent change in u. For example the function u=ax, will have a logarithmic derivative of one.9Large errors in small derivativesExample: y=10+(x-5)2. Compare the accuracy of forward difference derivatives with a step size of one at x=10 and x=6. Relate to size of logarithmic derivative

We use the function y=10+(x-5)2 as an example. At x=10, the function value is 35, and the value of its derivative is 10. So a change of 1% in x (0.1) will cause approximately a change of 1 in the function, which is 2.8% for a logarithmic derivatives of 2.8. When we use a large step size of Deltax=1 for estimating the derivative by forward differences we get 11 instead of 10, or 10% error.

At x=6, the function value is 11, and the value of the derivative is 2. The logarithmic derivative is calculated to be 1.1. Now a step size of 1 gives us a derivative estimate of 3. This is the same absolute error, but it is now an error of 50% in the derivative value.10Some uses of logarithmic derivativesThe logarithmic derivative of y=xn is nIf you link design variables to a single variable, their logarithmic derivatives will add upIf you scale up all the cross-sectional areas of a truss, or all the moment of inertias of a frame, displacements and stresses will scale inverselySo the sum of the logarithmic derivatives of displacement of a truss with respect to all cross-sectional areas is -1What are the implications on each derivative?

Here are some nice properties of the logarithmic derivatives. The first one, that the logarithmic derivative of xn is n, is good for estimation, when you know the approximate dependence of a function on a variable.

More useful is the fact that if you combine several design variables to a single one, the derivatives will not add, but the logarithmic derivatives will. This has implications some time when there are simple scaling effects.

For example, when you scale up all the stiffnesses of a structure by a constant, the displacement will scale by the inverse of this stiffness. That means that the sum of all the logarithmic derivatives is -1. That means that if the structure has many components, the effect of each one of them on the displacements tend to be small.11Problems logarithmic derivativeRather than the size of the logarithmic derivative what is the true ratio that determines the relative accuracy of finite difference derivative calculations?Logarithmic derivatives do not make much sense when the function changes sign. What else can you use to normalize the derivative in that case?