ws5-1 adm730, workshop 5, september 2005 copyright 2005 msc.software corporation workshop 5 results...

WS5-1ADM730, Workshop 5, September 2005Copyright 2005 MSC.Software Corporation

WORKSHOP 5

RESULTS INTERPRETATION

Response = 3 + 7X1 + X2 + 4X1X2 + 5X22

-5 <= X1 <= 5-5 <= X2 <= 5


WORKSHOP 5 – RESULTS INTERPRETATION

Problem statementUse ADAMS/Insight to design an experiment to discover a known response. Use visual cues and the ADAMS/Insight online help when considering statistics.

Response = 3 + 7X1 + X2 + 4X1X2 + 5X22

-5 <= X1 <= 5-5 <= X2 <= 5



Getting started First you will start ADAMS/Insight.

To start ADAMS/Insight:1. Change to the exercise_dir/mod_05_results_interpretation

directory.

2. Start ADAMS/Insight standalone from the command line as follows: UNIX: adams05r2 -c ainsight

Windows: adams05r2 ainsight

You'll use ADAMS/Insight to design an experiment aimed at

discovering this governing equation.



Response = 3 + 7X1 + X2 + 4X1X2 + 5X22

Consider the model being investigated

1. Look at the response that has been defined above. This equation is the real system that ADAMS/Insight will model. What is the highest order term in this equation? _____________________________________

2. ADAMS/Insight can fit Response Surfaces that are of linear, interactions, quadratic, or cubic form. Considering the know Response definition above, circle the model below that is likely the best fit for this experiment:

Linear

Interactions

Quadratic

Cubic



Building the model You will build a full factorial Response Surface Methodology (RSM)

experiment using two factors and a quadratic model to capture the squared term effects in the response definition equation.

To build the model:1. Create a new experiment in ADAMS/Insight:

From the File menu, select New.

2. Add two factors to the experiment by doing one of the following: From the Define menu, select Factor.

From the toolbar, select the Add factor button.



3. Modify the factors as follows: Name: leave default

Abbreviation: give meaningful name (factor_1, factor_2) Type: Continuous Delta Type: Absolute Settings: -5, 5

Leave all other settings at defaults.

4. Select Apply to save your changes.

5. Create a response by doing one of the following: From the Define menu, select Response.

From the toolbar, select the Add response tool.

6. Assign a meaningful abbreviation (response_1) to your response. Leave all other default values.



7. Select Apply.

8. Build the experiment by doing one of the following: Press the Design specification button.

From the Define menu, point to Experiment Design, and then select Set Design Specification.

9. Define the experiment as follows: DOE Response Surface Quadratic Full Factorial

Standard run order

10. Select Apply.

11. Select the Create Work Space button.



Using the Work Space Column Calculator Now that the experiment is built, the response values must be filled

in. To do this you can export a workspace from ADAMS/Insight as a comma-separated value (csv) file that can be read in to most spreadsheet applications, or use the workspace column calculator that is built into ADAMS/Insight.

In this module, we will use the workspace column calculator in ADAMS/Insight to enter the formula for the known response.

This calculator lets you enter simple expressions for any column of the workspace. So, you can use it to:

Calculate response values Scale or change factor values

The instructions for exporting a workspace in the form of a .csv file are included for reference only.



To use the workspace column calculator:

1. From the Tools menu, select Work Space then Column Calculator.

The calculator appears as shown next:

2. From the Column to Compute menu, select Response_1.



3. In the Expression text box, enter the following:

3 + 7*factor_1 + factor_2 + 4*factor_1*factor_2 + 5*factor_2**2

where factor_1 and factor_2 are the abbreviations for the two factors.

4. Select OK.

Your workspace should now be complete and look as shown next.



To export the work space: Following are the instructions for exporting a workspace in the form

of a .csv file. They are included for reference only, as we used the workspace calculator instead. If you had data coming from a different source, or the workspace calculator didn't suit your needs, you can use the following steps to export the incomplete workspace and then import the completed workspace.

1. Export the incomplete workspace as follows: From the File menu, point to Export, and then select Work Space. Assign a name, such as workspace_export. ADAMS/Insight

automatically appends the .csv extension.

2. Import this new .csv file into a spreadsheet application and fill in the response column using the formula from the problem definition:

Response = 3 + 7X1 + X2 + 4X1X2 + 5X22

3. Save the completed spreadsheet as workspace_completed.csv and import the workspace into ADAMS/Insight (from the File menu, point to Import, and then select Work Space).



Fitting Results Before analyzing the results, the fit must be done on the work

space.

To fit the results: Fit the model to the data using the Fit results tool on the

toolbar.

Analyzing results Now you can analyze the results by looking at the model analysis

summary.

To analyze the results:1. Under Analysis, select the Model_01 node.

2. Select the response from the Regression list in the form, then select the Fit Display.

The green icons indicate that the predicted response surface fit match the test data very well.



Results interpretation: find definitions in the online help:1. From the Help menu, select ADAMS/Insight Help.

The online help for ADAMS/Insight appears.

2. On the left side of the window, select Glossary.

A list of ADAMS/Insight terms appears.

3. Select a term and view its definition.

You can also search the entire ADAMS/Insight help by selecting the Search button near the top of the page.

Now you will step through some of the more commonly used statistics and see if they make sense in the context of this example.



To review summary results:

1. Expand the Analysis node, and select Model_01.

2. In the form, under Regression, select Summary.

3. Under Display, select Rules-of-thumb summary.

4. Considering the problem definition, along with the chosen model fit type, does it seem reasonable that the Fit and Residuals items have green icons indicating a good fit?

_____________________________________________________



To review goodness-of-fit results:1. Inspect the Fit statistics by selecting the Goodness-of-fit item. All

of the statistics presented indicate how well the model fits the measured data. Open the ADAMS/Insight glossary and look up the definition of R-squared (the R2 statistic under Goodness-of-fit). Does the model fit the experimental data well?

_____________________________________________________

2. Next look at the Adjusted R-squared definition in the ADAMS/Insight glossary. The present model should have a value of 1, indicating a perfect fit. If you had fit a linear model and found poor R2 and R2adjusted statistics, you could go back and re-fit a quadratic model to your results.



3. The P statistic indicates whether or not a regression or term is insignificant. In the model summary context, the P statistic refers to whether or not the regression fit is insignificant (see Probability and Regression Significance in the ADAMS/Insight glossary for more information). What is the value for the present model? ____________________

4. Does this mean that there is a high or low probability that the model fit is a good one for this experiment (circle one)?

High Low

5. The R/V or Range-to-variance ratio, indicates how well the model predicts values at the data points. The ADAMS/Insight glossary indicates that a high value for the R/V ratio is what? ________________



Reviewing Term significances

The statistics presented in this node indicate which terms of your model are useful and which aren't. This means that if you've used a quadratic model and all of the squared terms related to your factors are insignificant, then perhaps you should reduce the order of your model (to an interactions model, or a linear one).

Before looking at the significance values, investigate the terms displayed. Remember the general form of the model that you have fit in this experiment. The single response is to be approximated as a quadratic response surface equation in the two unknowns (factors). Therefore, ADAMS/Insight fits the trial run data to an quadratic equation of the form:

Response = A + BX1 + CX2 + DX1X2 + EX12 + FX2

2

where the least-squares fitting process determines the A, B, C, D, E, and F coefficients. Take a look at the description of each line in the Term column.



To review significance results:1. Inspect the statistics by selecting the Term significances entry.

2. Does it make sense that there are 6 terms? _______________

3. How many terms were actually specified in the problem statement equation? _______________

As stated in the ADAMS/Insight glossary, the Term Significance values under response_1 show the probability that the particular term does not affect the response.

4. Is it proper that the terms with very small values have green (good) icons? __________________



5. Is it proper that the only term having a bad icon (red with an X) is one of the squared terms? (Refer to the response definition equation specified earlier.) Why is the squared term bad?

___________________________________________________________

6. To emphasize the above, suppose the response definition equation that was initially specified had looked like this:

Response = 3 + 7X1 + 7X2 + 4X1X2

And you had fit the same quadratic model. Which of the terms in the Significances table would likely have had bad icons?

__________________________________________________



Reviewing Coefficients To review coefficient results:

1. Scroll down to the Term coefficients entry. Consider the assumed response surface equation:

Response = A + BX1 + CX2 + DX1X2 + EX12 + FX2

2

and the response definition equation:

Response = 3 + 7X1 + X2 + 4X1X2 + 5X22

It's quite clear from this table the term coefficients that ADAMS/Insight has determined for the unknowns (A, B, C, D, E, F) in the quadratic model.

2. All of the coefficients are correct (the constant term is 3, the X1 term is 7, and so on) except for the X1

2 term. Shouldn't this term be zero?

_____________________________________________



Reviewing residuals The last item that you’ll look at under the Summary node is

Residuals. This column is a quick way to verify that your response surface is accurate, showing the difference between the actual trial value and the value as predicted by the response surface equations. These values are all essentially zero for this experiment, as expected.

Each response has a Terms node underneath it that quickly summarizes the important points discussed previously.

To look at the responses:1. Under Regression, select response_1.2. Under Display, select Terms. You can quickly see the:

Coefficient for each term Probability that the term is not insignificant (icons show this at a

glance) Definition of the term Other information, such as Standard Error and T test scores



The Fit entry gives model statistics at a glance. The R2 and R2adjusted values are likely of the most interest; you can review the other statistics if the R2 and R2adjusted values don’t look good.

The difference between the least squares predicted response surface and the actual model value is shown under the Residuals node for each of the trial runs that was performed.

3. Look at the values under the Residuals entry to ensure that the residual (error) values are about the same for each trial.

An even distribution of error throughout all the trial runs is likely a sign of a reasonable response surface fit to the design space. If only a few of the runs contained most of the error, perhaps your design space is too complex for the order of model that you’re trying to fit.

The ADAMS/Insight glossary defines the condition number, warning against results that present large condition numbers.



Refining the model The simple example that you’ve been looking at presents all the

ideal statistics (R2 = 1, P = 1.58E-30, and so on) that likely won’t be found in a real experiment. What can you do if the statistics indicate problems with your model? You can refine your model by doing the following:

From the Tools menu, point to Refine Model. Notice that you have four choices:

Remove Outliers: Lets you completely remove a single trial from the results. This is more applicable in a physical DOE when you have inadvertently altered other factors and want to remove the trial.

Remove Terms: Lets you remove a term from the fit. Perhaps your fit has an interactions term, but there is no actual confounding in the experiment. The interactions term has a high probability of not affecting the response (the red circle with black X), so you want to remove this term completely from the fit model.



Transform Response: Lets you scale your results.

Change Order: Lets you change the entire model order. This does the same thing as the Remove Terms function, changing the assumed equation of the response surface.

Now refine the existing model by getting rid of the bad term in the response equation.

To make the fit even better:1. Referring back to the Terms node under response_1, which term

could be removed from the assumed response equation? ______________________________________

2. Remove the X12 term from the model. Before doing this, write down

the value of P in the Goodness-of-fit node. ___________________



3. Remove the X12 factor:

From the Tools menu, point to Refine Model, and then select Remove Terms.

Select factor_12.

4. What is the new P-value for the model? __________________________

Note how the factor_12 row in the Terms node has been removed.

The response surface equation and the defined response now have the exact same terms so the model fit is even better. What if you had originally chosen an interactions fit for the model? How good of a fit could you achieve with a linear model for this problem? This can be investigated as well using the Refinements node.



To reduce the model order to an interactions fit: 1. Put the factor_12 term back into the model:

From the Tools menu, point to Refine Model, and then select Remove Terms.

Clear the selection of factor_12.

Select OK.

Note how the factor_12 row reappears.

2. From the Tools menu, point to Refine Model, and then select Change Order.

3. Select Interactions.

4. Select OK.



To look at the new fit statistics:1. Investigate the Fit and Terms nodes under response_01. How

well does the interactions model approximate the response? __________________

2. Do the term coefficients look correct? __________________

3. Which term changed the most in the new fit?

___________________________________________

4. A linear model isn’t appropriate for fitting the current design space. What if the factor range was reduced to a smaller interval such as -0.5 <= X <= 0.5. Could a linear response surface be used on a smaller portion of the design space?

___________________________________

ws5-1 adm730, workshop 5, september 2005 copyright 2005 msc.software corporation workshop 5 results...

Documents