1. Business Statistics:A First Course(3rd Edition)Chapter 10Simple Linear Regression 2003 Prentice-Hall, Inc. Chap 10-1

2. 2003 Prentice-Hall, Inc.Chap 10-2Chapter Topics Types of Regression Models Determining the Simple Linear RegressionEquation Measures of Variation Assumptions of Regression and Correlation Residual Analysis Measuring Autocorrelation Inferences about the Slope 3. 2003 Prentice-Hall, Inc.(continued)Chap 10-3Chapter Topics Correlation - Measuring the Strength of theAssociation Estimation of Mean Values and Prediction ofIndividual Values Pitfalls in Regression and Ethical Issues 4. Purpose of Regression Analysis Regression Analysis is Used Primarily toModel Causality and Provide Prediction Predict the values of a dependent (response)variable based on values of at least oneindependent (explanatory) variable Explain the effect of the independent variables onthe dependent variable 2003 Prentice-Hall, Inc.Chap 10-4 5. Types of Regression Models 2003 Prentice-Hall, Inc.Chap 10-5Positive Linear RelationshipNegative Linear RelationshipRelationship NOT LinearNo Relationship 6. Simple Linear Regression Model Relationship Between Variables is Describedby a Linear Function The Change of One Variable Causes theOther Variable to Change A Dependency of One Variable on the Other 2003 Prentice-Hall, Inc.Chap 10-6 7. Simple Linear Regression ModelPopulation regression line is a straight line thatdescribes the dependence of the aavveerraaggee vvaalluuee((ccoonnddiittiioonnaall mmeeaann)) of one variable on the otherPopulationY intercepti i i Y b b X e 0 1 = + + 2003 Prentice-Hall, Inc.Chap 10-7PopulationRegressionLine(conditional mean)PopulationSlopeCoefficientRandomErrorDependent(Response)VariableIndependent(Explanatory)VariableY|X m(continued) 8. Simple Linear Regression Model 2003 Prentice-Hall, Inc.(continued)i i i Y b b X e 0 1 = + +Chap 10-8= Random ErrorYX(Observed Value of Y) =Observed Value of YY|X i m b b X 0 1 = +i eb0b1(Conditional Mean) 9. Linear Regression EquationSample regression line provides an eessttiimmaatteeof the population regression line as well as apredicted value of YY =b +b X = Simple Regression Equation 2003 Prentice-Hall, Inc.Chap 10-9SampleY InterceptSampleSlopeCoefficienti 0 1 i i Residual Y = b + b X + e0 1(Fitted Regression Line, Predicted Value) 10. Linear Regression Equation0 b 1 b and are obtained by finding the values of0 b 1 band that minimizes the sum of the n n - =Y Y e0 b b01 b b1 2003 Prentice-Hall, Inc.Chap 10-10squared residuals provides an eessttiimmaattee of provides and eessttiimmaattee of(continued)( )2 2i i i1 1i i= = 11. Linear Regression Equationi 0 1 i i Y = b + b X + e 2003 Prentice-Hall, Inc.(continued)i i i Y b b X e 0 1 = + +Chap 10-11YXObserved ValueY|X i m b b X 0 1 = +i eb0b1Y = b + b Xi 0 1i i e1 b0 b 12. Interpretation of the Slope andY |X 0 b m 0 = =|m 2003 Prentice-Hall, Inc.Chap 10-12Intercept is the average value of Y whenthe value of X is zero. 1measures the change in theaverage value of Y as a result of a one-unitchange in X.Y XXbD=D 13. Interpretation of the Slope and| 0 Y X b m 0 = =Dmb =Y |X 2003 Prentice-Hall, Inc.(continued)Chap 10-13Intercept is the eessttiimmaatteedd average value ofY when the value of X is zero. 1DXis the eessttiimmaatteedd change in theaverage value of Y as a result of a one-unitchange in X. 14. Simple Linear Regression: 2003 Prentice-Hall, Inc.Chap 10-14ExampleYou wish to examinethe linear dependencyof the annual sales ofproduce stores on theirsizes in squarefootage. Sample datafor 7 stores wereobtained. Find theequation of the straightline that fits the databest.AnnualStore Square SalesFeet ($1000)1 1,726 3,6812 1,542 3,3953 2,816 6,6534 5,555 9,5435 1,292 3,3186 2,208 5,5637 1,313 3,760 15. Scatter Diagram: Example12000100008000600040002000 2003 Prentice-Hall, Inc.Chap 10-1500 1000 2000 3000 4000 5000 6000Square FeetAnnual Sales ($000)Excel Output 16. Simple Linear Regressioni i1636.415 1.487 2003 Prentice-Hall, Inc.Chap 10-16Equation: Example0 1iY b b XX= += +From Excel Printout:CoefficientsIntercept 1636.414726X Variable 1 1.486633657 17. Graph of the Simple LinearRegression Equation: Example12000100008000600040002000 2003 Prentice-Hall, Inc.Chap 10-1700 1000 2000 3000 4000 5000 6000Square FeetAnnual Sales ($000)Yi = 1636.415 +1.487Xi 18. Interpretation of Results:Yi = 1636.415 +1.487Xi 2003 Prentice-Hall, Inc.Chap 10-18ExampleThe slope of 1.487 means that each increase of oneunit in X, we predict the average of Y to increase byan estimated 1.487 units.The equation estimates that for each increase of 1square foot in the size of the store, the expectedannual sales are predicted to increase by $1487. 19. Simple Linear Regression in 2003 Prentice-Hall, Inc.Chap 10-19PHStat In Excel, use PHStat | Regression | SimpleLinear Regression EXCEL Spreadsheet of Regression Sales onFootageMicrosoft ExcelWorksheet 20. Measures of Variation:The Sum of SquaresSST = SSR + SSETotalSampleVariability 2003 Prentice-Hall, Inc.Chap 10-20= ExplainedVariability+ UnexplainedVariability 21. Measures of Variation:The Sum of SquaresY 2003 Prentice-Hall, Inc.(continued)Chap 10-21 SST = Total Sum of Squares Measures the variation of the Yivalues aroundtheir mean, SSR = Regression Sum of Squares Explained variation attributable to the relationshipbetween X and Y SSE = Error Sum of Squares Variation attributable to factors other than therelationship between X and Y 22. Measures of Variation:The Sum of Squares 2003 Prentice-Hall, Inc.(continued)Chap 10-22XiYXYSST = (Yi - Y)2SSE =(Yi - Yi )2SSR = (Yi - Y)2___ 23. The ANOVA Table in Excel 2003 Prentice-Hall, Inc.Chap 10-23ANOVAdf SS MS FSignificanceFRegression k SSR MSR=SSR/kMSR/MSE P-value ofthe F TestResiduals n-k-1 SSE MSE=SSE/(n-k-1)Total n-1 SST 24. Measures of VariationThe Sum of Squares: ExampleExcel Output for Produce StoresDegrees of freedom 2003 Prentice-Hall, Inc.Chap 10-24ANOVAdf SS MS F Significance FRegression 1 30380456.12 30380456 81.17909 0.000281201Residual 5 1871199.595 374239.92Total 6 32251655.71SSRSSERegression (explained) dfError (residual) dfTotal dfSST 25. The Coefficient of Determinationr SSR2 Regression Sum of Squares= =SST 2003 Prentice-Hall, Inc.Total Sum of SquaresChap 10-25 Measures the proportion of variation in Y thatis explained by the independent variable X inthe regression model 26. Coefficients of Determination (r 2)r = +1 r = -1^^r = +0.9 r = 0 2003 Prentice-Hall, Inc.Chap 10-26and Correlation (r)r2 = 1, r2 = 1,r2 = .81, Y r2 = 0,^Yi = b0 + b1XiXYYi = b0 + b1XiX^YYi = b0 + b1XiXYYi = b0 + b1XiX 27. Standard Error of EstimateS SSEYX== =n n 2003 Prentice-Hall, Inc.Chap 10-27n( )21iY -Y- -2 2i The standard deviation of the variation ofobservations around the regression equation 28. Measures of Variation:Produce Store ExampleExcel Output for Produce Stores 2003 Prentice-Hall, Inc.Chap 10-28Regression StatisticsMultiple R 0.9705572R Square 0.94198129Adjusted R Square 0.93037754Standard Error 611.751517Observations 7r2 = .9494% of the variation in annual sales can beexplained by the variability in the size of thestore as measured by square footageSyx 29. Linear Regression Assumptions 2003 Prentice-Hall, Inc.Chap 10-29 Normality Y values are normally distributed for each X Probability distribution of error is normal 2. Homoscedasticity (Constant Variance) 3. Independence of Errors 30. Variation of Errors Around the 2003 Prentice-Hall, Inc. Y values are normally distributedaround the regression line. For each X value, the spread orvariance around the regression line isthe same.Chap 10-30Regression LineX1X2XYf(e )Sample Regression Line 31. 2003 Prentice-Hall, Inc.Chap 10-31Residual Analysis Purposes Examine linearity Evaluate violations of assumptions Graphical Analysis of Residuals Plot residuals vs. X and time 32. Residual Analysis for Linearitye e 2003 Prentice-Hall, Inc.Chap 10-32XNot Linear LinearXYXYX 33. Residual Analysis forHomoscedasticityYHeteroscedasticity Homoscedasticity 2003 Prentice-Hall, Inc.Chap 10-33SRXSRXYX X 34. Residual Analysis:Excel Outputfor Produce Stores ExampleExcel Output 2003 Prentice-Hall, Inc.Chap 10-34Residual Plot0 1000 2000 3000 4000 5000 6000Square FeetObservation Predicted Y Residuals1 4202.344417 -521.34441732 3928.803824 -533.80382453 5822.775103 830.22489714 9894.664688 -351.66468825 3557.14541 -239.14541036 4918.90184 644.09816037 3588.364717 171.6352829 35. Residual Analysis fore e( ) 2003 Prentice-Hall, Inc.Chap 10-35Independence The Durbin-Watson Statistic Used when data is collected over time to detectautocorrelation (residuals in one time period arerelated to residuals in another period) Measures violation of independence assumption21221ni iiniiDe-==-=Should be close to 2.If not, examine the modelfor autocorrelation. 36. Durbin-Watson Statistic in 2003 Prentice-Hall, Inc.Chap 10-36PHStat PHStat | Regression | Simple LinearRegression Check the box for Durbin-Watson Statistic 37. Obtaining the Critical Values ofDurbin-Watson StatisticTable 13.4 Finding critical values of Durbin-Watson Statisticn dL dU dL dU15 1.08 1.36 .95 1.5416 1.10 1.37 .98 1.54 2003 Prentice-Hall, Inc.a = .0 5k=1 k=2Chap 10-37 38. Using the Durbin-Watson: No autocorrelation (error terms are independent): There is autocorrelation (error terms are notindependent)Reject H0(positiveautocorrelation) 2003 Prentice-Hall, Inc.Inconclusive Reject H0(negativeautocorrelation)Chap 10-38StatisticAccept H0(no autocorrelatin)0 H1 H0 d 2 4 L 4-dL dU 4-dU 39. Residual Analysis forGraphical ApproachCyclical Pattern No Particular Pattern 2003 Prentice-Hall, Inc.Chap 10-39IndependenceNot Independent Independent e eTime TimeResidual Is Plotted Against Time to Detect Any Autocorrelation 40. Inference about the Slope:t b S S1 1 2003 Prentice-Hall, Inc.Chap 10-40t Test t Test for a Population Slope Is there a linear dependency of Y on X ? Null and Alternative Hypotheses H0: b1 = 0 (No Linear Dependency) H1: b1 0 (Linear Dependency) Test Statistic1121whereYX( )b nbiiSX Xb== - = -d. f . = n - 2 41. Example: Produce Store 2003 Prentice-Hall, Inc.Chap 10-41Data for 7 Stores:Estimated RegressionEquation:The slope of thismodel is 1.487.Does SquareFootage AffectAnnual Sales?AnnualStore Square SalesFeet ($000)1 1,726 3,6812 1,542 3,3953 2,816 6,6534 5,555 9,5435 1,292 3,3186 2,208 5,5637 1,313 3,760Y =1636.415 +1.487Xi 42. Inferences about the Slope:: b1 0a = .05df = 7 - 2 = 5Critical Value(s):From Excel Printout1 b b1 S tIntercept 1636.4147 451.4953 3.6244 0.01515Footage 1.4866 0.1650 9.0099 0.00028Reject Reject.025.025 2003 Prentice-Hall, Inc.Coefficients Standard Error t Stat P-valueChap 10-42t Test ExampleH0: b1 = 0H1Test Statistic:Decision:Conclusion:There is evidence thatsquare footage affectsannual sales.-2.5706 0 2.5706 tReject H0 43. Inferences about the Slope:Confidence Interval ExampleConfidence Interval Estimate of the Slope:Intercept 475.810926 2797.01853X Variable 11.06249037 1.91077694 2003 Prentice-Hall, Inc.Chap 10-431 n 2 b1 b t S - Excel Printout for Produce StoresLower 95% Upper 95%At 95% level of confidence the confidence intervalfor the slope is (1.062, 1.911). Does not include 0.Conclusion: There is a significant linear dependencyof annual sales on the size of the store. 44. Inferences about the Slope:SSR12F SSE( n)=- 2003 Prentice-Hall, Inc.Chap 10-44F Test F Test for a Population Slope Is there a linear dependency of Y on X ? Null and Alternative Hypotheses H0: b1 = 0 (No Linear Dependency) H1: b1 0 (Linear Dependency) Test Statistic Numerator d.f.=1, denominator d.f.=n-2 45. Relationship between a t Test 2003 Prentice-Hall, Inc.Chap 10-45and an F Test Null and Alternative Hypotheses H: b= 0 (No Linear Dependency)01 H1: b1 0 (Linear Dependency)( t )2=F n - 2 1,n - 2 46. Inferences about the Slope:F Test ExampleReject 2003 Prentice-Hall, Inc.From Excel PrintoutReject H0Chap 10-46ANOVATest Statistic:df SS MS F Significance FRegression 1 30380456.12 30380456.12 81.179 0.000281Residual 5 1871199.595 374239.919Total 6 32251655.71Decision:Conclusion:H0: b1 = 0H1: b1 0a = .05numeratordf = 1denominatordf = 7 - 2 = 5There is evidence thatsquare footage affectsannual sales.0 6.61a = .051,n 2 F - 47. Purpose of Correlation Analysis Correlation Analysis is Used to MeasureStrength of Association (Linear Relationship)Between 2 Numerical Variables Only Strength of the Relationship is Concerned No Causal Effect is Implied 2003 Prentice-Hall, Inc.Chap 10-47 48. Purpose of Correlation Analysis(continued) Population Correlation Coefficient r (Rho) isUsed to Measure the Strength between theVariables Sample Correlation Coefficient r is anEstimate of rand is Used to Measure theStrength of the Linear Relationship in theSample Observations 2003 Prentice-Hall, Inc.Chap 10-48 49. Sample of Observations fromVarious r Valuesr = -1 r = -.6 r = 0 2003 Prentice-Hall, Inc.r = .6 r = 1 Chap 10-49YXYXYXYXYX 50. Features of r and r Unit Free Range between -1 and 1 The Closer to -1, the Stronger the NegativeLinear Relationship The Closer to 1, the Stronger the PositiveLinear Relationship The Closer to 0, the Weaker the LinearRelationship 2003 Prentice-Hall, Inc.Chap 10-50 51. t Test for Correlation 2003 Prentice-Hall, Inc.Chap 10-51 Hypotheses H0: r = 0 (No Correlation) H1: r 0 (Correlation) Test Statisticwhere( ) ( )( ) ( )2 2 12 21 12ni iin ni ii it rrnX X Y Yr rX X Y Yr== == -1 --- -= =- - 52. Example: Produce StoresIs there anyevidence of linearrelationship betweenAnnual Sales of astore and its SquareFootage at .05 levelof significance? H0: r = 0 (No association) 2003 Prentice-Hall, Inc.From Excel Printout rRegression StatisticsMultiple R 0.9705572R Square 0.94198129Adjusted R Square 0.93037754Standard Error 611.751517Observations 7H1: r 0 (Association)a = .05df = 7 - 2 = 5Chap 10-52 53. Example: Produce Stores= - r = =1- --Critical Value(s):Reject Reject.025.025 2003 Prentice-Hall, Inc.Chap 10-53Solution-2.5706 0 2.5706Decision:Reject H0Conclusion:There is evidence of alinear relationship at 5%level of significance2.9706 9.00991 .94202 5t rrnThe value of the t statistic isexactly the same as the tstatistic value for test on theslope coefficient 54. Estimation of Mean ValuesConfidence Interval Estimate for :The Mean of Y given a particular XiY t S + X -XY|X Xi m = 1 ( ) - t value from tablewith df=n-2 2003 Prentice-Hall, Inc.2Chap 10-5422n X X1i( )i n YX nii-=Standard errorof the estimateSize of interval vary according todistance away from mean, X 55. Prediction of Individual ValuesPrediction Interval for IndividualY t S + + X -X 1 1 ( ) 2003 Prentice-Hall, Inc.Chap 10-55Response Yi at a Particular XiAddition of 1 increases width of intervalfrom that for the mean of Y222n X X -1i( )i n YX nii-= 56. Interval Estimates for Different 2003 Prentice-Hall, Inc.ConfidenceInterval for themean of YChap 10-56Values of XYXPrediction Intervalfor a individual YiA given XYi = b0 + b1XiX 57. Example: Produce Stores 2003 Prentice-Hall, Inc.Consider a storewith 2000 squarefeet.Chap 10-57Data for 7 Stores:Regression EquationObtained:AnnualStore Square SalesFeet ($000)1 1,726 3,6812 1,542 3,3953 2,816 6,6534 5,555 9,5435 1,292 3,3186 2,208 5,5637 1,313 3,760Y =1636.415 +1.487Xi 58. Estimation of Mean Values:Confidence Interval Estimate for Y|X Xi m =Y t S X X + - = 1 ( ) 4610.45 612.66n X X - 2003 Prentice-Hall, Inc.Chap 10-58ExampleFind the 95% confidence interval for the average annualsales for stores of 2,000 square feet2221i( )i n YX nii-=Predicted Sales 1636.415 1.487 4610.45( $000) i Y = + X =2 5 2350.29 611.75 2.5706 YX n X S t t - = = = = 59. Prediction Interval for Y :Prediction Interval for IndividualY t S X X + + - = 1 1 ( ) 4610.45 1687.68n X X 2003 Prentice-Hall, Inc.Chap 10-59ExampleFind the 95% prediction interval for annual sales of oneparticular store of 2,000 square feetPredicted Sales)222 -1i( )i n YX nii-=X Xi Y = 1636.415 1.487 4610.45( $000) i Y = + X =2 5 2350.29 611.75 2.5706 YX n X S t t - = = = = 60. Estimation of Mean Values andPrediction of Individual Values in PHStat In Excel, use PHStat | Regression | SimpleLinear Regression Check the Confidence and Prediction Interval forX= box EXCEL Spreadsheet of Regression Sales onFootage 2003 Prentice-Hall, Inc.Chap 10-60Microsoft ExcelWorksheet 61. Pitfalls of Regression Analysis Lacking an Awareness of the AssumptionsUnderlining Least-squares Regression Not Knowing How to Evaluate the Assumptions Not Knowing What the Alternatives to Least-squares 2003 Prentice-Hall, Inc.Chap 10-61Regression are if a ParticularAssumption is Violated Using a Regression Model Without Knowledgeof the Subject Matter 62. Strategy for Avoiding the Pitfalls 2003 Prentice-Hall, Inc.Chap 10-62of Regression Start with a scatter plot of X on Y to observepossible relationship Perform residual analysis to check theassumptions Use a histogram, stem-and-leaf display, box-and-whisker plot, or normal probability plot ofthe residuals to uncover possible non-normality 63. Strategy for Avoiding the Pitfalls 2003 Prentice-Hall, Inc.(continued)Chap 10-63of Regression If there is violation of any assumption, usealternative methods (e.g., least absolutedeviation regression or least median ofsquares regression) to least-squaresregression or alternative least-squares models(e.g., curvilinear or multiple regression) If there is no evidence of assumption violation,then test for the significance of the regressioncoefficients and construct confidence intervalsand prediction intervals 64. 2003 Prentice-Hall, Inc.Chap 10-64Chapter Summary Introduced Types of Regression Models Discussed Determining the Simple LinearRegression Equation Described Measures of Variation Addressed Assumptions of Regression andCorrelation Discussed Residual Analysis Addressed Measuring Autocorrelation 65. 2003 Prentice-Hall, Inc.(continued)Chap 10-65Chapter Summary Described Inference about the Slope Discussed Correlation - Measuring theStrength of the Association Addressed Estimation of Mean Values andPrediction of Individual Values Discussed Pitfalls in Regression and EthicalIssues