multiple regression analysis (mra)

35
MULTIPLE REGRESSION ANALYSIS (MRA) Design requirements Multiple regression model R 2 Comparing standardized regression coefficients

Upload: gayora

Post on 05-Jan-2016

66 views

Category:

Documents


7 download

DESCRIPTION

Multiple Regression Analysis (MRA). Design requirements Multiple regression model R 2 Comparing standardized regression coefficients. Steps in data analysis. Look first at each variable separately Then at relationships among the variables - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Multiple  Regression Analysis (MRA)

MULTIPLE REGRESSION ANALYSIS (MRA)

Design requirements

Multiple regression model

R2

Comparing standardized regression coefficients

Page 2: Multiple  Regression Analysis (MRA)

STEPS IN DATA ANALYSIS Look first at each variable separately Then at relationships among the variables

Examine the distribution of each variable to be used in multiple regression to determine if there are any unusual patterns that may be important in building our regression analysis.

Page 3: Multiple  Regression Analysis (MRA)

DISTRIBUTION OF VARIABLES

Page 4: Multiple  Regression Analysis (MRA)

CORRELATION ANALYSIS If interested only in determining whether a relationship exists, use

correlation analysis. Example: Student’s height and weight.

Plot of Height vs Weight

100 140 180 220 260

Weight

4.6

5

5.4

5.8

6.2

6.6

7

Hei

ght

Plot of Height vs Weight

100 140 180 220 260

Weight

5.3

5.6

5.9

6.2

6.5

6.8

Hei

ght

Plot of Height vs Weight

100 140 180 220 260

Weight

5.4

5.8

6.2

6.6

7

Hei

ght

Plot of Height vs Weight

100 140 180 220 260

Weight

5

5.4

5.8

6.2

6.6

Hei

ght

Page 5: Multiple  Regression Analysis (MRA)

CORRELATION ANALYSIS

Correlation coefficient close to +1=strong positive relationship.

Correlation coefficient close to -1= strong negative relationship.

Correlation coefficient close to 0= no relationship.

Page 6: Multiple  Regression Analysis (MRA)

EXAMPLE: SELF CONCEPT AND ACADEMIC ACHIEVEMENT (N=103)

CORRELATION

Page 7: Multiple  Regression Analysis (MRA)

MULTIPLE REGRESSION ANALYSIS (MRA)

Method for studying the relationship between a dependent variable and two or more independent variables.

Purposes: Prediction Explanation Theory building

Page 8: Multiple  Regression Analysis (MRA)

DESIGN REQUIREMENTS

One dependent variable (criterion)

Two or more independent variables (predictor variables).

Sample size: >= 50 (at least 10 times as many cases as independent variables)

Page 9: Multiple  Regression Analysis (MRA)

ASSUMPTIONS

Independence: The scores of any particular subject are independent of the scores of all other subjects

Normality: In the population, the scores on the dependent variable are normally distributed for each of the possible combinations of the level of the X variables; each of the variables is normally distributed

Page 10: Multiple  Regression Analysis (MRA)

ASSUMPTIONS Homoscedasticity: In the population, the

variances of the dependent variable for each of the possible combinations of the levels of the X variables are equal.

Linearity: In the population, the relation between the dependent variable and the independent variable is linear when all the other independent variables are held constant.

Page 11: Multiple  Regression Analysis (MRA)

HOMOSCEDASTICITY(HOMOGENEITY OF VARIANCE)

Page 12: Multiple  Regression Analysis (MRA)

LINEAR REGRESSION

In simple linear regression the relationship between one explanatory variable (IV) and one response variable (DV).

In multiple regression, several explanatory variables work together to explain the dependent variable.

Page 13: Multiple  Regression Analysis (MRA)

MODELS

Page 14: Multiple  Regression Analysis (MRA)

WHAT IS A MODEL?

Representation of Some PhenomenonRepresentation of Some Phenomenon

(Non-Math/Stats Model)(Non-Math/Stats Model)

Page 15: Multiple  Regression Analysis (MRA)

WHAT IS A MATH/STATS MODEL?

Describe Relationship between Variables

Types- Deterministic Models

(no randomness)

- Probabilistic Models

(with randomness)

Page 16: Multiple  Regression Analysis (MRA)

DETERMINISTIC MODELS

1. Hypothesize Exact Relationships

2. Suitable When Prediction Error is Negligible

3. Example: Body mass index (BMI) is measure of body fat based on this formula.

Non-metric Formula: BMI = Weight (pounds)x703 (Height in inches)2

Page 17: Multiple  Regression Analysis (MRA)

PROBABILISTIC MODELS

1. Hypothesize 2 Components Deterministic Random Error

2. Example: Systolic blood pressure (SBP) of newborns is 6 Times the Age in days + Random Error

SBP = 6xage(d) + Random Error May Be Due to Factors

Other than age in days (e.g. Birth weight)

Page 18: Multiple  Regression Analysis (MRA)

TYPES OF PROBABILISTIC MODELS

ProbabilisticModels

RegressionModels

CorrelationModels

OtherModels

ProbabilisticModels

RegressionModels

CorrelationModels

OtherModels

Page 19: Multiple  Regression Analysis (MRA)

REGRESSION MODELS

Page 20: Multiple  Regression Analysis (MRA)

TYPES OF PROBABILISTIC MODELS

ProbabilisticModels

RegressionModels

CorrelationModels

OtherModels

ProbabilisticModels

RegressionModels

CorrelationModels

OtherModels

Page 21: Multiple  Regression Analysis (MRA)

REGRESSION MODELS Relationship between one dependent variable

and explanatory variable(s)

Use equation to set up relationship Numerical Dependent (Response) Variable 1 or More Numerical or Categorical

Independent (Explanatory) Variables

Used Mainly for Prediction & Estimation

Page 22: Multiple  Regression Analysis (MRA)

REGRESSION MODELING STEPS

1. Hypothesize Deterministic Component Estimate Unknown Parameters

2. Specify Probability Distribution of Random Error Term

Estimate Standard Deviation of Error

3. Evaluate the fitted Model

4. Use Model for Prediction & Estimation

Page 23: Multiple  Regression Analysis (MRA)

MULTIPLE REGRESSION

Very popular among social scientists.Most social phenomena have more than

one cause.

Very difficult to manipulate just one social variable through experimentation.

Social scientists must attempt to model complex social realities to explain them.

Page 24: Multiple  Regression Analysis (MRA)

MULTIPLE REGRESSIONAllows us to:

Use several variables at once to explain the variation in a continuous dependent variable.

Isolate the unique effect of one variable on the continuous dependent variable while taking into consideration that other variables are affecting it too.

Write a mathematical equation that tells us the overall effects of several variables together and the unique effects of each on a continuous dependent variable.

Control for other variables to demonstrate whether bivariate relationships are spurious

Page 25: Multiple  Regression Analysis (MRA)

*** MULTIPLE REGRESSION For example:

A researcher may be interested in the relationship between Education and Income and Number of Children in a family.

Independent Variables

Education

Family Income

Dependent Variable

Number of Children

Page 26: Multiple  Regression Analysis (MRA)

MULTIPLE REGRESSION For example:

Research Hypothesis: As education of respondents increases, the number of children in families will decline (negative relationship).

Research Hypothesis: As family income of respondents increases, the number of children in families will decline (negative relationship).

Independent Variables

Education

Family Income

Dependent Variable

Number of Children

Page 27: Multiple  Regression Analysis (MRA)

MULTIPLE REGRESSION For example:

Null Hypothesis: There is no relationship between education of respondents and the number of children in families.

Null Hypothesis: There is no relationship between family income and the number of children in families.

Independent Variables

Education

Family Income

Dependent Variable

Number of Children

Page 28: Multiple  Regression Analysis (MRA)

MULTIPLE REGRESSION

Model Summary

.757a .573 .534 2.33785Model1

R R SquareAdjustedR Square

Std. Error ofthe Estimate

Predictors: (Constant), Income, Educationa. ANOVAb

161.518 2 80.759 14.776 .000a

120.242 22 5.466

281.760 24

Regression

Residual

Total

Model1

Sum ofSquares df Mean Square F Sig.

Predictors: (Constant), Income, Educationa.

Dependent Variable: Childrenb.

Coefficientsa

11.770 1.734 6.787 .000

-.364 .173 -.412 -2.105 .047

-.403 .194 -.408 -2.084 .049

(Constant)

Education

Income

Model1

B Std. Error

UnstandardizedCoefficients

Beta

StandardizedCoefficients

t Sig.

Dependent Variable: Childrena.

57% of the variation in number of children is explained by education and income!

Page 29: Multiple  Regression Analysis (MRA)

Predictable variation by combination of independent variables

EXPLAINING VARIATION: HOW MUCH?

Total Variation in Y

UnpredictableVariation

Page 30: Multiple  Regression Analysis (MRA)

PROPORTION OF PREDICTABLE AND UNPREDICTABLE VARIATION

X1

Y

(1-R2) = Unpredictable (unexplained) variation in Y

X2

Where:Y= # ChildrenX1 = EducationX2 = Income

R2 = Predictable (explained) variation in Y

Page 31: Multiple  Regression Analysis (MRA)

MULTIPLE REGRESSIONNow… More Variables! The social world is very complex. What happens when you have even more variables?

For example:

A researcher may be interested in the effects of Education, Income, Sex, and Gender Attitudes on Number of Children in a family.

Independent Variables

Education

Family Income

Sex

Gender Attitudes

Dependent Variable

Number of Children

Page 32: Multiple  Regression Analysis (MRA)

SIMPLE VS. MULTIPLE REGRESSION

One dependent variable Y predicted from one independent variable X

One regression coefficient

r2: proportion of variation in dependent variable Y predictable from X

One dependent variable Y predicted from a set of independent variables (X1, X2 ….Xk)

One regression coefficient for each independent variable

R2: proportion of variation in dependent variable Y predictable by set of independent variables (X’s)

Page 33: Multiple  Regression Analysis (MRA)

DIFFERENT WAYS OF BUILDING REGRESSION MODELS

Simultaneous (Enter): All independent variables entered together

Stepwise: Independent variables entered according to some order (Determined by researcher) By size or correlation with dependent variable In order of significance (theory)

Hierarchical (Forward, Backward): Independent variables entered in stages

Page 34: Multiple  Regression Analysis (MRA)

MULTIPLE REGRESSION:BLUE CRITERIA

Regression forces a best-fitting model onto data. If the model is appropriate for the data, regression should be used.

How do we know that our model is appropriate for the data?

Criteria for determining whether a regression model is appropriate for the data are nicknamed “BLUE” for best linear unbiased estimate.

Page 35: Multiple  Regression Analysis (MRA)

MULTIPLE REGRESSION:BLUE CRITERIA

Violating the BLUE assumptions may result in biased estimates or incorrect significance tests. (However, OLS is robust to most violations.)

Data (constellation) should meet these criteria: The relationship between the dependent variable and its

predictors is linear No irrelevant variables are either omitted from or included in

the equation. (Good luck!) All variables are measured without error. (Good luck!)